Penn Library Assessment Program

1
Penn Library Assessment Program
Paul H. Mosher
Vice Provost
Director of Libraries University of
Pennsylvania
2

• Issues for this morning
• The Library Data Farm
• Use Data
• Service Impact Quality (LibQuals Daughter)

3
Early Work on the Grand Canyon
from Great Moments in Architecture by David
Macaulay
4

5
The Data Farm
6
(No Transcript)
7
Schematic of the Data Farm As of May 2002
8

Data Farm Projects and Data Sets

9
Use Data
10
Information Base, Data, E-Journal Usage
Statistics

11


12
Use of Licensed Resources
What Databases Do Our Clients Use at What Cost?
15 Most Frequently Used Index/Abstract/Full-text
Databases in FY 2001
Database
Log-ins Pct Total Cost Per Login
13
Use of Licensed Resources
How Does Use Scatter Across Databases
Use Measured in Log-ins for FY 2001
14
ScienceDirect Articles Viewed, FY 2001
15
Academic Press (Ideal) Articles Viewed, FY 2001
16
Use of Licensed Resources

How Much Bang Do We Get on the Dollar For
E-Journals?
E-Journal Subscription Costs Per Log-In, FY2002
(July-April)
Publisher Log-ins Pct
of Total Cost Per Login
ScienceDirect 139,727 27.1 0.63 ECO
70,730 13.7 0.09 JSTOR 48,668 9.4 0.35 Wil
ey 38,255 7.4 0.09 ACS
31,865 6.2 0.12 Ideal 30,568 5.9 5.51 Blac
kwell/Munksgaard 28,940 5.6 0.27 Journals_at_Ovid
26,982 5.2 n/a Oxford 14,819 2.9 0.20 Sprin
gerLINK 13,507 2.6 n/a ABI/Inform
12,785 2.5 3.08 Project Muse
11,438 2.2 1.22 AIP 7,873 1.5 5.01 Cambrid
ge 7,835 1.5 n/a Annual Reviews
7,215 1.4 0.08 IEEE 7,132 1.4 6.73 RSC
5,661 1.1 n/a Others 11,451 2.2 Total 515
,451 100 11 publishers
17
Database Use by Penns Schools Centers
Use of Licensed Resources
School Pct of Log-ins
How Does Database Use Distribute By Communities?
Per Capita Use of Databases by Penns Schools and
Centers, FY 2001
55
50
45
40
35
30
Log-ins Per Capita
25
20
15
10
5
0
LAW
VET
ASC
MED
NUR
SAS
GSE
SSW
SEAS
GSFA
WHRT
ADM
DENTAL
School and Center Domains
Does not include resources licensed by the Law
Library for Law school affiliates
18
Use of Licensed Resources
Database E-Journal Log-ins by Discipline (based
on log samples from FY2001)
Discipline
Human. Life Social Business Physical Total Sc
ience Science Science Administration 21.1 36.
5 13.9 07.0 21.6 100.0 Wharton 02.9 74.3
03.2 19.2 00.5 100.0 Annenberg
15.2 32.1 42.3 08.9 01.5 100.0 Medical 0
2.3 86.0 01.9 01.0 08.8 100.0 Dental 01.8
87.7 08.9 00.2 01.4 100.0 Veterinary 01.7
96.0 00.6 00.4 01.3 100.0 Dialin 08.5 63.
2 09.9 15.4 02.9 100.0 Education 24.6 13.1
61.5 00.8 00.0 100.0 Fine
Arts 29.0 18.5 45.7 5.6 01.2 100.0 Law 13
.0 26.6 20.9 37.0 02.4 100.0 Library 21.3
54.8 09.1 08.5 06.3 100.0 Nursing 15.9 73
.1 07.8 03.2 00.0 100.0 Student
Residences 18.9 57.0 12.6 09.0 02.5 100.0 A
rts and Sciences 08.2 26.3 5.7 09.9 49.9 100.
0 Engineering 0 1.5 29.5 2.3 01.2 65.6 10
0.0 Social Work 20.6 29.1 41.6 06.1 02.7 1
00.0 Unresolved 18.9 44.7 17.8 10.0 08.6 1
00.0 Total 14.7 50.7 11.9 8.6
14.1 100.0
Network Domain
19
Use of Licensed Resources
Where Do Our Clients Access Information?
Database Log-ins by Domain, FY2001
Campus Residences 10
Off-Campus 15
In-Library 25
On-Campus Depts 50
20
Use of Licensed Resources
Where Do Communities of Clients Work?
Database Log-ins from Off Campus as a Percent of
Total Log-ins, FY2001
Pct. of Log-ins
School or Center
21
Use of Licensed Resources
When Are they Working?
Database Use by Time of Day, FY2001
22
Use of Licensed Resources
How Does Audience Composition Change Through the
Day?
Database Use by hour, FY2001
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
10.5
10.0
9.5
9.0
8.5
Files Views Saves Per Session
8.0
7.5
7.0
6.5
6.0
January 01
May 01
November 01
January 02
March 02
April 02
27
Distribution of Ebook Use by Network Domain.
Data for April 2002
Summary Statistics
Total Book Sessions 1,592
Total Activity Downloads (saves) Views
15,968
Total Views 15,104
Total Downloads (saves) 864
Total Books Accessed 533
28
Penn Library Franklin (OPAC) Activity Statistics
29
(No Transcript)
30
Circulation Statistics
31
(No Transcript)
32
(No Transcript)
33
Service Quality Impact Assessment
34
2002 Quality/Impact Survey

• Response
• Faculty 488
• Grad/Professional 1,107
• Undergraduate 898
• Total 2,493

Overall Quality Scores by Group

(Scale 7 highest) Group 2002 Mean
2000 LibQual Faculty 5.7 5.4 ranked
2nd of 12 Undergraduate 5.6 5.5 ranked
1st of 12 Grad/Professional 5.3 5.1
ranked 7th of 12 All 5.5 5.4 ranked 3rd
of 12
Scores adjusted to fit a 7 point scale
35
Overall Scores by School
Satisfaction Quality Impact Nursing n80
5.7 5.6 5.9 Law n117
5.7 5.8 5.8 Fine Arts n108
5.3 5.5 5.8 Arts Sci. n963
5.6 5.7 5.8 Dental Med. n73 5.3 5.4 5.7 Social
Work n40 5.5 5.6 5.7 Annenberg
n24 5.7 5.6 5.6 Education n119 5.4 5.5 5.6 Medic
ine n283 5.3 5.3 5.5 Wharton n399 5.3 5.4 5.5 En
gineering n172 5.1 5.2 5.3 Veterinary Med.
n115 4.8 5.0 5.0 Total n2,493 5.4 5.5 5.6
Overall Scores by Group Mean Faculty 5.6
Undergraduate 5.6
Grad/Professional 5.2

All 5.4 Positive Service
Impact Faculty 5.9
Grad/Professional 5.6
Undergraduate 5.5 All 5.6
Positive
User Satisfaction (7highest)
36
2002 results in bold
Boxes highlight instances where users'
perceptions of service were lower than
their minimum service expectations.
37
1 Availability of items in catalog 15 2
Providing electronic tools for my efficiency
25 3 Electronic info I need for work 9 4
Willingness to help me 6 5 Print info I need
for work 12 6 Use info when/where needed 21 7
Provide resources needed for achievement 24 8
Provide services as promised 4 9 Order of books
on Library shelves 5 10 Service hours 16 11
Knowledgeable Staff 13 12 Staff who understand
information needs 3 13 Timely document delivery
20 14 Available working photocopiers 2 15
Providing prompt service 11 16 Help locating
retrieving information effectively on my own
23 17 Dependability in handling service
problems 14 18 Computer/printer availability
19 19 Staff consistently courteous 7 20
Facilities for individual study 10 21 Help me
handle software technologies for information
access 22 22 Help me stay current with
research 26 23I nfo resources for
interdisciplinary study 18 24 Space group
study/research needs 8 25 Staff instills
confidence 17 26 Help me compare evaluate
info sources 27 27 Provide training in use of
info 1
Difference between Perceived and Minimum scores
NB Below this line, perception of service was
rated lower than minimum service expectation
Data labels correspond to rank order
Importance Rating 14 highest possible rating
(Minimum Desired score, averaged)
38
1 Willingness to help me 6 2 Provide resources
needed for achievement 24 3 Staff who
understand information needs 3 4 Print info I
need for work 12 5 Providing prompt service
11 6 Help locating retrieving information
effectively on my own 23 7 Availability of
items in catalog 15 8 Providing electronic
tools for my efficiency 25 9 Provide services
as promised 4 10 Electronic info I need for
work 9 11 Service hours 16 12 Knowledgeable
Staff 13 13 Dependability in handling service
problems 14 14 Staff consistently courteous
7 15 Help me handle software technologies
for information access 22 16 Use info
when/where needed 21 17 Order of books on
Library shelves 5 18 Help me stay current
with research 26 19 Info resources for
interdisciplinary study 18 20 Timely document
delivery 20 21 Facilities for individual study
10 22 Available working photocopiers 2 23
Computer/printer availability 19 24 Staff
instills confidence 17 25 Help me compare
evaluate info sources 27 26 Provide training
in use of info 1 27 Space group study/research
needs 8
Difference between Perceived and Minimum scores
NB Below this line, perception of service was
rated lower than minimum service expectation
Data labels correspond to rank order
Importance Rating 14 highest possible rating
(Minimum Desired score, averaged)
39
1 Provide resources needed for achievement 24 2
Willingness to help me 6 3 Providing
electronic tools for my efficiency 25 4
Provide services as promised 4 5 Order of
books on Library shelves 5 6 Print info I
need for work 12 7 Staff consistently
courteous 7 8 Electronic info I need for work
9 9 Availability of items in catalog 15 10
Knowledgeable Staff 13 11 Providing prompt
service 11 12 Timely document delivery 20 13
Available working photocopiers 2 14 Use info
when/where needed 21 15 Staff who understand
information needs 3 16 Help me stay current
with research 26 17 Dependability in handling
service problems 14 18 Service hours 16 19
Help locating retrieving information
effectively on my own 23 20 Facilities for
individual study 10 21 Help me handle software
technologies for information access 22 22
Computer/printer availability 19 23 Info
resources for interdisciplinary study 18 24
Space group study/research needs 8 25 Help me
compare evaluate info sources 27 26 Staff
instills confidence 17 27 Provide training in
use of info 1
NB Below this line, perception of service was
rated lower than minimum service expectation
Data labels correspond to rank order
Difference between Perceived and Minimum scores
Importance Rating 14 highest possible rating
(Minimum Desired score, averaged)
40
1
Providing electronic tools for my efficiency 25
2
Electronic info I need for work 9
3
Availability of items in catalog 15
4
Willingness to help me 6
5
Staff consistently courteous 7
6
Provide resources needed for achievement 24
7
Print info I need for work 12
8
Timely document delivery 20
9
Dependability in handling service problems 14
10
Use info when/where needed 21
11
Order of books on Library shelves 5
12
Service hours 16
Difference between Perceived and Minimum scores
13
Provide services as promised 4
14
Staff who understand information needs 3
Available working photocopiers 2
15
NB Below this line, perception of service was
rated lower than minimum service expectation
Knowledgeable Staff 13
16
Providing prompt service 11
17
Help locating retrieving information
effectively on my own 23
18
Help me stay current with research 26
19
Space group study/research needs 8
20
Data labels correspond to rank order
Facilities for individual study 10
21
Info resources for interdisciplinary study 18
22
Computer/printer availability 19
23
Help me compare evaluate info sources 27
24
Staff instills confidence 17
25
Help me handle software technologies for
information access 22
26
Importance Rating 14 highest possible rating
(Minimum Desired score, averaged)
Provide training in use of info 1
27
41
2002 Quality/Impact Survey How Did Faculty
Score the Library?
Faculty Respondents n 488
1
Electronic info I need for work 9
Providing electronic tools for my efficiency 25
2
Use info when/where needed 21
3
Print info I need for work 12
4
5
Availability of items in catalog 15
Provide resources needed for achievement 24
6
Provide services as promised 4
7
Order of books on Library shelves 5
8
9
Willingness to help me 6
Providing prompt service 11
10
Timely document delivery 20
11
Dependability in handling service problems 14
12
NB Below this line, perception of service was
rated lower than minimum service expectation
Difference between Perceived and Minimum scores
13
Help locating retrieving information
effectively on my own 23
Staff consistently courteous 7
14
Knowledgeable Staff 13
15
Available working photocopiers 2
16
Data labels correspond to rank order
17
Staff who understand information needs 3
Help me handle software technologies for info.
access 22
18
Service hours 16
19
Help me stay current with research 26
20
21
Info resources for interdisciplinary study 18
Staff instills confidence 17
22
Help me compare evaluate info sources 27
23
Computer/printer availability 19
24
Provide training in use of info 1
25
Importance Rating 14 highest possible rating
(Minimum Desired score, averaged)
Facilities for individual study 10
26
27
Space group study/research needs 8
42
Availability of items in catalog 15
1
Providing electronic tools for my efficiency 25
2
Willingness to help me 6
3
Electronic info I need for work 9
4
Print info I need for work 12
5
Order of books on Library shelves 5
6
Use info when/where needed 21
7
Provide services as promised 4
8
Service hours 16
9
Provide resources needed for achievement 24
10
Computer/printer availability 19
11
Difference between Perceived and Minimum scores
Facilities for individual study 10
12
NB Below this line, perception of service was
rated lower than minimum service expectation
Available working photocopiers 2
13
Knowledgeable Staff 13
14
Help locating retrieving information
effectively on my own 23
15
Staff who understand information needs 3
16
Providing prompt service 11
17
Data labels correspond to rank order
Staff consistently courteous 7
18
Dependability in handling service problems 14
19
Timely document delivery 20
20
Space group study/research needs 8
21
Help me handle software technologies for info.
access 22
22
Info resources for interdisciplinary study 18
23
Help me stay current with research 26
24
Help me compare evaluate info sources 27
25
Importance Rating 14 highest possible rating
(Minimum Desired score, averaged)
Staff instills confidence 17
26
Provide training in use of info 1
27