Assessment in the Emory University Libraries

1
Assessment in theEmory University Libraries

• Lurching Toward Sustainability
• Susan Bailey and Charles Forrest
• September 25, 2006

2
Emory Libraries

• University Libraries
• 300 staff
• 3.1M volumes
• 29M annual budget
• Offsite Collection Annex
• General Libraries
• Health Sciences Center
• Law
• Oxford College
• Theology

3
Emory Libraries

• General Libraries
• 170 staff members
• 62 professional staff
• 108 support staff
• Robert W. Woodruff Library (Main)
• Business Library
• Chemistry Library
• Manuscript, Archives and Rare Book Library
  (MARBL)
• Mathematics and Science Center Library
• Music and Media Library

4
Context for Assessment and Planning

• 2001
• March Organizational redesign
• Proposed
• Market Integration Coordinating Group
• Market Analysts
• Market Councils
• Established
• Office of Program Assessment and Coordination
  (OPAC) (originally called Office of Market
  Integration)
• Jane Treadwell, Director
• Amy Williamson, Information Analyst
• Market Councils

5
Context for Assessment and Planning

• 2002
• Interim Co-Coordinators for OPAC
• Charles Forrest
• Director, Library Facilities Planning
  Construction
• Betsey Patterson
• Librarian for Research and Special Initiatives
• ARL Statistics
• Ejournal metrics
• LibQUAL
• Market Councils

6
Context for Assessment and Planning

• 2003
• Coordinator, Program Assessment and Process
  Improvement
• Recruited
• Unsuccessful

7
Context for Assessment and Planning

• 2004
• University Strategic Planning
• Vision
• Goals
• Measures and Targets
• Measurement, evaluation and assessment
• Strategic Initiatives
• Tactics
• Library liaison to University strategic planning
  process
• Ruth Pagell, Executive Dir, Goizueta Business
  Library
• Charles Forrest, Dir, Library Facilities Planning
  Construction

8
Context for Assessment and Planning

• 2005
• Recruited for Coordinator, Library Assessment
• Susan Bailey started September 2005
• Created Office of Library Planning and Assessment
  (OPA)
• Strategic Planning
• Library Assessment and Evaluation
• Library Assessment Council
• Assessment Integration Group
• Facilities Planning Project Management
• ARL VPO visit in December

9
Context for Assessment and Planning

• 2006
• Student assistant began in January, providing
  some operational support
• Library Fellow began in September

10
Office of Library Planning and Assessment

• Mission
• The Office of Library Planning and Assessment
  (OPA) provides coordination and management
  services to the rest of the organization in the
  three related areas of library strategic
  planning, library assessment and evaluation, and
  library facilities planning and project
  management.

11
The Library Environment

• Two groups exist to oversee and support
  assessment activities LAC and AIG
• Staff are asked to report library assessment
  activities to the OPA OPA can ok or indicate
  that the activity needs additional vetting or
  approvals
• Connections with the Emory Office of
  Institutional Research (OIR)
• Connections with the Emory Institutional Review
  Board (IRB)

12
Library Assessment Council (LAC)

• Provides oversight for the librarys assessment
  program
• Coordinate program elements
• Common body of knowledge about the programs
  direction and priorities
• Membership
• Director, Office of Finance and Budget
• Director, Office of Planning and Assessment
• Human Resources Officer
• Library Assessment Coordinator

13
LAC work in 2005/2006

• Discussed priorities
• Engaged in planning for the ARL VPO visit
  including preparing the senior leadership group
• Did significant preliminary work on the Balanced
  Scorecard measures in advance of the broader
  discussions with the leadership group

14
Assessment Integration Group (AIG)

• Coordinates, promotes, and monitors assessment
  efforts in the library, supporting and sustaining
  a user-centered environment in which decisions
  are based on facts, research and analysis, and
  where services are planned and delivered in ways
  which maximize positive outcomes and impacts for
  customers and stakeholders
• Composed of volunteers from throughout the library

15
AIG work in 2005/2006

• Focused on developing a common understanding of
  assessment in the various units of the library
• Worked with the data and contributed to the final
  document presented to ESG on LibQUAL
• Developed an Assessment Activities form and
  posted it as a web form to enable creation of an
  Activities Database
• Participated in several all staff presentations,
  including the June InfoForum on Assessment
  Activities Update, and the July Assessment 101
  session

16
Assessment Activity Form

• http//fd2.formdesk.com/beaverbailey/activityform
• Submit proposals for new activity in advance
• Submit information on existing activities
• Submitted proposals can be found here
• http//fd2.formdesk.com/beaverbailey/activityform?
  results

17
Assessment 101 content

• Context for assessment at Emory
• Methods (surveys, focus groups, etc.)
• Assessment form
• Relationship to Office of Institutional Research
  (presentation from OIR Director)
• Relationship to Institutional Review Board
• Data Management Considerations in the Use of
  Survey Monkey

18
Strengths

• Creation of an Office devoted to planning and
  assessment in the Library
• Creation of Full-time position allocated to
  library assessment
• Presence and commitment of staff for whom
  assessment is important--in the OPA, in the LAC,
  and in the AIG
• Relationship with Making Library Assessment Work
  (MLAW) project
• Development of new relationships with other
  significant campus offices
• Office of Institutional Research
• Institutional Review Board
• Emory Healthcare Strategic Planning Office
  (already working with the Balanced Scorecard
  process)
• Arrival of a new library director who has a
  demonstrated commitment to strategic thinking,
  planning, and assessment

19
Weaknesses

• Newness of the office and programthe program
  is not yet well-established
• Difficulty of moving the library culture toward
  one of assessmentit is challenging to get staff
  members to adopt new policies and practices that
  are needed in programmatic assessment
• Data gathering and documentation challenges
  existthere are many questions about our current
  practices and the validity of much of the data
  that we collect and have used over the yearsthe
  derivation of many of our existing statistics is
  not documented or is poorly documented
• LA Coordinator is still working on development of
  needed skills and competencies
• Difficulty in identifying and providing the
  metrics that really speak to university
  administrators, library staff, and the user
  community

20
Pulling It All Together The Balanced Scorecard

• University Strategic Planning
• Vision
• Goals
• Measures and Targets
• Measurement, evaluation and assessment
• Strategic Initiatives
• Tactics

21
Pulling It All Together The Balanced Scorecard

• Executive Strategy Group initially identified
  measures and targets in Library Strategic Plan
  (May 2005)
• Refined from about forty to sixty measures
• University requests not more than twenty key
  measures for annual report
• LAC decides to introduce Balanced Scorecard to
  Library as tool to help focus and refine measures
  and targets, and move toward twenty key measures

22
(No Transcript)
23
Pulling It All Together The Balanced Scorecard

• Library Assessment Council met with staff from
  the Emory Healthcare Strategic Planning Office
• LAC took drafts of the Scorecard to the ESG
• Office of Institutional Research staff met with
  OPA staff
• Three strategic indicators
• Dashboard
• Thick indicators

24
Purposeful Connected to the Librarys Strategic
Plan!
25
What a Year! Hows it Been?

• Balanced Scorecard and the strategic planning
  process have encouraged us to become more
  data-driven
• There is evidence of more concern about our
  statistics in terms of comparability over time,
  developing meaningful measures that are reliable,
  standardization
• Staff members seek information or guidance when
  thinking about surveys

26
What a Year! Hows it Been?

• Pace of getting the program up and visible has
  been slower than we wanted!
• Developing a database of library statistical data
  has been more complex and challenging than
  anticipated!
• Organizational culture/social issues impact the
  speed and degree to which people adopt the new
  expectations

27
Assessment in the Emory University Libraries

• Susan Bailey
• Library Assessment Coordinator
• libsbb_at_emory.edu
• Charles Forrest
• Director, Office of Planning and Assessment
• charles.forrest_at_emory.edu