The Role of Assessment in Achieving Institutional Effectiveness

1
The Role of Assessment in Achieving Institutional
Effectiveness

• Assessment in Administrative and Educational
  Support Services
• Created by Dennis George

2
Administrative Units

• Units that provide services which maintain the
  institution and are essential to its operations,
  but do not directly impact the institutions
  instructional programs, e.g.
• Accounting Office
• Office of the Registrar
• Physical Plant

3
Educational Support Units

• Units which, while not primarily instructional in
  nature, contribute directly to student learning
  or instruction.
• Academic Advising Center
• Library
• Information Technology
• Provide services which directly relate to the
  student and are part of the institutions overall
  learning environment.

4
Overview of Assessment Process

• Each Administrative and Educational Support (AES)
  Unit identifies what is intends to accomplish to
  support the Institutional Mission/Purpose.

5
Overview of Assessment Process

• AES Unit assesses or measures the extent to which
  these objectives are being met
• Client satisfaction
• External Evaluations
• Direct measure of unit performance
• Assessment of student outcomes following receipt
  of services (I.e. what the student knows, thinks,
  or is able to do as a result of receiving the
  service)

6
Overview of Assessment Process

• AES Unit uses results of assessment to improved
  services to students or other clientele.

7
Overview of Assessment Process

• AES Unit Closes the loop by
• Describing the use of results
• Documenting the use of results to improve services

8
Working DefinitionAES Outcomes

• Statements from an Administrative and Educational
  Support Unit that identifies what it intends to
  accomplish to support the Institutional
  Mission/Purpose.

9
Administrative Units

• Objectives for Administrative Units are primarily
  process oriented and describe the support
  process or service which the unit intends to
  accomplish
• Improve services to handicap students
• Monthly accounting reports to academic and
  administrative departments will be processed and
  distributed promptly
• Transcript requests will be filled and returned
  promptly

10
Educational Support Units

• Objectives for Educational Support units contain
  both process and outcomes oriented statements
• Outcomes oriented statements
• Concern the units direct impact upon students or
  clients served
• Describe what the client will be able to do after
  receipt of the service

11
Educational Support Units

• Examples of Outcomes Oriented Objectives
• Students will be able to utilize the librarys
  reference services efficiently
• Before their last semester in attendance at the
  institution, graduates seeking employment will
  prepare an acceptable resume for presentation to
  potential employers
• Students will be proficient in logging on the
  Internet through the academic computing center.

12
Additional Sample Objective Statements

• Outcome statements
• Students will learn how to use library resources
• Graduates will have the ability to write a resume

13
Additional Sample Objective Statements

• Process Statements
• Library will be efficient in book acquisitions
• The Accounting Office will promptly process
  vendor statements

14
Additional Sample Objective Statements

• Satisfaction Statements
• Students will be satisfied with Library
  circulation service
• Vendors will report prompt payment

15
Long List Short List

• Identify a long list of administrative objectives
  supporting the activities described in the Unit
  Mission Statement
• From the long list, identify two or three
  objectives for assessment to focus on each
  assessment cycle

16
Long List Short List

• Selected objectives should be
• Targeted on those areas which the unit staff
  believe can be improved by using the currently
  available resources and personnel
• Related to the services the unit provides
• Relatively easy to assess within one assessment
  cycle
• Directly under control of the AES unit

17
Working Definitions

• Criteria for Success The benchmark that the
  program sets for the objective and against which
  the objectives performance is judged by the
  faculty within the department
• These criteria are most often stated in terms of
  percentages, percentiles, averages, or other
  quantitative measures

18
Working Definitions

• Means of Assessment The method, tool, or
  instrument used to determine whether (and/or the
  extent to which) the Objectives Criteria for
  Success has been achieved

19
Types of Assessment used in AES Units

• Attitudinal measures of client satisfaction
• Direct measures or counts of units services
• External validation
• Outcomes measures observation, performance

20
Types of Attitudinal Assessment

• Standardized
• Noel Levits
• ACT
• NCHEMS

• Locally Developed
• Alumni Surveys
• Graduating Student
• Employer
• Point of Contact

21
Primary and Secondary Criteria

• For example, if you are doing a survey that has a
  1 5 scale (1 very dissatisfied and 5 very
  satisfied)
• Primary criteria Students will report an average
  of 3.4 or higher score on overall satisfaction
• Secondary criteria On no component of the survey
  will the average rating be less than 3.0

22
Other Examples of Attitudinal Assessments

• University of Mississippi Graduating Student
  Survey
• University of Mississippi Undergraduate Alumni
  Survey
• Career Center (point of contact survey)

23
Direct Measures or Counts

• Volume of Activity (e.g. number of persons
  served, attending, etc.)
• Level of Efficiency (e.g. average time for
  response)
• Measure of Quality (e.g. average errors per audit)

24
More Examples

• Admissions Application Response Timing
• Campus Security Availability of Parking
• Career Services Resume Distribution
• Computer Services Information Distribution
• Food Services Cafeteria Services

25
External Validation

• Periodic assessment of the relationship of the
  department/units efforts to good and acceptable
  practice by a neutral person who is
  knowledgeable in the field
• Auditor
• Consultant
• Public health inspector
• Fire Marshall
• Peer-Professional Association

26
Review Issues for Means of Assessment and
Criteria for Success

• Specific identification of the means
• To whom and when it will be administered
• By whom and based upon what will results be
  judged
• How well the group of students or unit ought to
  score

27
Example Library

• Administrative Objective
• The Library will acquire adequate collections of
  information resources to support curriculum needs

• Means of Assessment and Criteria for Success
• Institutional faculty survey will indicate an
  average ranking of at least 3.5 ( on a 5 point
  scale) on items related to satisfaction with
  library collections. On no single item will
  satisfaction be reported as less than 3.0

28
Example Library

• Administrative Objective
• Students will know how to use library resources

• Means of Assessment and Criteria for Success
• 80 of freshmen students enrolled in UC 101 will
  locate 9 of the 10 resources explained in the
  Library Orientation workshop

29
Who is Involved?

• The AES Unit Director or Head
• Lead staff in assessment planning,
  implementation, and service improvement
• Submit assessment plan for the unit
• Resist procrastination
• Avoid closing their office door and emerging with
  the assessment plan

30
Who Is Involved?

• Professional Staff
• Identify and select administrative objectives
• Determine means of assessment
• Determine criteria for success

31
Suggested Steps

• Formulating Unit Assessment Plan
• Establish a linkage to the Institutions
  Statement of Purpose. Identify which portion of
  the State of Purpose the AES Unit supports
• Prepare the AES Unit Mission Statement
• Formulate Administrative Objectives
• Identify Unit Means of Assessment and Criteria
  for Success

32
Suggested Steps

• Moving from Planning to Implementation an Use of
  Results
• Conduct Assessment Activities
• Document Use of Results for Service Improvements

33
Components of an Assessment Plan Report

• Reference to the institutions mission and goals
  and its support by the units mission statement
• Formulation of administrative objectives
• Identification of means of assessment and
  criteria for success for each administrative
  objective
• Summary of the data actually collected when the
  planned assessment took place
• Description of how these data were used to
  improve services

34
Common Problems

• No match of means of assessment with objective
• Lack of a criteria for success
• Clarity of means of assessment description and/or
  to general terminology in describing what is
  anticipated from the means of assessment
• Using words like all or 100
• Choosing appropriate method for measuring
  outcome/objective
• Failure to describe specifically how measurements
  will be made