EASTFIELD COLLEGE GOALS SETTING AND MEASUREMENT

1
EASTFIELD COLLEGEGOALS SETTING AND MEASUREMENT

• PRESENTED BY
• THE EASTFIELD OFFICE OF INSTITUTIONAL RESEARCH
  PLANNING

2
SLIDES BASED ON

• THE DEPARTMENTAL GUIDE RECORD BOOK FOR STUDENT
  OUTCOMES ASSESSMENT AND INSTITUTIONAL
  EFFECTIVENESS
• BY JAMES O. NICHOLS

3
OUTLINE OF INSTRUCTION

• STATING INTENDED OUTCOMES

• ASSESSMENT

• DEMONSTRATING USE OF ASSESSMENT FOR
  IMPROVEMENT

• SETTING GOALS FOR ADMINISTRATIVE AND
  INSTRUCTIONAL SUPPORT AREAS

4
WHAT ARE STATEMENTS OF INTENDED STUDENT OUTCOMES?
STATEMENTS BY THE PROGRAM OR DIVISIONS OF
EXPECTED OUTCOMES WHEN THE STUDENT COMPLETES THE
PROGRAM OR CORE CURRICULA

• COGNITIVE (KNOW)

• ATTITUDINAL (THINK)

• BEHAVIORAL (DO)

5
HOW MANY OUTCOMES?

• THREE TO FIVE FOR INSTRUCTIONAL AREAS
• BE REALISTIC
• CHOOSE THE MOST BASIC OBJECTIVES FIRST - YOU CAN
  ADD MORE AS YOU ATTAIN ORIGINAL GOALS

6
MEASURABLE DOES NOT ALWAYS MEAN QUANTIFIABLE

• CHOOSE MEASURES WHICH CAN AT LEAST BE OBSERVED,
  COMPARED, OR DUPLICATED
• MANY MEASURES CAN BE QUANTIFIED

7

• DONT SET YOUR OUTCOME MEASUREMENT STANDARDS TOO
  HIGH - YOUR PEOPLE WILL BECOME DISCOURAGED

8

• DONT SET YOUR OUTCOME CRITERIA TOO LOW - YOUR
  AREA WONT BE ABLE TO DEMONSTRATE IMPROVEMENT

9
ROLE OF THE GROUP LEADERS
10
GATHERING INFORMATION REGARDING WHAT IS EXPECTED
11
MOTIVATING THE FACULTY TO TAKE PART
12
PROVIDING COORDINATION AND LOGISTICAL SUPPORT
13
GETTING THINGS MOVING IN A PARTICIPATIVE,
NONCONFRONTATIONAL MANNER
14
PROBLEMS?FACULTY OWNERSHIP

• INVITE ALL TO PARTICIPATE
• GET ENDORSEMENT OF FACULTY GOVERNING BODY
• RESPECT PROFESSIONAL JUDGEMENT OF MAJORITY OF
  FACULTY IN SETTING OUTCOMES AND MEASURES

15
PROBLEMS?LIMITING NUMBER OF STATEMENTS

• COMBINE COMMON THEMES
• GENERALIZE OUTCOMES
• CREATE SHORT LIST FROM LONG LIST
• SHORT LIST 3 OR 4 MOST IMPORTANT OUTCOMES
• AS OUTCOMES ARE ACCOMPLISHED THEY CAN BE REPLACED
  BY OTHERS

16
PROBLEMS?BACKING INTO INTENDED OUTCOMES FROM
ASSESSMENT MEANS

• IF YOU IMITATE OTHERS MEANS OF ASSESSMENT, YOU
  SHOULD MATCH THEIR INTENDED OUTCOMES
• BY USING OUTCOME MEASURES DEVELOPED BY OTHER
  INSTITUTIONS ENTITIES MAY HAVE TO ALTER THEIR
  INTENDED OUTCOMES

17
PROBLEMS ? REVISING INTENDED OUTCOMES

• SOME OUTCOMES WILL NEED TO BE CHANGED ONCE IT IS
  DETERMINED THAT THERE IS NO PRACTICAL MEANS OF
  ASSESSMENT
• LOOKING FOR MEANS OF ASSESSMENT IS A GOOD WAY TO
  IDENTIFY INTENDED OUTCOMES WHICH ARE TOO VAGUE

18
ADVANTAGES

• COMMON FORMAT
• RELATIONSHIP TO EXPANDED STATEMENTS OF
  INSTITUTIONAL PURPOSE
• SIMPLE WORDING

19
ASSESSMENT OF STUDENT LEARNING

• BRIDGE BETWEEN INTENDED EDUCATIONAL (STUDENT)
  LEARNING AND USE OF THE RESULTS OF ASSESSMENT TO
  IMPROVE INSTRUCTIONAL PROGRAMS

20
QUALITATIVE QUANTITATIVEMEANS OF ASSESSMENT

• PORTFOLIO REVIEWS
• PUBLIC PERFORMANCES
• ORAL EXAMS
• DISSERTATION DEFENSES

• COGNITIVE MEANS OF ASSESSMENT
• ASSESSMENT OF BEHAVIORAL CHANGE PERFORMANCE
• ATTITUDINAL ASSESSMENT

21
TIE IN SELECTED CORE CURRICULUM INDICATORS AND
OTHER EVALUATION ASSESSMENT OF PROGRAMS

• BUT REMEMBER - CURRICULUM IS AN INPUT, STUDENT
  LEARNING IS AN OUTPUT

22
PROBLEMS WITH QUALITATIVE MEANS OF ASSESSMENT

• INDENTIFICATION OF CRITERIA FOR SUCCESS
• OBJECTIVITY OF EVALUATORS
• CONSISTENCY OR RELIABLILITY OF JUDGMENT

23
COGNITIVE STANDARDIZED EXAMINATIONS

• PROFESSIONAL/GRADUATE SCHOOL ADMISSIONS TESTS
• LICENSURE EXAMINATIONS
• STANDARDIZED ACHIEVEMENT TESTS

24
ADVANTAGES OF STANDARDIZED EXAMINATIONS

• EASY TO ADMINISTER
• ACCEPTABLE TIME INVESTMENT
• LESS SUBJECTIVITY OR BIAS
• NATIONALLY NORMED
• STATISTICALLY VALID RELIABLE

25
PROBLEMS WITH STANDARDIZED EXAMINATIONS

• MAY NOT MATCH INTENDED OUTCOMES
• OFTEN ONLY HAVE AGGREGATE SCORES WHICH DONT
  MATCH SPECIFIC INTENDED OUTCOMES
• NATIONAL NORM GROUP IS OFTEN A BAD MATCH
• EXPENSIVE

26
ADVANTAGES OF LOCALLY DEVELOPED COGNITIVE
EXAMINATIONS

• GOOD MATCH WITH INTENDED OUTCOMES OF DEPARTMENT
• DETAILED ANALYSIS POSSIBLE
• FLEXIBILITY OF FORMAT
• FACULTY OWNERSHIP

27
DISADVANTAGES OF LOCAL COGNITIVE MEANS OF
ASSESSMENT

• LACK OF EXTERNAL CREDIBILITY
• LACK OF COMPARATIVE DATA
• HIGH FACULTY EFFORT TO DEVELOP, ADMINISTER AND
  MAINTAIN

28
ASSESSMENT OF BEHAVIORAL CHANGE AND PERFORMANCE

• WHAT A STUDENT DOES OR CAN DO

29
ASSESSMENT OF BEHAVIORAL CHANGE AND PERFORMANCE

• EXAMPLES
• EMPLOYMENT
• TRANSFER
• VOTING
• COMMUNITY SERVICE
• SERVICE ORGANIZATION PARTICIPATION

30
ADVANTAGES TO BEHAVIORAL CHANGE PERFORMANCE

• DIRECT LINKS TO INTENDED OUTCOMES
• WITH BEHAVIOR - LACK OF INTRUSIVENESS
• WITH PERFORMANCE - CONTROL OF EXTERNAL VARIABLES

31
DISADVANTAGES TO BEHAVIORAL CHANGE PERFORMANCE

• LIMITED APPLICABILITY
• INFORMATION GATHERING
• COST

32
CLOSE THE LOOP

• SHOW USE OF ASSESSMENT FOR IMPROVEMENT AND CHANGE

33
INSTITUTIONAL EFFECTIVENESS WITHIN ADMINISTRATIVE
OR EDUCATIONAL SUPPORT DEPARTMENTS
34

• ADMINISTRATIVE AND SUPPORT AREAS MAY NEED TO HAVE
  THEIR OWN MISSION STATEMENTS WHICH SUPPORT THE
  INSTITUTIONAL STATEMENT OF PURPOSE

35
ADMINISTRATIVE SUPPORT MEANS OF ASSESSMENT

• CLIENTELE SATISFACTION OR ATTITUDE
• DIRECT MEASURES OR COUNTS OF DEPARTMENTAL
  OPERATIONS
• USE OF EXTERNAL EVALUATOR

36
ATTITUDINAL MEASURES OF CLIENT SATISFACTION

• NOEL LEVITZ SURVEY (SURVEY AGAIN IN FALL, 2001)
• CAN COMPARE FIRST NOEL LEVITZ TO NEW SURVEY TO
  MEASURE IMPROVEMENT
• INDIVIDUALIZED AT POINT QUESTIONAIRE
• FOCUS GROUPS

37
DIRECT MEASURES COUNTS

• VOLUME OF ACTIVITY
• NUMBER OF STUDENTS REGISTERED
• LEVEL OF EFFECIENCY
• AVERAGE TIME FOR STUDENTS TO REGISTER
• PERFORMANCE QUALITY
• NUMBER OF ERRORS

38
GETTING STARTED
39
GETTING STARTED

• EXAMINE CURRENT DEPARTMENTAL MISSION
• REVISE IF NECESSARY
• CHOOSE THREE TO FIVE MOST IMPORTANT FUNCTIONS
• DETERMINE HOW TO SHOW THAT YOU ARE ACCOMPLISHING
  THESE FUNCTIONS

40
(No Transcript)
41
(No Transcript)