ASSESSMENT

1
ASSESSMENT
2
WHAT IS ACADEMIC ASSESSMENT ?

• THE SYSTEMATIC COLLECTION, REVIEW, AND USE OF
  INFORMATION ABOUT ACADEMIC PROGRAMS IN ORDER TO
  IMPROVE STUDENT LEARNING

Source Catherine Palomba Ball State University
3
WHAT IS ACADEMIC ASSESSMENT ?

• THE APPLICATION OF MANAGERIAL AND EDUCATIONAL
  CONCEPTS TO THE DESIGN, ANALYSIS, AND IMPROVEMENT
  OF CURRICULUM DEVELOPMENT AND LEARNING PROCESSES.

4
EVOLUTION OF ASSESSMENT

• PRIVATE COLLEGES
• PUBLIC UNIVERSITIES
• REGIONAL PROFESSIONAL ACCREDITATION AGENCIES
• STATE LEGISLATURES AND BOARDS OF EDUCATION

5
PRIVATE COLLEGES

• MAJOR PROBLEM
• THEY WERE BEING SQUEEZED OUT OF THE UNDERGRADUATE
  MARKET AND WERE GOING BROKE
• THEIR BOARDS, MADE UP MAINLY OF BUSINESS AND
  INDUSTRY LEADERS, DEMANDED PROMPT AND POSITIVE
  RESULTS

6
PRIVATE COLLEGES

• CONCLUSIONS
• UNDERGRADUATE EDUCATION IS AN ACADEMIC AND A
  SOCIALIZATION PROCESS
• HAD TO SHOW PROOF OF ADDITIONAL VALUE ADDED IN
  ORDER TO WARRANT TUITION RATES 4-5 TIMES STATE
  UNIVERSITY RATES

7
PRIVATE COLLEGES

• CONCLUSIONS
• THEY WERE IN THE KNOWLEDGE BUSINESS NOT THE
  RESEARCH, TEACHING, AND SERVICE BUSINESS
• THEIR FOCUS SHOULD BE ON THE UNDERSTANDING/ORGANIZ
  ATION/ STRUCTURING AND DISSEMINATION PARTS OF THE
  KNOWLEDGE MODEL OF HIGHER EDUCATION

8
PRIVATE COLLEGES

• SOLUTIONS
• EMPLOYED ABILITY BASED LEARNING
• STUDENTS SHOULD BE ABLE TO DO SOMETHING WITH WHAT
  THEY KNOW (Handout 1)
• REVISED CURRICULUM DESIGN
• CHANGED FROM MY COURSE MUST BE IN THE PROGRAM
  BECAUSE IT IS WONDERFUL AND OUR DEPARTMENT NEEDS
  THE ENROLLMENT

9
PRIVATE COLLEGES

• SOLUTIONS
• REVISE CURRICULUM DESIGN
• TO A PROCESS THAT INVOLVED
• AN ANALYSIS AND DETERMINATION OF THE KNOWLEDGE
  SETS, SKILLS, AND VALUES (KSVs) NEED TO BE A
  SOCIALLY AND ECONOMICALLY VIABLE GRADUATE
• DEVELOPMENT OF THE LEARNING OUTCOMES REQUIRED TO
  ACCOMPLISH THE KSVs
• ORGANIZATION OF KSVs AND ASSOCIATED LEARNING
  OBJECTIVES INTO COURSE OR MODULE UNITS

10
PRIVATE COLLEGES

• SOLUTIONS
• CHANGED MISSION STATEMENTS AND REWARD SYSTEMS TO
  REFLECT EMPHASIS ON ORGANIZING AND DISSEMINATION
  OF KNOWLEDGE
• UTILIZED BOYERS IDEAS ABOUT SCHOLARSHIP

11
PRIVATE COLLEGES

• SOLUTIONS
• ADAPTED AND USED BUSINESS MANAGEMENT
  CONCEPTS/METHODS
• FUNCTIONS OF MANAGEMENT
• SYSTEMS THEORY
• CONTINUOUS IMPROVEMENT

12
MANAGEMENT CONCEPTS

• MANAGEMENT FUNCTIONS
• SYSTEMS THEORY
• CONTINUOUS IMPROVEMENT

13
MANAGEMENT FUNCTIONS
PLANNING
CONTROLLING
ORGANIZING
DIRECTING
14
PLANNING

• EXTERNAL INTERNAL ANALYSIS
• MISSION VISION
• GOALS, OBJECTIVES STRATEGIES
• DEVELOP ACTION PLANS
• ALLOCATION OF RESOURCES TO PLANS

15
ORGANIZING

• STRUCTURE
• FUNCTION
• GEOGRAPHIC
• PRODUCT
• CUSTOMER
• MATRIX
• STAFFING

16
DIRECTING

• ADMINISTRATIVE LEADERSHIP SKILLS
• COMMUNICATION
• MOTIVATION
• GROUPS
• CHANGE

17
CONTROLLING

• ESTABLISH STANDARDS
• MEASURE PERFORMANCE
• COMPARE PERFORMANCE AGAINST STANDARDS
• DETERMINE AND APPLY PROPER CORRECTIVE ACTION

18
MANAGEMENT FUNCTIONS
PLANNING
1.Establish Standards 2. Measure Performance 3.
Compare Performance To Standards 4. Take
Corrective Actions
ORGANIZING
DIRECTING
19
SYSTEMS THEORY
A SYSTEM IS DEFINED AS A GROUP OF OBJECTS THAT
ARE INTERRELATED AND INTERDEPENDENT WITH EACH
OTHER AND THEIR ENVIRONMENT SO AS TO FORM A
WHOLE.
20
SYSTEMS THEORY
1. OBJECTS/PARTS AND THEIR ATTRIBUTES 2.
INTERRELATIONSHIPS/INTERDEPENDENCIES 3.
BOUNDARIES 4. GOAL SEEKING 5. TRANSFORMATION
PROCESS - INPUTS OUTPUTS 6. SYNERGISM/GESTALT 7.
HIERARCHY OF SUBSYSTEMS 8. REGULATION 9.
ENTROPY/DISORDER
21
FUNCTIONS OF MANAGEMENT - SYSTEMS PERSPECTIVE
INTERNAL ANALYSIS
LONG RUN PLANS - - - - - - - - - - - SHORT
RANGE PLANS
GOALS OBJECTIVES STRATEGIES
MISSION VISION
IMPLEMENT ACTION PLANS
COMMITMENT OF RESOURCES
EXTERNAL ANALYSIS
STANDARDS, MEASUREMENT CORRECTIVE ACTION
E
22
CONTINUOUS IMPROVEMENT

• PART OF THE QUALITY MOVEMENT
• ANY PROCESS CAN BE IMPROVED
• KEY METHOD TO REMAIN COMPETITIVE IN TODAYS
  CHANGING BUSINESS WORLD
• PROCESS REENGINEERING IS NEEDED WHEN INCREMENTAL
  IMPROVEMENT IS NOT SUFFICIENT

23
PUBLIC UNIVERSITIES

• LAND-GRANT CONCEPT
• EMERGENCE OF STATE RESEARCH UNIVERSITIES
• QUESTIONS ABOUT CURRICULUM UNDERGRADUATE
  EDUCATION (Handouts 2 3)
• ACCOUNTABILITY MOVEMENT

24
PROFESSIONAL ACCREDITING AGENCIES

• AACSB
• MISSION DRIVEN STANDARDS
• ONE OF THE FIRST TO REQUIRE LEARNING OUTCOMES AND
  ASSESSMENT
• MUST HAVE PROCESSES IN PLACE TO ASSURE HIGH
  QUALITY AND CONTINUOUS IMPROVEMENT

25
REGIONAL ACCREDITING AGENCIES

• SACS (Handout 4)
• ESTABLISH CLEARLY DEFINED INSTITUTIONAL PURPOSE
• FORMULATE EDUCATIONAL GOALS
• EVALUATE ACHIEVEMENT OF GOALS
• USE RESULTS TO IMPROVE PROGRAMS
• MEASURES TO EVALUATE PROGRAMS AND GENERAL
  EDUCATION

26
STATE LEGISLATURES AND STATE BOARDS OF CONTROL

• LEGISLATURE
• HOT BUTTONS - ACCESS, AFFORDABILITY
  ACCOUNTABILITY
• MORE THAN 40 STATES REQUIRE ASSESSMENT
• PASS LAWS THAT REQUIRE STATE HIGHER ED
  INSTITUTIONS TO TAKE SPECIFIC ACTIONS - SB 148
• THECB (Handout 5)
• EACH INSTITUTION MUST REVIEW AND EVALUATE ITS
  CORE CURRICULUM AND REPORT THE RESULTS TO THECB

27
WHY DO WE HAVE TO DO IT?

• LEGISLATURE
• TOLD US TWO TIMES - THEN PASSED A LAW
• THECB
• IMPLEMENTED THE LAW (SB148) BY REQUIRING EACH
  INSTITUTION TO REVIEW AND EVALUATE ITS CORE
  CURRICULUM AND REPORT THE RESULTS
• SACS
• REQUIRES A PROCESS FOR ASSESSMENT AND CONTINUOUS
  IMPROVEMENT OF PROGRAMS AND GENERAL ED CORE

28
WHY SHOULD WE DO IT?

• HELPS US CLARIFY OUR GOALS FOR THE CORE
• SHIFTS OUR THINKING FROM INPUTS TO PROCESSES AND
  OUTPUTS
• A PROVEN TOOL TO HELP US IMPROVE THE CORE AND THE
  LEARNING ENVIRONMENT

29
GENERAL EDUCATION CORE

• UNTS PLAN FOR ASSESSMENT AND CONTINUOUS
  IMPROVEMENT

30
WHAT IS EXPECTED OF US ?
AS THE STEERING COMMITTEE, WE ARE RESPONSIBLE
FOR 1. DESIGNING AN ASSESSMENT AND CONTINUOUS
IMPROVEMENT PROCESS FOR THE UNIVERSITYS
GENERAL EDUCATION CORE, 2. DEVELOPING A PLAN
FOR IMPLEMENTING THE PROCESS, AND 3.
IMPLEMENTING THE PLAN AND MONITORING THE
PERFORMANCE OF THE ACI PROCESS.
31
DEVELOPING A PLAN
A good place to start is with the answers to the
following

• WHAT NEEDS TO BE DONE ?
• HOW WILL WE DO IT ?
• WHO WILL DO IT ?
• WHEN WILL IT BE DONE ?

32
DEVELOPMENT OF OUR ACI PROCESS AND PLAN

• WHAT ARE WE GOING TO ASSESS ?
• HOW ARE WE GOING TO CONDUCT ASSESSMENT - WHAT
  TOOLS OR METHODS WILL WE USE ?
• WHO IS GOING TO DO THE ASSESSMENT ?

33
DEVELOPMENT OF OUR ACI PROCESS AND PLAN

• WHEN AND HOW MUCH ?
• HOW WILL THE RESULTS BE USED TO IMPROVE THE CORE
  CURRICULUM ?
• WHO IS GOING TO PREPARE THE REPORTS FOR THECB AND
  SACS ?

34
DEVELOPMENT OF OUR ACI PROCESS AND PLAN

• WHAT KIND OF DOCUMENTATION WILL WE NEED ?
• PROCESS ITSELF
• RESULTS OF THE PROCESS
• EVIDENCE TO PROVE THAT THE ASSESSMENT RESULTS ARE
  USED BY CURRICULUM PLANNING OR OTHER PROCESSES TO
  IMPROVE THE CORE

35
CURRICULUM PROCESS
INTERNAL ANALYSIS
- - - - - - - - - - -
LEARNING OBJECTIVES
GOALS OBJECTIVES STRATEGIES
MISSION GEN ED CORE
IMPLEMENT GEN ED CORE
COMMITMENT OF RESOURCES
GEN ED CORE CURRICULUM
EXTERNAL ANALYSIS
STANDARDS, MEASUREMENT, ANALYSIS AND CORRECTIVE
ACTION
E
36
ASSESSMENT TOOLS

• SURVEYS
• TESTS
• PORTFOLIOS
• INSTITUTIONAL DATA
• CAPSTONE COURSE PROJECTS
• CLASSROOM ASSESSMENT
• CURRICULUM REVIEW

Source Catherine Palomba Ball State University
37
SURVEYS

• ADVANTAGES
• COVER MANY DIFFERENT TYPES OF INFORMATION AND
  GROUPS
• SOMETIMES ENABLE NATIONAL COMPARISONS
• DISADVANTAGES
• TAKE TIME TO DESIGN
• RESPONDENTS MAY NOT BE REPRESENTATIVE

Source Catherine Palomba Ball State University
38
TESTS

• ADVANTAGES
• PROVIDES INFORMATION ABOUT COGNITIVE OUTCOMES
• CAN COVER A WIDE RANGE OF CONTENT
• CAN ASSESS A LARGE GROUP
• GOOD FOR LONGITUDINAL DATA
• GOOD FOR BEFORE AND AFTER MEASURES
• NATIONAL EXAMS ARE AVAILABLE

Source Catherine Palomba Ball State University
39
TESTS

• DISADVANTAGES
• NATIONAL EXAMS
• MAY NOT MATCH CURRICULUM
• ARE USUALLY EXPENSIVE
• LOCALLY DEVELOPED EXAMS
• CAN BE TIME CONSUMING TO DEVELOP
• MAY BE DIFFICULT TO GET CONSENSUS
• LACK OF EXTERNAL NORMS

Source Catherine Palomba Ball State University
40
PORTFOLIOS
A collection of student work generated over time

• ADVANTAGES
• CAN BE INTEGRATED WITH COURSEWORK
• PROVIDES INFORMATION OVER TIME
• DISADVANTAGES
• RAISE MANY PRACTICAL QUESTIONS
• CAN BE DIFFICULT TO EVALUATE

Source Catherine Palomba Ball State University
41
INSTITUTIONAL DATA

• ADVANTAGES
• AVAILABILITY
• MAY ALREADY BE IN A DATABASE
• DISADVANTAGES
• MAY NOT MATCH CURRICULUM
• MAY BE TIME CONSUMING AND EXPENSIVE TO ANALYZE

Source Catherine Palomba Ball State University
42
CAPSTONE COURSE

• ADVANTAGES
• PROVIDES A TIME AND PLACE TO UNDERTAKE ASSESSMENT
• DISADVANTAGES
• IS NOT ASSESSMENT BY ITSELF
• NEEDS TO BE PART OF AN OVERALL PLAN

Source Catherine Palomba Ball State University
43
CLASSROOM ASSESSMENT
Use of small-scale assessment techniques
conducted by teachers to determine what students
are learning in particular classes

• ADVANTAGE
• A LOT OF GOOD DATA
• DISADVANTAGE
• HARD TO USE THE DATA

Source Catherine Palomba Ball State University
44
CURRICULUM REVIEW

• ADVANTAGES
• COMPARES COURSE LEARNING OBJECTIVES TO PROGRAM
  OBJECTIVES
• EVALUATES THE WHOLE PROGRAM, NOT JUST THE PARTS
• DISADVANTAGES
• TAKES A LOT OF TIME
• POLITICALLY CHARGED

Source Catherine Palomba Ball State University
45
ASSESSMENT TOOLS( )
Also See SACS Handout

• SURVEYS
• TESTS
• PORTFOLIOS
• INSTITUTIONAL DATA
• CAPSTONE COURSE PROJECTS
• CLASSROOM ASSESSMENT
• CURRICULUM REVIEW

Source Catherine Palomba Ball State University
46
OUR ACI PROCESS AND PLAN MUST

• MEET ALL THE THECB REQUIREMENTS
• BE DESIGNED TO ALSO ACCOMPLISH SACS REQUIREMENTS
  
• MEET OUR OWN NEEDS FOR A PROCESS THAT WILL ASSURE
  EXCELLENCE OF UNTS GENERAL EDUCATION CORE

47
THECB REQUIREMENTS ( )
SEE THECB HANDOUTS 5 6

• Extent to which the institutions core
  curriculum is consistent with the elements of the
  core curriculum recommended by the Board
• Extent to which the institutions core
  curriculum is consistent with the Texas Common
  Course Numbering System

48
THECB REQUIREMENTS

• Extent to which the curriculum is consistent with
  the elements of the core curriculum component
  areas, intellectual competencies, and
  perspectives as expressed in Core Curriculum
  Assumptions and Defining Characteristics adopted
  by the Board and

49
THECB REQUIREMENTS

• Extent to which the institutions core
  curriculum educational goals and the exemplary
  educational objectives of the core curriculum
  recommended by the Board are being achieved.

50
THECB REPORT REQUIRES( )
SEE THECB HANDOUTS

• A table that compares the institutions core
  curriculum with the core component areas and
  exemplary educational objectives of the core
  curriculum recommended by the Board
• A brief description of the purpose and substance
  of the institutions core curriculum

51
THECB REPORT REQUIRES( )
SEE THECB HANDOUTS

• A description of the processes and procedures
  used to evaluate the institutions core
  curriculum and
• A description of the ways in which the evaluation
  results are utilized to improve the core
  curriculum at the institution.

52
SACS REQUIREMENTS( )
SEE SACS HANDOUT
The institution must

• Establish a clearly defined purpose appropriate
  to collegiate education,
• Formulate educational goals consistent with the
  institutions purpose,

53
SACS REQUIREMENTS( )
SEE SACS HANDOUT

• Develop and implement procedures to evaluate the
  extent to which these educational goals are being
  achieved, and
• Use the results of these evaluations to improve
  educational programs, services, and operations.

54
OUR REQUIREMENTS

• Is our core curriculum an integrated whole or a
  collection of parts ?
• Are the current goals and objectives of UNTS
  general education core sufficient ?
• Does the design and content of the general
  education core meet the needs of our students ?

55
WHAT IS EXPECTED OF US ?
AS THE STEERING COMMITTEE, WE ARE RESPONSIBLE
FOR 1. DESIGNING AN ASSESSMENT AND CONTINUOUS
IMPROVEMENT PROCESS FOR THE UNIVERSITYS
GENERAL EDUCATION CORE, 2. DEVELOPING A PLAN
FOR IMPLEMENTING THE PROCESS, AND 3.
IMPLEMENTING THE PLAN AND MONITORING THE
PERFORMANCE OF THE ACI PROCESS.
56
DEVELOPMENT OF OUR ACI PROCESS AND PLAN

• WHAT ARE WE GOING TO ASSESS ?
• EXAMPLES
• HOW ARE WE GOING TO CONDUCT ASSESSMENT - WHAT
  TOOLS OR METHODS WILL WE USE ?
• WHO IS GOING TO DO THE ASSESSMENT ?

57
DEVELOPMENT OF OUR ACI PROCESS AND PLAN

• WHEN AND HOW MUCH ?
• HOW WILL THE RESULTS BE USED TO IMPROVE THE CORE
  CURRICULUM ?
• WHO IS GOING TO PREPARE THE REPORTS FOR THECB AND
  SACS ?

58
DEVELOPMENT OF OUR ACI PROCESS AND PLAN

• WHAT KIND OF DOCUMENTATION WILL WE NEED ?
• PROCESS ITSELF
• RESULTS OF THE PROCESS
• EVIDENCE TO PROVE THAT THE ASSESSMENT RESULTS ARE
  USED BY CURRICULUM PLANNING OR OTHER PROCESSES TO
  IMPROVE THE CORE