Assessment of the Core

1
Assessment of the Core Science

• Charlyne L. Walker
• Director of Educational Research and Evaluation,
• Arts and Sciences

2
Assessing the Major

• SACS requires us to assess all degree programs
  bachelors through doctorate as part of
  Institutional Effectiveness
• FIU has been assessing our degree programs since
  1999-2000
• Recent changes in SACS has moved us toward more
  direct measures
• Departments assess programs in a variety of ways
• Sample IE document

3
Science Core Category Description

• Our technologically dependent world requires an
  understanding of the processes that led us here.
• Learning the basic concepts and ideas of
  scientific fields provides contact with not just
  those fields but with how science is done.

4
Science Core Category Description

• In these courses students study the scientific
  method through examination of the foundational
  theories of modern scientific thought.
• Students apply scientific principles and theories
  to problem solving, evaluate scientific
  statements, and incorporate new information
  within the context of what is already known.
• Emphasizing the essential connection between
  theory and experiment, the hands-on laboratory
  experience provides the context for testing
  scientific theories.
• Students will be able to describe the scientific
  method through examination of the foundational
  theories of modern scientific thought.

5
Foundations of Science

• Students will be able to
• Apply scientific principles and theories to
  problem solving,
• Evaluate scientific statements,
• Interpret new information (e.g. scientific data,
  journal articles, media reports of scientific
  findings, etc.) within the context of what is
  already known, and
• Apply the scientific method by conducting
  experiments to test scientific hypotheses.

6
Objectives and Measures

• There should be objectives for the courses in
  course syllabi
• As objectives are developed, we may be asking for
  you to include them into course syllabi
• As measures are developed, you might be asked to
  include them into course syllabi to let students
  know what measures are being used
• This is not a short term solution for the 2010
  SACS visit
• The measures must be on going as part of SACS
  requirements beyond 2010

7
Types of Direct Measures

• Embedded questions
• Set of locally developed questions intended to
  measure specific student learning outcomes
• Placed within tests of all sections of the same
  course
• Used to track cognitive or skill development
  through a program, sets of embedded questions,
  with each expecting a higher level of proficiency
  than the previous, may be used across sequential
  courses
• Papers or short answer questions using rubrics
  a rubric is used to standardize the grading of
  work

8
Types of Direct Measures

• Locally developed exams - exams produced by
  faculty within a discipline
• Pre/Post Test - a locally developed test that
  measure students incoming and post program
  levels of knowledge, skills, behaviors and
  attitudes to measure students gains
• You can find more types of measures at
  http//oeas.ucf.edu/alc/dir_measures.htm

9
Examples of Objectives and Measures from UCF

• PSC 1121 Physical Science - Demonstrate an
  understanding of the scientific method. In
  particular, to demonstrate an ability to assess
  which claims presented as "scientific" are indeed
  scientific and not pseudoscientific.
• Measures
• 7.1 PSC 1121 Physical Science - A pre-test and a
  post-test will be given. A gain of 30 will be
  considered satisfactory, a gain of 50 will be
  considered well above satisfactory, and a gain of
  65 or more will be considered outstanding. The
  gain is defined as the net change in the average
  score divided by the average of the pre-test. If
  full tests cannot be given, few questions will be
  sufficient.
• 7.2 PSC 1121 Physical Science - In each exam,
  questions will be embedded to enable an
  assessment of the extent to which this objective
  is met. The questions may be multiple choice,
  long answer, or quantitative problems. An overall
  score of 70 on these questions will be
  considered acceptable.

10
Examples of Objectives and Measures from UCF

• GLY 1030 Geology and its Applications - To
  demonstrate an understanding of multifaceted
  scientific terminology and an ability to apply
  the definitions involved. 
• Measures 
• 19.1 GLY 1030 Geology and its Applications - A
  set of 8 tightly-written definitions, several
  involving at least 3 characteristics, will be
  presented and illustrated in class, in
  conjunction with graded homework assignments. A
  7-question quiz (or quiz section) will follow,
  testing knowledge of precisely what is included
  in each definition. Each quiz question will
  require more than one answer and no partial
  credit will be given, in order to stress the
  importance of learning the entire definition for
  application later in the course. Because any
  error or omission can render an answer incorrect,
  a score of at least 4 out of 7 on these questions
  by 100 of students who earn a "C" or higher in
  the course will be considered acceptable. A
  similar, but smaller, set of questions will be
  included on a pretest in Spring 2007 to establish
  a baseline for use in subsequent years.

11
Examples of Objectives and Measures from UCF

• GLY 1030 Geology and its Applications - To
  demonstrate an understanding of multifaceted
  scientific terminology and an ability to apply
  the definitions involved. 
• Measures 
• 19.2 GLY 1030 Geology and its Applications - A
  subsequent test will include an embedded set of 7
  questions that require simultaneous application
  of all the characteristics of each definition
  from measure 19a that is tested. For example,
  determining the number of minerals in a
  collection described in the question-set requires
  consideration of whether to count types or
  pieces, natural and/or synthetic materials, and
  crystalline and/or noncrystalline materials. In
  addition, knowledge of the basic mineral
  compositions of the igneous rocks is needed.
  Again, no partial credit will be given, in order
  to promote the learning of knowledge as a
  cohesive whole, rather than as a set of
  disconnected fragments. Because any error or
  omission can render an answer incorrect, a score
  of at least 4 out of 7 on these questions by 100
  of students who earn a "C" or higher in the
  course will be considered acceptable. A similar,
  but smaller, set of questions will be included on
  a pretest in Spring 2007 to establish a baseline
  for use in subsequent years.

12
Examples

• UCF General Education Science Foundation
• http//iaaweb.ucf.edu/assessment/visitor/view_200
  6_07.asp?program_ID1240
• UCF Assessment Plans for 2006
• http//iaaweb.ucf.edu/assessment/visitor/y67.asp?
  groupvisitor