CAC, SRC, APWT, RFC, and SSP?

1
CAC, SRC, APWT, RFC, and SSP?

• OEAA Conference 2006 Day 2
• MI-Access Coordinators

2
Session Presenters

• Alison Peterson, BETA, Inc.
• Sheila Potter, BETA, Inc.
• Desiree Spikings, BETA, Inc.
  

3
Goals for the Presentation

• Provide an overview of MI-Access assessment
  development, highlighting committee involvement
  in the process.
• Explain the composition of the committees and the
  benefits of participating.
• Describe roles/functions/tasks of the APWT, CAC,
  SRC, RFC, SSP.
• Tell how interested participants can apply.

4
MI-Access Advisory CommitteesComposition

• Educators, parents, and other advocates
  dedicated to learners with special needs
• Well-balanced groups
• Geographically demographically diverse

5
MI-Access Advisory CommitteesComposition

• Special general education teachers
• Content specialists
• Teacher consultants
• Administrators
• School psychologists, counselors, diagnosticians
• Community business representatives

6
MI-Access Advisory CommitteesAdvantages of
Participating

• Active role in shaping the assessments
• Networking and collegiality
• Learning and professional development
• Contributions to the quality of education in
  Michigan

7
Advisory Roles and Functions

• Assessment Plan Writing Team (APWT)
• Content Advisory Committee (CAC)
• Sensitivity Review Committee (SRC)
• Rangefinding Committee (RFC)
• Standard Setting Panel (SSP)

8
Assessment Development Process
Identify Assessable Content. Committee APWT
Design Assessment Plan. Committee APWT
Develop Assessment Blueprint. Committee APWT
Write Passage or Stimulus Materials.
Train Item Writers.
Develop Items.
9
APWT?
10
Assessment Plan Writing Teams

• Function Recommendations for the design of the
  assessment plan
• Activities
• Examine research base resources.
• Extend MI Content Standards, Benchmarks, Grade
  Level Content Expectations (GLCEs).
• Using the Extended GLCEs and Benchmarks (EBs),
  identify content assessable at the state level.
• Develop the assessment plan.

11
Assessment Plan Writing Teams

• The Assessment Plan includes
• Underlying assumptions
• Content area constructs to be assessed
• Format of assessment prototype items
• Administration, scoring, reporting
  recommendations
• Preliminary assessment blueprint (length, number
  of items per EGLCE/EB, administration, time,
  accommodations, item specifications)

12
Item Review Process
Edit Items.
State Internal Item Review Meeting
External Item Review Meeting Committees CAC
and SRC
Field-Testing
Data Review Meeting Committees CAC and SRC
New Form Construction
13
CAC?
14
Content Advisory Committee

• Function Review of assessment items for
• Alignment with Content Standards, Benchmarks,
  EGLCEs/EBs
• Appropriate grade difficulty level
• Quality, accuracy, completeness
• Wording clarity

15
Content Advisory Committee

• Decisions
• Is the item important enough to ask?
• Does the item accurately match the content of the
  EGLCEs and Extended Benchmarks?
• Is the item coded correctly?
• Is the item appropriate for the grade
  developmental level of the students?

16
Content Advisory Committee

• Decision What makes an assessment item good?
• Clearly worded stem
• Plausible distractors
• Concise language
• Defensible correct answer

17
CAC Review

• Mary and Dad want to go out to eat. They go to
  a _______ to have dinner.
• A library
• B doctor
• C restaurant

18
CAC Review

• Who was the 7th President of the United States?
• A Andrew Jackson
• B Henry Ford
• C George Bush
• D Elvis Presley

19
CAC Review

• Who was the 7th President of the United States?
• A Andrew Jackson
• B John Quincy Adams
• C Abraham Lincoln
• D James Madison

20
CAC Review

• An employer is
• A a person who hires people for a job.
• B a person who works for a newspaper.
• C a person who looks for a job.

21
SRC?
22
Sensitivity Review Committee

• Function
• To review all assessment items materials for
  fairness potential bias issues
• To make sure assessments are sensitive to
  cultural other differences among students

23
SRC What is item bias?

• Item bias is any characteristic of an assessment
  passage or item that takes the focus off the
  content and causes a difference in performance
  between groups of students.

24
SRC Types of Item Bias

• Religious
• Age
• Socioeconomic
• Disability

• Racial/Ethnic
• Cultural
• Gender
• Controversial material or language

25
SRC What should we look for?

• Stereotypes or negative perceptions
• Knowledge that is elite or private more or less
  readily available to one or more groups, but not
  all
• Cultural insensitivity misrepresentation
• Balance of gender race

26
SRC Review

• The game in this passage is like
• A Duck, Duck, Goose.
• B Hopscotch.
• C Marbles.
• D Hide and Seek.

27
SRC Review

• Shayna was unable to catch the drift of what
  Rashid was saying because of a
• A bad phone connection.
• B siren from a fire truck.
• C loud crashing sound.
• D terrible ear infection.

28
SRC Review

• Which ball is NOT shaped like a sphere?
• A football
• B tennis ball
• C basketball
• D none of the above

29
SRC Review

• Allan was playing golf. He bogied 5 holes and
  double bogied the other holes. His final score
  was 99. . . .

30
Rangefinding and Scoring
Assessment Administration
Data Review Meeting Committees CAC and SRC
New Form Construction
Rangefinding for Field- Tested Constructed
Response Items Committee RFC
Scoring the Assessment
31
RFC?
32
Rangefinding Committee

• Function From a large sample of student
  responses for each assessed prompt, the RFC
  selects sets of papers that can be used for
  training and qualifying scorers.

33
Rangefinding Committee

• These sets are
• Practice/Training
• Qualifying
• Anchor

34
Rangefinding and Scoring

• FI Expressing Ideas Scoring Rubric
• Focus on topic
• Development with details examples
• Organization of ideas
• Use of conventions (e.g., grammar, usage,
  mechanics, spelling, perspective, shape, clarity,
  etc.)
• Writing Drawing

35
RFC

• Tasks
• Independently score a variety of actual student
  responses to a particular Expressing Ideas
  prompt.
• With the group, discuss attempt to reach
  consensus on scores for papers that represent the
  whole range (low, medium, high) of each of the 4
  score points.
• Repeat process with each prompt.

36
Standard Setting and Reporting
Standard Setting is Conducted. Committee SSP
The TAC Reviews and Approves Recommendations of
the SSP.
The Performance Standards are Presented to the
SBE.
Assessment Results Are Released to the Public.
37
SPP?
38
Standard Setting Panel

• Function To recommend the cut scores that
  delineate achievement at 3 performance levels
• Emerging
• Attained
• Surpassed

39
SSP

• Panels include educators, parents, business
  leaders. They
• represent geographic ethnic diversity and
• are grouped by grade cluster and Sp. Ed.
  population.

40
SSP

• Tasks
• Learn the standard-setting technique.
• Define refine the Performance Level Descriptors
  (PLDs).
• Take the assessment.

41
SSP

• Engage in 2-3 practice rounds of standard
  setting.
• Between rounds, discuss defend ratings with
  fellow panelists.
• Consider actual student performance data
  implications of proposed cut scores.
• Participate in final round, the one that counts!

42
MI-Access Advisory Committees

• So . . . what have we learned??
• Why Apply to be a MI-Access committee member?

43
Committee Membership -- Benefits

• Active role in shaping the assessment process
• Networking and collegiality
• Learning and professional development
• Contributions to the quality of education in MI

44
MI-Access Advisory Committees
45
MI-Access Committees

• How do we apply?
• Application Form
• www.mi.gov/mi-access
• Questions
• Peggy Dutcher
• dutcherp_at_mi.gov