Background questionnaires: Why ask about social identity

1
Background questionnaires Why ask about social
identity

• Ruth Childs (OISE-UT)
• Orlena Broomes (OISE-UT)

2
Social Identity

• Social Identity is a potentially important factor
  in our attempts to understand student achievement
  on large scale tests.
• We typically do not collect enough information
  about this variable.
• It takes time and effort to develop good
  questionnaire items

3
Questions

• How do we deal with those groups who do not want
  the information that could be obtained by such
  items?
• Aboriginal population
• What are the implications of options such as I
  prefer not to Respond
• Can respondents distinguish between
  confidentiality and anonymity

4
Four Perspectives on the Issue of Motivation as
it Relates to Large-scale Assessment

• Christina van Barneveld (Lakehead University)
• Sherri-Lynne Pharand (Lakehead University)
• Lori Ruberto (Lakehead Public Schools)
• Daphne Haggart (Lakehead Public Schools)

5
Motivation

• Motivation can have an impact on students test
  performance.
• The importance of expectation and value
• There are a variety of methods to detect lack of
  motivation.
• There is a real value in examining the
  perspectives of different stakeholders.

6
Questions

• Which students are not motivated?
• Are there other ways to raise motivation rather
  than through the use of incentives and support?
• What do you mean by the notion that motivation
  is a content irrelevant factor?
• Have you been able to get the perspectives of
  others within your stakeholder groups?
• Does increased motivation lead to change?

7
Testing of English- and French Speaking Students
in Canada

• Kadriye Ercikan, UBC
• Marielle Simon, University of Ottawa
• María Elena Oliveri, UBC
• Normand Dufour, Quebec Ministry of Education

8
Comparability of Translated Tests

• It is difficult to do this successfully.
• Minority language population are often small
  (lt5)
• There are a variety of procedures, both
  statistical and descriptive to try to address
  these comparability issues.
• We have a long way to go.

9
Questions

• Why is there such a large section on DIF?
• Should we be promoting comparability across
  languages or discouraging it?
• When can we be confident in the comparability of
  results across languages?

10
Complexities, Challenges and Opportunities in
Scoring Provincial Assessments A Perspective
from British Columbia

• María Elena Oliveri (University of British
  Columbia)
• Britta Gundersen-Bryden (B. C. Ministry of
  Education)
• Kadriye Ercikan (University of British Columbia)

11
Scoring

• The methods used to score large-scale assessments
  can impact the accuracy of the results.
• The case of British Columbia.

12
Questions

• Which kinds of tests and items are more
  susceptible to scoring errors?.
• What are some of the real challenges of
  decentralized scoring?
• Move beyond consistency
• Downloading of responsibility
• What have other researchers found out here?

13
Improving the Ways we Report Test Scores
Research Findings, and Important Steps and
Methods.

• Ron Hambleton (University of Massachusetts)
• April Zenisky (University of Massachusetts)

14
Reporting

• There is a need for score reports to include
  important technical information?
• Scales
• Measurement Error
• We need to develop reports that make it easier
  for accurate interpretations.
• Potential of benchmarking.
• Diagnostic Information

15
Questions

• But, students, teachers, etc. want more
  information!
• What accurate information can we provide?
• The need for large-scale test honesty.

16
Summary Across Papers

• Each chapter highlights an important issue in
  large-scale testing
• These topics often do not receive sufficient
  attention in large-scale testing systems
• The chapters are well written summaries of what
  we know
• Differences in the writing, structure, and depth
  of each chapter.
• There is not much new in the chapters. Most of
  the information has been reported in other forums.