PerformanceBased Accountability in Qatar

1
Performance-Based Accountability in Qatar

• Sonia Ben Jaafar (MORE)
• International Congress for School Effectiveness
  and Improvement
• January 7, 2009

1
2
Why PBA in Qatar?

• Large-scale student testing as a mechanism for
  educational accountability is an international
  norm
• (Popham, 1999)
• Performance-based accountability (PBA) When
  student performance on tests are indicators of
  school effectiveness and used for accountability
• (Firestone, Mayrowetz, Fairman, 1998
  Fitz-Gibbon Kochan, 2000)
• Education is a national priority in Qatar
• Qatar holds a prominent and leading role in the
  Arab world.
• Only country in the Arab world that uses
  standardized measures and methods to track
  student performance over time in an attempt to
  review the progress of the reform
• (Supreme Education Press Release, 2007)

2
3
This Paper

• Position of PBA in Qatars reform
• Five-dimensional PBA
• Canada UK
• Methods
• Results
• Implications for understanding Qatars PBA
• Adoption of foreign educational approaches
• Practicality of parameters of developing country
• Next Steps

3
4
4
5
Qatar

• Emiri state HH Sheikh Hamad Bin Khalifa Al Thani
• Population 824,789 (July 2008)
• National population Approx 200,000
• National GDP 71.42 billion (2007 est.)
• GDP per capita 87,600 (2007 est.)

5
6
Qatars Major Educational Reform

• Human development of the national population
  through education (General Secretariat for
  Development Planning GSDP, 2008)
• 2002
• started to implement the K-12 education reform
  including
• developing a new governance structure
• new curriculum standards
• new assessment system
• 2004 SEC opened first Independent schools
  (Brewer et al., 2007)
• 2008
• 3313 Qatari students enrolled in Independent
  secondary schools (70.9 nationals)
• 916 of them are expected to graduate (Statistics
  Authority, 2008).

6
7
Performance-Based Accountability

• Five Dimensions
• Testing Structure
• Standard setting
• Consequential use of results
• Reporting
• Professional involvement

7
8
Method

• SEC policy text
• 66 documents
• N-Vivo
• Dimension Tree node
• Elements nodes
• Emerging nodes Free nodes
• Summary by dimension

8
9
Testing Structure (D1)

• Qatar Comprehensive Educational Assessments (QCEA)

QSCC Exit Examinations

• Grades 4 11
• In 2004 All students in school receiving
  government support assessed
• Since 2007 Only Independent schools
• Subjects Arabic, English, Mathematics, and
  Science
• Multiple-choice short-answer questions
• NB. Participation is required but is not
  mandated
• Administered in spring
• Results released a month later

• Grade 12
• Independent schools only
• Senior Tests (STs)
• Subjects Arabic, English, Mathematics and
  Science
• Multiple-choice short-answer questions
• Administered in May
• Results released a month later

10
Standard Setting (D2)

• Qatar Comprehensive Educational Assessments (QCEA)

QSCC Exit Examinations

• Curriculum Own
• Purpose Improvement accountability
• Performance Scales and levels
• 5 levels
• Below Standards (1-3)
• Approaching standards (4)
• Meeting standards (5)
• Acceptability Target Meeting Standards

• Curriculum Own
• Purpose Credentialing
• Performance Scale scores (Relative)
• Acceptability Target Unspecified

11
Consequential Use of Results (D3)

• Qatar Comprehensive Educational Assessments (QCEA)

QSCC Exit Examinations

• For students 3
• For system (teacher school state
  policymaker) merged with QCEA
• Overall score 10/15

• For students 0
• For system (teacher school state
  policymaker) 7
• Overall score 7/15

12
Reporting (D4)

• Qatar Comprehensive Educational Assessments (QCEA)

QSCC Exit Examinations

• 4 reports
• Results aggregation
• Indicators Leadership information,
  infrastructure, teaching methods, facilities,
  demographic information, parent and student
  school satisfaction
• Comparisons Student nationality, gender, school
  type, cohort, subject area, across schools,
  between students, between classrooms

• 1 report
• Results aggregation N/A
• Indicators Student non-academic accomplishments
• Comparisons Gender

13
Professional Involvement (D5)

• Pre-testing phase

Testing/post-testing phase

• Standard setting (0)
• Test construction (1)
• Test preparation (3)

• Test administration (0)
• Scoring (2)
• Results interpretation (3)

• Pre-testing phase 4/9
• Testing/post-testing phase 4/9
• Total involvement 8/18

14
3 Types of PBA Models

• Maximizing consequential use of results relative
  to professional involvement
• Mid consequential use of results relative to
  professional involvement
• Minimizing consequential use of results relative
  to professional involvement

14
15
Qatars PBA Model

• Maximizing consequential use of results relative
  to professional involvement
• Mid consequential use of results relative to
  professional involvement
• Minimizing consequential use of results relative
  to professional involvement

15
16
Complexities In Comparisons

• Economy Politics
• 4 year terms Vs Emiri state
• Hierarchy Vs Autonomy
• Education for All
• Capacity Sustainability
• Appropriate identification and attention to
  capacity issues
• Baseline capacity for professional Involvement
  (5D)
• Qatarization Non-national teachers
• Adoption Adaptation
• Cultural assumptions embedded in PBA policies
• Responsive policy implementation given context

16
17
Next Thoughts

• Embedded assumptions surface
• Re-conceptualizing contextualizing PBA For
  understanding
• Socio-cultural expectations
• Economic expectations
• Developing countries and dynamic processes
• Implications
• Theoretical
• Comparative methodology
• Interpretive

17