Dr. Algirdas Zabulionis

1
Examinations from Mission to Reality

• Dr. Algirdas Zabulionis
• Anglia Assessment Ltd
• Lithuania

2
From soft assessment to hard exam!
3
in mass media
4
Who is interested in examinations?

• Politicians
• Ministry of education
• Local educational authorities
• Education specialists
• Schools
• Teachers
• Students
• Parents
• Business companies (IT, printing, logistics,)

5
Why are educational assessment and examinations
so popular now?

• Reforms of Educational Systems in most countries
  include the changes in educational assessment
• World Bank, ADB, EU, British Council, Soros Fund
  projects have subcomponents on Educational
  Assessment
• Armenia, Azerbaijan , Bulgaria, Czech R, Georgia,
  Hungary, Lithuania, Latvia, Estonia, Kazakhstan,
  Kyrgyzstan, Macedonia, Moldova, Montenegro,
  Poland, Romania, Russia, Slovakia, Slovenia,
  Ukraine,

6

• COMPLEXITY OF THE PROBLEM
• Educational issues
• Policy issues
• Technical problems and solutions
• Security, transparency
• Public Relation campaign

GOALS OF ASSESSMENT!!!
EXAMS what is behind this tasty word?
7
Examinations reforms might be focused on

• Pure educational goals (quite different by
  their nature monitoring, quality assurance,
  certification, motivation,)
• New ways of selection applicants to Universities
• Fight against corruption in the area of admission
  to Universities

Different priorities different impact on
students, teachers, educational process, society
8
Examination vs. educational assessment
EdA targets system

? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
? ? ? ?
Exam targets individuals
9
EXAMINATIONS ASSESSMENT
May be taken by all students. Conducted using a small, representative sample.
Generally test across a broad curriculum. Generally focus on a limited number of selected 'core competencies'.
Do not investigate background variables and attitudes. Results are systematically analysed against a broad range of background variables.
The same pattern of examinations has to be repeated annually. The same pattern of assessment does not have to be repeated regularly.
Exam 'standards' are rarely fixed by sophisticated psychometric techniques. Require psychometric techniques for test calibration, year-on-year comparisons, etc.
The associated stakes (for students) are high. The associated stakes (for students) are low.
Stakes are high teaching may be distorted by 'teaching to the test'. Stakes are low teaching and learning should be distorted less.
Stakes are high exams may be used to influence classroom practice. Stakes are low - the backwash effect is weakened.
Stakes are high security risks are great. Stakes are low - security is rarely a problem.
In many countries, students pay examination fees. Students will not pay for national assessment
10
Why do we need Matura exams?

• Helps restore meaning to a high school diploma
• Increases the public confidence in the school
• Certifies that the student has particular
  competences
• Promotes the teaching of particular knowledge and
  skills
• Serves as a motivational element for both
  educators and students

11
Does one size fit all?Why do we need EXTERNAL
exams?
Secondary schoolleaving examinations
University entrance examinations

• Certification
• Criteria-based
• All students leaving the school
• Test corresponds to the course learnt by student
  (basic/advanced levels)
• Educational goals to look back to the course
  taught
• Test certificates the maturity (pass/fail?)
• Low stake

• Selection
• Norm-based
• Students applying for university only
• One test for all
• Some freedom for the content
• Test intends to show the potential abilities
  (motivation?)
• Test should differentiate the students
• High stake

12
Unified HEI admission system

• Information about the places available in HEI
  and the rules for calculating weighted rank score

Information of students wishes to apply to HEI
Central Info System (Db)
EXAMS
Centralized informationon students achievement
13
Why do we like/dislike external high stake
examinations?

• Psychological discomfort (student, teacher,
  parents)
• Responsibility for the students (bad?) results
• Preparation for exam (private tutoring)
• Test-driven curricula, teaching for exam
• Comparison (unfair?) of schools, teachers
• Costs (financial, time, human resources)
• Consolidation of the schools
• Possibilities to see equity, equal opportunity
  issues
• Society asks for transparency, reliability,
  validity, professionalism

14

• More information
• who could use this information about the
  education system, school, student, teacher?
• how might this information be used?
• breaks myths on the quality of education
• calls for actions

15
Common principles of all examination systems

• Conformity with the main objectives of the exam
  system (clear priorities selection OR
  certification)
• Child-centered approach (in many cases - as
  declaration only ?)
• Professionalism, security, transparency
• Advanced technological solutions
• Use of information for secondary analysis (for
  education quality monitoring more in wishes
  than in reality ...?)
• Links with the test theory (classical and modern
  test theory IRT)
• International cooperation, analysis of the best
  practice

16
Some current issues and fashions...

• Use of results by Universities
• Students rights
• Reliability (security) of the exams system
• Relationship between Exam Centre and teachers
• Could we trust somebody in the exam?
• Overestimation of the ICTs role
• Wish to measure higher order skills
  (competencies?)
• Attempts to use exam data for secondary analysis
• Blossom of private tutoring

17
New technologies in service for examinations

• Why NEW TECHNOLOGIES?
• the Stone Age did not end for lack of stone
• goals (educational/anti-corruptional) and
  national (socio/cultural/educational) contexts of
  exams
• objectives to use new technologies
• professionalism (target the highest quality)
• effectiveness of works
• public trust, transparency

All computerized IT systems were developed and
are administrated by humans! Who will guard the
guards themselves? asked Roman poet Juvenal,
55 AD-127AD
18
Share of 1st year students in universities who
entered HEI through private tutoring
International Survey, OSI, 2006
19
Sorry to tell, but

• Exam itself doesnt improve the quality of
  education
• Exams cant be a tool to fight against corruption
  in education
• One size doesnt fit all exams for selection
  differs from exams for certification
• All computerized testing systems were developed
  and are administrated by human
• In most cases teachers were taught how to teach,
  but havent got even basic knowledge on
  educational assessment
• Perfect examinations do not exist

20
IN AN IDEAL WORLD, EXAMS SYSTEM WOULD BE
CAREFULLY DESIGNED TO

• meet goals for which it was set
• be fair to all students
• reflect curriculum priorities and traditions
• provide feedback that guides teaching/learning
• provide results that are reliable and comparable
  across students
• be professional and transparent
• be administratively convenient and inexpensive.

21
Summary

• Exams are not short-time fashion, as well as not
  panacea
• What are the main goals of the exam system?
  (short/long term goals might be different, but
  both should be taken into account)
• Exams system should be carefully (professionally)
  designed, implemented and maintained
• Do not forget about the back-wash effects
• Exams are not the island in the ocean!
• It might be DONE!

22

• THANK YOU for your attention!
• Algirdas Zabulionis
• eMail algiz_at_mail.lt