GRAMMAR ASSESSMENT THROUGH PERFORMANCE: WHAT DO TEACHERS THINK

1
GRAMMAR ASSESSMENT THROUGH PERFORMANCE WHAT DO
TEACHERS THINK?

• Sener ES
• senere_at_anadolu.edu.tr
• Anadolu University, TURKEY

2
Organization

• Performance Assessment
• The Present Study
• Findings and Implications

3
Performance Assessment
4
Performance Assessment

• a form of on-going assessment that requires
  students to perform written and spoken tasks as
  classroom activities and teachers to monitor
  their performance and grade them according to
  previously established criteria
• (Bachman, 2002 H. D. Brown, 2004 Norris et
  al., 1998)

5
Why Performance Assessment?

• How we teach and how we assess should match.
• (Heaton, 1990, p 7-8)
• Assessment must not be an isolated step in the
  instruction cycle.
• No single form of assessment works well in all
  situations and for all purposes.
• It may be a more valid indicator of students
  knowledge and abilities. (Sweet et al., 1992)

6
Traditional Assessment vs. Alternative Assessment

• One-shot, standardized exams Continuous
  long-term assessment
• Timed, multiple-choice format Untimed,
  free-response format
• Decontextualized test items Contextualized
  communicative tasks
• Scores suffice for feedback Individualized
  feedback and washback
• Norm-referenced scores Criterion-referenced
  scores
• Focus on the right answer Open-ended, creative
  answers
• Summative Formative
• Oriented to product Oriented to process
• Non-interactive performance Interactive
  performance
• Fosters extrinsic motivation Fosters intrinsic
  motivation
• (H.D. Brown, 2004, 13)

7
Key Points in Performance Assessment 1

• Practicality
• Time / thought / effort both in planning and
  application
• Reliability
• Raters judgments / Subjectivity
• Criteria (guidelines used to judge the quality
  of student performance)
• Fairness
• Rater / Students
• Raters subjectivity in scoring
• Students may not be given equal opportunities
  to show what they know
• Validity
• Variety of assessment types
• Reference to the objectives of the curriculum
• Integration
• Integration of assessment into instruction
• Integration all four skills into assessment
  period

8
Key Points in Performance Assessment 2

• Continuous Assessment
• Not a different time slot is allocated for
  assessment
• Keeps going as the instruction goes on
• Observation
• Checklist
• Rating Scale
• Tasks
• essay/paragraph writing
• oral presentations
• pair/group discussions
• open-ended problems
• real-world simulations
• and other authentic tasks

9
Ways of Keeping a Record

• Checklist Approach
• Narrative/Anecdotal Approach
• Rating Scale Approach
• Memory Approach
• (Stiggins, 1994)

(not recommended)
10
What we do

• Grammar In-context Assessment
• Midterm ? 50
• Quizzes ? 20
• In-class ? 30 (Performance Assessment)

11
What we do

• In-Class Assessment
• 1) Written and Oral Tasks as classroom activities
• 2) In-class Assessment Criteria
• 3) Checklist (optional)
• 4) Grading Sheet

12
The Present Study
13
Purpose of the Study

• to find out the Grammar teachers attitudes
  towards the present application of In-class
  Assessment in Grammar in Context classes and
  prospective alternative assessment types.

14
Setting

• Anadolu University, TURKEY
• School of Foreign Languages
• Lower-Intermediate Level
• Grammar in Context

15
Participants

• 15 teachers
• 3 18 years of experience in teaching English
• (mean 9,2 years)
• 3 17 years of experience in teaching Grammar
• (mean 7,5 years)
• 8 of 15 have MA degree in ELT
• 4 of 15 have done research on Grammar

16
Design of the Study

• Questionnaire
• Follow-up interviews (5 of the participants)

17
Findings

• The weight of In-class Assessment (30) in total
  assessment.
• OK ? 87
• NO ? 13
• Participant 11
• I think 30 for In-class Assessment is high.
  It must be lower than the percentage of quizzes
  at least because we grade more specific items in
  the quizzes, and they are more objective. But,
  In-class Assessment is rather subjective. So, it
  must be 20 perhaps.

18

• of Written Tasks for one assessment period.
• less than 3 ? 0
• 3-5 ? 60
• 5-7 ? 27
• more than 7 ? 13
• of Oral Tasks for one assessment period.
• less than 3 ? 20
• 3-5 ? 20
• 5-7 ? 33
• more than 7 ? 27

19

• I use the In-class Assessment Checklist
  (optional)
• Yes ? 80
• No ? 20
• This checklist helps me observe students
  performance more easily and keep a good record of
  my observations.
• Strongly Agree ? 33
• Agree ? 50
• Undecided ? 17
• Disagree ? 0
• Strongly Disagree ? 0

20

• Participant 12
• I dont use the checklist because its a waste
  of time for me. As students do a lot of
  activities, I cant use it in the classroom all
  the time. So, I have to use it after every
  lesson, and I dont think it is convenient. So, I
  consider students general performance throughout
  the term, and give their scores according to
  this.
• Participant 13
• I dont use the Checklist because it is not
  user friendly. I am for holistic assessment,
  rather than focusing on particular products. I
  think what we are supposed to assess has to be
  the development of students over time, rather
  than their performance on single tasks.

21

• The application of In-class Assessment is in
  agreement with the Course Objectives.
• Strongly Agree ? 27
• Agree ? 46
• Undecided ? 20
• Disagree ? 7
• Strongly Disagree ? 0

22

• In-class Assessment is a valid way of Grammar
  assessment.
• Strongly Agree ? 20
• Agree ? 40
• Undecided ? 33
• Disagree ? 7
• Strongly Disagree ? 0
• Participant 15
• Of course, the longer the assessment period
  takes and the more varied assessment types we
  use, the better. The results are more reliable
  and more valid. It creates chances for frequent
  use of test-retest applications.

23

• It takes a lot of time and effort to implement
  In-class Assessment.
• Strongly Agree ? 13
• Agree ? 28
• Undecided ? 13
• Disagree ? 33
• Strongly Disagree ? 13
• Participant 15
• Moreover, I cant use all of the tasks because
  of time constraints. So, I give students fewer
  speaking tasks than writing tasks because they
  take a lot of time in the classroom.

24

• The Written/Oral Tasks in the book and the study
  pack do not promote the use of the target
  structure.
• Strongly Agree ? 0
• Agree ? 0
• Undecided ? 13
• Disagree ? 40
• Strongly Disagree ? 47
• Participant 12
• Actually, Im not happy with some of the tasks
  in the course book. They dont help me to assess
  students productivity. But, I take them as
  examples, and use them after making some
  improvements or changes.
• Participant 15
• Some tasks are not interesting and productive
  at all. Sometimes students do these activities
  just to practice grammar. Not as a real-life
  tasks. So sometimes I dont want to use them,
  sometimes students dont either.

25

• It is easy to observe and then to consider the
  students performance in these tasks while
  scoring their In-class grades.
• Strongly Agree ? 13
• Agree ? 40
• Undecided ? 20
• Disagree ? 20
• Strongly Disagree ? 7

26

• Participant 4
• It is really difficult to observe all the
  students in crowded classes. So, grading process
  is really tough for me.
• Participant 13
• In-class Assessment causes too much assessment
  in the classroom. It leaves no space for the
  teacher in the classroom. This In-class
  Assessment thing creates a kind of atmosphere in
  which the students are to be observed and
  assessed by the teacher every second of the
  lesson. This creates a kind of atmosphere where
  students and teachers work under a lot of
  pressure. We also give students multiple quizzes.
  So, the grammar classes involve too much
  assessment.
• Participant 5
• Its not easy to observe and assess students
  performance The focus of the class shifts. The
  teacher needs to be able to adapt different
  roles. One being the moderator, facilitator and
  the other assessor.

27

• I am fair towards every student while assessing
  their In-class performance.
• Strongly Agree ? 33
• Agree ? 54
• Undecided ? 13
• Disagree ? 0
• Strongly Disagree ? 0
• Participant 9
• I believe there are different applications of
  in-class assessment in grammar courses, which
  results in unfair grades among students from
  different classes. The reason for this could be
  that the assessment process is rather
  subjective.

28

• The In-class Assessment Criteria helps teachers
  assess their students communicative skills.
• Strongly Agree ? 13
• Agree ? 60
• Undecided ? 20
• Disagree ? 7
• Strongly Disagree ? 0
• Participant 13
• About 80 of our students in our school have
  weak language skills, especially communicative
  skills. One reason for this may be that they
  werent assessed correctly and appropriately in
  high school. I mean their communicative skills
  maybe werent assessed at all. So, In-class
  Assessment might be filling this gap .

29

• The In-class Assessment Criteria fosters
  students classroom participation in general.
• Strongly Agree ? 73
• Agree ? 27
• Undecided ? 0
• Disagree ? 0
• Strongly Disagree ? 0
• Participant 11
• In-class Assessment makes students active in
  the classroom. It increases students
  participation in the activities through fear of
  grades.
• Participant 12
• When compared to earlier times, students are
  more active in the classroom after In-class. I
  think it is thanks to the In-class Assessment. I
  observe that they have been learning better.
• Participant 5
• Its not the criteria that fosters students
  participation in the classroom, rather the
  grades students get because of their performance.
  They do it to get higher grades.

30
General Reflections on the Criteria 1

• Participant 4
• We use this criteria in all levels from
  beginner to advanced. Does this criteria provide
  a valid and reliable grading for all levels?
• Participant 5
• Effective, yet the Task Achievement component
  is not contextualized. Its too broad to be
  applicable to all tasks.
• Participant 6
• The lowest grade should be zero in the
  criteria.
• Participant 13
• The last bends of the criteria describes a
  never or no behaviour. However, it is still
  worth 5 or 3 points. Why do we give a score for
  nothing?

31
General Reflections on the Criteria 2

• Participant 4
• It is not user friendly. It is difficult to use
  this criteria to assess 30-35 students at one
  time. It doesnt eliminate a teachers bias
  towards a specific student. Each grader can
  interpret the descriptives as he wishes.
• Participant 11
• Criteria must be improved. Language use and
  Task Achievement components are confusing. We
  are supposed to do a general (holistic)
  assessment, but these components are analytic.
• Participant 1
• There are some inconsistencies between the
  components of the criteria. It is not clear what
  we expect from students. Lets say a student who
  is very silent and who never talks in the
  classroom may receive 15 from the Classroom
  Conduct component. Is this really what we want?
  Do we want silent students? Or do we want active
  participants?

32

• Peer-assessment can successfully be implemented
  in Grammar classes in the following years.
• Strongly Agree ? 13
• Agree ? 40
• Undecided ? 27
• Disagree ? 7
• Strongly Disagree ? 13
• Self-assessment can successfully be implemented
  in Grammar classes in the following years.
• Strongly Agree ? 40
• Agree ? 27
• Undecided ? 20
• Disagree ? 13
• Strongly Disagree ? 0

33

• Participant 12
• I dont think self or peer-assessment can be
  implemented in this school because students cant
  know what to assess. They have to be better than
  their peers to assess them. We do this in writing
  lessons, you know, we make them give
  peer-feedback. But, they cant find their peers
  mistakes both in terms of grammar, vocabulary,
  mechanics and content. I mean they have to be
  able to write better topic sentences to have an
  opinion about their friends success. So, they
  must know grammar better to assess their friends
  
• Participant 3
• The success of any new application depends
  mainly on the intention and appreciation of the
  teacher who uses that application.

34
Implications

• In-class Assessment should continue
• with some adjustments and improvements
• Criteria should be improved or rewritten
• especially the component Task Achievement
• with more explicit and clearer descriptives
• more user-friendly
• more holistic
• scoring

35
Implications

• The syllabus or the application of In-class
  should be reorganized to create more space and
  time for teachers.
• The productive tasks in the coursebook and the
  study pack should be reconsidered to increase
  their productivity and authenticity.
• A greater planning should be carried out for the
  prospective application of self or
  peer-assessment.

36
Thank youAnadolu University,
TURKEYsenere_at_anadolu.edu.tr