Formative eassessments for KS3 ICT

1
Formative e-assessments for KS3 ICT

• Sue Walton
• IEA Conference, London, 3 May 2007

2
Background

• QCA remit was to develop an onscreen summative
  test to assess pupil performance in ICT at the
  end of key stage 3
• The test was to be delivered to about 600,000
  pupils in approximately 4000 schools each year in
  a 4-week test window in the summer term
• The practice test would provide pupil reports
  which were intended to be used formatively
• The test was piloted in 2004, 2005 and 2006. The
  final pilot is taking place in May 2007
• In January the project was given a new remit

3
The new remit

• The test will not become a statutory end of key
  stage assessment but rather an assessment for
  learning tool available for use throughout the
  key stage
• Revised assessment materials will provide
  formative feedback to help pupils progress and
  inform teaching
• Schools will be able to use the materials at any
  point when pupils are ready, as a resource which
  can indicate students strengths and weaknesses,
  propelling them to further success
• Jim Knight, BETT show, January 2007

4

• Revised focus is in line with other documents
  released in January
• 2020 Vision (the Gilbert report)
• Making Good Progress consultation
• Emphasis on
• Progress
• Ongoing assessment
• Personalised learning

5
NAAs vision for the new assessments

• New assessments will
• Cover the programme of study
• Be stand-alone tasks
• Be available to schools throughout the key stage
• Provide levelled formative information
• Output pupil and group reports
• Have support materials
• Include a grid to track progress

6
Task overview

• Short standalone tasks that can be fitted into
  lessons
• Tasks focused on
• content areas
• particular levels
• Tasks generate reports which include information
  that feeds into a progression grid

7
Purpose of the new tasks

• To provide information to help inform teaching
  and learning in Key Stage 3 ICT, by allowing
  teachers and pupils to plan the next stages of
  learning using the outcomes of the assessment
  together with other evidence

8
So what will the tasks be like?

• A look at the 2007 practice test

9
Task detail (1)

• 15-30 minutes long
• Bank to contain a mix of tasks either comprising
  short questions and short activities or
  containing one longer activity
• Will have a 2-level focus 3-4, 4-5, 5-6
• Will be either single application/concept or
  multiple applications/concepts

10
Task detail (2)

• Task categorisation
• Data handling
• Modelling
• Presentation/Information handling
• Control
• Delivered via the existing KS3 ICT delivery
  software but with reduced administration
  requirements e.g. no need for pupil passwords

11
What are we assessing?

• NC programme of study
• Processes as well as skills, knowledge and
  understanding
• The NC programme of study covers
• Finding things out
• Developing ideas and making things happen
• Exchanging and sharing information
• Reviewing, modifying and evaluating work as it
  progresses

12
Recording and mapping progress to the ICT
capabilities

• Identifying problems and defining tasks
• Searching and selecting information
• Organising and structuring information
• Analysing and interpreting information
• Combining and refining information
• Modelling
• Controlling events and devices
• Exchanging information
• Presenting information
• Reviewing, testing and evaluating
• Assessing the impact of ICT

13
Tasks reports

• Levelled statements
• Table with clear indication of when statement has
  been evidenced
• Clear link to the progression grid
• Individual pupil task report
• Group task report

14
Sample pupil reporting
15
Task support

• Task outline
• NC coverage
• Level focus
• Information about marking what is being
  assessed and what evidence is expected
• Guidance on how to use the task

16
Overall support

• Road map of the tasks i.e. what will be available
  and when
• Grid of available tasks
• Progression grid
• Guidance on e.g.
• Purpose of each task and how to use
• Task selection
• Using information from the tasks to inform
  teaching and learning and to help with teacher
  assessment

17
Marking

• Evidence is collected and automatically marked
• Underpinning rules base breaks down the
  national curriculum level descriptions in smaller
  processes
• These subprocesses are broken down into
  meaningful chunks of ICT evidence opportunities
• Currently opportunities are aggregated for level
  awarding for the summative test

18
Level description

• Level 5Pupils select the information they need
  for different purposes, check its accuracy and
  organise it in a form suitable for processing.
  They use ICT to structure, refine and present
  information in different forms and styles for
  specific purposes and audiences. They exchange
  information and ideas with others in a variety of
  ways, including using email. They create
  sequences of instructions to control events, and
  understand the need to be precise when framing
  and sequencing instructions. They understand how
  ICT devices with sensors can be used to monitor
  and measure external events. They explore the
  effects of changing the variables in an ICT based
  model. They discuss their knowledge and
  experience of using ICT and their observations of
  its use outside school. They assess the use of
  ICT in their work and are able to reflect
  critically in order to make improvements in
  subsequent work.

19
Rules base example

• Statement of attainment
• Pupils select the information they need for
  different purposes, check its accuracy and
  organise it in a form suitable for processing.
• Process indicator
• Select information/assets for specific purposes
• Elaboration
• Use a variety of ICT tools/ techniques or
  functions to achieve consistency

20
Gathering evidence of ICT capability
ICT capability is about having the technical and
cognitive proficiency to access, use and
communicate information using technological
tools.   Pupils demonstrate this capability by
purposefully applying technology to solve
problems, analyse information, develop ideas,
create models and exchange information.   They
are discriminating in their use of information
and ICT tools.
ICT capability in the KS3 ICT onscreen test and
tasks
Task problems allow pupils to answer in a
variety of ways.
All possible valid ways of answering are
mapped to NC Level Descriptions
As a pupil starts to answer a question
so evidence is collected
All the evidence from the test is used to
produce an overall summative level
21
Gathering evidence of pupil performance
Pupils demonstrate their ICT capability in
answering a task in a variety of ways. There is
no one correct answer but many correct answers.
These answers and parts of these answers will
demonstrate evidence of the pupil working at
various national curriculum ICT levels.
A Open e-mail Application
Evidence in the KS3 ICT test Simple example of
two valid routes producing evidence of opening an
email. This evidence will contribute to the
pupil working at level 3.
C Use File-Open menu route
B Double Click on e-mail icon
D Read instructions
22
Gathering evidence of pupil performance
Use File- Send to mail recipient menu route
Open e-mail application
Use Messages-Reply menu route
Use File- New menu route
Use Reply button
Click New Message Button
Enter equinn_at_waltywood.co.uk in To
Add atijani_at_waltywood.co.uk to cc field
Evidence in the KS3 ICT test Example of the
variety of routes a pupil could take to send a
reply e-mail with attachment to
multiple recipients. The evidence produced by
a pupil answering a complete task will be many
times greater than this. Also remember pupils
are applying their knowledge to answer problems.
Use Messages-Add Attachment Menu Route
Enter Subject
Click Add attachment button
Browse for file and click Add button
Use Messages Send menu route.
Click send button
Enter Message
23
Gathering evidence of pupil performance
The KS3 ICT onscreen test consists of a number of
different tasks. Each task will allow pupils to
demonstrate evidence of their ICT capability from
a range of national curriculum levels. All the
evidence produced by a pupil in the test is used
to award a summative national curriculum
level. The new tasks will allow pupils to
demonstrate evidence at the 2 levels covered by
the tasks. At the end of the task the pupil
report will show which pieces of evidence the
pupil actually showed.
Schematic diagram of possible evidence from a
task
Evidence in the KS3 ICT test Example of the
variety of routes a pupil could take to answer a
task. The green route is of a pupil mainly
demonstrating level 3-4 evidence. The yellow
route is of a pupil mainly demonstrating level 5
evidence. These two pupils are choosing to
answer the same task in different ways.
3
4
3
3
4
4
4
4
4
4
5
4
5
5
4
24
Evidence of pupil performance
Example task How much profit is made if 9 oranges
are sold? Evidence example Demonstrate the
ability to use a model e.g. use a model to help
the pupil find things out.
The national curriculum attainment target for ICT
gives details of expected pupil behaviour in
levels. The KS3 ICT onscreen test and new
assessment tasks further expand these levels into
detailed behaviours that still represent pupil
behaviour at the level.
Schematic diagram of possible evidence from a
task
3
4
3
Example task How much profit is made if 9 oranges
are sold? Evidence example Question
plausibility of conclusions from a model e.g.
are the results realistic? Did the total change
when I entered 9? Is the total a formula?
3
4
4
4
4
4
4
5
4
5
5
4
25
Gathering evidence of ICT capability summary
Task problems allow pupils to answer in a
variety of ways.
Pupils become more focused, efficient and
rigorous in their use of ICT, and carry out a
range of increasingly complex tasks. (PoS KS3
ICT)
All possible valid ways of answering are
mapped to NC level descriptions
Pupils apply technology to solve problems,
analyse information, develop ideas, create models
and exchange information. (Applying ICT
Capability)
As a pupil starts to answer a question
so evidence is collected
Pupils develop their ability to judge when and
how to use ICT and where it has limitations.
(PoS KS3 ICT)
All the evidence from the test is used to
produce an overall summative level
Pupil test reports detail the evidence collected
and used to award the summative level.
26
Timescales

• Further details to schools summer 2007
• First batch of materials late autumn
• First batch of tasks to be data handling followed
  by modelling in Spring 2008

27
Further information

• www.naa.org.uk/ks3ictpilot