An Orientation to The Surveys of Enacted Curriculum Wisconsin ELL Project

1
An Orientation to The Surveys of Enacted
Curriculum Wisconsin ELL Project

• Carolyn Karatzas
• Technical Assistance Provider
• SEC ELL Grant

2
Questions that will be addressed

• What are the Survey of Enacted Curriculum tools?
• Why are SEC data useful to educators?
• How are data collected, analyzed, reported?

3
What are the Surveys of Enacted Curriculum and
why do educators use them?
4
Survey- Defined

• 1 a to examine as to condition, situation, or
  value appraise b to query (someone) in order
  to collect data for the analysis of some aspect
  of a group or area
• www.merriam-webster.com
• 1.to examine for some specific purpose inspect
  or consider carefully review in detail2.to look
  at or consider, esp. in a general or
  comprehensive way view
• www.yourdictionary.com

5
The Surveys of Enacted Curriculum-Defined

• An educational tool to examine, inspect or
  consider the instructional content and practices
  used by teachers
• The data or information from the SEC provides a
  vehicle for discussion among teachers about their
  collective instruction
• Provides the individual teacher with information
  about their specific instruction to state
  standards

6
The Surveys of Enacted Curriculum are

• a research-based, data analysis tool for
  analyzing the content of state standards and
  assessments
• online, web-based surveys that collect teacher
  reported information about what content they
  teach and what instructional practices they use
  in one class
• graphed reports representing the data collected
  from teacher surveys and coding.

7
How many educators are using the Surveys of
Enacted Curriculum?

• 10,393 teachers took a Survey of Enacted
  Curriculum in the 2006-07 school year
• 4674 Math
• 1993 Science
• 3646 ELAR
• 80 Soc Stud
• 131 standards and assessments were coded.

8
Why are educators using the Surveys of Enacted
Curriculum?
9
Could you improve student achievement if
you had answers to these questions?

• How do I know that what teachers are teaching is
  aligned with what students are expected to learn
  as identified by state standards?
• How do I know that teachers are spending
  instructional time on the right things?
• What do I know about our instructional program
  in a low performing area?
• Are our instructional practices consistent with
  research on effective practices?
• What professional development do my teachers
  need?

10
Could you improve student achievement if
you had answers to these questions?

• Am I teaching what students are expected to
  learn?
• Am I spending my instructional time on the right
  things?
• Are my instructional practices consistent with
  research on effective practices?
• What professional development do I need?

11
Surveys of Enacted Curriculum
The intended curriculum State content
standardsWhat students should learn
The assessed curriculum State (and other)
assessmentstested learning
The learned curriculum Student outcomes based
on school learning
The enacted curriculum What teachers teach
12
Enacted (3rd Grade Teacher Reports)
Intended (3rd Grade Math Indicators)
13
Enacted (9th Grade Teacher Reports)
Intended (9th Grade Indicators)
14
How do the surveys work?

• Content specialists code the standards and
  assessments
• Teachers take an online survey about what they
  teach, how they teach it and what professional
  development they have had

15
How are standards coded?

• Teams of content experts from state departments
  are trained to code content standards using
  neutral language to describe the content.
• The resulting descriptions are graphed so that
  comparisons can be made between standards and
  assessments or between enacted and intended
  curriculum.

16
How are standards coded?
SEC utilizes a two-dimensional taxonomy based on
Topic by Cognitive Demand
17
The Content Matrix
18
The teacher online survey

• Collects data on what content is taught and how
  it is taught.
• Produces reports indicating the degree of
  alignment between the taught (enacted) curriculum
  and state assessments or standards.
• Collects data on teacher beliefs, readiness to
  teach the content and readiness to instruct
  special groups of students.
• Provides a rich source of information to support
  teachers analysis of student learning challenges.

19
The Surveys allow teachers

• to compare instructional content they are
  teaching to the state standards and assessments.
• to compare their own practice with results for
  their school, district, and state.
• SEC data is never used for teacher evaluations.
  Individual teacher responses remain confidential
  and teacher ID information is never reported.

20
Needs Assessment
Content Descriptions
Curriculum Management
SEC Taxonomy
Monitoring Change
Program Evaluation
21
(No Transcript)
22
Enacted (3rd Grade Teacher Reports)
Intended (3rd Grade Math Indicators)
23
SEC Online reports instructional content
using tile charts. Drop-down menus allow you to
select grade-specific and course-specific results
for your school, district and state, in addition
to reports of your own results.
24
Survey results can be summarized using a
collection of Instructional Practice and
Characteristics Scales Results are displayed
using floating bar charts to report mean and
standard deviation results to describe variations
I teacher responses
25
(No Transcript)
26
Wisconsin is one of nine states to participate in
the SEC - ELL Grant, an enhanced assessment grant
awarded by the U.S. Department of Education to
CCSSO and a consortium of states to better
understand and support the achievement of English
Language Learners.
27

• In too many schools, English Language Learners
  are a subgroup whose performance is not meeting
  the AYP targets. Schools, districts, and states
  across the country are trying to find answers to
  how to best provide access to and support ELL
  students in attaining state standards and NCLB
  targets.

28
To help our ELL students, we need to better
understand

• What is the nature of the language that students
  need to meaningfully engage with and achieve
  academic content?
• Given the expected student outcome, how do we
  expect students to use (receive, produce)
  language to acquire/demonstrate understanding of
  the content?
• What are the language demands and complexity of
  our state content standards?
• What do our state content standards require in
  their language complexity and demands?

29
To help our ELL students, we need to better
understand

• Are our ELL students being taught the state
  standards?
• What instructional strategies and activites are
  being used to help ELL students learn the
  content?
• Are regular education teachers supporting the
  language needs of their ELL students in any ways?
• Are the language proficiency skills aligned with
  the ELA state content standards?

30
This SEC ELL Study is addressing the following
questions

• To what extent do ELLs have opportunity to learn
  academic content and skills in state standards?
• What is relationship of state ELD standards and
  assessments to state academic standards?
• What instructional practices/strategies are used
  to teach English language skills across content
  areas?
• What is relationship of alignment of instruction
  (to standards) with student achievement?

31
What can Wisconsin educators gain from
participating in the Surveys?
32
School and district leaders can

• Identify the extent to which instruction is
  aligned to state standards and assessments
• Understand what instructional activities and
  strategies are being used in classrooms
• Compare the strategies used in classrooms with
  ELL students to strategies used in classrooms
  without ELL students.

33
School and district leaders can

• Identify the extent to which language proficiency
  standards are aligned to language proficiency
  assessments
• Identify alignment of language complexity and
  demand in content standards
• Identify priority needs for professional
  development

34
Teachers can

• Identify the extent to which what they teach is
  aligned with state standards and assessments
• Better understand what they need to be teaching
  more or less
• See how their instructional practices compare to
  other teachers in their school or district
• Identify the language complexity and demand of
  the standards they teach
• Identify priority needs for professional
  development

35
What is your role in this project?What is the
states role?
36
The States Role

• To participate in a nine state collaborative with
  CCSSO (Council for Chief State School Officers)
  and WCER (Wisconsin Center for Educational
  Research) to identify study questions, implement
  the survey in their respective states, and
  analyze the results.
• To identify which districts will participate in
  taking the survey, the target number of
  participants and content areas and grade levels
  that will be included in the project.
• To coordinate the coding of selected state
  content standards and the language proficiency
  standards for their state and send content and
  linguistic specialists to participate in the
  coding workshops.

37
Your role

• Select which schools and teachers participate in
  taking the survey and the timeframe for
  completion
• Plan and lead an orientation presentation to the
  participating teachers
• Administer or monitor the administration of the
  survey
• Attend training on how to read and interpret the
  data results from the survey
• Facilitate a discussion with the participating
  teachers about their data results

38
Administering the SurveysWhat is involved?
39
Online Survey Administration www.seconline.org

• Approximately 30-120 minutes to complete
  (depending on which surveys)
• May be completed in multiple sittings
• Data is saved as each section is completed
• Teachers may use their planning book or calendar

40
To set up the online survey for your schools,
you need to email WCER the following information
two weeks in advance

• Name of Project/Group/Region
• Participating District(s)
• Participating School(s)
• Survey Start Date
• Survey End Date
• Contact name, email and phone

41
Lets take a look at the online registration
process and the survey process to see what
teachers actually need to do.
42
SEC Online Registration
www.seconline.org
From the Home Page, click on Registrar
43
SEC Online Registration
Registrar Area
If not registered,
select your group
from the drop-
down menu, then
click on Register
button.
If registered,
enter your
username and
password, then
click on Login
button.
44
SEC Online Registration
Registration Page
45
SEC Online Registration
Final Step
After submitting your
personal information
you will be prompted
to enter a
username
and password
Enter a valid email address. This will be
important if you forget your username or password.
You will receive an email verification of your
username and password after registration is
completed. Save for your records.
Both username and Password are required to log
back in.
46
SEC Online
Survey List
47
SEC Online
Survey Menu
Sections assigned
for your group will
be pre-checked.
Sections you
have completed
will appear with
green text.
Sections you
have not
completed will
appear in white
text.
Note Instructions for Selecting the Target Class!
48
ELL-SEC Online
Survey Menu

49
SEC Online
Instructions for Selecting the Target Class

• For all questions, please respond only for the
  selected subject.
• If you teach more than one class in this subject,
  respond only for the first class that you teach
  each week for this subject.
• If that is a split class (i.e., the class
  contains more than one group for instruction, and
  each group is taught separately), respond for
  only one group.

50
Your data is saved each time you click on a
Submit button.
51
You may log-off at any time by simply closing
your browser.
52
Reporting Instructional Content

• Encourage use of lesson plans, grade books,
  text, etc. to recall
• Topic Coverage organized and reported by Content
  Area (e.g. math)

53
Reporting Instructional Content

• Cognitive Demand reported for each topic covered
  (e.g. math)
• Memorize Facts, Definitions, Formulas
• Perform Procedures
• Demonstrate Understanding of Mathematical Ideas
• Conjecture, Generalize, Prove
• Solve non-routine problems, Make Connections

Avoid assigning same level of emphasis across
categories of cognitive demand
54
Reporting Instructional Content
Step 1 Report time spent on topics taught
Review the list of topics
presented for the
current Content Area.
For each topic in the list
that is taught to the
target class, select a
radio button
corresponding to 1,2, or
3 based on the
following definitions
0 Not covered
1 Less than 1 lesson
2 1-5 lessons
3 more than 5 lessons
55
Reporting Instructional Content
Step 2 Set expectations for students for each
topic taught.
Focus on target class reporting period.
For each topic selected
from the previous
screen set the cognitive
expectations for
students for each of 5
categories of cognitive
demand, using the
following definitions
0 No emphasis
1 Slight emphasis
2 Moderate emphasis
3 Sustained emphasis
56
ELL-Reporting Instructional Content
57
Reporting Instructional Content
Step 3 Repeat steps 1 2 for each content area
presented.
Step 1 Report time spent on topics. Step 2
Report expectations for students.
58
Survey Completion
Editing Results

• Completed survey sections may be reviewed/edited
  until
• The survey completion window has closed
• You review results for a given survey section
• To review/edit responses
• Check the box on the Survey Menu Screen next to
  the section you want to review/edit. (Completed
  sections will be identified by green text.)

59
Reviewing Results
Online Report Generator
ELL data will be available Early Fall
60
(No Transcript)
61
(No Transcript)
62
SEC Online reports instructional content
using tile charts. Drop-down menus allow you to
select grade-specific and course-specific results
for your school, district and state, in addition
to reports of your own results.
63
SEC Online Contour Maps are also
available. Drop-down menus allow you to select
grade-specific and course-specific results for
your school, district and state, in addition to
reports of your own results.
64
Taking a look at the survey Items

• Please take the next 10 minutes to read through
  some of the questions in each sections of the
  survey. Record any questions you have, and we
  will address them to the entire group at the end
  of this time.

65
Planning and Leading the Orientation Session
66
What do you need to tell teachers about taking
the survey?

• What do teachers need to know to be able to do a
  good job taking the survey?
• Where and when will teachers take the survey?
• What do teachers need to bring?
• What data will teachers receive after they take
  the survey?
• How will their confidentiality be protected?
• Are there any additional incentives for their
  participation?

67
What happens after teachers take the survey?

• The state, districts, and schools will have
  invaluable data to analyze for a variety of
  purposes.
• Next October, you will participate in a training
  workshop on how to read and interpret your data.
• After that training, you will facilitate a data
  discussion with your schools about how to read
  their SEC data and what lessons they are learning
  as a result of their survey data.

68
Who should you contact for additional information?

• At the state.
• Shelley Lee
• Shelley.Lee_at_dpi.wi.gov
• Jacqueline Iribarren
• Jacqueline.Iribarren_at_dpi.wi.gov
• Aubree Potter
• Aubree.Potter_at_dpi.wi.gov
• At CCSSO
• Carolyn Karatzas, Technical Assistant
• 603.231.8534 carolyn_at_ekaratzas.com

69
Where can you find additional information?