Postschool Data Collection: Training Interviews

1
Post-school Data Collection Training Interviews

• Potsdam Institute for Applied Research (PIAR)
• At SUNY Potsdam, J. Pat Turbett, Director
• Center for Change in Transition Services (CCTS)
• At Seattle University, Cinda Johnson, Principal
  Investigator

2
New York

• Potsdam Institute for Applied Research (PIAR)
• At SUNY Potsdam
• J. Pat Turbett, Director
• 315-267-2718 piar_at_potsdam.edu

3
New York State

• Urban-Suburban-Rural
• 700 districts
• 3.3 Million Students 1.1 Million in NYC
• 25,000 exits Special Education each year
• 8,000 from NYC
• 2005-2006 SPP14 includes 107 NYC
• (73 districts had 25 or fewer in their survey
  pool)

4
Central Interviewing/ Call Center What works
well (strengths)

• Efficient to train and supervise staff
• Consistent, onsite, training and supervision
• Interviewing 7 days a week morning-evening
• Security
• Simplifies mid-survey modifications
• Improves interviewer and respondent objectivity
• Computerized onsite management tools
• Consistent messages/T.A. to districts

5
Training to interviewers

• Voluntary Confidential Accuracy
• Required Online Course (short) SUNY RF
• Conducted by PIAR Staff
• At hire, and regularly throughout survey cycle
• Provide and Review an Interviewers Manual
• Expectations/ Responsibilities/ Schedules
• Orientation to the Computer based surveys
• Practice Practice Practice

6
Training to interviewers

• Voluntary Confidential Accuracy
• Protecting respondent rights
• Initiating the survey
• Consent
• Strategies for objective interviewing
• Consistent survey administration
• Disability awareness and appropriate
  accommodation strategies

7
Survey Formats

• Computer assisted phone survey
• CASES Software
• Web-based survey
• Mail Survey
• English
• Spanish
• Large Print

8
Contact Rates

• Statewide 65
• New York City 52
• Rest-of-State 69
• Districts with 10 or more in the survey pool
• 27 to 90
• ED 52
• Minority 50 Dropped Out 42

9
Reporting data

• All data are securely stored and backed up onsite
  
• Automated and customized reports
• Preliminary reports are prepared for NYSED
• Final Reports are generated for NYSED
• Demographic Representativeness
• Response Rates by Demographic Characteristic
• Statewide, District Sub-population Reports

10
Limitations and areas of improvement

• Interviewers are calling all over the state and
  may not understand all local references.
• Respondents do not know SUNY/interviewers and are
  sometimes suspicious or reluctant to participate.
• We have a limited number of interviewer stations
  and equipment
• It is difficult to find interviewers who speak
  every language spoken by respondents/families.

11
Limitations and areas of improvement General

• Out-of-Date Contact Information provided by
  districts is more of a factor than refusals
• Students with more stable lives are more likely
  to be interviewed during the federal survey
  cycle.
• Improvement testing earlier contact and
  surveying to improve response rates among at risk
  youth

12
Washington State

• Center for Change in Transition Services
• www.seattleu.edu/ccts
• Seattle University
• Cinda Johnson, Principal Investigator
• 206-296-5888 cinda_at_seattleu.edu

13
Washington State Census Data

• All districts, all leavers
• Districts with high schools 248
• Graduates and drop-outs 5,000 plus
• 80 contact rate with all leavers

14
Post-school Outcome Data

• 12th year of post-school data collection
• Consistent survey with district input
• Paper/pencil to web-based survey
• Districts use data for LEA application for funds
• Districts develop goals for program improvement

15
Training for interviewers

• Assist districts in developing system for data
  collection
• Training in person, video, teleconferencing and
  web-based
• Expectations, responsibilities, timelines
• Training to the computer-based surveys
• Voluntary and confidential

16
Training to interviewers

• Protecting respondents rights (IRB)
• Assuring consent
• Identifying respondent
• Consistency of interview
• Timing the calls and initiating the survey (at
  least 3 attempts at different times of day)
• Accommodation strategies, LEA identifies

17
Training Activities

• Directors meetings (fall and spring)
• Conferences
• State department of education monthly bulletins
• Email list serve
• CCTS website

18
Post-school Outcome Survey

• Web-based survey
• Training modules online
• Research coordinator at CCTS
• District manager
• District users

19
Reporting data

• All data are securely stored and backed up onsite
  
• Reports provided for SPP, state, districts and
  agencies
• Disaggregated by disability, gender, ethnicity
  and race, urban to rural, educational region,
  county.
• Comparison to state level data and by year.

20
Suggestions for Improvement

• Assure that emergency contact information is up
  to date and reliable.
• Be prepared to offer resources and information to
  former students and families.
• Share post-school data beyond special education
  personnel.

21
Conclusion The Top Ten

• Things you should never do as an interviewer!

22
Top Ten

• Interview the wrong person.
• Make up questions and codes of your own.
• Talk to other people while on the phone (other
  than people you should be interviewing).
• Talk so fast that even you dont know what you
  are asking.
• Tell someone how much you like them or their
  answers.

23
The Top Ten, continued

• Tell your friends how silly someones answers
  were it might have been them you were talking
  to.
• Tell an anonymous participant you like being
  their neighbor and once dated their brother.
• Laugh at any answers.
• Accept offers of dates from participants (or
  propose one).
• (Okay, top 11!) Swear or use slang or make sexual
  comments (unless they are in the survey).