Healthy Youth Survey

1
Healthy Youth Survey

• Using your Data Workshop
• Spring 2009

2
Welcome!

• Review agenda and objectives for the day
• Is there anything else you hope to get out of
  this training?
• Logistics
• Break
• Workbook follows this powerpoint
• Do you have a local report???

3
20 Years of Student Surveying!!!!
What were you doing 20 years ago?Share with your
neighbor.
4

• Group Discussion

How have you used HYS results? Who do you plan on
using your results with?
5
New Ways to Use Your Results
1

• With schools,
• communities,
• and youth

6
Using Healthy Youth Survey Data with School
Administrators
7
Points to Include..

• Connections to School
• Risk and Protective Basics
• Link Student Behavior/Attitudes with Academics
• Overview HYS
• Example of Action
• Resources

8
Why is linking academics and child well-being
important?

• What we do in the name of health, safety, and
  well-being are linked with teaching and learning.
  Teaching and learning cant take place if
  students arent healthy, arent physically and
  mentally fit, or arent safe.
• William Modzeleski, Director, Safe and Drug
  Free Schools Program,
• U.S. Department of Education

9
Make the Connection Student Behaviors
Academic Success
10
Taking Actionsome examples

• Share data with parents/students
• Implement targeted or school-wide initiatives to
  address areas of interest
• Set school improvement goals (or strategies
  embedded within goals) related to risk and
  protective factors

11
Regional Prevention Contacts

• http//www.k12.wa.us/PreventionIntervention/
  CoordinatorDirectory.aspx
• Curriculum
• Professional Development
• Technical Assistance
• Data Support

12
Community Mobilization

• Using HYS data to help change community
  perspectives about important issues among kids

13
1. Bring the community together to discuss how
to reduce underage drinking 2. Motivate
community to talk with children about not using
alcohol before age 21.
14
(No Transcript)
15
(No Transcript)
16
Keys to success

• Well organized and strong coalition
• Close collaboration with the school district
• Involvement of the kids
• They planned a great meeting, and fed everybody!
   
• Resulted in a new parent network

17
To get their perspective on whats important to
them and their ideas for improving their
community.
Sharing HYS with Youth
18
Example Wahkiakums Healthy Youth Summits

• In 2005 and 2007, Wahkiakum held Healthy Youth
  Summits
• A community-wide group of youth and community/
  school adult leaders met
• Youth and adults shared their interpretations of
  HYS data

19

• In the morning, the YOUTH.
• Reviewed HYS data and discussed
• If they believed the results
• Reasons for their results
• What needed to change in their community
• Prioritized issues and recommendations for adult
  leaders

20

• In the afternoon, the ADULTS.
• Reviewed the data what youth said
• Discussed priorities and their recommendations
• Began to think about coming year what could be
  done to address youth issues

21
Keys for a successful summit

• Create a comfortable and confidential environment
  for youth
• Assure youth that they will be taken seriously -
  they are the experts and their opinions are
  highly valued
• Develop concrete ideas for addressing youth
  issues
• Recognize that everything doesnt have to be
  solved immediately
• Appreciate the importance of starting this type
  of dialog between youth and adults

22
Replicating something similar

• There are tools available to it easy to talk
  about HYS data
• Powerpoint slides of results
• (sent by RMC, see page 15 in your workbook)
• AskHYS.net topical fact sheets
• This could be done with in many ways
• With a school or community youth group
• Within any geographic location a county, a
  city, a school district, or a school
• It could even be done in a single classroom

23
Understanding Your Local Report and Results
2

• 2008 administration
• Local report overview
• Understanding your results

24
HYS 2008 administration

• A collaborative effort among the Joint Survey
  Planning Committee, made up of 6 state agencies
  and the survey contractor
• Office of the Superintendent of Public
  Instruction
• Department of Health
• Department of Social Health Services/
  Department of Alcohol Substance Abuse
• Department of Community, Trade and Economic
  Development
• Family Policy Council
• Liquor Control Board

25
2008 HYS participation

• The 2008 Healthy Youth Survey was completed

By 211,244 students In all 39 counties. In 247
school districts In 1,097 schools
26
Local Report Overview
Types of reports Your report Summary of selected
results Selected results by gender Item
groupings Risk and protective factors List of
core items item index
27
Types of reports

• Individual School reports
• Districts, counties ESDs
• Report of Results
• gt70 participation rate
• Report of Participating Schools
• 40-69 participation rate

28
Your report
Healthy Youth Survey 2008Survey Results
Sample Middle School, Grade 8
Number of students surveyed
173 165 206 80
Number of valid responses
Estimate of enrolled students
Survey participation rate
The impact of adolescent health risk behaviors
remains a primary concern of citizens throughout
the country.
29
Your report, cont.

• Key to the Notes

A  wording on Form AB  wording on
Form BC  wording on Form C  optional item
30
Summary of selected results

• Reports include same questions and graphs as in
  2002 through 2006 reports

31
Cross-tab results by gender

• Same as in 2002 through 2006 reports
• Results suppressed to protect anonymity

If p value lt 0.05 the difference between local
males and local females is statistically
significant

• Do you have asterisks instead of numbers for
  gender?

32
Item groupings

• General Information
• ATOD Use
• Other Health Concerns
• School Climate
• Quality of Life
• Risk and Protective Factor Items

33
Item examples (Grade 10)
34
Why is item n lt overall n?

• Not a core item (Grades 8, 10, and 12)
• Optional look for symbol
• Students did not reach item near end of survey
• Students skipped or did not mark clearly

35
Why asterisks instead of numbers for frequencies?

• No students responded (n0)
• Suppressed to protect anonymity
• Overweight (Item 65)
• Results by Gender (cell nlt10)

36
Risk and Protective Factors

• Framework and Reporting Schedule
• Scale Results
• Graphs
• Individual Item Results

37
Last page

• List of Core Items
• Secondary only
• Helps interpret ns
• Item Index

38
Fundamentals for Understanding Your Results
Validity and reliability Generalizability
Confidence intervals Comparing state and local
results Comparing data over time
39
Reliability

• Reliability Does the survey consistently produce
  the same results under the same conditions?
• How we assure reliability
• Standardized administration procedures

40
Validity

• Does the item measure what it is intended to
  measure?
• How we assure validity
• Items from established instruments, validity
  checks

41
Generalizability

• What is generalizability?
• Why 70 participation is important
• Challenges to generalizability
• School level
• ESD, County, or District level
• How do these challenges affect data
  interpretation?

42
Do I have to generalize?

• Yes, if you want to apply the results to a larger
  population.
• 8th graders in our district said.
• Yes, if you want to compare to others or results
  over time
• No, if you want to just describe the students
  surveyed, in that moment, without confidence
  intervals
• Students at our school who took the survey said.

43
Confidence Intervals
What are they? How to interpret the numbers How
to interpret the graphs Comparing State and Local
Results Practice Activity 1
44
What is a confidence interval?

• The reported value is unlikely to be exactly the
  same as the true value for all your students
• We are 95 confident that the true value is
  within the /- range, called the confidence
  interval

45
Why do we need confidence intervals?

• Confidence intervals account for variability
  among students, NOT validity of the data
• Variability is inherent in any population worth
  studying
• Variability causes uncertainty in the results
• Doubt is not a pleasant condition, but certainty
  is absurd Voltaire
• They help us compare our results to others and to
  ourselves over time

46
What do confidence intervals look like?
Smoked cigarettes (Grade 10, 2008) 14.4 (
1.6) 14.4 1.6 12.8, 14.4 1.6
16.0 Between 12.8 and 16.0 smoked cigarettes
47
Why are confidence intervals different sizes?

• Number of students
• Inherent variability
• Level of confidence (All 95 for HYS)
• Sampling design

48
Significant differences

• Smoked cigarettes in the state 14.4 ( 1.6)
• Between 12.8 and 16.0
• Smoked cigarettes at my school 20.0 ( 2.0)
• Between 18.0 to 22.0
• Conclusion Difference IS statistically
  significant
• (confidence intervals do not overlap)

49
Significant differences
50
Non-significant differences

• Smoked cigarettes in the state 14.4 ( 1.6)
• Between 12.8 and 16.0
• Smoked cigarettes at my school 20.0 ( 10)
• Between 10.0 to 30.0
• Conclusion Difference is NOT statistically
  significant
• (at least 1 confidence interval overlaps the
• other point estimate)

51
Non-significant differences
52
Significance inconclusive

• Smoked cigarettes in the state 14.4 ( 1.6)
• Between 12.8 and 16.0
• Smoked cigarettes at my school 20.0 ( 5.0)
• Between 15.0 to 25.0
• Conclusion Inconclusive, more testing required
• (confidence intervals overlap each other but not
• the point estimates)

53
Significance inconclusive
54
Exercise 1Comparing State and Local Results

• Directions
• Page 38 in your workbook
• Question 1
• Determine if the difference between the local
  result and state result provided statistically
  significant
• Question 2-4
• Calculate the highest and lowest percentages
• Determine if the difference between your local
  result and state result is statistically
  significant
• What it mean about your local students?

55
Example question 1
Example What do the statistics suggest are the
lowest and highest percentages of students who
said Yes to the question Has a doctor or nurse
ever told you that you have asthma?
Are the state and local rates different?
56
Example question 1 graphically
24.0 to 32.0
19.3 to 22.3
57
What if I am in a small school, and we have large
confidence intervals?

• Having a confidence interval protects you (and
  your program) from appearing to be ineffective
  when just a few students can make big changes
• Consider the input of teachers staff from
  small-school environments when interpreting data
  the data should be used to complement what they
  already know about their kids

58
New Tool Using Excel and CIs to perform a
statistical test

• Conduct a statistical test using this Excel file
  to determine if two groups are significantly
  different.  
• Use this method if there are 30 or more
  respondents for the item for both the local and
  state results.
• http//www.hys.wa.gov/Reporting/Default.aspx

59
Using Excel and CIs to perform a statistical test
(contd.)
The p-value is less than 0.05, so the local
result is significantly higher compared to the
state result.
20.0 14.4
5.0 1.6
60
Combining item responses
To determine the confidence interval for Any
days. Use the same confidence interval as the
0 days" response. ( 2.0)
To determine the result for Any days Subtract
the 0 days result from 100 100.0 - 97.0
3.0
61
Combining item responses cont.
To determine the result for YES Add the
Probably yes and Definitely yes responses
together 29.5 59.0 88.5
When you combine 4 response options down to 2
groups like this, you cannot determine the
confidence interval
62
Comparing data over time

• Things to think about
• Did the questions change?
• http//www.hys.wa.gov/Reporting/Default.aspx
• Were the challenges to generalizability similar?
• Is there a reason to think that things would have
  changed? (a prevention program)

63
Final notes about significance

• Even if a difference is statistically
  significant, it might not be practically
  significant
• For example
• State use of some drug 12.8 (0.2)
• Local use of that drug 14.4 (1.2)
• The difference is statistically significant, but
  is it different enough to influence program
  planning?
• CIs are quick and available, but there are more
  precise tests to determine significance. EXCEL

64
Dont worry, help is available from many sources

• Local Health Department
• Local ESD
• JSPC agencies
• Web materials

65
Exploring AskHYS.net
3
Introduction to the website Topic specific survey
fact sheets Q x Q data query analysis system
66
Welcome to AskHYS.net
67
Log on page
68
Log in site
Need permission? Click here to find out more.
69
How to get access
70
Results on AskHYS.net
71
Available results?
72
Available results
No results available
73
Fact sheets on AskHYS.net
74
Check the topic(s) grade(s)
X
75
Select geography, year gender
76
2008 Sample Fact Sheet

77
Information and background
78
Current measure chart
79
State Comparison Chart
Rode with Drinking Driver Compared to the State,
All Grades, 2008
Significantly lower local rate for 6th
grade. Significantly higher local rate for
12th graders.
80
Trend chart
NO Significant changes change from 2006 to 2008
81
Academic Achievement
82
Analysis with Q x Q on AskHYS.net
83
Running frequencies and cross-tabs

• Frequencies
• Use to get results for a single variable
• Cross-tabs
• Use to get results for a variable crossed with
  another variable. Even gender, is a cross-tab
• Minimum respondents per cell required
• State analysis must have 5 or more
• Sub-state analysis must have 10 or more
• Questions must be on the same survey form or a
  core question
• E.g., cant cross cigar smoking (only form B)
  with meth use (only form A)

84
Select year, grade, gender location
85
Starting your analysis
86
Select variable in dropdown menu
87
Select your variable
88
Drag your variable into 1st box
89
For frequency results
90
Your results
91
For crosstab results
92
Your crosstab results
Race / Ethnicity
Current Alcohol Drinking
93
Switch your variables around
94
Your new crosstab results
95
Selecting a sub-state location with same variables
96
Error message if cell size is too small
97
Error message if variables are not on the same
form, or core
98
Right click on variable for more information or
to remove it from your analysis
99
www.AskHYS.net
100
Communicating Your HYS Results
4

• How to talk about your results

101
Simplify when talking about numbers

• Dont say 17.4 plus or minus 3.1 of students
  said
• Round whole numbers
• Use language to convey that these percents are
  estimates
• About 17
• About 17 plus or minus three percent
• Between 14 to 21 percent

102
Use the correct terminology

• How does your response rate effect how you talk
  about your results?
• 70 Participation
• About 17 of 8th graders said
• 45 Participation
• About 17 of the 8th graders in our community
  who took the survey said

103
Consider different ways to say it

• You could say
• About 17 of our youth feel unsafe in school
• About one in six youth feel unsafe in school
• Remember you can present it positively
• About 83 of our youth feel SAFE in school
• About five out of six youth feel SAFE in school

104
Communication planning

• Whats your SOCO?
• (Single Overriding Communication Objective)
• The so what of your message
• Meaningful to your target audience
• Tells them what you want them to learn or do

105
What does a SOCO look like?

• For broadcast media 10-12 word sound bite
• For print media 1-3 line quotation
• For real people people just wont remember more
  than 3-4 ideas

106
SOCO example

• Community leaders should speak out in support of
  Safe Drug Free Schools Programs
• A SOCO needs to be supported by
• Details Healthy kids learn better
• Provide logical justification
• These are reasons why
• The details need to be proven by
• Facts HYS data show strong associations between
  substance abuse and lower academic achievement
• These are the evidence for your details
• Use your data here

107
Message map
108
Example Tobacco program results
109
Exercise 2 Using your data to communicate

• Pick one scenario (A,B,C) on page 64
• Work in a small group
• Use the data in the grid on page 65
• Create a SOCO - page 66
• Share

110
Exercise 2 Discussion

• Some SOCOs that we came up with
• The school board needs to take action to
  prioritize alcohol prevention programs for our
  middle school children
• Every parent in our community should talk to
  their kids about drinking and driving
• Funding this program will reduce the excess
  danger that children in our community face from
  alcohol

111
Exercise 2 Discussion, cont.

• Details (supported by facts in the data)
• Alcohol use in 8th graders is higher for our
  children compared to the state
• Both driving while drinking and riding with a
  driver who had been drinking are significantly
  higher for 10th and 12th graders compared to the
  state
• Our 8th, 10th and 12th graders are more likely to
  report their parents did not talk to them about
  the dangers of drinking alcohol.
• Our 10th and 12th graders are less likely than
  children statewide to believe that regular
  alcohol use is dangerous

112
Potential traps.

• Including non-relevant information or providing
  too much detail (be brief!)
• Speculating beyond the data or treating
  differences as significant when they are not
• Forgetting to double-check your numbers
• Using too many graphics, or graphics that distort
  the data
• Talking about numbers instead of the people they
  represent

113
Helpful tips

• Make your first words count
• Have someone you trust critique your message for
  understandability and accuracy
• Practice saying everything, including the
  numbers, out loud
• Practice saying I dont know, but I can get back
  to you about that
• Be prepared to discuss the limitations of your
  data
• Speak with clarity and compassion

114
Keep on message

• The answer to every question is your SOCO.
• Thats an interesting point, but what Id really
  like people to know is,
• my SOCO

115
Remember.Data are only ONE piece of the
puzzle.But sometimes Without data, youre
just another person with an opinion - unknown
DATA
116
Final thoughts

• Dont flip through your HYS report to find the
  story, find your story and use data to support it
  (most of the time)
• If your story is we have important data to
  describe our kids (overview) then use the local
  slide sets provided to you
• Share your HYS results!

117
Why share your results?

• Awareness of data availability
• Gain community, school, and youth perspectives
• Planning
• Change perceptions of issues
• Make informed decisions
• Needs assessment
• Evaluation
• Establish baselines
• Monitor and evaluating progress
• But watch out for unreasonable expectations

118
Obtaining more information

• Survey participation information and training
  materials
• www.hys.wa.gov/
• State and county level results, state level
  online data query, and survey background
  information
• https//fortress.wa.gov/doh/hys/
• Preformatted fact sheets with HYS results
• http//www.AskHYS.net

119
Questions about this training?

• DASA, DSHS, Linda Becker 360-725-3705,
  BeckeLG_at_dshs.wa.gov
• Department of Health, Kevin Beck 360-236-3492,
  Kevin.beck_at_doh.wa.gov
• OSPI Dixie Grunenfelder 360-725-6045
  dixie.grunenfelder_at_k12.wa.us
• CTED, Ramona Leber 360-725-3033
  ramonal_at_cted.wa.gov
• RMC Research Corporation, Eric Einspruch
  1-800-788-1887, www.rmccorp.com

120
Thanks!from the Joint Survey Planning Committee

• OSPI Jeff Soder, Dixie Grunenfelder
• DOH Kevin Beck, Diane Pilkey, Susan Richardson,
  Lillian Bensley, Juliet VanEenwyk
• CTED Ramona Leber
• Family Policy Council Bill Hall
• DSHS, DASA Steve Smothers, Linda Becker
• Liquor Control Board Tony Masias
• RMC Research Corporation Eric Einspruch,
  Jennifer Lembach

Dont forget to fill out an evaluation!