A Delphi Study of Critical School Counseling Research Questions

1
A Delphi Study of Critical School Counseling
Research Questions

• ASCA-ACES Research Summit
• St. Louis, Missouri
• June 28, 2003

2
National Center for School Counseling Outcome
Research

• John Carey
• Carey Dimmitt
• http//www.umass.edu/schoolcounseling/

3
Center Mission

• The National Center for School Counseling Outcome
  Research is dedicated to improving the practice
  of School Counseling by developing the research
  base that is necessary for responsible and
  effective practice.

4
Center Mission

• The Center provides national leadership in the
  measurement and evaluation of the outcomes of
  school counseling interventions and programs. The
  Center helps K-12 leaders and practitioners use
  research findings and methods to improve school
  counseling practice by enabling them to set
  standards, establish benchmarks, implement
  evidence-based interventions, measure student
  learning outcomes, and evaluate program
  effectiveness.

5
Why the Need for Empirical Studies of School
Counseling Outcomes?

• Need to Identify Best Practices
• Need to Identify Evidence-based Interventions
  (NCLB)
• Need to be Accountable
• Need to Demonstrate Impact on Student Achievement
• To Develop as a Profession

6
Leadership Needed

• To Evaluate Existing Evidence
• To Identify Critical Questions
• To Identify the Most Important Outcomes
• To Identify or Develop Good Outcome Measures
• To Develop New Research Approaches
• To Coordinate Multi-Site Efforts
• To Communicate the Implications of Research
  Findings to Practitioners

7
Center ActivitiesNational Focus

• Research Brief Listserve, 10,000 subscribers
• Summer Institute, Using Data to Help All
  Children Succeed (July 14 thru 18)
• Review of Outcome Studies and Measures
• Delphi Study of Critical School Counseling
  Research Questions

8
Center ActivitiesRegional Focus

• Urban Guidance Directors Leadership Group
• Task Force on a Massachusetts School Counseling
  Model
• Program Evaluation and Leadership Consultation
• Curriculum Development Consultation
• Research Partnerships with School Districts

9
Center New Initiatives

• National Board for Evidence-Based Practice in
  School Counseling
• Establish Criteria
• Select Critical Outcomes
• Evaluate Evidence
• Identify Needed Research
• Make Recommendations for Counselor Education

10
Center New Initiatives

• Electronic Journal School Counseling Outcomes
• Promote empirical identification of best
  practices
• Promote the development of effective research
  approaches
• Promote the development of quality outcome
  measures
• Disseminate findings broadly to multiple
  constituencies
• Establish credibility for school counseling
  practice

11
Delphi Study of Critical School Counseling
Research Questions

• Delphi Research Uses Expert Opinion to
• Forecast Future Trends and Directions
• Identify Critical Issues
• Achieve Consensus and/or Map Points of
  Disagreement
• First Phase in Evidence-Based Professional Boards
• Delphi Research is Especially Appropriate for
  Identifying Approaches to Complex Problems in
  Areas with Multiple Constituencies
• Delphi Research Has Been Used Successfully by
  National Agencies to Develop Priorities for
  Research Funding

12
Delphi Study of Critical School Counseling
Research Questions

• Panel Selection
• Phase I Identify the Purposes of School
  Counseling Research
• Phase II Identify Specific Research Questions
  For Each Purpose
• Phase III Prioritize Research Questions and
  Identify Criteria for Judging Importance

13
Expert Panel

• Judy Bowers
• Carol Dahir
• Teesue Fields
• Norman Gysbers
• Trish Hatch
• Richard Hayes
• Peggy Hines
• Reese House
• Curly Johnson
• Dawn Kay
• Judy Madden

• Brenda Melton
• Linda Miller
• Rachelle Perusse
• Lonnie Rowell
• Russell Sabella
• Susan Sears
• Toni Tollerud
• Jose Villaba
• Susan Whiston
• Richard Wong

14
Purposes of School Counseling Research Phase I

• Delphi Study Question 1 Given the history and
  current political context of school counseling,
  what ends should research achieve?

15
Purposes of School Counseling Research Phase I
Responses

• To identify evidence-based interventions and best
  practices in terms of school counseling
  activities, interventions, programs, and models.
• Research can provide practitioners with
  information on which interventions really help
  students.

16
Purposes of School Counseling Research Phase I
Responses

• To document effectiveness in order to be
  accountable to external constituencies for school
  counseling activities, interventions, programs,
  and models.
• Research needs to show the results of a
  comprehensive school counseling program and
  demonstrate to stakeholders the effectiveness of
  the program.

17
Purposes of School Counseling Research Phase I
Responses

• To understand how to effect change in school
  counselor practice, school counseling programs,
  and school counselor roles in schools.
• More research is needed to discover what
  factors are contributing to or influencing the
  changes in school counselors beliefs and
  behaviors regarding the performance of counseling
  and non-counseling activities.

18
Purposes of School Counseling Research Phase I
Responses

• To identify the most effective approaches to
  educating and supervising school counselors in
  training.
• If best practices could be determined, they
  might help provide some kind of standard by which
  all school counselors could be prepared
  consistently.

19
Specific Research Questions Phase II Questions

• 1. Do your ideas about the purposes of school
  counseling research fit into one or more of these
  categories?
• 2. Would you add to or redefine the definitions
  of any of these purposes?
• 3. Please identify what you consider to be an
  important, specific research question for each
  purpose, if possible.

20
Specific Research Questions Phase II Responses

• Responses were collapsed into 42 distinct
  Research Questions.

21
Specific Research Questions Phase III Ratings

• Participants were asked to rate the importance of
  each Research Question
• 1Not Important
• 2Important
• 3Very Important
• 4Critically Important
• Participants were asked to identify criteria used
  in judging importance
• Participants were asked to supply additional
  questions

22
Survey

• Please Complete
• We would like your completed survey

23
Phase III Results

• 23. Which specific School Counseling
  interventions result in the greatest gains in
  students academic development and achievement?
  3.8
• 27. What specific School Counseling
  interventions are most effective at preventing
  school dropout? 3.7
• 30. What is the impact of enhancing School
  Counselors ability to use data in planning and
  accountability on student outcomes? 3.7
• 26. What specific school counseling
  interventions are most successful at preventing
  school failure? 3.6

24
Phase III Results

• 37. What are the most effective approaches to
  changing school leaders opinions about the value
  and legitimacy of school counseling? 3.6
• 2. What factors contribute to the success of a
  school counseling program? 3.6
• 8. Whats the impact of implementing a
  results-based school counseling program on
  students academic development and achievement?
  3.6
• 4. Whats the impact of implementing a
  Comprehensive Developmental School Counseling
  Program on students academic development and
  achievement? 3.6
• 36. What are the most effective approaches to
  changing school leaders opinions about school
  counselor role and function? 3.5

25
Phase III Results

• 24. Which specific School Counseling
  interventions result in the greatest gains in
  students career development? 3.5
• 25. Which specific School Counseling
  interventions result in the greatest gains in
  students personal/social development? 3.5
• 6. What is the impact of implementing a
  Comprehensive Developmental School Counseling
  Program on students personal/social development?
  3.5
• 9. What is the impact of implementing a
  Results-based School Counseling Program on
  students career development? 3.5
• 10. What is the impact of implementing a
  Results-based School Counseling Program on
  students personal/social development? 3.5

26
Phase III Results

• 12. What is the impact of implementing an ASCA
  National Model School Counseling Program on
  students academic development and achievement?
  3.5
• 35. What is the impact of enhancing school
  counselors abilities to use data in planning and
  accountability on school leaders perceptions of
  the value and legitimacy of school counseling?
  3.5
• 1. What is a successful School Counseling
  program? 3.4
• 11. What is the impact of implementing a
  Results-based School Counseling Program on school
  leaders perceptions of the value and legitimacy
  of school counseling? 3.4
• 3. What are the factors that contribute to the
  efficacy of a school counseling programs
  implementation? 3.4

27
Phase III Results

• 13. What is the impact of implementing an ASCA
  National Model School Counseling Program on
  students career development? 3.4
• 28. Under what circumstances are whole
  classroom, small group, and individual counseling
  maximally effective? 3.4
• 5. What is the impact of implementing a
  Comprehensive Developmental School Counseling
  Program on students career development? 3.3
• 15. What is the impact of implementing an ASCA
  National Model School Counseling Program on
  school leaders perceptions of the value and
  legitimacy of school counseling? 3.3
• 16. What are the best ways to measure the extent
  to which a school counseling program model is
  fully implemented? 3.3

28
Phase III Results

• 19. What is the impact of reducing
  student/counselor ratios on program outcomes?
  3.3
• 31. What are the most effective approaches to
  enhancing the leadership skills of School
  Counselors? 3.3
• 41. What are the most effective professional
  development approaches to enhance School
  Counselors competence? 3.3
• 7. What is the impact of implementing a
  Comprehensive Developmental School Counseling
  Program on school leaders perceptions of the
  value and legitimacy of school counseling? 3.2
• 14. What is the impact of implementing an ASCA
  National Model School Counseling Program on
  students personal/social development? 3.2

29
Phase III Results

• 22. What research and evaluation skills are
  necessary for effective School Counseling
  practice? 3.2
• 38. What is the impact of collaboration between
  guidance leaders and school counselor educators
  on the quality of school counseling training and
  the competence of school counseling trainees?
  3.2
• 39. What is the impact of aligning a school
  counselor education program with a school reform
  agenda on the quality of school counseling
  training and the competence of school counseling
  trainees? 3.2
• 40. What is the impact of aligning a school
  counselor education program with the ASCA
  National Model on the quality of school
  counseling training and the competence of school
  counseling trainees? 3.1

30
Phase III Results

• 17. What is the impact of aligning a school
  counseling program with the school-reform agenda
  on students academic development and
  achievement? 3.0
• 33. To what extent do School Counselors trained
  in leadership and systemic change models attempt
  to effect change in their schools? 3.0
• 42. What is the impact of school counselor
  educators work experience in schools on the
  orientation and outcomes of counselor education
  programs? 3.0
• 18. What is the impact of implementing the use
  of specific competency-based student learning
  objectives in a School Counseling Program on
  student outcomes? 2.9
• 32. What are the most effective approaches to
  developing and enhancing the professional
  identity of school counselors? 2.8

31
Phase III Results

• 34. What are the most effective approaches to
  enhancing the program organization and management
  skills of school counselors? 2.7
• 21. Do school counselors who use research to
  guide practice have a greater impact on student
  outcomes? 2.6
• 29. What is the impact of peer helper programs
  on students attendance, grades and school
  behavior? 2.3
• 20. What is the impact of different methods of
  assigning students to counselors on different
  outcomes? 2.1

32
Criteria for Judging Importance

• Research that demonstrates impact on outcomes is
  needed because school counseling is at risk
• Research that would significantly add to our
  understanding of best practices
• Research that is intrinsically interesting to
  me
• Research that advances the efficacy and
  effectiveness of school counselor training
• Research that helps the profession meet the
  challenges associated with NCLB and implementing
  Transforming School Counseling Initiative and
  ASCA National Model reforms.

33
Additional Questions

• What is the technological literacy of school
  counselors? How do school counselors implement
  technology? How does implementing technology
  impact outcomes?
• What is the process through which school
  counselors make decisions and how can
  decision-making be improved?
• What is the impact of school counselor
  self-reflection on efficacy and effectiveness?

34
National Center for School Counseling Outcome
Research

• Thank You

http//www.umass.edu/schoolcounseling/