Title: Professionalization of evaluation
1Professionalization of evaluation recent
activities within DeGEval
- Dr. Jan Ulrich HenseDepartment Psychologie,
Empirische Pädagogik und Pädagogische
Psychologiejan.hense_at_psy.lmu.de - Jahreskongress SEVAL 2012, Workshop 2 Freitag,
7. September 2012Université de Fribourg
2- (1) Some comments and questions on the
professionalization of evaluation
3- Two uses of professional
- Referring to a job or designation (a
professional painter) - Is evaluation developing into a full-blown
profession? - Referring to performance on a task (a
professional repair) - How to carry out evaluations professionally?
4Evaluators?
5- Two Classes of Professions (Mieg, 2010)
- Care and relationship orientation
- Clients or patients trust professionals
expertise and integrity - Readiness to work cost free in emergency
situation - e.g. Doctors, Lawyers, Psychotherapists
- Market orientation
- Price competition
- Income orientation
- e.g. Engineers, Consultants
6- Professionalization as regulation of autonomy
- Definition of education and training demands
- Control of market entry
- Determination of fees and charges
- Monopoly of performance evaluation
(Mieg, 2010)
7- Conditions for the development of a new
profession - Combination of
- a problem relevant for society and
- knowledge on how to explain and deal with the
problem - Relation to central values of society (e.g.
health, justice) - (Mostly) academic education and training
- Professional societies
(Mieg, 2003)
8- Phases of development of a profession (Wilensky,
1964) - Some are working in a new area (mostly) full time
- Education and training programs are developed
- Professional societies emerge
- Shaping profile by delegating lower tasks
- Conflict between established and graduates of new
training programs - Competition with neighboring professions
- Professional Codes und Standards
- Move to formal closure
- Rules and conditions for working in the new
profession - Protection of designation
- Declaring non licensed work illegal
9- (2) Recent DeGEval activities in the context of
professionalisation - Competencies for evaluators
- Didactics of Evaluation
- Further projects and plans
10- Competencies for evaluators
11- Why develop "Recommendations on Education and
Training in Evaluation"? - Growing demand for evaluation services
- Lack of qualified personnel
- Lack of a professional profile
- Guidelines as contribution to the
professionalization of evaluation
12- A bit of history...
- 2004 First German edition
- Authors A. Caspari, M. Hennen, D. Scheffler, U.
Schmidt O. Schwab - Published by the DeGEVal working group
"Education and Training in Evaluation - 2008 English translation
download/order on www.degeval.de
13- Goals and intended uses
- Standards for
- Training programmes
- Direction for programme developers and teachers
- Evaluation of existing programmes
- Assessing evaluators
- Choice of an evaluator by clients
- Self-evaluation of evaluators
14- "A competency is more than just knowledge and
skills. It involves the ability to meet complex
demands, by drawing on and mobilising
psychosocial resources (including skills and
attitudes) in a particular context." - (OECD/OCDE DeSeCo-Project)
- Which knowledge, skills and attitudes are needed
for conducting professional evaluations?
15(No Transcript)
16(No Transcript)
17(No Transcript)
18(No Transcript)
19(No Transcript)
20(No Transcript)
21- Development of the recommendations
- Theoretical approach
- Based on DeGEval standards of evaluation
- Practical procedure
- Expert panel
- Feedback cycles from DeGEval members
- Officially adopted by DeGEval conference
22- Who uses the guidelines?
- Usage in advanced study programmes
- Berne ?
- Saarbrücken ?
- Bonn ?
- Further education courses ?
- Evaluation courses within domain-specific
curricula ?
23Coverage of competencies in psychology diploma
programmes
Soellner, R., Hapkemeyer, J. Scheibner, N.
(2010). Evaluationsausbildung in der Psychologie
eine Bestandsaufnahme. Zeitschrift für
Evaluation, 9 (2), 297-308.
24- Need for revision (to be discussed)
- Concept of competencies (KSA)
- Field-specific interpretation and differentiation
- Elaboration of methodological competencies
- Compatibility with international models of
evaluator competencies, e.g. - King et al. (2001 2005) Mertens (1994)
- Zorzi, McGuire Perrin (2002)
- Russ-Eft et al. (2008)
25- 2. Didactics of Evaluation
26http//www.degeval.de/arbeitskreise/aus-und-weite
rbildung-in-der-evaluation/publikationen-und-prod
ukte
27- Why didactics of Evaluation?
- Competency profiles
- Contents and learning goals of training
programmes - ? What to teach
- Didactics of evaluation
- Practical guidelines for teachers and trainers
- ? How to teach
28- Goals
- Provide guidance for
- training and education programmes
- courses in evaluation
- self-learning materials (textbooks etc.)
- Based on current state of theories on learning
and instruction - Supplement existing recommendations on competency
profiles
29- 0 Executive Summary
- Introduction
- Learning contents and goals Competence profiles
for evaluators - Programmes
- Types of programmes and target groups
- Didactical foundation
- Learning and teaching methods
- Requirements for teachers/trainers
- Outlook
CondensedCookbook guidelines
30- Important Messages
- Cover whole range of competencies (KSA)
- Apply a mix of construction and instruction
- Emphasize active, self-directed learning
- Allow for learners own practical experiences
- Provide instructional support
- Embed all contents in authentic evaluation
contexts - Match method and learning goals
- Adapt to learners individual needs and
preconditions
31- Examples
- Chocolate cookie exercise
- Exercise on the logic of evaluation
- Reflection exercise An evaluator is like a ...
- Discuss different evaluator roles and
responsibilities - Role play stakeholder resistance
- Awareness for possible sources of resistance to
evaluation
32- 3. Further projects and plans
33Future projects
- Other topics and activities of the DeGEval
working group Education and Training in
Evaluation - Overview of training courses and seminars
(online-database) - Textbook overview and review
- Evaluation capacity building
- Certification and credentialing of evaluators
34Future projects
- Overall DeGEval efforts
- Work group on ethics in evaluation (ethical
codes) - Revision of evaluation standards (until 2014)
35- Thank you for your interest!
- contactjan.hense_at_psy.lmu.de