A Framework for Assessing Professional Dispositions

1
A Framework for Assessing Professional
Dispositions

• Mary Lee Hall
• University of Tennessee at Martin
• Brenda Gulledge
• University of Tennessee at Martin

Accreditation, Accountability, and Quality
Institutional Orientation Sponsored by AACTE and
NCATE Spring 2005
2
Framing Questions

• What are dispositions?
• What are the primary sources of disposition
  outcomes for programs?
• How can disposition outcomes be included in the
  program curriculum?
• What are important decisions in assessing
  dispositions?
• How can disposition assessment data be gathered
  and used?

3
What Are Dispositions?

4
NCATEs Definition

• DISPOSITIONS. The values, commitments, and
  professional ethics that influence behaviors
  toward students, families, colleagues, and
  communities and affect student learning,
  motivation, and development as well as the
  educators own professional growth... (NCATE
  2000)

5
NCATE explains further

• Dispositions are guided by beliefs and
  attitudes related to values such as caring,
  fairness, honesty, responsibility, and social
  justice. For example, they might include a
  belief that all students can learn, a vision of
  high and challenging standards, or a commitment
  to a safe and supportive learning environment.
  (NCATE 2000)

6
Dispositions are similar to but different from

• Traits, thought processes, skills, habits,
  attitudes, and beliefs.
• For more on these distinctions see
• Katz, L. G. (1993). Dispositions Definitions and
  Implications for Early Childhood Practice. ERIC
• http//ericeece.org/pubs/books/disposit.html

7
What are Sources of Disposition Outcomes?

8
Conceptual Framework The Primary Source for
Dispositions

• Conceptual Framework
• What are the values, commitments, and
  professional ethics that guide and influence
  faculty behavior?
• How does the units mission, vision,
  philosophical orientation, and professional
  knowledge base influence outcomes?

9
Vision and Dispositions

• Vision - the view of what professional practice
  in your area should look like. A vision
  statement represents the programs future what
  the program wants to become. The vision
  statement responds to the question What should
  social studies education, counseling, etc. look
  like?
• What would you like to see the program become?
  What reputation would it have?
• What contributions would the program make to
  candidates and its community?
• What characteristics or values would the program
  embody?

10
Given this Vision

• What are the patterns of observable behavior that
  our candidates would be demonstrating that are
  consistent with our vision?
• For example, if we envision teaching as a
  profession where practitioners demonstrate
  respect for human diversity, what could we
  observe candidates doing that would be consistent
  with this vision?

11
Values and Commitments as Dispositions

• What values and commitments about professional
  practice do the faculty hold? For example,
• Faculty value human diversity and are committed
  to expressions of respect for human diversity in
  its various forms.
• Faculty value and are committed to professional
  ethical responsibility.
• Faculty value and are committed to belief in the
  ability, talent, and learning potential of all
  students.

12
Given these values and commitments

• What are the patterns of observable behavior that
  our candidates would be demonstrating that are
  consistent with these values and commitments?
• For example, if we value and are committed to a
  profession where practitioners value human
  diversity and are committed to expressions of
  respect for human diversity in its various forms,
  what could we or others observe candidates doing
  that would be consistent with these values and
  commitments?
• What could we observe candidates doing to
  indicate that they are committed to ethical
  responsibility?

13
Philosophical orientation and Dispositions

• If the program philosophy is the lens through
  which the program sees the world of its
  professional practice, then what is the
  philosophical orientation, or system of
  principles that guides decision-making in your
  program?

14
For example

• The mission of the Special Education Teacher
  Education Program is to prepare teachers who
  demonstrate the commitment and capacity to reform
  schooling and education for children with special
  educational needs from birth through young
  adulthood. This mission is built upon a set of
  principles that guides faculty teaching, research
  and service
• To collaborate with faculty within the COE and
  across the University, candidates, practitioners,
  and other community members and agencies
• To model and promote research-based and
  reflective practice
• To act as advocates for children with special
  educational needs and their families
• To support healthy development and well-being
  of children special educational needs and their
  families and,
• To promote social justice in the school and
  larger community.

15
Given this orientation or these principles

• What are the patterns of observable behavior that
  our candidates would be demonstrating that are
  consistent with this orientation and principles?
• For example, if our program is guided by the
  principle of ethical responsibility, what could
  we or others observe candidates doing that would
  be consistent with this principle or orientation?
  

16
Other Sources of Disposition Outcomes

• State and Institutional Policies
• Professional Standards
• What disposition outcomes are specified in unit
  and program specialized professional association
  standards?
• For example, From professional standards
  Candidates engage in continuous, collaborative
  learning to inform practice. NAEYC, 2001
• See also INTASC Disposition outcomes.

17
How can Disposition outcomes be included in the
curriculum?

18
Re-thinking Disposition outcomes in teacher
education programs

• If we adopted the notion of dispositions as the
  frame for our outcomes, we could ask and expect
  candidates to behave in ways consonant with the
  selected dispositions, regardless of what they
  believed about them (Raths, 2001).
• For example, the disposition to engage in
  continuous professional development.

19
Including Dispositions in the Program Curriculum

• Opportunities to learn pre-requisite knowledge
  and skills For example, knowledge about
  professional organizations, journals, and
  conferences skills necessary to conduct action
  research
• Opportunities to see target dispositions modeled
  For example, faculty share their conference
  activities with classes conduct action research
  on their own teaching and share insights,
  processes, and findings with classes refer in
  class to professionally related local newspaper
  articles
• Opportunities to practice and demonstrate target
  dispositions For example, faculty make a point
  of letting students know about professional
  conferences, letting them miss class to attend
  help initiate a literacy circle

20
What are important decisions in assessing
Dispositions?

21
When should Dispositions be assessed?

• Admission - Like many meaningful knowledge and
  skill outcomes, there are limited disposition
  outcomes that can be assessed before candidates
  enter the program. Assessment at admission may
  establish a baseline for measuring future growth.
  Assessment at this point may be viewed as a
  needs assessment.
• Appropriate transition points - Keep
  pre-requisite knowledge and skills in mind.
• Program completion - Opportunity for assessing
  advanced and complex disposition outcomes.

22
What Dispositions should be assessed?

• Dispositions that have been defined in terms of
  clearly observable behavior (e.g., What behaviors
  show respect for human diversity?).
• Dispositions that candidates have an opportunity
  to learn, practice, and demonstrate throughout
  the program.
• Dispositions that are appropriate to expect at
  each program phase.

23
Defining Dispositionsas Observable Behaviors

• The process of defining dispositions should
  include professional education faculty and
  members of the professional community.
• Together, this group should agree on a list of
  actions candidates would be taking when
  demonstrating the target disposition.
• For example, What actions could candidates take
  to demonstrate a commitment to continuous
  professional development?

24
Candidates are engaging in professional
development and learning when they

• read current professional journals and magazines
• read professionally related articles in popular
  magazines
• read professionally related articles in local
  newspapers
• read professional books not required for school
  or work
• participate in literature circles or study groups
• attend professional training sessions, workshops,
  seminars, etc.
• attend local professional conferences
• attend state, regional or national professional
  conferences
• conduct classroom-based action research to inform
  their practice
• regularly prepare professional development goals,
  plans to meet those goals, and assess progress
  toward attaining those goals
• participate in classroom observations of other
  professionals
• invite observation of their own teaching by
  others
• demonstrate use of feedback to change teaching
  for the better

25
How Should Dispositions be assessed?

• Using clearly observable (low-inference)
  behaviors that are meaningful indicators of the
  disposition outcome
• Using different types of assessments and over
  time
• Using internal and external sources of assessment
  data
• Using assessment strategies characterized as
  accurate, reliable, meaningful, valid, objective,
  and defensible

26
Assessments are Less Accurate, Reliable,
Meaningful, Valid, Objective, Defensible

• When the assessment is based on a continuous
  criteria. For example, The candidate is
  self-motivated to engage in professional
  development. Circle one of the following Never
  / Some of the time / Much of the time / Always
  Or, Assess candidates professional involvement
  on a scale of 1 to 5 where 1 is low and 5 is
  high.
• When assessments are based on evaluating
  performance using global rather than specific
  indicators. For example, rating the candidate by
  responding to this statement The candidate
  engages in continuous professional development
  and learning rather than by rating them on
  specific behaviors that characterize engaging in
  continuous professional development.
• When the assessment is based on many (5-10)
  performance levels.

27
Assessments are More Accurate, Reliable,
Meaningful, Valid, Objective, Defensible

• When the assessment consists of a fixed
  measurement scale, and a list of criteria that
  describe the characteristics of performance for
  each score point. For example, as is now the
  case in NBPTS, NCATE, and most SPA program
  standards.
• When the assessment expresses performance
  criteria in terms of observable behaviors 
• When the assessment uses fewer (2-4) performance
  levels

28
Constructing Low-inference Assessments of
Dispositions

• The process of constructing assessments should
  involve members of the P-12 professional
  community. improves validity
• Use a list of observable behaviors
  low-inference generated by the faculty and P-12
  professional community. meaningful indicator
• First identify those behaviors that describe an
  Acceptable performance standard for the
  professional role and time being assessed. Clear
  distinction between Acceptable and Unacceptable
  is important.
• Finally, identify those behaviors that describe
  an Optimal performance standard for the
  professional role and time being assessed.

29
A RUBRIC TO ASSESS THE DISPOSITION TO ENGAGE IN
CONTINUOUS PROFESSIONAL DEVELOPMENT
UNACCEPTABLE Does not yet meet expectations ACCEPTABLE Meets Expectations OPTIMAL Exceeds Expectations
Candidates show little or no evidence of an orientation toward continuous, self-motivated inquiry aimed at professional learning and development. They do not establish and follow through with annual professional development plans. They do not show that they read professional literature beyond that required for school or work. They do not take advantage of professional training opportunities. They show little or no evidence of using feedback from others to improve their teaching, and they do not observe others in order to learn. Candidates work shows clear evidence of an orientation toward continuous, self-motivated inquiry aimed at professional learning and development. Candidates regularly establish professional development goals, take action to attain those goals, and assess the outcomes of action they have taken. They read a variety of professionally related periodical literature, including professional journals, professional magazines, and articles about professional topics in popular magazines and newspapers. Candidates attend professional training sessions, workshops, seminars, including attending local professional conferences. They demonstrate that they use feedback to change their professional practice. They participate in classroom observations of other professionals Candidates show clear and consistent evidence of an orientation toward continuous, self-motivated inquiry aimed at professional learning and development. In addition to reading a variety of professionally related periodical literature, candidates read professional books not required for school or work, or participate in collaborative literature circles or study groups focusing on professional topics. Candidates attend state, regional or national professional conferences or other training opportunities. Candidates conduct classroom-based action research to inform their practice. They invite observation of their own teaching by others
30
Disposition Evaluation Form
Admission to Student TeachingCandidate Bobbi
Sue Vygotsky Date Fall, 2003

• Rate candidates on each item as either Ob
  (Observed) or Nob (Not Observed)
• _____Regularly establishes professional
  development goals, takes action to attain those
  goals, and assesses the outcomes of action they
  have taken
• _____Reads a variety of professionally related
  periodical literature (e.g., professional
  journals, professional magazines, and articles
  about professional topics in popular magazines
  and newspapers
• _____Attends professional training sessions,
  workshops, seminars, including attending local
  professional conferences
• _____Uses feedback to change professional
  practice
• _____Participates in classroom observations of
  other professionals

31
Avoiding Legal IssuesAround Dispositions

32
Put Dispositions in the Catalog

• In programs where candidates are
  specializing in a professional area, awarding a
  degree or recommending for a professional license
  does not merely attest to the accumulation of the
  specified number of hours in the classroom or
  other professional setting but also to the
  demonstration of professional knowledge, skills,
  and dispositions. The faculty has the
  responsibility to both the public and the
  profession to award a degree or license only when
  the candidate has demonstrated a satisfactory
  level of professional knowledge, skills, and
  dispositions as judged by the program faculty.
  Further, candidates must exhibit integrity and
  character consistent with the standards of
  ethical principles set forth by appropriate
  professional associations and Tennessee law.

33
Adopt Technical Performance standards

• For example,
• Candidates display evidence of commitment to the
  profession of teaching by . . . Displaying
  exemplary attendance in class and field
  experiences, completing work in a timely manner
  for classes and field experiences, and accepting
  feedback from university faculty and school
  practitioners for professional growth and
  development

34
Be clear and document!

• Assessments based on clear, observable behavioral
  indicators of the disposition outcomes
• Assessments linking observable behavioral
  indicators with performance standards for each
  level.
• Documented multiple assessments, at appropriate
  times and from internal and external sources
• Documented feedback of assessment data to the
  candidate and documented opportunities for
  improving performance

35
How Can Disposition Assessment Data be Gathered
and Used?

36
Gathering Disposition Assessment Data

• Internal sources
• Candidates demonstrated routine behavior (e.g.,
  responsible professional behavior regarding
  attendance, meeting deadlines, working as part of
  a team, etc.)
• Periodic use of disposition rating forms by
  university faculty (e.g., at end of each
  semester, at identified gateways)
• Candidate performances in response to portfolio
  prompts (e.g., document and reflect on on-going
  professional development).
• Candidate performance in response to case studies
  or simulations (e.g., What would you do in this
  situation? Comment on what the teacher did in
  this situation.)
• Candidate reflections, journals, self-evaluations
• Candidate surveys and exit interviews

37
Conceptual Framework

• What is the basis for your dispositions?
• P-12 educators were asked for their input
  regarding dispositions for our teacher education
  candidates as we reviewed our CF for impending
  NCATE visit.
• What is your vision of what a professional
  educator should look like as relates to attitude,
  values, beliefs, behaviors?

38
Standards, Standards, Standards

• Which professional organizations guide our
  dispositions?
• INTASC Standards (basis for State Knowledges,
  skills, and dispositions)
• NEA Code of Ethics
• Specialized Professional Association Codes of
  Ethics (ACA, ISSLC, CEC)

39
Data used that relates to Dispositions

• Survey of 400 K-12 teachers in rural Northwest
  Tennessee revealed that 67 had taught students
  with limited English proficiency. However, 85
  indicated that an improved understanding of
  international cultures would enhance their
  effectiveness in the classroom. How does this
  relate to dispositions?

40
Diversity, cultural awareness and community
service learning

• A field experience developed to increase teacher
  education candidate awareness of diversity,
  cultural awareness and community service learning
  was the result of the above data. You will see
  how this fits into our outcome for dispositions
  related to the CF and the list of dispositions we
  are developing in candidates

41
Conceptual Framework

• The theme Educators as Facilitators of Learning
  is a modification of the original theme,
  representing the Units belief that teachers do
  more than deliver content. The purpose of the
  Teacher Education Program (TEP) at UTM is to
  provide education professionals with the
  knowledge, skills, and dispositions necessary to
  become effective facilitators of learning.

42
Outcome that relates to Dispositions

• Professional and Ethical Behavior the candidate
  will be able to articulate their basic values and
  beliefs, value professional interactions,
  demonstrate self-respect and respect for others
  and help students examine and understand the
  ethical dimensions of the content of study. The
  candidate will be expected to discuss such
  matters as intellectual freedom, equity,
  tolerance, and due process. They will also know
  the professional codes of ethics in their
  specialty and the consequences of violating those
  propositions. Professionally, the candidate is
  expected to participate in collegial activities
  designed to make the entire school a productive
  learning environment while maintaining some type
  of activity in professional organizations.

43
Our Dispositions at UTM

• Interacts positively with peers, faculty, and
  P-12 personnel.
• Demonstrates self-respect and respect for others
• Accepts constructive criticism and is willing to
  modify ones behavior
• Assumes responsibility when appropriate
• Attempts to solve problems independently and in a
  fair-minded manner
• Exhibits interest in the learner and enthusiasm
  for the learning process
• Behaves in a professional manner and in
  accordance with TEP guidelines

44
A Little Quiz

• How do our dispositions correlate with Educators
  as Facilitators of Learning?

45
Observable Behaviors

• Lets look at some examples of observable
  behaviors and how we assess them.
• How do our candidates learn about the expected
  dispositions?
• TCED 301 Introduction to Teacher Education,
  foundations, portfolio process
• Website tutorial
• http//www.utm.edu/acoleman/TCEDProgramCF.htm

46
Admissions to TEP

• Where do dispositions fit into the process of
  admission to TEP?
• Candidates are interviewed by a team of unit
  faculty members and faculty or administrators for
  P-12 schools.

47
TEP Interview

• Likert-type scale that has 4 indicators from 1
  (Weak) to 4(Strong) and a fifth indicator Not
  Observed
• C. Professional/Personal Attitudes
• Awareness of responsibilities which are a
  necessary part of the profession
• Displays an interest, enthusiasm, and commitment
  to teaching and continual professional growth
• Friendliness/manner/presentation of self
• Willingness to accept the responsibility to
  adhere to the highest ethical standards
• Displays maturity in judgment and ability to make
  realistic decisions
• Capacity for development

48
Gathering Disposition Assessment Data

• External Sources
• Candidates demonstrated routine behavior in
  field and clinical settings (e.g., responsible
  professional behavior regarding attendance,
  meeting deadlines, working as part of a team,
  response to supervision, etc.)
• Periodic use of disposition rating forms by P-12
  field and clinical teachers (e.g., at end of
  field experiences and student teaching)
• Alumni surveys can generate data about candidate
  preparation for work, program satisfaction, and
  curriculum relevancy
• Employer surveys can generate data about employer
  satisfaction levels with the abilities,skills,
  and dispositions of recent graduates

49
Dispositions Rubric

• 4 point scale
• Not exhibited
• Exhibited some of the time
• Exhibited most of the time
• Clearly exhibited

50
Dispositions Rubric

• Used in key classes
• Positive interaction w/peers, faculty and P-12
  personnel
• Demonstrates self respect respect for others
• Accepts constructive criticism willing to
  modify behavior
• Assumes responsibility
• Solves problems in a fair minded manner
• Exhibits interest enthusiasm in the learner
  learning
• Behaves professionally in accordance with TEP
  guidelines

51
Early Field Experience Evaluation

• Field Experience form used in Level I Field
  Experiences (Early)
• Likert-type scale
• 1. Below Expectations
• 2. Needs Improvement
• 3. Meets Expectations
• 4. Exceeds Expectations
• N/A Not applicable

52
C. Professional and Ethical Behavior

• Notice how even the titles for indicators relate
  back to CF
• What do the indicators remind you of?
• Interacts positively and appropriately with peers
  and P-12 school personnel
• Demonstrates self-respect and respect for others
• Accepts constructive criticism and is willing to
  modify ones behavior accordingly
• Assumes responsibility when appropriate
• Exhibits interest in the learner and enthusiasm
  for the learning process
• Conducts oneself in a professional manner and in
  accordance with Teacher Education Program (TEP)
  guidelines

53
Level II Field Experiences

• Form is used during midpoint evaluation
• Level II methods classes
• Likert-type scale same
• 1. Below expectations
• Needs Improvement
• Meets Expectations
• Exceeds Expectations
• N/A Not applicable

54
Professional and Ethical Behavior

• How should they be assessed differently in Level
  II? Or should they?
• Interacts positively and appropriately with peers
  and P-12 school personnel
• Demonstrates self-respect and respect for others
• Accepts constructive criticism and is willing to
  modify ones behavior accordingly
• Assumes responsibility when appropriate
• Exhibits interest in the learner and enthusiasm
  for the learning process
• Conducts oneself in a professional manner and in
  accordance with Teacher Education Program (TEP)
  guidelines

55
Teacher Education Continuous Evaluation

• This form is applicable to candidates seeking
  teacher licensure concerning their retention in
  the Teacher Education Program at UTM. Any
  instructor of a teacher education program course
  may file a Teacher Education Continuous
  Evaluation form if a candidates suitability to
  enter or continue in the program is questioned.
  This form will be used in such instances as when
  there is evidence of a candidates inability to
  perform the professional competencies of ethical
  conduct, professional attitudes and behaviors, or
  essential mental or physical functions of a
  teacher. Copies of this form signed by the
  instructor and preferably by the candidate must
  be submitted to the Office of Education-Services,
  which will distribute all appropriate copies.

56
Continuous Evaluation

• Course Enrolled Date
• Candidates Name
• ID Number Cumulative GPA
• Major Teacher Education Advisor
• Please state specifically the nature of the
  deficiency. Provide specific examples which will
  support the evaluation.
• Suggested Action/Remediation
• Signature, faculty Date
• Signature, Candidate Date
• The candidates signature indicates only that
  s/he has read the report, not that s/he agrees
  with it.

57
What happens next?

• How do we plan to use the Continuous Evaluation
  form?
• We are working on that 3 copies in the
  candidates file indicates need for hearing
  before a retention committee
• Who monitors this? Good question

58
Mid-session Student Teaching Progress Report

• Culminating Experience check for Dispositions
• Cooperating teacher assessment
• Checklist 5 points

59
Examples

• Relations with others
• Exceptionally well accepted
• Works well with others
• Gets along satisfactorily
• Has some difficulty working with others
• Works very poorly with others

60
More Examples

• Attitude/Application to Work
• Outstanding in enthusiasm
• Very interested and industrious
• Average in diligence and interest
• Somewhat indifferent
• Definitely not interested

61
Back to CF and Dispositions

• How does the mid-session report relate to the
  list of dispositions and the CF?
• The candidate is also ranked on attendance as
  regular or irregular and on punctuality as on
  time or tardy.

62
Student Teaching Comprehensive Assessment
Performance Observation Evaluation

• By University Supervisor
• Based on Tennessee Framework for Evaluation and
  Professional Growth
• Scale is
• 4. Exceeds Expectations
• 3. Meets Expectations
• 2. Requires Improvement
• 1. Unacceptable

63
Professional Growth

• Collaborates with colleagues and appropriate
  others
• Engages in professional development
• Performs professional responsibilities
  efficiently
• Profits from evaluative feedback

64
Professional Responsibilities

• University supervisor also uses the four point
  scale to assess this statement
• Candidates performance reflects disposition
  expected of professional educator

65
What about follow-up of candidates and
dispositions?

• Teacher Warranty UT-Martin has with area school
  systems signed agreement
• We guarantee the quality of teachers who complete
  our program
• One of the statements we make is that all
  candidates completing an approved teacher
  education program
• Can demonstrate success in accommodating students
  from diverse cultural, ethnic, international, and
  socioeconomic groups.
• How does this relate to dispositions?

66
Using Dispositions Data

• Program faculty can monitor candidate growth and
  development throughout the program
• Self-assessment by candidates at key points in
  their program (e.g., At admission, where do I
  stand with respect to these expected disposition
  standards?)
• Verify change or lack of change in dispositions
  demonstrated by candidates engaged in documented
  efforts to improve their performance
• Program evaluation and improvement

67
Teacher Education Interview Summary

• Example of using data
• University of Tennessee At Martin
• Oral Communication and Dispositions Interview
  Evaluations
• Sem/year Admitted Admitted with Conditons
  Rejected Total
• Fall 00 78 3 0
  81
• Spring 01 88 0 0
  88
• Fall 01 87 3 0
  90
• Spring 02 84 3 0
  87
• Fall 02 108 0 0
  108
• Spring 03 83 5 1
  89

68
Summary and Questions

• Can you see some difficulty as I talked about our
  dispositions and the rating instruments, etc. as
  far as using the data for program evaluation and
  improvement?
• How could that be changed?
• In closing, thank you for attending our session
  and isnt this a wonderful time to be in Teacher
  Education?