Michigan Quality Leadership Award

1
Michigan Quality Leadership Award New Examiner
Training 2014
2
Agenda

• Welcome and Introductions
• Begin with the End in Mind
• Code of Ethical Conduct
• Key Factors
• Six-Step Process
• Evaluating Items
• Preparing for March 12

3
Clever Introductions

• Find a partner to introduce to the class.
• Share name, affiliation, Baldrige-like
  experience.
• Share another item of your choice, or complete
  this sentence

My zodiac sign is ______ and it will help me be
an awesome Examiner because
4
Examiner Roles and Responsibilities for Todays
Learning

• Take an active role in your own learning by
  asking questions, sharing information,
  viewpoints, and roles during the training
  program.
• Take an active role in helping others to increase
  knowledge and develop skills.

5
(No Transcript)
6
Award Process Cycle
Selected
7
Begin with the End in Mind

• Feedback Report
• Constructive, actionable feedback that helps
  applicant
• Identify and leverage Strengths
• Prioritize and address Opportunities for
  Improvement (OFIs)

8
Baldrige Criteria Framework A Systems
Perspective
9
Ensuring Examiner Ethical Behavior

• A key success factor for the Award program is
  promoting and ensuring ethical behavior.
• Code of Ethical Conduct
• Conflict of Interest

10
Code of Ethical Conduct

• Four Principles
• Protect the Integrity of the Award Process.
• Exhibit Professional Conduct at All Times.
• Protect the Promise of Confidentiality.
• Protect the Programs Intellectual Property.

11
Conflict of Interest Statement

• Upon notification of application, immediately
  review for conflicts of interest.
• Sign and email back the Conflict of Interest
  statement.
• Call Geri Markley, Michigan Quality Council, with
  all questions.

12
Exercise Code of Ethics

• In your small table groups
• Assign roles.
• Discuss your assigned ethics scenario.
• Prepare to share your consensus response.

13
Scenario 1 the answer is
Examiners should not exploit their selection to
the Board of Examiners. There are no secrets
for receiving the award and no guarantees for
receiving a site visit. Also, the program prefers
to understate the winning aspect of the
process it encourages a focus on continued
self-improvement and the sharing of best
practices to help organizations achieve
performance excellence. Examiners may not use
the MQC logo in any advertising/promotion (and
business cards may not include MQLA examiner
or the MQC logo).   Principles Protecting the
integrity of the award process, the programs
intellectual property, and the reputation of the
BOE.
14
Scenario 2 the answer is

• Any assistance with an organizations application
  would result in a conflict of interest for an
  examiner. The examiner would be considered an
  employee of the organization. If the organization
  reapplies, the examiner would not be permitted to
  evaluate the application.
•  Examiners advising or participating with an
  organization in preparing an award application
  shall not reveal or discuss that participation
  with other examiners during training or at any
  other time.
•  Principles Protecting the promise of
  confidentiality and the integrity of the award
  process.

15
Scenario 3 the answer is

• No. At no time during Independent Review or
  Consensus Review should an examiner contact the
  applicant. Examiners shall not communicate with
  the applicant organization or in any manner seek
  additional documentation, information, or
  clarification. This includes Internet searches or
  other external sources of information.
• Examiners shall safeguard the confidences of all
  parties involved in the judging or examination of
  current or former applicants.
•  Principles Protecting the promise of
  confidentiality and the integrity of the award
  process.

16
Scenario 4 the answer is

• No. Examiners must avoid disclosures that may in
  any way influence the award integrity or process,
  currently or in the future, and treat as
  confidential all information about the applicant.
  While this information has been publicly
  released, it is not part of the application, and,
  therefore, it is not relevant to the evaluation.
  While potentially interesting, this information
  should not be discussed and must not be
  considered in the evaluation.
• The program shares with the team leader any known
  information that could affect selection as a role
  model, and the team leader seeks clarification
  during the site visit.
• Principle protecting the integrity of the award
  process.

17
Scenario 5 the answer is

• Yes. Examiners may use the Internet and other
  sources to familiarize themselves with common or
  generic terminology. Examiners shall not
  communicate with the applicant organizations or
  in any manner seek additional documentation,
  information, or clarification specific to the
  applicant. This includes Internet searches or
  other external sources of information.
• Principle protecting the promise of
  confidentiality.

18
Master List of Key Factors

• Facts about the organization
• Indicates relevance and importance
• Not processes or evaluation of processes
• Mainly from the Organizational Profile

19
Examiners View Applicants through Two Lenses
Criteria and Key Factors
20
Examples of Key Factors

• Size
• a) 39 employees located in one building
• b) 73,000 employees located in twelve states
• Strategic Challenges
• a) Heavily regulated and intensely competitive
• b) Rapid expansion from acquiring many small
  competitors
• Products
• a) Highly technical and changes often
• b) Has not changed in 37 years

21
Key Factors

• How are key factors used?
• Why are they important?
• Case Study Key Factors

22
Exercise Key Factors

• In your small table groups
• Assign roles.
• Identify key factors for your assigned section of
  the Organization Profile.
• Prepare to share your key factors.

23
Six-Step Item Evaluation Process
24
Independent Review Steps

• Ground yourself in the criteria.
• Select the relevant 4-6 key factors for the item.
• Read the relevant section of the application.
• Draft around 6 strengths OFIs.
• Write two feedback-ready comments.
• One strength
• One OFI
• Determine the scoring range and score for the
  item.

25
Step 1 Ground yourself in the Criteria item.
26
Step 1 Ground Yourself in Criteria Requirements

• Read the Criteria item to understand the Item
  requirements.
• Look at 1.1a(1) Senior Leadership (p.7)
• How do SENIOR LEADERS set your organizations
  VISION and VALUES? How do SENIOR LEADERS DEPLOY
  the VISION and VALUES through your LEADERSHIP
  SYSTEM, to the WORKFORCE, to KEY suppliers and
  PARTNERS, and to CUSTOMERS and other
  STAKEHOLDERS, as appropriate? How do SENIOR
  LEADERS actions reflect a commitment to those
  VALUES?

27
Step 2 Select the relevant key factors for the
item.
28
Step 2 Select the Relevant Key Factors for the
Item

• Review master list of key factors determine the
  attributes of the organization that would
  influence its responses to the item requirements.
• Select the most relevant 4-6 key factors for that
  item.
• Example 1.1a(1) would likely have a key factor
  related to the organizations vision and values.

29
Step 3 Read the relevant section of the
application.
30
Step 3 Read the Relevant Section of the
Application

• Identify the processes or approach the applicant
  uses to meet item requirements.
• Flag, mark up and/or take notes as needed.
• Item 1.1a(1) may describe an approach used to set
  vision and values, how they are deployed, etc.

31
Step 4 Draft around six strengths and
opportunities for improvement.
32
Step 4 Draft Around Six Strengths and OFIs

• Select relevant key factors for each approach.
• Provide brief statement of approach.
• Provide evidence.
• Select appropriate evaluation factors (ADLI).
• Provide reference to specific areas of the item
  (e.g., a(1), b(3)).

33
Step 5 Draft feedback-ready comments.
34
Step 5 Draft one each Feedback-Ready Strength
and OFI

• Use Comment Guidelines to help craft comments
  that capture your findings and analysis.
• Include these key elements
• Brief nugget
• An example or two
• Why this feedback is important

35
Step 6 Determine the scoring range and score for
the item.
36
Step 6 Determine Scoring Range and Score for
the Item

• Review balance of observations.
• Use scoring guidelines (pp. 32-33).
• Determine best fit scoring range.
• Determine score.

37
Analyzing a Process Item

• The purpose of process items is to permit
  diagnosis of your organizations most important
  processes
• the ones that contribute most
• to organizational performance improvement
• and to key outcomes or performance results.

38
Assessing Process Maturity

• Process the methods the applicant uses to
  address item requirements in Criteria categories
  1-6. Four factors are used to evaluate process
  maturity
• Approach
• Deployment
• Learning
• Integration

39
Approach

• Approach refers to
• Appropriateness of methods
• Effectiveness of methods
• Degree to which the approach is systematic
  (repeatable and based on reliable data and
  information)
• Degree to which the approach addresses basic,
  overall, or multiple requirements of the criteria

40
Criteria Item Format
41
Item 1.1 Approach (p. 7)

• a.(1) How do SENIOR LEADERS set your
  organizations VISION and VALUES? How do SENIOR
  LEADERS DEPLOY the VISION and VALUES through your
  LEADERSHIP SYSTEM, to the WORKFORCE, to KEY
  suppliers and PARTNERS, and to CUSTOMERS and
  other STAKEHOLDERS, as appropriate? How do
  SENIOR LEADERS actions reflect a commitment to
  those VALUES?

42
Deployment

• Deployment refers to the extent to which
• The approach is applied consistently
• The approach is used by all appropriate work
  units
• The approach is applied to all appropriate areas,
  customers, suppliers, partners, etc.
• Tip Key factors are important in assessing
  deployment.

43
Learning

• Learning refers to
• Refining the approach through cycles of
  evaluation and improvement
• Encouraging breakthrough change to the approach
  through innovation
• Sharing refinements and innovations with other
  work units and processes in the organization

44
Process Maturity - Learning

• Three dimensions of organizational learning
• Continuous improvement of existing processes
• Innovation leading to breakthrough changes
• Knowledge-sharing of such improvements and
  innovations

45
Integration

• Integration refers to the extent to which
• The approach is aligned with organizational
  needs.
• Measures, information, and improvement systems
  are complementary across processes and work
  units.
• Plans, processes, results, analyses, learning,
  and actions are harmonized across processes and
  work units to support organization-wide goals.

46
Process Maturity - Integration
47
Alignment and Integration
48
Capturing Observations
User Type Double Description Evidence ADLI Item Ref
BR Strength The applicant has an integrated and systematic approach for assessing its workforce capability and capacity needs, including the skills, competencies, certifications, and staffing levels. - HR Capability Assessment is conducted annually- The quarterly Capacity Study ensures a match between position requirements, required competencies, and qualified EOs for each position.- The capabilities are a specific element of the strategic (long-term) and action (short-term) plans resulting from the SPP- action (short-term) plans resulting from the SPP (Figure 2.1-1) A,D,I a(1)
49
How do I evaluate Process Items?
By using the Six-Step Process!!

• Ground yourself in the criteria item.
• Select the relevant key factors for the item.
• Read the relevant section of the application.
• Draft around 6 strengths and opportunities for
  improvement (OFIs).
• Draft feedback-ready comments (Two for IR).
• Determine the scoring range and the score for the
  item.

Independent Review
50
Exercise Evaluate a Process Item (Steps 1 - 4)

• In your small table groups
• Assign roles.
• Complete Steps 1 - 4.
• Ground yourselves in the Criteria by summarizing
• Identify 4 6 most important Key Factors
• Review the applicants response
• Identify around six observations with evidence
• Prepare to report out.

51
Well-Written Comments

• Why are comments important?
• Feedback is the basis for improvement.
• They demonstrate understanding of the applicant
  and its key factors.
• What are characteristics of well-written
  comments?
• Concise
• Precise
• Show insight
• Actionable

52
Applicant Requirements

• A concise opening sentence that expresses a
  single thoughtthe nugget (or essence, or main
  point) of the comment
• One or a few examples (not every example)
• The relevance or importance of the nugget to the
  applicant (think key factors)

53
Sample Comment
6.2b(1) The organization ensures that work
processes meet key requirements day-to-day by
utilizing technology for real-time monitoring of
patient length of stay, test results, turnaround
time, and bed availability. Other methods of
evaluation include day-to-day rounding by
administration, audits, and scorecard reporting.
This approach supports the core competency of
operational excellence and helps address the
strategic challenge of cost control.
54
Comments Should Not

• Be prescriptive
• Parrot the application
• Go beyond the criteria

55
Exercise Write a Comment (Step 5)

• In your small table groups
• Assign roles.
• Review observations.
• Select one and write a feedback-ready comment.
• Prepare to report out.

56
Scoring System

• Each Item is assigned a point value.
• Scoring guidelines describe percentage range of
  the points based on evaluation factors.
• The Examiner selects a scoring range that is
  most descriptive of the applicants achievement
  level in that item.
• The Examiner selects the best score within that
  range.

57
Exercise Score (Step 6)

• In your small table groups
• Assign roles.
• Review observations and comment.
• Discuss and agree on scoring range.
• Agree on score.
• Prepare to report out.

58
How do I evaluate Results Items?
By using the Six-Step Process!!

• Ground yourself in the criteria item.
• Select the relevant key factors for the item.
• Read the relevant section of the application.
• Draft around 6 strengths and opportunities for
  improvement (OFIs).
• Draft feedback ready comments (Two for IR).
• Determine the scoring range and the score for the
  item.

Independent Review
59
What Should You Be Looking For?

• Lets See LeTCI
• Levels
• Trends
• Comparisons
• Integration

60
Importance

• What results are required?
• How do you know what results are important to
  this applicant?
• What comparisons are appropriate?
• What segmentation should you expect?

61
Appropriate Comparisons

• Industry mean
• Best competitor
• Best-in-class performance

62
Rate of Improvement Comparisons
63
Meaningful, Well-Written Results Observations

• Describe
• How you grouped various Key Results
• Whether you think Levels and Trends shown
  represent favorable or unfavorable performance
  and why
• Whether you think the Comparisons provided are
  appropriate and represent favorable or
  unfavorable performance and why
• Whether you think the results demonstrate
  Alignment and Linkage to the applicants key
  factors and key process requirements

64
Meaningful, Well-Written Results Observations

• What Missing Results (Gaps) you noted and why it
  matters to the applicant, including expected
  results based on Criteria, process, or other
  stakeholder requirements
• What Overall Item observations you have noted, if
  any.
• Tip Be sure to consider results embedded in the
  text as well as the charts and graphs provided.

65
Results OFI Comment
7.1a Comparisons to local competitors are limited
for customer outcomes, although there are
comparisons to top -decile performance levels
from national or state databases. Regularly
comparing performance to others in the
marketplace may help the organization find
opportunities to outperform competitors and gain
market share, a key strategic challenge.
66
Exercise Evaluate a Results Item (Steps 16)

• In your small table groups
• Assign roles.
• Complete Steps 1-6 for assigned Item.
• Prepare to report out.

67
Preparing for March 12

• Prework using Baldrige Online Scoring System
  (BOSS)
• Demo

68
Prework

• Read Collin Technology Case Study.
• In BOSS
• Complete master list of key factors.
• Complete evaluation (Steps 1 6) for these
  items 1.1, 5.2, 6.2, 7.4.
• In Word
• Write a brief description of these items (format
  provided) 1.2, 2.1, 2.2, 3.1, 3.2, 4.1, 4.2,
  6.1, 7.1, 7.2, 7.5
• 5.1 and 7.3 were provided as examples.

69
Prework Tips

• Dont procrastinate and expect to do it all at
  once.
• Use notes from today to help you complete your
  item evaluations.
• Contact Geri to take a look at your prework
  progress and provide feedback or offer tips.
• Dont spend more than 20 hours.
• If youre at 16 hours and think youll go over,
  contact Geri to help brainstorm ways to move
  forward.

70
Resources

• Participant Notebook
• Online Baldrige presentations (prework)
• Welcome to BOSS Powerpoint
• Conference Call Feb 7 at 1000
• Examiner Preparation Team Mentors
• Email Geri
• BOSS monitoring and follow-up emails by Geri

71
Questions?