Strategies for implementing successful IL action plans

1
Strategies for implementing successful IL action
plans

• Barbie E. Keiser
• UNESCO IL TTT Workshop
• Wuhan University
• October 2008

2
What well cover during this session

• Strategic planning for IL
• Environmental scanning
• SWOT/TOWS analysis
• Critical Success Factors (CSFs)
• The team
• Needs assessment
• Methods for conducting

3
And what we wont

• Organizational structure and programmes within
  UNESCO
• Education
• Communication information (IFAP)
• Bangkok (http//www.unescobkk.org/index.php?id662
  )
• Differentiating among education, training, and
  guidance
• ACRL Information Literacy IQ (Institutional
  Quotient) Test (http//www.ala.org/ala/acrl/acrlis
  sues/acrlinfolit/professactivity/iil/immersion/inf
  olitiqtest.cfm)

4
What do we know about strategic planning?

• Concerns the relationship of an organization to
  its environment
• Involves wide-range scanning of external and
  environmental factors
• Flexible, dynamic and continually reworked plans
  maximize results
• Participative
• Shorter and longer-range plans are interwoven
  into a continuous strategy
• Forward-looking future-oriented

• Iterative, ongoing effort
• Proactive seek opportunities
• Bottom-up decision process
• Environment considered ever-changing and dynamic
• Integrated focus
• Requires creativity to deal with new
  opportunities and choices
• Incentives given for overall performance of the
  organization

5
Key planning issues

• Review past performance
• Understand reasons for past failures
• Identify opportunities
• Determine client/customer and learner preferences
• Understand the impact of IL training on existing
  operations and staff function
• Marketing

6
Potential planning pitfalls

• Inability to get management and/or staff involved
• Lack of clear objectives
• Not relating IL goals and objectives directly to
  those of our organization/ institution
• Other?

7
What strategies can we use to assure that we do
not fall into these traps?

• Establish strong partnerships (Jamaica)
• Early, strong, consistent, and growing
• Be careful in selecting your champion (Quebec)
• Listen
• Be responsive
• Learn how to say no
• Appreciate the art of persuasion
• Tell stories (Quebec)
• Storytelling trumps statistics
• Demonstrate value to all stakeholder groups
  (WIIFM and ROI)
• Share the information expertise of your staff
  with other knowledge workers
• Added benefits?
• Employ the vocabularies of target stakeholder
  groups (i.e., no library jargon)
• Changing the message and the focus

• Demonstrate the need
• Place IL skills in context
• Identify where IL skills are already being
  taught/in use
• An added bonus A focus for our advocacy efforts
• New product(s) to market
• Reinvented service
• A case for proactivity in a way we havent seen
  before
• Opportunity to market

Sources Caroline Stern
8
Where are we in the strategic planning process
for IL?

• Perform an environmental scan
• Conduct a SWOT/TOWS analysis
• Initial assumptions (and their bases in fact)
• Never assume!
• Extant data (collected and reviewed)
• Identify Critical Success Factors (CSFs)
• Indicators Measures
• Develop vision, mission, and values statements
• Envisioning your IL programme

9
Environmental scanning

• Detects trends and events important to the
  project (IL training programmes)
• Provides early warning of changing external
  conditions
• Defines potential threats, opportunities, changes
  implied by trends and events
• Promotes a future orientation in the thinking of
  stakeholders
• Enables decision-makers to understand current
  (and potential) changes to determine
  organizational strategies
• What are the triggers in your institutions/
  organizations/communities (i.e., indicators that
  IL training is needed that will resonate with
  your community)?

10
Performing a comprehensive environmental scan

• IL models and standards
• Methods? (MyBookmarks)
• Your institution/organization/community
• Methods?
• Previous IL (and other training) efforts
• Understanding reasons for success/failure
• What kind of extant data do you collect (and
  review)?

11
IL models and standards guidelines

• Big6 Information Problem-Solving Process
  (http//www.big6.com/what-is-the-big6)
• Task definition
• Information seeking strategies
• Location and access
• Use of information
• Synthesis
• Evaluation
• AASL (http//www.ala.org/ala/aasl/aaslproftools/in
  formationpower/InformationLiteracyStandards_final.
  pdf) and CASL (http//www.cla.ca/casl/literacyneed
  s.html)
• ACRL (higher education) guidelines (competencies)
  - http//www.ala.org/ala/acrl/acrlstandards/standa
  rdsguidelines.cfm, toolkit http//www.ala.org/ala/
  acrl/acrlissues/acrlinfolit/infolitstandards/stand
  ardstoolkit.cfm and http//www.ala.org/ala/acrl/ac
  rlstandards/informationliteracycompetency.cfm
• Five standards
• Performance indicators (and outcomes for each)
• Seven Pillars (http//www.sconul.ac.uk/groups/info
  rmation_literacy/sp/sp/model.html)
• Information Inquiry, Problem-Solving and Research
  Process

• Country models
• National Information Literacy Framework
  (Scotland)
• Information Literacy Framework for Schools (Hong
  Kong)
• Australian and New Zealand IL Framework
  Principles, Standards, and Practice (ANZIL)
• US School Library Media Center Questionnaire
  (http//nces.ed.gov/surveys/sass/pdf/0304/sass_ls1
  a.pdf)

12
Conducting a SWOT analysis

• Useful when you need to understand your own
  competitive advantages
• Provides information helpful in matching your
  organizations resources and capabilities to the
  environment in which it operates
• Scan of the internal and external environment,
  beginning externally
• Environmental factors internal to the
  institution/organization/library/community
  Strengths or Weaknesses
• Environmental factors external to the
  institution/organization/library/community
  Opportunities or Threats
• Sometimes can be too inward

13
SWOT Analysis
14
TOWS

• An extension of the SWOT analysis
• Analyze the external environment (threats and
  opportunities) and your internal environment
  (weaknesses and strengths) to help you think
  about the strategy of your organization
• Useful for marketing campaigns
• Threats and opportunities
• External environmental factors over which you do
  not have control (changing demographics)
• Weaknesses and strengths
• Internal factors (poor location bad reputation)

15
TOWS Strategic Alternatives Matrix
16
Balanced Business Scorecard

• Perspectives
• Financial/ stakeholder
• Customer/service
• Internal/process
• Innovation/learning

• Goals
• Measures

Source Dr. Sheila Corrall, University of
Sheffield
17
What are the CSFs for your IL training programme?
18
Key questions

• Is your community ready?
• What to do if it is not
• Is your institution/ organizations culture a
  barrier?
• How to deal with that
• Where did the idea for IL training originate?
• You or others

• ACRL
• What do you want learners to be able to do?
• What do learners need to know in order to do this
  well?
• What type of instruction will best enable the
  learning?
• How will the student demonstrate the learning?
• How will you know that the learner has learned?

Source Ruth Pagell, SMU
19
Scope of your IL training

• Comprehensive or narrow(er)?
• Pilot project approach
• One subject, that can then be extended to others
• One tool, that can then be extended to similar
  tools
• Our responsibility is to help learners make those
  connections!
• Focus on what the learner needs to know, teaching
  them how to ask the right question (if they want
  to get the right answer)
• Begin by asking what the learner already knows
  about the subject, and then.

20
Questions for the learner (Framework for IL
Scotland)

• Whats the most likely place you will find the
  answer? Was this choice the best?
• What words can you use to search effectively to
  improve on your existing knowledge? Was the
  strategy the best?
• How do you know when youre finished? Did
  learner assess correctly?
• Have you learned something new?
• Who else should know this (and how should this be
  shared - ethically)?
• How will you apply this now?
• What have you learned from this experience that
  you can apply elsewhere?

21
Overview of the process

• Goals and objectives
• Strategies and tactics
• Target population
• Type of training
• Alternatives
• Project planning

22
Team approach

• Who should be included on the team?
• Who is responsible?
• When is it due?
• Consider whats needed to gain institutional
  commitment and stakeholder buy-in

23
Needs assessment Knowing your market

• Definition a planned, systematic approach to
  determining the information needs of each
  distinct customer group
• Purpose Help you develop training targeted
  specifically to each group and need

• Diverse set of learners, each with distinct set
  of needs
• Understand your targets
• Why they need to improve their IL skills (direct
  impact)
• What would persuade them that IL is important to
  their success
• Identify groups with related needs

24
Methods

• At-the-elbow
• Usability-lite testing
• Easter egg hunt
• Interviews
• Surveys
• Focus Groups

• Questions
• Analysis

25
Interviewing techniques

• Know the lingo enough so that you can ask
  sensible questions and understand the answers
• Decide in advance what each person is likely to
  be able to tell you
• What he/she would be interested in talking about
• Show respect for each individuals expertise and
  explain your objectives
• Introduce yourself and why you are calling (in
  general terms)
• Do not lie! You are not a student doing a
  research project.
• You are not conducting a survey the objective is
  to engage them in conversation
• Allow enough time to do the right job 2-3 hours
  to find the right source, exchange voicemails,
  conduct the interview, and write the summary
• If you are covering a field on a continuing
  basis, make interviews part of your routine
• Listen!
• Benefits of a two-on-one approach

26
The interview

• Begin with the general and work your way towards
  the items in which you are really interested
• Ask the least sensitive questions first, and when
  they are comfortable, proceed towards the more
  sensitive. Ask the most sensitive questions last.
• Develop models or hypotheses that address key
  questions
• Focus on whats most important
• Dont take up too much precious time (lt 30
  minutes)
• Write a complete summary of the conversation.
  Capture both facts and nuances.
• Do not tape the interview. (Illegal in many
  jurisdictions, unless you notify the party, and
  that will put them on their guard and theyll
  likely not want to continue, or at least not be
  as forthcoming as youd like.)
• Be an attentive listener
• Get referrals Ask who else you should contact
  about the subject next
• Mention who suggested that you call that
  individual

27
Characteristics of .

• Surveys
• Success based on the length, knowledge of
  questioner (if telephone survey is used), and
  list
• Important to pretest items/flow
• Introduction to participate/
• Intro to survey/Follow-up with non-respondents
• Anonymity/confidentiality
• Response rate
• Online
• Incentives to complete
• Time
• Timing of release
• Time required to complete
• Sampling
• Significance

• Focus groups
• Less formal way of soliciting consumer feedback
  on products and services
• 6-10 people experienced facilitator
• Need for ground rules
• 1.5 hours
• Record the session
• Disadvantage small sample, so composition is key

28
When to use surveys and focus groups for
information gathering

• Use surveys when
• You need quantitative estimates/confidence
  intervals
• You have a clear idea as to the questions you
  want to ask (and how)
• You have the time to develop and test the
  instrument, and analyze and present the results

• Use focus group when
• You need insights quantitative estimates are
  less important
• Statistical analysis is not a necessity
• You want flexibility in pursuing issues
  discovered during your inquiry
• Limited in terms of time and/or budget

29
Finding participants for surveys and focus groups

• And how those methods influence the validity of
  your analysis

30
Tips for conducting successful interviews and
focus groups

• Let participants talk, but facilitate the
  discussion
• May have a set of questions but do not force a
  slavish go-through
• Assure complete confidentiality notes are
  aggregated, no names ever given out
• If need to prime the pump, refer to
  observations (we noticed ) and ask for
  comments
• Be aware of interpersonal dynamics and politics
• Recognize that participants may not want to look
  bad may tailor comments to what they think is
  correct
• Validate Interesting, you are not the first to
  say so
• Use others-find technique (you too?)

31
Tips for conducting effective surveys

• Short - Fast Easy did I mention short!
• Clear, unambiguous (in terms of questions posed)
• Ask only one question at a time
• Logical flow of survey sections
• Ranking of personal priorities (What means more
  to you?)
• Minimize the number of open-ended questions
• http//www.qsrinternational.com/
• Do you agree with these statements made by your
  peers?
• Lead with interesting questions, enticing people
  to respond
• Include quick demographic questions at the end to
  aid in analysis

32
Questions to pose concerning facts

• Why should I believe it?
• Does the claim need evidence to support it?
• If there is evidence provided, how good is the
  evidence?
• Other plausible interpretations?
• What reasonable alternative conclusions are
  possible?

33
What well cover during this session

• Moving from goals and objectives to who will do
  what (and when)
• Plus a bit on.
• How the training should be delivered
• Options available
• Monitoring performance and measuring success
• Student assessments
• Overall programme, including cost/benefit (ROI)
  and value analysis

34
The overall goal is information fluency and
developing the lifelong learner
1 Establish specific objectives for your IL
training program
General/Basic, stand-alone
Integrated, subject-specific
IMPROVE!
2 Develop IL training strategies
4 Evaluate IL training accomplishments
BI, ITC, and more
One-time or semester?
Required or optional?
Grade level undergrad/grad/ researcher/ worker
Classroom/Online
3 Implement IL training programme(s)
Credit/other reward/incentive?
35
The Plan

• Context
• Goals
• Objectives
• Positioning statement

• Key message(s)
• Target audiences
• Strategies
• Evaluation measures

36
1a. Establish objectives

• What gaps must be addressed?
• Identify required and desired proficiencies
• Identify deficiencies (and their causes)
• Identify non-training (e.g., availability
  equipment) and training (e.g., skills of
  trainers) issues
• What innovative approaches can be used?
• Benefits of blended learning
• Social networks for viral spreading knowledge
  gained
• Web 2.0 to reinforce over time

• What (specific) competencies must your audience
  possess?
• Focus on the ends, not the means
• From four perspectives
• Audience(s)/learners
• Competence
• Condition under which performance will be
  observed
• Criteria for success

37
Worksheet 1
38
The Plan

• Positioning statement How do you want the
  project to be perceived?

• Key message(s) What is the most important
  message that you wish to deliver?

39
Creating priorities within competencies sought
(ISD)

• Based on the importance of knowing what your
  target learners need to DO
• How frequently is the task performed?
• How critical is the task to performance?
• How difficult or complex is the task?
• If a subset of collective tasks, what is the
  relationship among tasks?
• To what extent will training for this task be
  encountered elsewhere? Ability to apply knowledge
• What prerequisite skills, knowledge, and
  abilities are required to perform the task?
• What is the current/desired criteria for
  acceptable performance?
• What behaviors distinguish good performers from
  poor?
• What behaviors are critical to the performance of
  the task?

40
Non-training and training issues
41
People learn differently!

• Doers
• Thinkers
• Feelers

• Seeing is believing vs. auditory (Wharton study)
• Importance of reinforcement
• Storyboarding, scenario building, case
  studies/examples

42
1b. Establish objectives

• How does each contribute to the overall strategic
  goal for information fluency?
• What approach(es) will you take?
• What has been successful for you in the past (and
  why)?
• What specific results (outcomes) must be
  accomplished so that you can get closer to your
  goal of information fluency?
• How those results will be achieved is explained
  in Step 2
• How will you market this effort?

43
Worksheet 2
44
2a. Develop IL training strategies

• What training (content) could address IL
  competence gaps?
• What format should that training take?
• How do you make those decisions?
• Staff competencies and time available
• Generate alternative training strategies for
  addressing (specific) IL gaps
• List all trainings considered/selected (and
  rationale)
• What innovative approaches can be used?

45
Is classroom training appropriate?
Tool MS Visio
46
Training worksheet
47
2b. Develop IL training strategies

• What are the projected (life-cycle) costs for
  (developing and implementing) each type of
  training to be offered?
• Specific benefits anticipated, both tangible and
  non-tangible?
• What are the consequences to the organization and
  library strategic goals of not offering IL
  training?

48
3. Implement IL training programme(s)

• Write your IL training goals and make them known
• Collaboration, teamwork, marketing/pr
• Identify performance measures and indicators
  (outcomes and impact)
• How will you benchmark performance prior to
  taking the training (e.g., pre-testing)?
• Mechanisms for assessing IL post-training
• Immediate and longer-term
• Continuous improvement process
• Measuring self-sufficiency achievements
• Understanding Top Box scores
• Develop an action plan

49
Action plans

• Strategy
• Tactics
• Evaluation and control
• Results

50
Develop an action plan

• What will be done?
• Key tactics to support the strategies
• Identify specific tasks to be completed
• By whom?
• For whom?
• By when?
• Timelines for each objective
• What resources are required (including
  financial)?
• Who should know/be involved?
• Collaborators and stakeholders
• How will you market the effort?

51
Worksheet 3
52
Sample Gantt chart presentation
53
4. Evaluate IL training goal accomplishments (ISD)

• Did you achieve the training goal?
• How much did it cost?
• Did accomplishing your IL training goal help the
  organization/institution achieve larger goals?
• What modifications should be made to the plan,
  based on the evaluation findings?

54
Training costs

• Development costs (personnel and equipment)
• Direct implementation costs (e.g., training
  materials, instructor travel/per diem,
  facilities)
• Indirect implementation costs (overhead, GA)
• Compensation for participants
• Lost productivity or costs of backfilling
  positions during training
• Developer
• Instructor
• Faculty

55
Training benefits

• Time/resource savings
• Improved quality
• Error reduction
• Allow the learner to do something not possible
  before

56
Your business case

• Are the projected benefits (to the individual,
  library, faculty, school, organization,
  community) consistent with strategic performance
  goals?
• What are the consequences if IL training did not
  occur (or did not occur here)?
• Do the potential benefits outweigh the costs?
• What is the value added from closing IL
  competency gaps?

57
Best practices for implementing training
programmes

• Demonstrate results Performance measures should
  tell each target group how well its achieved its
  goals (individual, faculty, library, school)
• Limited to the vital few Measures should cover
  key performance dimensions Too much data may
  obscure rather than clarify (expensive)
• Link to departments Performance measures should
  be linked directly to offices responsible for
  making training work (library and faculty)

58
How can we assess learning and training goal
achievement?

• Pre- and post-testing
• Delayed post-tests
• Anonymity
• Interviews (in-person, phone)
• Survey
• Work samples/co-grading reports
• Existing monitoring and reporting mechanisms
  (extant data)
• Each data collecting method has advantages and
  disadvantages

59
Moving from teacher-centered to learning and
learner-centered training
Source Cox and Lindsay
60
Measuring targeted learning behaviors
Source Cox and Lindsay
61
Worksheet 4
62
Types and quality of assessments (ACRL)/Examples

• Quality
• Collaborative
• Multi-dimensional
• Holistic
• Assess the thinking process
• Include critical thinking elements
• Managed

• Types
• Formal
• Informal
• Traditional (test)
• Authentic (real life task)
• Integrated
• Knowledge/Content-based
• Formative
• Summative
• Self-Assessment (pre-and post)
• Peer
• Portfolio

63
Creating your assessments

• Create a list of KSAs What should learners know
  and be able to do?
• Identify standards that the learners need to meet
• Design some tasks that will illustrate whether
  learners have grasped concepts
• Determine what signifies good performance
• Develop rubrics to be used in grading and course
  redesign

64
Assessment models and samples

• TRAILS Tool for Real-time Assessment of
  Information Literacy Skills (http//www.trails-9.o
  rg/)
• http//www.paccd.cc.ca.us/library/ilhandbook/il_as
  sessform.htm
• http//jonathan.mueller.faculty.noctrl.edu/toolbox
  /
• http//school.discoveryeducation.com/schrockguide/
  assess.html
• iSkills from ETS
• Additional assessment resources
  (http//www2.acs.ncsu.edu/UPA/assmt/resource.htm)

65
Monitoring your performance as well

• Establish a tracking system to monitor both plan
  execution and impact
• For each scheduled milestone, compare the actual
  performance with the anticipated, and report
  results
• For all variances in scheduled performance

66
Ask the following

• How does the reported performance compare with
  the previous performance?
• Benchmark the starting line
• Is the performance/schedule variance likely to
  prevent goal achievement?
• Particularly when the stepped approach has been
  used
• Are external factors affecting performance?
  Which?
• Is the variance due to unrealistic expectations
  (from planning stage)? What adjustments should be
  made?
• What modifications should be made to the action
  plan?
• What performance information should be collected
  now?

67
Establishing the worth

• Of training
• Subjective method for establishing the worth of
  improved performance (as a result of IL training)
• Comparing costs and benefits by calculating total
  Return on Investment (ROI)
• Assessing results

68
A more detailed illustration of the process
69
What you should have at the end of this session

• Model worksheets for creating action plans
• Ideas to use as the starting points for your IL
  training efforts
• Resources to consult
• My bookmarks
• How you can share yours

70
Comments? Questions? Suggestions?

• Thank you!
• Barbie E. Keiser
• barbieelene_at_att.net
• bkeiser1_at_jhu.edu