PRR 389

1
PRR 389

• Module 1 Concepts, Philosophy, Process
• Module 2 Goals, Objectives, Perf. Measures
• Module 3 Data Gathering Analysis
• Module 4 Forecasts Impacts
• Module 5 Communication/Reporting

2
Planning/Research/Evaluation

• Concepts Definitions Philosophy
• Purposes Why plan, evaluate?
• Processes Systematic Steps
• Types of Plans, Evaluations, Research

3
Definitions

• Planning preparation for decision making
• Evaluation process of judging the merit or
  worth of something
• Research application of scientific methods to
  answer questions
• Pairs Evaluation research, research planning,
  planning evaluation

4
Related terms

• Design
• Policy
• Management
• Data gathering
• Decision making
• Analysis

5
Comprehensive planning

• Systems of facilities and programs
• Multiple providers and stakeholders

6
Why Plan?

• Prepare for/anticipate future
• Establish goals, directions
• Coordinate various activities programs
• Balance needs/wants with available resources
• Establish systematic steps to achieve goals
• Requirements for funding

7
Messes

• 1. Not well defined
• 2. Multiple ill-defined objectives
• 3. Multiple decision makers
• 4. Incomplete information.
• 5. Dynamic.
• 6. Multiple stakeholders
• 7. Uncertainty

8
Characteristics of Recreation Tourism

• Nature of R/T
• Diversity of activities
• Hard to define
• Freedom of choice
• A frill, worthy of serious study?
• Supply Characteristics
• Services
• Fragmented providers/industries
• Public-private non-profit mix
• Role of consumer in providing experience
• Externalities, public goods, common property
  resources

9
Characteristics (cont)

• Demand Characteristics
• Demand for what?
• Dispersed use
• Substitution not understood
• Multi-purpose trips
• Peaking of demand
• Prices/markets
• Methodological
• Fuzzy concepts
• Weak Measures/measurements
• Planning- management-research gaps
• Interdisciplinary, young as sciences go

10
Trends

• Government-private roles, privatization
• Federal, state, local roles
• Partnerships, cooperation vs competition
• Increasing accountability
• Convergence of public private/business
  approaches
• Scale, pricing
• Shifting markets, activities
• Linkages with health, crime, aging, economic
  dev., diversity other societal issues

11
Incremental vs Comprehensive Planning

• Lindblom, science of muddling through planning
  is ..
• Incremental
• Remedial
• Serial
• Exploratory
• Fragmented
• Satisficing
• Iterative

12
Patton Sawicki Tips for planning/policy
analysis

• Focus quickly on central decision criteria
• Avoid tool box approach
• Learn to deal with uncertainty
• Say it with numbers
• Make analysis simple and transparent
• Check the facts
• Learn to advocate positions of others
• Give client analysis, not decisions
• Push boundaries of policy envelope
• No such thing as correct, rational, complete,
  comprehensive analysis

13
Steps in Planning Process

• Define the system
• Establish goals and objectives
• Gather information
• Generate Alternatives
• Evaluate Alternatives
• Select Implement
• Monitor
• Iterate

14
Types of plans

• By geographic scope/extent
• Building, Facility, Site Master Plan (Design)
• Community, Regional, State, National, System of
  Parks/Programs
• By Activity/Function
• Housing, transportation, recreation tourism, land
  use, open space, physical plan, comprehensive
  plan
• Program, maintenance, safety, security, risk
  management, budget personnel, marketing
• By Time
• Feasibility, Short Long Range, Strategic Plan,
  annual performance plan, EIS.

15
Specific Types

• Community recreation plan
• SCORP
• Environmental impact statement
• Business Plan
• Feasibility study
• GMP, LMP,
• Community or Regional tourism plan

16
Why Evaluate - the Academic list

• To assess merits of alternative programs
• To discover whether how well objectives are
  being fulfilled
• To determine the reasons for successes
  failures.
• To uncover the principles underlying a successful
  program.
• To refine, revise, update or track a program

17
Why Evaluate- "The Real List"

• 1. Because we are required to
• 2. To make better decisions.
• 3. To learn from experience
• 4. To justify programs
• 5. To kill programs

18

• Research Evaluation

Evaluation Research
19
Purposes of Research

• Answer management questions - applied research
• For sake of knowing - pure or basic research
• Answer research questions - methodological
• Develop/test management alternatives -
  developmental research
• Assess worth or merit of programs - evaluation
  research

20
Research Purposes - II

• Exploratory
• Descriptive
• Explanatory
• Predictive

21
Ways of Knowing or Establishing belief or truth

• Tradition
• Authority
• Repetition/Tenacity
• Science

22
Science

• Body of Knowledge
• systematic
• abstract
• general
• parsimonious

• Method of Inquiry
• logical
• induction
• deduction
• self-corrective
• empirical

23
Scientific Management

• Application of scientific principles to
  management and decision making
• systematic information gathering
• empirical, objective, self-corrective

24
Process -- Steps

• Research
• define problem
• objectives/hypotheses
• literature review
• research methods
• gather data/analysis
• conclusions
• further research

• Evaluation
• describe program
• evaluation criteria
• program scoping
• evaluation methods
• gather data/analysis
• conclusions
• modify program?

• Planning
• define system
• specify goals/objectives
• gather info
• generate alternatives
• evaluate alternatives
• choose implement
• monitor, evaluate
• Iterate

25
Types of Evaluationby Program Stage

• formative (conceptualization/design)
• process (implementation)
• summative (outcomes, impacts, efficiency)

26
Types - By Approach

• Standards
• norm-based
• criterion-referenced
• Goals and objectives
• Impacts or effects

27
Types- by method

• Qualitative - quantitative
• Secondary data - Primary
• Survey - Experiment
• Internal vs external
• Who - Peers, participants, superiors,
  subordinates, scientists

28
Evaluation Criteria

• Effort - qnty and qlty of inputs
• Performance - qnty and qlty of outputs
• Adequacy - meet needs?
• Efficiency - benefits/costs
• Equity - distributional issues, fairness

29
Process Evaluation

• Identifies how and why program works
• attributes
• recipients
• conditions
• effects
• single or multiple
• intended or side effects
• timing duration, long/short term
• cognitive, affective or behavioral

30
Types by program area/subarea

• Emphasis Parks, Tourism, Interp, Program Mgmt,
  Commercial Rec, ...
• Program lands, facilities, personnel, budget,
  marketing, PR, maintenance, design, policies,
  plans

31
Types of Research

• Basic - Applied
• Exploratory, Descriptive, Explanatory, Predictive
• In-house / out- house
• by discipline - sociological, psych, economics
• by topic - boating, wilderness, legal, tourism,
  ...
• by method - survey, expmt, ...

32
Examples of Evaluation Studies

• Community needs assessment
• Feasibility study, SWOT analysis
• Program Audits, Market audit
• Cost/benefit, Cost effectiveness analysis
• Impact assessment
• PPBS, MBO, GPRA, CAPRA, LAC, ...
• Peer review
• Importance-performance analysis

33
AEA Principles for Evaluators

• Systematic Inquiry
• Competence
• Integrity/Honesty
• Respect for people
• Responsibility for public welfare

34
Scriven- main points

• Merit vs Worth
• Grading vs Ranking
• Facts and values
• Big six program, personnel,performance, policy,
  proposal, product
• Need vs market assessment
• Holistic vs analytic
• Indicators vs true measures of merit
• Goal Achievement Eval vs Goal Free Evaluation

35
Goal free evaluation (GFE)

• Dont worry about goals look at all effects
• Problems with goal achievement evaluation
• Whose goals, how measured?
• Unintended/side-effects interaction effects
• Values left behind
• Costs, alternatives/competing programs
• Generalizability
• Synthesis