Research Process Define Problem, Research Objectives

1
Research Process Define Problem, Research
Objectives

• HOW?
• Overall Method
• Survey
• Experiment
• Case Study
• Secondary Data

• What?
• Concepts
• Variables
• Measures

• Who?
• Population
• Sampling

Data Gathering Analysis Application
2
STEPS IN A SURVEY
1. Define problem and study objectives 2.
Identify information needs study
population(s) 3. Determine basic
design/approach - cross sectional vs
longitudinal - on-site vs household vs other -
self-admin. vs personal interview vs phone -
structured or unstructured questions 4.
Questionnaire design 5. Choose sample (frame,
size, sampling design) 6. Estimate time,
costs, manpower needs, etc.
3
Survey Implementation
7. Proposal Human subjects review 8.
Line up necessary resources 9. Pre-test
instruments and field procedures 10. Data
gathering and follow-up procedures 11. Coding,
cleaning and data processing 12. Analysis
preliminary, then final. 13. Communication and
presentation of results.
4
Definition Measurement
measurement is the beginning of science, until
you can measure something, your knowledge is
meager and unsatisfactory Lord Kelvin
Nominal/Conceptual Definition - define concept in
terms of other concepts, links concepts without
tying them to real world Operational definition -
equates definition with measurement, specify
procedures/operations to generate the concept.
5
Levels of Measurement
6
Examples of Measurement scales- Income
NOMINAL Middle income if income is between 30K
and 50K Not middle if less than 30K or more
than 50K. ORDINAL LOW is less than 30K MID
if between 30K and 50K HIGH if greater than
50K INTERVAL Income in dollars from Line 17 of
1997 tax return. RATIO Income in dollars
10,000.
7
Questionnaire Design
1. Preliminary Info Information needed Who are
subjects Method of communication 2. Question
Content 3. Question Wording 4. Response Format 5.
Question Sequencing/Layout
8
What Info?
Demographic, Socioeconomic, Physical Cognitive -
Knowledge beliefs Affective - attitudes,
feelings, preferences Behavioral - actions
9
Question Content
1. Is this question necessary? useful? 2. Are
several questions needed on this subject?
Avoid double barreled questions. 3. Do
respondents have information to answer the
question? Use filter questions to screen. 4.
Does question need to be more concrete, specific
and related to subject's personal experience? Is
a time referent provided? 5. Is question
sufficiently general? Do you want recent behavior
or "typical behavior"? 6. Do replies express
general attitudes or specific ones? 7. Is content
loaded or biased 8. Are subjects willing to
answer? 9. Can responses be compared with
existing information?
10
Wording
1. Will words be uniformly understood? Simple
language. Avoid technical phrases, jargon and
abbreviations. 2. Does question adequately
express the alternatives? 3. Is the question
misleading due to unstated assumption or unseen
implications. 4. Is wording biased, emotional, or
slanted? 5. Will wording be objectionable to
respondents? 6. Should you use more or less
personalized wording? 7. Ask in a more direct or
more indirect way?
11
Form of Response

• 1. Open or closed-ended
• 2. If closed,
• ordered or unordered
• number of categories,
• type of cue,
• forced or unforced choice
• 3. Response categories
• mutually exclusive
• exhaustive.

12
Sequencing layout
1. Will this question influence responses to
others? 2. Is question led up to in a natural
way? 3. Placement to create interest, improve
response rate. 4. Branching, skipping, and
transitions on questionnaires.
13
1. Simple fill in the blank. Obtaining a
straightforward number or other easily understood
response. How old are you? ___________
(years). In what county is your permanent
residence? _______________ (
county) How much money did you spend on this
trip? ________________
14
2. Open ended To avoid leading subject, to
obtain wide range of responses in subjects own
words, or when you dont know kinds of responses
to expect. What is your primary reason for
visiting the park today? ________________________
_______________.
15
3. Partially closed ended. List major response
categories while leaving room for others. Which
of the following community recreation facilities
do you most frequently use? (check one). ?
neighborhood parks/playgrounds ? swimming
pools ? community centers ? natural areas ?
tennis courts ? other (please specify)
___________________
16
4. Checklists Allow subjects to check multiple
responses. Categories exhaustive mutually
exclusive Which of the following winter
recreation activities have you participated in
during the past month? (check all that
apply) ? Cross-country skiing ? Downhill
skiing ? Snowmobiling ? Ice Skating ? Sledding or
Tobogganing
17
5. Likert Scales Versatile format for measuring
attitudes. Please check the box that best
represents your level of agreement or
disagreement with each of the following
statements about downhill skiing
Strongly Strongly Agree
Agree Neutral Disagree
Disagree Downhill skiing is... exciting ? ? ? ? ?
dangerous ? ? ? ? ? expensive ? ? ? ? ? Can
replace agree with importance satisfaction,
interest preference and other descriptors to
fit the attitude you wish to measure.
18
6. Rank Ordering To measure preferences or
priorities. Limit to short lists. Rank the
following states in terms of your interest as
possible travel destinations for a summer
vacation trip. (Place a 1 beside the state you
would most like to visit, place a 2 besides your
second choice, and a 3 beside your third
choice.) ______ Michigan ______ Wiscon
sin ______ Minnesota
19
7. Filter Question. To screen for eligibility
or knowledge prior to asking other questions.
Did you stay overnight on your most recent trip?
? NO ? YES
If Yes, How many nights did you spend away from
home? ________
To next question
20
8. Semantic Differential scale. Measure
perception or image of something using a set of
polar adjectives. For each of the
characteristics listed below, mark an X on the
line where you feel downhill skiing falls with
respect to that characteristic.
exciting ___ ___ ___ ___ ___ ___
dull expensive ___ ___ ___ ___ ___ ___
inexpensive safe ___ ___ ___ ___
___ ___ dangerous
(Could repeat with cross country ski and
snowmobiling and compare perceptions or Coke and
Pepsi.
21
AGE Operational Definition B Give the subject
a slip of paper with the following
question. ENTER YOUR AGE IN YEARS AT YOUR LAST
BIRTHDAY ______. The number written in the
blank is the person's age.
22
EXAMPLE 1. CONCEPTUAL DEFINITION - Length of
table is the distance from one end to the other.
OPERATIONAL DEFINITION - Place a yardstick
along one edge of the table, place additional
yardsticks end to end until one extends over the
other edge. Read the numeric marking for inches
on the final yardstick at the point where it is
exactly over the other edge of the table (call
this X inches). Count the number of yardsticks
you have used (call this N). Compute (N-1) 36
X. Repeat this process on the edge perpendicular
to this one. The length of the table in inches is
the larger of these two numbers.
23
EXAMPLE 2. CONCEPTUAL DEFINITION - Age- the
number of years that have passed since a person's
date of birth. OPERATIONAL DEFINITION - A.
Ask an individual for his name, place of birth
(county and state), and names of both parents. Go
to the county records office for the given county
and find the record for the person with the given
name. If there are more than one such records,
check for names of parents. Identify the date of
birth from this record. Subtract the year of
birth from 1998. If the month and day of birth
has not yet been reached in the current year,
subtract 1.
24
Levels of Measurement
25
Reliability and Validity of Measures
RELIABILITY Absence of random error (noise),
refers to repeatability of the measure. Do we get
same answer each time we make measurement? Inter-
rater, test-retest, split half, parallel forms,
Internal consistency- Cronbachs alpha,
KR VALIDITY Absence of systematic error
(bias). Are we measuring what we purport to be
measuring? Content validity, face
validity Criterion-related validity predictive,
concurrent, convergent, discriminant Construct
validity MTMM
26
Validity vs Reliability
27
Statistics measurement level
A dichotomous variable (0 or 1) can be treated as
interval
28
Misuse of StatisticsVisitor Characteristics
across Wilderness Areas
Variable San Gorgonio John Muir F p
Gender 1.2 1.1 .43 .51
Age 3.9 3.8 .52 .47
Educ 3.9 4.2 6.2 .01
Ethnicity 1.2 1.2 .00 .96
Income 3.49 3.5 .98 .32
29
Notes

• Gender 1male, 2female
• Age ranges 10-12, 213-18, 319-30,
  431-50, 551-65, 666
• Educ 10-8th, 2HS, 32 yr
  coll, 44 year college,
  5postgrad
• Ethnicity 1Anglo, 2Latino, 3Black, 4 Asian
• Income 1 0-13,00, 214-25,
  326-50, 451-75, 576