Primary Data and Measurement

1
Primary Data and Measurement

• Jeremy Kees, Ph.D.

2
Formulate Problem
Stages in the Research Process
Determine Research Design
Design Data Collection Method and Forms
Design Sample and Collect Data
Analyze and Interpret the Data
Prepare the Research Report
3
Primary Data Overview

• Types of Primary Data
• Demographic / Socioeconomic Characteristics
• Psychological / Lifestyle Characteristics
• Attitudes / Opinions
• Awareness / Knowledge
• Intentions
• Motivation
• Behavior
• What, how much, where, when, how, who
• Purchase behavior vs. use behavior
• E.g., --- basic hierarchy of effects models

4
Primary Data Overview

• Qualities of Primary Data
• Versatility
• Can be obtained quickly
• Quality
• Objectivity
• Accuracy

5
Primary Data Overview

• Two Classes of Primary Data
• Observation
• Communication

6
Decisions, Decisions
7
Structure and Disguise Decisions
8
Trade-Offs Among Methods of Survey Administration
HIGHLY VERSATILE
LIMITED
phone
fax
web
email
mail
personal (e.g., mall, home)
LOW COST
HI COST
personal
web
fax
mail
phone
email
FAST TURN- AROUND
SLOW
web
mail
fax
phone
personal
email
9
Mall Intercepts vs. E-Panels
10
E-Panels vs. Phone
Jeff Miller and Alan Hogg Internet vs. Telephone
Data Collection Burke White Paper series 2 (4)
(www.burke.com). Also see Ashok Ranchhod and Fan
Zhou Comparing Respondents of E-Mail and Mail
Surveys, Marketing Intelligence Planning 19
(2001), 254.
11
Basic Question Formats

• Open-ended question
• Unprobed format seeks no additional information
• Probed format includes a response probe
  instructing the interviewer to ask for additional
  information

12
Basic Question Formats

• Close-ended question
• Dichotomous has only two response options, such
  as yes or no
• Multiple response has more than two options for
  the response
• Scaled-response utilizes a scale developed by
  the researcher to measure the attributes of some
  construct under study
• Labeled vs. Unlabeled

13
Other Considerations

• The nature of the property being measured
• Previous research studies
• The data collection mode
• The ability of the respondent
• The scale level desired

14
Types of Scales

• Nominal scales those that use only labels
• Ordinal scales those with which the researcher
  can rank-order the respondents or responses
• Interval scales those in which the distance
  between each descriptor is equal
• Ratio scales ones in which a true zero exists

15
Examples
15
16
Thinking Ahead

• What analysis technique is used when
• There are 2 nominally scaled variables
• There are 2 intervally (or ratio) scaled
  variables
• There is one DV that your trying to predict or
  explain with multiple interval variables (the
  independent variables)
• You are checking for differences between 2 groups
  (nominal) for an interval scaled variable
• You are checking for differences between 4 stores
  on total sales across 4 categories?
• Potential Answers --- t-test multiple
  regression, oneway ANOVA, factorial ANOVA,
  correlation, crosstabs, multivariate analysis of
  variance

17

• Survey (Questionnaire) Design

18
Procedure for Developing a Questionnaire
19
Procedure for Developing a Questionnaire
20
Desirable Characteristics

• Brief
• Objective
• Specific
• Relevant

21
Other Important Issues

• Respondent qualification
• Frame of reference
• Question loading

22
QUESTION SEQUENCING - General Guidelines

• Use (more) simple, interesting opening questions
• Use the funnel approach, asking broad questions
  first, and follow with more specific questions
• Carefully design branching questions
• Ask for classification information last
• Place more difficult or sensitive questions near
  the end

23
QUESTION SEQUENCING - General Guidelines
Question ordering 1 1 EVALUATION OF FAT LEVEL
OF A PRODUCT 2 EVALUATION OF OVERALL PROD.
NUTRITIOUSNESS 3 EVALUATION OF OVERALL PRODUCT
ATTITUDE AND INTENTIONS TO PURCHASE
Question ordering 2 1 EVALUATION OF OVERALL
PRODUCT ATTITUDE AND INTENTIONS TO PURCHASE 2
EVALUATION OF LEVEL OF PROD. NUTRITIOUSNESS 3
EVALUATION OF FAT LEVEL OF PRODUCT
24
QUESTION WORDING - General Guidelines

• Use simple words and questions
• Avoid ambiguous words and questions
• Avoid leading questions---be objective
• Avoid implicit alternatives
• Avoid generalizations and estimates ---Be
  specific
• Avoid double-barreled questions

25

• What is your income?
• 10,000 or less.1
• 10,000 to 25,000..2
• 25,000 to 50,000..3
• 50,000 to 75,000...4
• 75,000 to 100,000.....5
• 100,000 or more..6

What is the problem and how would you revise the
question?
26
2. Do you agree with many experts that the
United States will run out of oil by the year
2020? Yes.1 No...2
27
3. How important was price compared to service
when your company decided to buy its current
computer network? Very important.4 Somewhat
important.3 Somewhat unimportant.2 Very
unimportant......1
28
Please rate the importance of the following
factors in making the choice among
vendors. Selling price Not Very
Important ___ ___ ___ ___ ___ ___ ___
Important Vendors reputation for service Not
Very Important ___ ___ ___ ___ ___ ___
___ Important
29

• Is the speed and efficiency of the drive-in
  teller services at your regular bank..(READ
  CATEGORIES)
• Very Satisfactory4
• Somewhat satisfactory3
• Somewhat unsatisfactory2
• Very unsatisfactory.1

PROBLEMS a. Speed and efficiency
(double-barreled) b. Do they use drive-in teller
service Do Rs know what drive in tellers are?
What about ATM drive-ins? c. What are regular
banks?
30
Itemized Rating Scales

• The respondents are provided with a scale that
  has a number or brief description associated with
  each category.
• The categories are ordered in terms of scale
  position, and the respondents are required to
  select the specified category that best describes
  the object being rated.
• The commonly used itemized rating scales are the
  Likert and semantic differential
• Others include Thurstone and Guttman

31
Itemized Rating Scales

• Likert Scales
• requires the respondents to indicate a degree of
  agreement or disagreement with each of a series
  of statements about the stimulus objects

•  
• Strongly Disagree Neither Agree Strongly
• disagree agree nor agree
• disagree
•  
• 1. Wal-Mart sells high quality merchandise.
  1 2X 3 4 5
•  
• 2. Wal-Mart has poor in-store
  service. 1 2X 3 4 5
•  
• I like to shop at Wal-Mart . 1 2 3X 4 5
• 4. Wal-Mart has low prices . 1 2 3X 4 5

32
Itemized Rating Scales

• Semantic Differential Scales
• End points associated with bipolar labels that
  have semantic meaning
• SEARS IS
• Powerful ---------X----- Weak
• Unreliable -----------X--- Reliable
• Modern -------------X- Old-fashioned

33
Key Decisions for Itemized Scales

• Number of categories
• Although there is no single, optimal number,
  traditional guidelines suggest that there should
  be between five and nine categories
• Balanced vs. unbalanced
• In general, the scale should be balanced to
  obtain objective data
• Odd/even no. of categories
• If a neutral or indifferent scale response is
  possible for at least some respondents, an odd
  number of categories should be used
• Forced vs. non-forced
• In situations where the respondents are expected
  to have no opinion, the accuracy of the data may
  be improved by a non-forced scale( DK/NR)
• Verbal description
• An argument can be made for labeling all or any
  scale categories. The category descriptions
  should be located as close to the response
  categories as possible

34
Measurement Theory

• Construct Validity
• Criterion (Face) Validity
• Predictive Validity
• Nominological Validity
• Reliability

35
Construct Validity
Construct Abstract Concept
Unobservable Latent Psychological
We often use Multi-item measures (scales) to
assess unobservable constructs (WHY???)
36
Reliability

• True Score Test Theory
• All measures have
• True Score
• Error (Random and Systematic)
• Good measures minimize the systematic error
  component of the score
• Types of Reliability
• Inter-Rater
• Test-Retest
• Internal Consistency (Cronbachs Alpha)

37
Developing Sound Measures
38
Tips for Maximizing Participation

• Personal communication
• Asking a favor
• Importance of the research project and its
  purpose
• Importance of the recipient, how recipient was
  selected
• How the recipient may benefit from this research
• Completing the questionnaire will take only a
  short time
• The questionnaire can be answered easily

39
Tips for Maximizing Participation

• A stamped reply envelope is enclosed
• Answers are anonymous or confidential
• Offer to send report on results of survey
• Appreciation of sender
• Importance of sender or senders organization
• Description and purpose of incentive
• Style, format and appearance
• Brevity

40
Tips for Maximizing Participation
Professional Mail Surveys Company 7432 East Court
Avenue Elveron, CA 90101 [phone] Dear
Mr. Smythe Will you do us a favor? We are
conducting a nationwide survey among executives
and managers in the metalworking industry. The
purpose of this research is to find out the
opinion of experts like you on the advantages and
disadvantages of using three new steel products.
Your answers will enable steel manufacturers to
be aware of the requirements of the users and the
opinions of the nonusers of these products, and
this in turn will help them design the products
you need. Your name appeared in a
scientifically selected random sample. Your
answers are very important to the accuracy of our
research, whether or not your company is a user
of one or more of the products described. It
will take only a short time to answer the simple
questions on the enclosed questionnaire and to
return it in the stamped reply envelope. Of
course all answers are confidential and will be
used only in combination with those of other
metalworking executives and managers. If you
are interested in receiving a report on the
findings of this research, just write your name
and address at the end of this questionnaire, or
if you prefer, request the results of the Survey
on Steel Products in a separate letter. We will
be glad to send you a complimentary report when
it is ready. Please return the completed
questionnaire at your earliest convenience. Thank
you for your help! Sincerely, James
B. Jones James B. Jones Director P.
S. The enclosed dollar bill is just a token of
appreciation.
41
And finally, remember the golden rule.

• Do unto your respondents as you would have them
  do unto you!!

42
Team Assignment 2

• Dont start on this assignment until youve
  read Fowler (CH 6-7)
• Based on your research design
• Write a paragraph about what your measurement
  instrument is supposed to accomplish
• Make a list of what should be measured to
  accomplish the goals of the study
• Develop your measurement instrument

43
Team Assignment 2

• Deliverables include
• A very clean, polished version that you could use
  to actually collect data
• This means you will need to carefully think
  through all of the issues we covered tonight
  (e.g., set-up, ordering, length, multi-item
  scales, etc.)
• Intro paragraph and variable list (see previous
  slide)
• (Note Dont worry about defining your
  sample--youll have your chance to do that next
  week)