Ateam Spring Session

1
A-team Spring Session 2Questionnaire Design

• February 28, 2007

2
Questionnaire or Survey?

• Questionnaire is an actual instrument I.e. web
  questionnaire (Perseus)
• Survey is actually a verb/method to study or
  examine comprehensively
• Your questionnaire is actually surveying
• Often used interchangeably

3
Yields Different Types of Data

• Descriptive
• Socioeconomic parameters to better understand the
  larger population represented by the sample.
  (e.g., income, age, college/school, major, class
  standing)
• Behavioral
• Patterns of use, recreation, entertainment,
  personal behavior. (e.g., UGA bus rides per
  day/week)
• Preferential
• Opinions preferences about socio-political
  issues. (e.g., opinion about new parking plan)

4
Types of Responses/Measurement Scales

• Nominal Scales
• Used to categorize objects name them
• Object is in a category or it is not
• No order implied along any dimension
• Response sets that are nominal
• Yes/No (dichotomous)
• Can be Choose one, Choose all that apply,
  Choose one and specify, or Choose all and
  specify from listing of characteristics

5
Examples of Nominal Scales

• Immediately after aerobic exercising I generally
• feel ___Exhausted ___Invigorated
  ___Thirsty
• ___Sweaty ___Overheated ___Nauseated
• Note make sure () and (-) options offered in
  listing
• Indicate your sex ___Male ___Female
• Have you ever resided in Brumby Hall?
• ___Yes ___No

6
Measurement Scales (Contd)

• Ordinal Scales (a.k.a. rank, order, rank-order)
• Used to rank objects according to amount of
  characteristic the object possesses
• Order reflects varying amounts or levels
• Rank reflects range from high to low amounts
• Ranking has no absolute zero
• Intervals from one rank to next not the same
• Likert scales are ordinal but sometimes treated
  as interval scales (judgment call)

7
Examples of Ordinal Scales

• Order of finish in a horse race
• Rank in class (e.g., achievement)
• Highest degree earned
• Order of preference
• A higher number indicates a higher rank, e.g.,
  more of characteristic possessed
• Watch for (reverse) coding

8
Examples of Ordinal Scales

• Rank-order your on-campus living preferences for
  the next academic year, with 1 first priority,
  2 second priority, and so on
• ___ Single Room ___One-bedroom
  Apartment
• ___ Double Room ___
  Multiple-bedroom Apartment
• ___ Suite (2 double rooms separated by a
  bathroom)
• Rank-order your reasons for attending this
  workshop, with 1most influential reason,
  2second greatest influence, and so on
• ___ Surveys are my life ___My boss sent
  me
• ___ To get the handouts ___ My thirst for
  knowledge
• ___ To earn CEUs ___ To get the free gift

9
Measurement Scales (Contd)

• Interval Scales
• A ranking/rating using interval score values
• The difference between intervals is equal
• The difference between 1 2 is the same as the
  difference between 4 5
• Still a focus on the amount of a characteristic
  an object possesses
• Likert-type (pronounced Lick-ürt) scales often
  treated like interval scales (although considered
  ordinal)
• 5strongly agree, 4agree, 3no opinion,
  2disagree, 1strongly disagree

10
Examples of Interval Scales

• Likert Scale Example
• Parking on campus should be free.
• __Strongly Agree __Agree __Neither Agree nor
  Disagree __Disagree __Strongly Disagree
• Non-Likert Scale Example
• When driving a UGA van, the safest following
  distance under ideal conditions (in seconds) is
• __1.5 __3 __4 __8 __10 __25

11
Examples of Interval Scales

• Non-Likert Scale Example (contd)
• Please rate your satisfaction with the
  following student activities on a scale of 1 to
  5, with 1very dissatisfied, 2dissatisfied,
  3neither dissatisfied nor satisfied, 4satisfied
  and 5very satisfied. could add not applicable
  or did not attend option.
• __Welcome Week __Dawgs After Dark
• __Movie-O-Rama __Concert on the Quad
• __ Halloween I Vant to Drink Your
  BlooooooDrive

12
Measurement Scales (Contd)

• Ratio Scales
• Empirically meaningful zero/absolute zero true
  absence of characteristic (e.g., height, weight
  more common in physical/biological sciences)
• Have all characteristics of nominal, ordinal, and
  interval scales
• Can be converted to ordinal scales
• Can be converted to categories
• Education examples income, age, years of
  education, meetings with academic advisor

13
Examples of Ratio Scales

• Indicate the number of times you accessed the
  University Health Center in the last 30 days
  ___
• Indicate your age ___

14
Likert Scales

• Present question/item stem in a declarative
  sentence (one statement under consideration).
• Response options represent varying degrees of
  agreement or endorsement of one statement.
• Response options should be worded to represent
  approximately equal intervals use equal
  positive and negative possibilities.
• The question stem doesnt have to span the range
  of the construct (as in Thurstone or Guttman)
  response options infer levels of phenomena.

15
Likert Scales

• Often 5, 7, or 9 response-options sets
• A 6 response-options set is also common
• Strongly disagree
• Moderately disagree
• Mildly disagree
• Mildly agree
• Moderately agree
• Strongly agree

16
Likert Scales

• Midpoint often used but optional
• What does midpoint wording imply?
• Neither agree nor disagree Apathy?
• Agree and disagree equally Strong paradox?
• Common midpoint wording
• Neither agree nor disagree
• Agree and disagree equally
• Neutral

17
Likert Scales

• Most used in surveys of opinions, beliefs,
  attitudes
• Useful if statements are fairly strong (but not
  extremely)
• Everyone can agree, have no opinion, or have
  little opinion about a mild statement
• Write clear statements that reflect true
  differences of opinion

18
Likert Scale Examples

• Exercise is an essential component of a healthy
  life-style.
• 1Strongly Disagree, 2Moderately Disagree, 3
  Mildly Disagree, 4Mildly Agree, 5Moderately
  Agree, and 6Strongly Agree
• Combating drug abuse should be a top national
  priority.
• 1Completely True, 2Mostly True, 3Equally True
  and Untrue, 4Mostly Untrue, and 5Completely
  Untrue

19
Semantic Differential Scales

• Response options consist of one but usually
  several adjective pairs
• One adjective is negative, the other positive
  each serves as the (-) or () end of a continuum
  that characterizes the stimulus

20
Semantic Differential Scales

• Individual lines/points are placed between the
  two extremes (adjectives)
• 7 or 9 lines/points are common
• Respondents check/select lines or points closest
  to the adjectives if they hold extreme views
• Respondents check/select lines or points toward
  the middle of the continuum if they hold more
  moderate views

21
Semantic Differential Examples

• Automobile Salesmen
• Honest __ __ __ __ __ __ __ Dishonest
• Quiet __ __ __ __ __ __ __ Noisy
• Friendly __ __ __ __ __ __ __ Not
  Friendly
• Fair __ __ __ __ __ __ __ Unfair
• Trustworthy __ __ __ __ __ __ __
  Untrustworthy

22
Length of a Survey

• Sufficient to capture needed data
• Short enough to hold participants attention
• Type of survey affects length
• Types of questions affect length
• Quantitative/Qualitative/Mixed approach affects
  length
• Participant characteristics affect length

23
Measurement Scales More Tips

• Avoid providing categories/options that overlap
  difficult or impossible to analyze
• Frequently happens with age, income, class hours,
  years of service, hours worked, etc.
• Example
• Select the category that best describes your
  annual, gross income __0-10,000
• __10,000 30,000 __30,000 - 60,000

24
Measurement Scales (Contd)

• Be thoughtful with Use of other or does not
  apply or not applicable in listing of
  characteristics/options
• Positive Obtain option you may not have
  considered
• Positive Prevents forced responses
• Negative Can give response already listed or
  spurious data
• Potential Negative Adds to analysis time

25
Analysis
26
Analysis (Contd)
27
References

• DeVellis, R. F. (1991). Scale development
  Theory and applications. Newbury Park, CA Sage.
• Miller, T. K., (1999). CAS The book of
  professional standards for higher education.
  Washington, DC Council for the Advancement of
  Standards in Higher Education.
• Payne, D. (1992). Measuring and evaluating
  educational outcomes. New York Macmillan.
• Rea, L. M. Parker, R. A. (1997) Designing and
  conducting survey research (2nd Ed.). San
  Francisco Jossey-Bass.
• Schuh, J. H. Upcraft, M. L. (2001). Assessment
  practice in student affairs An applications
  manual. San Francisco Jossey-Bass.
• Upcraft, M. L. Schuh, J. H. (1996).
  Assessment in student affairs A guide for
  practitioners. San Francisco Jossey-Bass.