ES2002 Report Designing Research Tools

1
Designing the Research Tool(s)
2
Designing the Research Tool(s)

• Research Design
• Types of Primary Research
• Questionnaire Construction

3
Research Design

• Primary vs. Secondary Research
• Reliability and Validity
• Quantitative vs. Qualitative Data

4
Research Design

• Primary Research
• Focus groups
• Interview
• Observation
• Questionnaire survey

• Secondary Research
• Unpublished
• Academic theses, reports
• Published
• Books, magazines, journals, newspapers, internet
  websites

5
Research Design

• Reliability
• Replicates research with same results
• Ensure a significant sample size

• Validity
• Test what you set out to test
• Think through, design and construct carefully

6
Research Design

• Quantitative Data
• Completely objective
• Easy to tabulate
• Discrete
• continuous

• Qualitative Data
• Qualities or attributes
• Difficult to measure quantitatively

7
Types of Primary Research

• Focus groups
• Observation
• Interview
• Questionnaire survey

8
Focus Group Discussions

• Can be an important tool for
• programme evaluation
• Marketing, advertising
• Policy-making
• communication
• Consists of a number of individuals
• Lasts between 1 to 2½ hours
• Records or observes the session

9
Focus Group Discussions

• Advantages
• Used for exploratory research
• Obtain data quickly and less costly
• Interact, probe and clarify
• Observe non-verbal behaviour

• Disadvantages
• Cannot extrapolate to a larger population
• Affect the thought processes of respondents and
  researcher
• Collating the information and its interpretation
  may be difficult

10
Observation

• Involves watching or seeing what is happening
• Obtain data through the use of the five senses
• Example
• Counting the number of
• buses that run on time

11
Observational Tally Sheet
12
Observation

• Advantages
• May be the only method at times
• Generally objective
• Easy to tally and work with

• Disadvantages
• Limited to those phenomenon observed
• Cannot explain why
• May be influenced by observer bias
• May be expensive and time-consuming

13
Interviews

• Used when in-depth understanding is needed
• Can be used with other research tools to
• Supplement information
• Clarify the problem, limit the scope etc.
• Help interpret unusual findings
• Put data into perspective

14
Interviews

• Structured Interview
• Cannot vary the way the questions are asked
• Can only repeat the question
• Speak in as neutral a tone as possible
• Purpose Limit the interviewer bias

• Unstructured interview
• Conversational in tone
• Informal way of eliciting information
• Example Job interview
• information to be elicited qualifications,
  experience, ability to work with others etc.

15
Interviews

• Advantages
• Relatively more flexible and adaptable
• Permits probing
• Can observe the non-verbal behaviour
• Ambiguity can be clarified

• Disadvantages
• Interviewer bias can affect the interviewees
  responses
• Time-consuming and expensive
• Difficult to tabulate

16
Questionnaire Survey

• An orderly list of questions to
• obtain facts, opinions, attitudes etc.
• provide answers to how and why people think or
  behave in a certain way
• Helps researchers
• make decisions
• improve products
• recommend policies/procedures
• suggest changes

17
Questionnaire Survey

• Advantages
• Eliminates researchers prejudices
• Time and cost effective
• Reaches a large number of respondents
• Respondents privacy maintained
• Easy to tabulate

• Disadvantages
• Returns may not be representative
• Answers may not be as desired
• May be invalid and unreliable
• Time-consuming to design and refine

18
Types of Questions
Open-ended question In your opinion, how can
Resident Committees meet the needs of people
living in the neighbourhood? Dichotomous
question Only people with degrees should be
promoted. Agree ___ Disagree ___ No opinion
___ Multiple-choice question Tick the radio
station you listen to most frequently Class 95
FM ____ FM92.4 ____ 93.8 FM
____ Gold 90FM ____
19
Types of Questions

• Rating question
• How do you rate the efficiency of this
  department?
• Excellent Good Average Fair
  Poor
• 2 3 4
  5
• Ranking question
• Rank the following subjects in order of
  preference.
• (1 being the most preferable)
• Applied Statistics __________
• Business Finance __________
• International Economics __________
• Managerial Accounting __________
• Management Sciences __________

20
Questionnaire Construction

• Be as clear as possible
• Designed to elicit as accurately and quickly as
  possible from the respondent
• obtain facts, opinions, attitudes etc.
• provide answers to how and why people think or
  behave in a certain way
• Helps researchers
• make decisions
• improve products
• recommend policies/procedures
• suggest changes

21
Example Purpose To evaluate the effect of
training programme on staff morale Hypothesis Wo
rkers are dissatisfied with the selection
procedures.
22
Hypothesis Workers are dissatisfied with the
selection procedures.
1. How would you rate the present selection
procedures used to identify staff for the new
training programme? good 1 2 3 4 poor
23
Hypothesis Workers are dissatisfied with the
selection procedures.
2. If you gave a rating of either 3 or 4, please
indicate your reasons (you may tick more than one
option) ? it is embarrassing to be nominated ?
it doesn't identify the people who really need
the training ? people who might want to attend
the course are not able to ? others please
elaborate ______________________
24
Structure of the Questionnaire

• Be personal and friendly
• Show appreciation
• Introduce yourself and subject
• Be logical
• Provide clear instructions
• Place simple questions first
• Categorise questions
• Make transitions smooth
• Use filter/classification questions

25
Content of Questions

• Make questions easy to answer
• Avoid sensitive or personal questions
• Avoid asking for difficult information
• Provide realistic options
• offer choices that are mutually exclusive
• avoid multi-topic questions
• include dont know, others, no opinion
  categories

26
Phrasing of Questions

• Phrase questions unambiguously
• Use question words
• Eg. Who? What? When? Where? Why? How?
• Avoid words with vague meanings
• Eg. Do you drink regularly?

27
Phrasing of Questions

• Use objective phrasing
• Phrase questions concisely
• Eg. Has it happened to you that over a long
  
• period of time, when you neither
  practiced
• abstinence nor birth control, you did
  not
• conceive?

28
Phrasing of Questions

• Use objective phrasing
• Avoid leading questions
• Eg. Is Phua Chu Kang your favourite sitcom?
• Avoid loaded questions
• Eg. Do you practise good dental hygiene?

29
Basic Principles

• Be brief
• Keep the questionnaire short
• Make each question count
• Be professional
• Make questionnaire visually appealing
• Use good quality paper
• Use correct grammar
• Plan for easy tabulation
• Avoid open-ended questions
• Provide range categories