Research Design

1
Research Design

• Week 4 Lecture 1

2
Agenda

• Triangulation
• What is research design
• Two broad categories of research methodology
• The data source
• Measurement instruments
• Ethical issues

3
Questions about assignment 1

• Where to find the research problem?
• What do I expect to review?
• How do I write the review?
• Word limit?

4
Triangulation

• Triangulation of measures
• Triangulation of observers
• Triangulation of theory
• Triangulation of method

5
Examples of triangulation

• Topic The amount of violence in popular
  Hollywood films
• Measures
• Observers
• Theory
• Method

6
What is research design

• A detailed blueprint specifying how the research
  will be carried out
• Basic format of all research process
• General criteria for a research project
• Universality
• Could be carried out by any competent person
• Replication
• Control
• Measurement
• Data can be measured

7
Quantitative and qualitative orientation

• Different research problems
• gtDifferent research design and method
• Collection of different types of data
• Different interpretation of data
• Two broad categories
• Quantitative research
• Used to answer questions about relationships
  among measured variables with the purpose of
  explaining, predicting and controlling phenomena
• Qualitative research
• Used to answer questions about the complex nature
  of phenomena, often with the purpose of
  describing and understanding the phenomena from
  the participants point of view

8
Research Process

• Quantitative
• Concepts are in the form of distinct variables
• Relationships are expressed in hypotheses
• Measures are systematically created before data
  collection and are standardized
• Data are in the form of numbers from precise
  measurement
• Procedures are standard, and replication is
  assumed

• Qualitative
• Concepts are in the form of themes, motifs,
  generalizations, and taxonomies
• Measures are created in an ad hoc manner and are
  often specific to the individual setting or
  researcher
• Data are in the form of words and images from
  documents, observations and transcripts
• Procedures are particular and replication is very
  rare

9
Types of Analysis

• Quantitative
• Findings are typically expressed in terms of
  relationships that are presented in tables and
  graphs
• Qualitative
• Findings are typically expressed by quoting
  interviews or relating experiences the researcher
  has had in the field
• Which approach should I use?
• Guidelines from practical application on page
  112 of textbook

10
Validity of the methodology

• Internal validity
• The extent to which its design and the data that
  it yields allow the researcher to draw accurate
  conclusions about cause and effect and other
  relationships within the data
• Concerns
• Lack of control
• Reactivity Hawthorne effect
• Experimenter expectancy
• 3 cases from textbook page 104

11
Validity of the methodology (cont)

• External validity
• The extent to which the conclusions drawn can be
  generalized to other context
• Real-life setting
• A representative sample
• Replication In a different context
• Validity in qualitative research
• triangulation

12
Planning for data collection

• Four fundamental questions about the data
• What data are needed?
• Where are the data located?
• How will the data be secured?
• How will the data be interpreted?

13
Data source

• Secondary data
• Information collected by others for a purpose
  other than the project at hand, but may be useful
  for the project
• Primary data
• Original data collected for the purpose of the
  research problem
• Observation, questionnaire, interview and tape
  recording, experiment results

14
Use of secondary data

• Answering research questions or solving some or
  all of the research problems
• Helping in problem formulation
• Providing benchmarking measures and other
  findings that can be compared later on with the
  results of the study at hand
• Building and testing analytical (mathematical,
  econometric) models

15
An example of data source

• From this weeks paper
• last-minute bidding and the rules for ending
  second-price auctions evidence from eBay and
  Amazon auction by Roth A. and Ockenfels A.
• What types of data were collected in the study?
• How did the authors use those data for?

16
Where to find secondary data

• Sources may be internal or external
• Internet sites
• Central and local government studies and report
• Australian Bureau of Statistics
• Australian Stock Exchange
• Commercial research companies selling data
• Studies and reports of institutions and
  departments such as universities,
  telecommunication departments, marketing and
  other research institutes

17
Measurement instruments

• Use existing measurement instruments
• A clock to measure execution time
• A published personality test to measure a
  persons tendency to be either shy or outgoing
• Develop our own measurement instruments
• A survey to assess peoples opinion about welfare
  reform
• A final exam to test students understanding of a
  certain subjects
• Determine clearly and definitively the nature of
  the measurement instruments
• Provide evidence that the the instruments have a
  reasonable degree of validity and reliability for
  the research purpose

18
Validity of measurement instrument

• Face validity
• A personal judgment that a measure seems to be
  reasonable.
• Content validity
• The extent to which a measure reflects the
  dimension(s) implied by the concept
• Always in connection with achievement testing

19
Validity of measurement instrument

• Criterion validity
• The extent to which the results of a measurement
  instrument correlate with another, presumably
  related measure (criterion)
• Construct validity
• The extent to which the measure reflects the
  theoretical concept it is supposed to measure

20
Reliability of measurement instrument

• Test-retest reliability (stability over time)
• The extent to which the same instrument yields
  the same result on two different occasions
• Inter-rater reliability (representative across
  subgroups)
• The extent to which two or more individuals
  evaluating the same product or performance give
  identical judgments
• Internal consistency reliability (equivalence
  across indicators)
• The extent to which all the items with in a
  single instrument yield similar results.

21
Ethical Issues

• Researchers ethical responsibility to
  respondents
• Voluntary participation
• People not be coerced into participating in
  research
• Informed consent
• Prospective research participants must be fully
  informed about the procedures and risks involved
  in research and must give their consent to
  participate.
• Do not place pressure on respondents

22
Ethical Issues (cont)

• Confidentiality and Anonymity
• Identifying information will not be made
  available to anyone who is not directly involved
  in the study.
• Debrief subjects
• Where experiments or field studies involve
  deception, subjects should have the study
  explained to them after the session

23
Ethical Issue(cont)

• Researchers responsibility to his or her
  discipline and society
• Represent research literature fairly
• Acknowledge all your sources
• Be aware of potential sources of bias
• Avoid of falsifying data

24
Summary

• Triangulation
• General criteria for a research project
• Two major types of research design
• quantitative and qualitative design
• Two different data sources
• Find and evaluate measurement instruments
• Ethical issues