Research design experiment

1
Research design experiment

• Week 4 Lecture 2

2
What is research design

• A detailed blueprint specifying how the research
  will be carried out
• Identifying unit of analysis
• Defining conceptual variables and their measures
• Stating hypotheses to be tested
• Choosing an appropriate design type
• creating a detailed program for collecting,
  analyzing, and interpreting data

3
Identifying unit of Analysis

• The entities under study
• Research topic --IT project failure
• Research question 1
• In the context of IS projects, what factors are
  associated with the transition from escalation to
  deescalation?
• -- Turning around troubled software projects,
  an exploratory study of the deescalation of
  commitment to failing courses of action
• Research question 2
• How do individuals make reporting decisions
  regarding negative project status information
• Keeping mum as the project goes under

4
Defining variables

• Characteristics of units that vary, taking on
  different values, categories or attributes for
  different observations
• How do we identify variables/factors
• Literature
• Variables discovered by other researchers
• Exploratory study
• Focus group
• Interview

5
Stating Hypotheses

• An expected but unconfirmed relationship between
  two or more variables
• Either explicit or implicit
• Where do hypotheses come from
• Theory, direct observation,guess, intuition
• Putting it all together
• Unit of analysis, variables, relationships
• Original statement Wireless network is
  valunerable
• How do we transform it into a testable hypothesis?

6
Common forms of hypothesis

• Conditional statement
• H3a Individuals will exhibit a greater
  reluctance to report negative information when
  they perceive a high risk of negative personal
  consequences for doing so
• Continuous statement
• H5 Perceived project risk will increase with
  perceived impact
• Difference statement
• H3c individuals will exhibit more reluctance to
  report through external than through internal
  reporting channels
• Keeping Mum as the Project Goes Under Toward an
  Explanatory Model
• -- H. Jeff Smith et al, Journal of Management
  Information Systems/Fall 2001

7
Research purpose and design

• Exploratory studies
• The research plan in an exploratory study is more
  open then in other kinds of research
• Descriptive studies/Causal studies
• Highly structured
• Require detailed research design

8
Types of research design

• Experiment
• Causal research
• Key features
• Manipulation
• Control
• Survey
• Descriptive and Causal research
• Typical features
• A large number of respondents are chosen through
  sampling procedures
• Systematic questionnaire or interview procedures
• Answers are numerically coded and analyzed with
  the aid of statistical software

9
Key elements in experimental designs

• Dependent variable
• A variable thought to be influenced by other
  variables
• The effect in a cause-effect relationship
• Independent variable
• A variable which has been selected as a possible
  influence on variations in the dependent variable
• A cause in a cause-effect relationship
• Extraneous variable/control variable
• Any variable other than the independent variable
  that influences the dependent variable
• Held constant during the course of observation or
  analysis

10
Causal research model
11
Experiment stages
12
Experimental manipulation

• Identify various levels of independent variable
• Often two or three treatment levels will be
  examined
• Manipulation methods
• Instructional manipulation
• Varying the independent variable by giving
  different sets of instructions to the subjects
• Event manipulation
• Affecting the independent variable by altering
  the events that subjects experience

13
Manipulation checks

• Obtain evidence that the manipulation of the
  independent variable was experienced by the
  subjects
• Conduct interviews with the subject after the
  independent variable has been introduced
• Manipulation check through pretesting/pilot study

14
Control in experiment

• Internal validity -- The extent to which we can
  accurately state that the independent variable
  produced the observed effect

15
Design elements and notations

• Observations or Measures
• Symbolized by O in design notation
• Treatments or Programs
• A manipulated independent variable
• Symbolized by X in design notation
• Time
• Used to indicate the time you make the
  observation or take the measure
• Time moves from left to right. Elements that are
  listed on the left occur before elements that are
  listed on the right

16
Design elements and notations(2)

• Groups
• Units of analysis are usually grouped to
  facilitate comparison
• Each group in a design is given its own line in
  the design structure.
• Assignment to group
• R random assignment
• N Nonequivalent groups
• C assignment by cutoff

17
Sample design notation

• pretest-posttest
• Treatment Versus comparison group
• Randomized
• Experimental design

http//trochim.human.cornell.edu/kb/desintro.htm
18
Two group experimental design

• The two-group posttest-only randomized experiment
• Random assignments ensure that two groups are
  probabilistically equivalent
• Main interest determining whether the two groups
  are different after the treatment

19
An example

• Research question
• The impact of distance learning on college
  students understanding and cognitive skills
• Units of analysis college students
• Independent variables
• Distance learning (2 level)
• Dependent variables
• Understanding
• Cognitive skills

20
Research model and hypotheses

• Research model
• Hypotheses
• H1. Students using distance learning system will
  achieve higher understanding than students with
  traditional classroom learning do
• H2. The cognitive skills of students using
  distance learning system will be higher than
  those of students with traditional classroom
  learning

Cognitive skill
Understanding
H1
H2
Distance learning
21
Experiment design and conduct

• Field experiment