Research Design

1
Research Design

• Minjuan Wang
• ED 690, EDTEC, SDSU

mwang_at_mail.sdsu.edu
2
(No Transcript)
3
Basic vs. Applied

• The development of Interpersonal relationships in
  online chat rooms
• How does playing video games affect childrens
  learning styles?

• How can these theories be used to help people
  develop romantic relationships in online chat
  rooms (e.g., eharmony)?
• Can playing video games help these children
  develop media literacy skills?

4
Examples of Applied Research

• Action (or practitioner) research
• ongoing improvement of practice
• a cycle of plan -gt act -gt study the act -gt learn
  from the study -gt act again -gt ...
• Evaluation research
• Formative How have I met the objectives of my
  life?
• Summative
• Has my life been worth living?

5
(No Transcript)
6
Qualitative (interpretive) Quantitative
Research

• Think in terms of how to do the study
• View of the world holistic vs. variables
• Primary data types
• Naturalistic settings or manipulation
• Researcher involvement
• Data analysis
• Report writing

7
Traditional Chinese vs. Western Medicine

• The "whole body" health approach
• sees the person as an integral Mind/body organism
• Emotional, mental life, physical symptoms
• stimulate the bodys natural healing potential by
  treating root causes rather than just symptoms

8
(No Transcript)
9
Quantitative Design

• Descriptive/Survey Research
• to describe
• customer service
• Correlational Research
• Not causation
• Causal-Comparative Research
• The personal, social, and family characteristics
  of angry teens
• Experimental Research
• Independent variable manipulated

10
Descriptive

• Collecting numerical data to test hypotheses or
  answer questions concerning current status.
• Through self-reports, questionnaires or
  interviews, or through observations.
• A study to determine the beliefs of EDTEC
  graduate students.

11
Correlational

• Are ages of EDTEC graduates related to job
  satisfaction?
• Do stock prices correlate with employee training?
  
• Do Big Bob test scores correlate highly with
  other intelligence test? Do they correlate highly
  with people's self-esteem and height?

12
Causal-Comparative

• The cause, or reason, for existing differences in
  the behavior or status of groups of individuals
• Gender, ethnicity, personality, height
• Behaviors
• Smoking
• Insomnia
• Lack of motivation

13
Experimental!!!

• One-Shot Case Study
• One-Group Pretest-Posttest
• Quasi-Experimental
• No random assignments
• Pretest-Posttest Control Group
• Posttest-Only Control Group
• Factorial Design
• two or more separate independent variables

14
Examples

• Bobs Online Experiments
• http//coe.sdsu.edu/ed690/mod/mod11/app.htm

15
Qualitative (Interpretive) Research

• Case Study
• The ABC Nightline programs
• Ethnography
• the cultural patterns and perspectives
• cultures of work and family women's employment
  and care of children media and culture gender,
  race, and generation education, race, and the
  American dream
• Grounded Theory
• Historical

16
Content Analysis

• Can be both
• Exam campaign speeches on the war and examine the
  number of positive and negative words.
• Is there a strong presence of positive or
  negative words in the Clinton speech?"
• What are the major themes in the campaign speech
  and how they are related?
• identifying, coding, and categorizing the primary
  patterns in the data

17
The research/evaluation dichotomy real or
contrived?

• Not method or subject matter
• But intent--the purpose for which it is done
• Evaluation leads to
• conclusions, and to get to them requires
  identifying standards and performance data and
  the integration of the two.

18
Types of evaluation questions

• Descriptive
• Goals, objectives, procedures, processes, and
  outcomes of a program
• Normative
• Evaluate the programs goals objectives by
  multiple values
• Impact
• Outcomes impact on client or stakeholder

19
Major types of evaluation design

• Survey
• Descriptive and normative
• Case study
• Field experiment
• Mostly in impact analysis
• TSA Cox Kaiser
• Secondary data analysis

20
Question from Danny

• According to the ADDIE model of designing
  instructions, I understand that the 'E'
  represents evaluation and it is an important
  element of the instructional design process. I
  considered myself new to the work of an
  instructional designer and would like to know
  what are the pertinent and frequently asked
  questions in the evaluation phase. Would also
  appreciate if someone can advise me how you
  actually go about implementing the evaluation as
  well as the difficulties encountered. Thanks a
  lot!Danny

21
Answer from Ummon

• There are many kinds of evaluation and many ways
  to proceed through instructional design...
• E-learning addition the software development
  cycle on top of that usability testing, quality
  testing, etc...
• Narrow down your question to something more
  specific.

22
Answer from TBH some models

• Kirkpatricks (1994) model
• Reaction, learning, behavior, results
• How?
• Outcome-focused
• Appropriate for all settings
• Intel attitude, achievement, performance, impact
• Philips (1997) model
• Go beyond cultural/organizational impact
• Return on Investment

23
Another model to evaluate E-Learning

• Systematic, flexible, multidimensional model
  (Mngania Hatcher)
• The Onion Model
• Look and feel of a program
• Peel its layers to see the actual quality
• Learner, technology, instruction, instructor,
  institution, and community
• The evaluation matrix
• Interaction of all these factors

24
Evaluating Distance Education (Simonson, 1997)

• The Best way to find things out is not to ask
  questions at all.
• Like firing off a gunbang.
• Sit still and pretend not to be looking
• Little facts peck around your feet
• Situations will venture forth
• Intentions will creep out
• If you are patient, youll see and understand a
  grat deal

25
Evaluating Distance Education (Simonson, 1997)

• The Best way to find things out is not to ask
  questions at all.
• Like firing off a gunbang.
• Sit still and pretend not to be looking
• Little facts peck around your feet
• Situations will venture forth
• Intentions will creep out
• If you are patient, youll see and understand a
  grat deal

26
Evaluating Distance Education (2) (Simonson,
1997)

• Woodley Kirkwoods Summary of Evaluation
  procedures
• Activity, efficiency, outcomes, program aims,
  policy, organization
• AEOIU approach to evaluation
• Accountability, effectiveness, impact,
  organizational context, unanticipated
  consequences

27
Models of evaluation

• Stufflebeam 1956--CIPP (context, input, process,
  product)
• Americas educational reform
• Tylers objectives-based model
• Sort of like the behaviorism in design
• Constructivist, heuristic approach
• Scrivens goal free model
• Participants-centered discovery
• Influenced by qualitative approach
• Wstake(1973) responsive model
• Adversary model in the 70s
• Developmental mode
• Involve people of color in the evaluation process
  of a multicultural education project