Research Seminars in IT in Education MIT6003 Research Methodology I

1
Research Seminars inIT in Education(MIT6003)Re
search Methodology I

• Dr Jacky Pow

2
Agenda

• Method evaluation critical review of research
  procedures

3
Typical research procedures

• Formulation of the research problem
• An emerging issue
• Review of the literature
• What has been done and what need to be done
• Data collection
• How to collect the necessary data to answer the
  research question

4
Typical research procedures

• Data analysis
• How to analyze the and organize the data
• Discussion
• To discuss the implication of the data and see
  whether the research questions can be answered
• Future research agenda
• Conclusion
• What this research has achieved

5
Class activity

• What questions would you ask if you were to
  evaluate the appropriateness of the research
  method in a study?
• How will you answer the questions you identified?

6
Issues in educational research

• Reliability (Trochim, 2002)
• In research context, the term reliability means
  repeatability or consistency. A measure (of the
  variables) is considered reliable if it would
  give us the same result over and over again
  (assuming that what we are measuring is not
  changing!)

7
Types of reliability

• Inter-Rater/Inter-observer Reliability
• Used to assess the degree to which different
  raters/observers give consistent estimates of the
  same phenomenon
• Test-Retest reliability
• Used to assess the consistency of a measure from
  one time to another
• Administer the same test to the same (or a
  similar) sample on two different occasions

8
Types of reliability

• Parallel-Forms Reliability
• Used to assess the consistency of the results of
  two-tests constructed in the same way from the
  same content domain
• Administer both instruments to the same sample
• Internal Consistency reliability
• Used to assess the consistency of results across
  items within a test
• Calculate the average correlation of the items
  splitting into half e.g., Cronbachs alpha (a)

9
Cronbachs alpha (a)
Step 1
Trochim, 2002
10
Cronbachs alpha (a)
Step 2
Trochim, 2002
11
Issues in educational research

• Validity
• Refers to what we measure is what we intend to
  measure (highly related to the research question)
• We have to make sure that the tools we use to
  measure the variables are valid
• Internal validity
• The degree to which correct conclusions about
  causal relations can be drawn

12
Reliability and validity
Trochim, 2002
13
Class activity

• Discuss with your group members to see whether
  the issues facing by quantitative and qualitative
  research are the same in terms of reliability and
  validity

14
Issues in educational research

• Applicability
• To be of any use educational research must
  generalize from the sample to the population
• The findings of the research should be able to
  inform teaching practice or policy formulation

15
Issues in educational research

• Feasibility
• There may be some excellent ideas in educational
  research but due to some reasons, the research
  may not be feasible
• These reasons may include resource, time or
  ethical considerations

16
Issues in educational research

• Significance
• Educational research should have impact or
  acceptable level of importance to the field
• Significance the value of the research

17
Issues in educational research

• Ethics
• The research we conduct should not violate the
  ethical standard
• For instance, it is not ethical to segregate a
  group of students during their programme in order
  to test out a new teaching technique

18
Common errors made in research

• Selective observation
• Inaccurate observation
• Overgeneralization
• Made-up information
• Ex post facto hypothesizing
• Illogical reasoning
• Ego involvement in understanding
• Premature closure of inquiry

19
Selective observation

• It happens when attention is drawn to
  observations or answers that confirm pre-existing
  beliefs
• A few ways you can try to avoid selective
  observation in your research
• Literature review
• Decide your research approach beforehand
• Take thorough notes
• Watch for "disconfirmatory" information
• Consider both "sides" of your study

20
Inaccurate observation

• It occurs when we misremember or misrecord
  data
• To increase your accuracy you have to plan your
  observation
• Using forms
• Doing time/area sampling
• Writing down as much information as you can
• To practice observing and recording your
  observations before actually doing it for real

21
Overgeneralization

• Overgeneralization is generalizing to others who
  are different from one's research population
• To avoid overgeneralizing
• Replicate one's study
• Support many tests of the same theory
• Attempt to use representative samples
• Recognize the limitations of one's research

22
Made-up information

• It happens when one fills in details without a
  scientific basis for doing so
• Instead of making up information, you could do a
  variety of things
• Ask someone who knows what's going on for their
  opinion
• Do a thorough literature review to help you
  understand possible outcomes and unique customs
• Rely on prior research studies and/or theory to
  guide your own interpretation of what was going on

23
Ex post facto hypothesizing

• It occurs when a researcher decides what happened
  after it happened and after the study was done
• In scientific research, we have to decide what
  will happen before we do our study, not after

24
Illogical reasoning

• Illogical reasoning is just that illogical
• to avoid illogical reasoning in your own
  research
• Base your decisions on prior research and theory
  
• Make extensive use of peer review

25
Ego involvement in understanding

• It happens when people let the human side of them
  dictate their findings and how they view findings
  by other researchers
• To avoid ego involvement
• To try to remain neutral
• To stay away from topics about which you can't be
  neutral
• To use the team approach

26
Premature closure of inquiry

• It occurs when we decide that we know enough
  about a topic and decide that it no longer
  warrants future study
• To deal with premature closure
• To keep looking for answers, even if it involves
  using different approaches
• To find non-controversial ways to do it as far as
  possible

27
Mystification

• It happens when we attribute results to the
  supernatural
• To avoid mystification
• Keep looking for answers when you feel tempted to
  rely on enchanted explanations
• Peer review will prevent you from embarrassing
  yourself

28
End of lesson 9