Research Methodology and Project Proposal Preparation

1
Research Methodologyand Project Proposal
Preparation
Master of System Engineering Faculty of
Engineering Gadjah Mada University
Adhy Kurniawan Faculty of Engineering Gadjah Mada
University
2
Adhy Kurniawan

• 1987-1990 SMA 3 Semarang
• 1990-1991 Fac. Of Economy, Diponegoro
  University, Semarang
• S1(1991-1996) Civil Engineering Dept. Gadjah
  Mada Univ.
• S3(1998-2003) Swiss Federal Institute of
  Technology Lausanne (EPFL), Swiss
• Post Doct (nov.2005-sept.2006) Kyoto University,
  Japan

3
My Goals for Course

• That each of you develop an intuition for the
  fundamental principles of research methodology
• That we have an enjoyable semester learning
  together

4
Lecture and Homework

• Lecture
• Presentation and discussion

• Homework
• Your chance to practice using the concepts
  presented in class
• Teamwork vs. Individual work?

5
References

• All of literature concerning Res Met
• Marczyk, DeMatteo, Festinger. 2005, Essentials
  of Research Design and Methodology, John Wiley
  and Sons.
• Day and Gastel, 2006, How to write and Publish a
  Scientific Report, Greenwood Press
• Metodologi Riset, Etc.

6
List of students

• Alif Ardy Saputra, Geodesi UGM
• Anik FR, TL, ITB
• Ashri Uswatun, TFisika,UGM
• Ayi Fajarwati, TL, ITB
• Corry Agustina, Perenc Wil, TA, UGM
• Dwi Astuti, TKimia, UGM
• Elva Nur , TF, UGM
• Erika Kezia, TL, ITB
• Fitri Wijayanti, Fisika, UNS
• I Nyoman Kusuma, TF, UGM
• Ihsan Hasan, T Industri, UII,
• Ihwan Ghazali, T Industri, UAD
• Iin Lestari, TL, ITB
• M Sony Abertiawan, TL, ITB
• Maria Auliana, T Sipil, UGM
• Norma Pradipta, TArsitektur, UGM
• Satrya Alrizki, TGeofisik, ITB
• Tatag Lindu Bhakti, TFisika, UGM

7
Contents

• The aims of research,
• the research topic,
• title and research problem,
• literature review,
• research design population and sampling types,
  types of quantitative research designs, validity
  of conclusions, data-collecting methods and
  measuring instruments in quantitative research,
  qualitative research designs,
• data analysis and interpretation of results,
• report writing and the research proposal,
• ethical consideration on research.

8
OVERVIEW OF SCIENCE AND THE SCIENTIFIC METHOD

• science can be defined as a methodological and
  systematic approach to the acquisition of new
  knowledge.
• This definition of science highlights some of the
  key differences between how scientists and
  nonscientists go about acquiring new knowledge.
• Specifically, rather than relying on mere casual
  observations and an informal approach to learn
• about the world, scientists attempt to gain new
  knowledge by making careful observations and
  using systematic, controlled, and methodical
  approaches (Shaughnessy Zechmeister, 1997).

Shaughnessy, J. J., Zechmeister, E. B. (1997).
Research methods in psychology (4th ed.). Boston
McGraw Hill.
9

• In addition, scientific knowledge is not based on
  the opinions, feelings, or intuition of the
  scientist.
• Instead, scientific knowledge is based on
  objective data that were reliably obtained in the
  context of a carefully designed research study.
• In short, scientific knowledge is based on the
  accumulation of empirical evidence (Kazdin,
  2003a)

Kazdin, A. E. (2003a). Methodology What it is
and why it is so important. In A. E. Kazdin (
Ed.), Methodological issues and strategies in
clinical research (3rd ed., pp. 522).
Washington, DC American Psychological
Association.
10

• The defining characteristic of scientific
  research is the scientific method .
• First described by the English philosopher and
  scientist Roger Bacon in the 13th century, it is
  still generally agreed that the scientific method
  is the basis for all scientific investigation.
• The scientific method is best thought of as an
  approach to the acquisition of new knowledge, and
  this approach effectively distinguishes science
  from nonscience.

11
The Scientific Method
The development of the scientific method is
usually credited to Roger Bacon, a philosopher
and scientist from 13th-century England, although
some argue that the Italian scientist Galileo
Galilei played an important role in formulating
the scientific method. Later contributions to
the scientific method were made by the
philosophers Francis Bacon and René Descartes.
12

• Although some disagreement exists regarding the
  exact characteristics of the scientific method,
  most agree that it is characterized by the
  following elements
• Empirical approach
• Observations
• Questions
• Hypotheses
• Experiments
• Analyses
• Conclusions
• Replication

13
Empirical Approach

• The scientific method is firmly based on the
  empirical approach. The empirical approach is an
  evidence-based approach that relies on direct
  observation and experimentation in the
  acquisition of new knowledge (see Kazdin, 2003a).
  
• In the empirical approach, scientific decisions
  are made based on the data derived from direct
  observation and experimentation.
• Contrast this approach to decision making with
  the way that most nonscientific decisions are
  made in our daily lives.
• For example, we have all made decisions based on
  feelings, hunches, or gut instinct.
  Additionally, we may often reach conclusions or
  make decisions that are not necessarily based on
  data, but rather on opinions, speculation, and a
  hope for the best.
• The empirical approach, with its emphasis on
  direct, systematic, and careful observation, is
  best thought of as the guiding principle behind
  all research conducted in accordance with the
  scientific method.

14
Observations

• An important component in any scientific
  investigation is observation. In this sense,
  observation refers to two distinct conceptsbeing
  aware of the world around us and making careful
  measurements.
• Observations of the world around us often give
  rise to the questions that are addressed through
  scientific research.
• For example, the Newtonian observation that
  apples fall from trees stimulated much research
  into the effects of gravity. Therefore, a keen
  eye to your surroundings can often provide you
  with many ideas for research studies.

15
Questions

• After getting a research idea, perhaps from
  making observations of the world around us, the
  next step in the research process involves
  translating that research idea into an answerable
  question.
• The term answerable is particularly important
  in this respect, and it should not be overlooked.
  
• It would obviously be a frustrating and
  ultimately unrewarding endeavor to attempt to
  answer an unanswerable research question through
  scientific investigation.
• It is therefore important to formulate a research
  question that can be answered through available
  scientific methods and procedures.

16
Hypotheses

• The next step in the scientific method is coming
  up with a hypothesis, which is simply an
  educatedand testableguess about the answer to
  your research question.
• A hypothesis is often described as an attempt by
  the researcher to explain the phenomenon of
  interest.
• Hypotheses can take various forms, depending on
  the question being asked and the type of study
  being conducted.
• A key feature of all hypotheses is that each must
  make a prediction.
• Remember that hypotheses are the researchers
  attempt to explain the phenomenon being studied,
  and that explanation should involve a prediction
  about the variables being studied.
• These predictions are then tested by gathering
  and analyzing data, and the hypotheses can either
  be supported or refuted on the basis of the data.

17

• Two types of hypotheses with which you should be
  familiar are
• the null hypothesis
• and the alternate (or experimental) hypothesis.
• The null hypothesis always predicts that there
  will be no differences between the groups being
  studied.
• By contrast, the alternate hypothesis predicts
  that there will be a difference between the
  groups.
• For example,
• the null hypothesis would predict that the
  exercise group and the no-exercise group will not
  differ significantly on levels of cholesterol.
• The alternate hypothesis would predict that the
  two groups will differ significantly on
  cholesterol levels.
• Homework Individual
• Please try to find one example. About the null
  hypotheses and alternate hypotheses.

18
Experiments

• After articulating the hypothesis, the next step
  involves actually conducting the experiment (or
  research study).
• For example, if the study involves investigating
  the effects of exercise on levels of cholesterol,
  the researcher would design and conduct a study
  that would attempt to address that question.
• As previously mentioned, a key aspect of
  conducting a research study is measuring the
  phenomenon of interest in an accurate and
  reliable manner.
• In this example, the researcher would collect
  data on the cholesterol levels of the study
  participants by using an accurate and reliable
  measurement device.
• Then, the researcher would compare the
  cholesterol levels of the two groups to see if
  exercise had any effects.

19
Accuracy vs. Reliability

• When talking about measurement in the context of
  research, there is an important distinction
  between being accurate and being reliable.
• Accuracy refers to whether the measurement is
  correct, whereas reliability refers to whether
  the measurement is consistent.
• An example may help to clarify the distinction.
• When throwing darts at a dart board, accuracy
  refers to whether the darts are hitting the
  bulls eye (an accurate dart thrower will throw
  darts that hit the bulls eye).
• Reliability, on the other hand, refers to
  whether the darts are hitting the same spot (a
  reliable dart thrower will throw darts that hit
  the same spot).
• Therefore, an accurate and reliable dart thrower
  will consistently throw the darts in the bulls
  eye. As may be evident, however, it is possible
  for the dart thrower to be reliable, but not
  accurate.
• For example, the dart thrower may throw all of
  the darts in the same spot (which demonstrates
  high reliability), but that spot may not be the
  bulls eye (which demonstrates low accuracy).

20
Analyses

• After conducting the study and gathering the
  data, the next step involves analyzing the data,
  which generally calls for the use of statistical
  techniques.
• The type of statistical techniques used by a
  researcher depends on the design of the study,
  the type of data being gathered, and the
  questions being asked.
• It is important to be aware of the role of
  statistics in conducting a research study.
• In short, statistics help researchers minimize
  the likelihood of reaching an erroneous
  conclusion about the relationship between the
  variables being studied.

21
Conclusions

• After analyzing the data and determining whether
  to reject the null hypothesis, the researcher is
  now in a position to draw some conclusions about
  the results of the study.
• For example, if the researcher rejected the null
  hypothesis, the researcher can conclude that the
  phenomenon being studied had an effecta
  statistically significant effect, to be more
  precise.
• If the researcher rejects the null hypothesis in
  our exercise-cholesterol example, the researcher
  is concluding that exercise had an effect on
  levels of cholesterol.

22

• It is important that researchers make only those
  conclusions that can be supported by the data
  analyses.
• Going beyond the data is a cardinal sin that
  researchers must be careful to avoid.

23
Replication

• One of the most important elements of the
  scientific method is replication.
• Replication essentially means conducting the same
  research study a second time with another group
  of participants to see whether the same results
  are obtained.
• The same researcher may attempt to replicate
  previously obtained results, or perhaps other
  researchers may undertake that task.

24

• Replication illustrates an important point about
  scientific researchnamely, that researchers
  should avoid drawing broad conclusions based on
  the results of a single research study because it
  is always possible that the results of that
  particular study were an aberration.
• In other words, it is possible that the results
  of the research study were obtained by chance or
  error and, therefore, that the results may not
  accurately represent the actual state of things.
• However, if the results of a research study are
  obtained a second time (i.e., replicated), the
  likelihood that the original studys findings
  were obtained by chance or error is greatly
  reduced.

25

• What are the three general goals of scientific
  research?

26
Answer

• description,
• prediction,
• and understanding/explaining

27
(No Transcript)
28
What Exactly is Research?

• we will focus on two of the most common types of
  research
• correlational research
• and experimental research

29
Correlational research

• In correlational research, the goal is to
  determine whether two or more variables are
  related. (By the way, variables is a term with
  which you should be familiar.
• A variable is anything that can take on different
  values, such as weight, time, and height.)
• For example, a researcher may be interested in
  determining whether age is related to weight. In
  this example, a researcher may discover that age
  is indeed related to weight because as age
  increases, weight also increases. If a
  correlation between two variables is strong
  enough, knowing about one variable allows a
  researcher to make a prediction about the other
  variable.
• It is important to point out, however, that a
  correlation
• or relationshipbetween two things does not
  necessarily mean that one thing caused the
  other.To draw a cause-and-effect conclusion,
• researchers must use experimental research.
• .

30
Experimental research

• In its simplest form, experimental research
  involves comparing two groups on one outcome
  measure to test some hypothesis regarding
  causation.
• For example, if a researcher is interested in the
  effects of a new medication on headaches, the
  researcher would randomly divide a group of
  people with headaches into two groups.
• One of the groups, the experimental group, would
  receive the new medication being tested.
• The other group, the control group, would receive
  a placebo medication (i.e., a medication
  containing a harmless substance, such as sugar,
  that has no physiological effects).

31
Experimental research

• Besides receiving the different medications, the
  groups would be treated exactly the same so that
  the research could isolate the effects of the
  medications. After receiving the medications,
  both groups would be compared to see whether
  people in the experimental group had fewer
  headaches than people in the control
• group.
• Assuming this study was properly designed (and
  properly designed studies will be discussed in
  detail in later chapters), if people in the
  experimental group had fewer headaches than
  people in the control group, the researcher could
  conclude that the new medication reduces
  headaches.

32
(No Transcript)
33
Task

• Compose your own brief research proposal.
• Try to determine your research topic for MST
  final project
• Format
• 1. In MS Word
• 2. In Power point

34
Task/assignment ? next week

• Review 1 International Publication (Journal,
  Conference paper,etc) related to Renewable energy
• Compose the summarize of your review
• Format
• 1. In MS Word
• 2. In Power point

35
Purpose of the research proposal

• 1. To inform the reader of nature of your
  proposed research.
• What is the problem?
• What is its extent?
• 2. To convince the reader, especially supervisors
  and reviewers, of the value of your proposed
  research.
• Is this project worth the time
• and money?
• Will it make a difference to the
• world?

36
Purpose of the research proposal

• 3. To demonstrate your expertise and competency
  in a particular area of study.
• Do you have the qualifications to conduct this
  research?
• Have you informed yourself of the existing theory
  and data relevant to your topic?
• Do you have the
• necessary skills to
• conduct the research?

37
Purpose of the research proposal

• 4. To plan the research project and provide a
  step-by-step guide to the tasks necessary for its
  completion.
• What are the key stages of the work?
• What are the priorities?
• How do the various components fit together?
• 5. To request support from individuals and
  agencies who provide supervision, oversight or
  funding for the research project.
• What kinds of support does the project need?
• Are all participants properly protected?

38
Purpose of the research proposal

• 6. To contract with the agencies and individuals
  involved, including supervisors, foundations and
  participants in the research team.
• How will tasks be assigned and resources
  expended?
• What does each contribute
• to the collective endeavor?

39
First things first

1. Basics
2. Topic ideas
3. Typical methodologies
4. Common pitfalls
5. Getting started and putting it all together
6. Questions/discussion

40
Basic steps of a research project

• Find a topic?What, When
• Formulate questions?What, Why
• Define population?Who, When
• Select design measurement?How
• Gather evidence?How
• Interpret evidence?Why
• Tell about what you did and found out

41
Selecting a Research Topic

• What are some considerations when selecting a
  research topic?

42
Considerations in Selecting a Topic

• Personal interest / Passion
• Importance / Contribution to the field
• Newness / Relevance
• Feasibility
• Tradeoff between rigor and practicality
• Time constraints
• Ethical constraints
• Organizational support
• Economic factors
• Availability of Subjects

43
Sources of Research Topics

• Peer-reviewed journals in your field
• Personal experiences
• Work setting experiences
• Existing literature
• Recommendations for future research

44
Refining Your Topic

• Refinement needed for effective and efficient
  research
• Narrow your topic
• Identify a theoretical framework
• Specifically and unambiguously define terms
• State research questions and hypotheses

45
Refining Your Topic (contd)

• A literature review will help you
• See if your idea has been tried
• Include all relevant constructs
• Select instruments
• Anticipate common problems

46
Components of a Concept Paper

• Title page
• Introduction
• Nature of the Problem
• Background and Significance of the Problem
• Preliminary Literature Review
• Initial Research Question or Questions

47
Components of a Concept Paper (contd)

• Brief Description of Methodology and Research
  Design
• Anticipated Outcomes
• Timeline
• References

48
(No Transcript)
49
(No Transcript)
50
The Literature Review

51
What is a Literature Review?

• According to Creswell (2005), a review of the
  literature is a written summary of journal
  articles, books and other documents that
  describes the past and current state of
  information, organizes the literature into topics
  and documents a need for a proposed study. (pp.
  79)

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
52
Focusing on Empirical Research

• What does Empirical Mean?
• Primary Sources
• Original Research Article
• Secondary Sources
• Newspapers
• Book chapters
• Television/Radio
• Magazines
• Wikepedia

53
Empirical Research

• All empirical research is inherently flawed
• Limitations
• Sampling
• Generalizability
• Representative
• Measurement
• Measurement Error
• Social Desirability
• Problem Identification
• Grasping the Whole Problem

54
Literature Reviews

• Well-written analytical narrative that brings a
  reader up-to-date on what is known on a given
  topic, but also provide fresh insights that
  advance knowledge
• Resolve conflicts between studies
• Identify new ways to interpret research results
• Creating a path for future research

55
Anecdotal Reports

• A description of an event or experience that
  happened to be noticed
• No control
• No comparison

56
Review of Key Elements of Previous Definition

• The LR is a summary of research
• It is not a list of found research but a
  coherent and articulate account of past and
  current research findings
• Suggestion read 2 or 3 LRs in order to become
  familiar with summary styles

57
Review of Key Elements of Previous Definition
(contd)

• The sources typically are journal articles, books
  and other documents that describe past and
  present status of research in a given field
• The LR should be exhaustive and as current as
  possible.
• How many articles?
• There is no set number. As long as the search is
  exhaustive and focused on the research topic, the
  review will be acceptable.

58
Review of Key Elements of Previous Definition
(contd)

• How far back should one search?
• A reasonable and widely accepted timeframe
  includes research conducted during the past 10
  years. Important studies (i.e., studies that had
  a significant impact on the field of study)
  should also be mentioned even if these go beyond
  the mentioned timeframe.

59
Review of Key Elements of Previous Definition
(contd)

• The LR should be organized
• The review should not only be coherent, but
  should organize the studies reviewed under themes
  or topics.
• The reviewer is a guide and should be able to
  provide readers with an in-depth and current
  status of research in a given area.
• This aspect is essential for readers to
  understand what the reviewer found during the
  search.

60
Review of Key Elements of Previous Definition
(contd)

• The LR should document the need for a proposed
  study
• Studies should not duplicate research that has
  been already done.
• Even in cases when research is duplicated
  (replicated is the appropriate term), one is
  responsible for documenting the need for
  replication, e.g., need to explore the same
  methodology with a different group or population,
  or need to change methodology with the same group.

61
Creswells 5 steps to Conduct a Literature Review

• Step 1 Identify Key Terms or Descriptors
• Extract key words from your title (remember, you
  may decide to change the title later)
• Use some of the words other authors reported in
  the literature

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
62
Step 1 Identify Key Terms or Descriptors
(contd)

• Use the Thesaurus of ERIC Descriptors to look
  for terms that match your topic go to
  www.eric.ed.gov and in Search select
  Descriptors (from Thesaurus)
• Scan both electronic and library journals from
  the past 10 years and look for key terms in the
  articles

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
63
Creswells 5 steps to Conduct a Literature Review
(contd)

• Step 2 Locate Literature
• Use academic libraries, do not limit your search
  to an electronic search of articles
• Use primary and secondary sources. A primary
  source is research reported by the researcher
  that conducted the study. A secondary source is
  research that summarizes or reports findings that
  come from primary sources

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
64
Step 2 Locate Literature (contd)

• It is best to report mostly primary sources (p.
  82)
• Search different types of literature summaries,
  encyclopedias, dictionaries and glossaries of
  terms, handbooks, statistical indexes, reviews
  and syntheses, books, journals, indexed
  publications, electronic sources, abstract
  series, and databases

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
65
Creswells 5 steps to Conduct a Literature Review
(contd)

• Step 3 Critically Evaluate and Select Literature
• Rely on journal articles published in national
  journals
• Prioritize your search first look for refereed
  journal articles, then, non-refereed articles,
  then books, then conference papers, dissertations
  and theses and then papers posted to websites

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
66
Step 3 Critically Evaluate and Select Literature
(contd)

• Look for research articles and avoid as much as
  possible opinion pieces
• Blend qualitative and quantitative research in
  your review

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
67
Creswells 5 steps to Conduct a Literature Review
(contd)

• Step 4 Organize the Literature
• Create a file or abstract system to keep
  track of what you read. Each article you read
  should be summarized in one page containing
• Title (use APA to type the title so that you can
  later copy-paste this into the References section
  of your paper)
• Source journal article, book, glossary, etc.

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
68
Step 4 Organize the Literature (contd)

• Research problem one or two lines will suffice
• Research Questions or Hypotheses
• Data collection procedure (a description of
  sample characteristics can be very handy as well)
• Results or findings of the study
• Sort these abstracts into groups of related
  topics or areas which can then become the
  different sections of your review

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
69
Creswells 5 steps to Conduct a Literature Review
(contd)

• Step 5 Write a Literature Review
• Types of Reviews
• Thematic Review a theme is identified and
  studies found under this theme are described.
  Major ideas and findings are reported rather than
  details.

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
70
Step 5 Write a Literature Review(contd)

• Study-by-study Review a detailed summary of each
  study under a broad theme is provided. Link
  summaries (or abstracts) using transitional
  sentences. Must be organized and flow coherently
  under various subheadings. Avoid string
  quotations (i.e., lengthy chunks of text directly
  quoted from a source)

Creswell, J.W. (2005) Educational Research
Planning, Conducting, and Evaluating Quantitative
and Qualitative Research
71
Preliminary Literature Review

• This succinct review of current literature
  should
• Provide further contextual background
• Reveal issues related to your study
• Describe similar problems in other organizations
• Provide significance to your approach to the
  study

72
Guidelines on Style, Mechanics, and Language Usage

• Does your draft follow the logic or idea that is
  presented in your intro and title?
• Avoid overusing direct quotations, especially
  long ones
• Check style manual for correct use of citations
• (Doe, 2005) Doe (2005) (Doe Smith, 2005) Doe
  and Smith (2005) (Black, 2005 Brown, 2006
  Yellow, 2007)

73
Guidelines on Style, Mechanics, and Language Usage

• Avoid using synonyms for recurring words
• This is not creative writing and stay consistent
  with terminology
• Group I, Phoenix Cohort, Experimental Group
• Spell out all acronyms when first using them
• Traditional - American Psychological Association
  (APA)
• Non-traditional - Collective Efficacy (CE)
• Yes - Do NOT use contractions No Dont use
  contractions
• Coined terms should be set off by quotes

74
Guidelines on Style, Mechanics, and Language Usage

• Avoid the following
• Slang cool
• Colloquialisms thing gtgt item or feature
• Idioms rise to the pinnacle gtgt to become
  prominent
• Use great care to avoid Plagiarism

75
What needs to be included in the Literature
review.

• Provides contextual background
• Reveals related issues
• Reviews similar problems elsewhere
• Provides significance to your approach to the
  study
• Includes major/seminar research articles
  pertaining to study
• Written in an integrated manner
• Uses peer-reviewed research
• Includes a Reference section

76
Writing Your Research Question(s)

• Reflect the problem that the researcher wants to
  investigate
• Can be formulated based on theories, past
  research, previous experience, or the practical
  need to make data-driven decisions in a work
  environment

77
Writing Your Research Question(s) (contd)

• Are vitally important because they, in large
  part, dictate what type of statistical analysis
  is needed, as well as what type of research
  design may be employed
• A research question should address only 1 concept
• Question must be measurable

78
Types of Questions Asked

• Once you have identified the topic of study, you
  will need to consider the type of question you
  want answered and how it will be answered
• Two paradigms
• Quantitative Paradigm
• Generally attempt to quantify variables of
  interest. Questions frequently address how well
  or how much.

79
Types of Questions Asked

• Qualitative Paradigm
• there are times when we wish to know not how
  many or how well, but simply how. (Shulman,
  1988, pg. 7)

80
Class Exercise

• Now youre ready to formulate your own research
  question(s)
• Sample questions
• Is there a relationship between participation in
  an Elluminate chat session and course grade?
• How do 5th grade students experience the
  anticipation of standardized testing?

81
Research Questions

• From Topic to Research Question A good research
  topic asks a clear, concise question. Asking a
  research question helps you keep a tight focus on
  your topic.
• Tweaking Your Research Question
• A good research topic is broad enough to allow
  you to find plenty of material, but narrow enough
  to fit within the size and time constraints of
  your paper.
• If your topic is either too broad or too narrow,
  consider adding or eliminating the following
  elements
• Time Period, century, decade, future, Population
  Type, age, gender, nationality, species,
  Geographic Location country, state, region, Point
  of View economic, social, cultural, biological

82
Assignment 2 Components (see syllabus for
details)

• Title Page
• Nature of the Problem
• Background and Significance of the Problem
• Literature Review
• Research Questions
• References

83
(No Transcript)
84
Figuring out your study

• What?
• Who?
• When?
• Why?
• And How?

85
Topic ideas

• Online chat reference
• Types of questions
• Subject? Type?
• of turnaways
• Difference in discourse
• In-person vs. chat
• Partnership studies
• Similar libraries with same software

86
Topic Ideas

• E-book usage
• Usability studies of
• Online tutorial(s)
• My Library portals
• Analysis of library web sites or library
  instruction sites or pathfinders by best
  practices
• Student learning outcomes in LI programs

87
Types of methodologies

• QuaLitative Measures
• Descriptive
• Numbers not the primary focus
• Interpretive, ethnographic, naturalistic
• QuaNtitative Measures
• N for numbers
• Statistical
• Quantifiable

88
QuaLitative measures

• Content Analysis
• Analyzed course syllabi of library use through
  discipline and level (Rambler)
• Studied online tutorials, applying best practices
  recommendations (Tancheva)

89
QuaLitative Measures

• Discourse Analysis
• Analyzed student responses in writing and
  discussions to a short film compared findings
  to parallel study with LIS grad Ss (Vandergrift)
• Focus Groups
• Discussed how participants experience use the
  library (Von Seggern Young)
• Studied why students use the Internet and how
  much time they use it (Wilson)

90
QuaLitative Measures

• Interviews
• Studied 25 HS students web use for research
  assignments (Lorenzen)
• Looked at what type of information first year
  students need and how they go about acquiring it
  (Seamans)
• Observation (obtrusive)
• Observed students as they conducted online
  research noted their activities (Dunn)
• Observation (Unobtrusive)
• Retrieval of discarded cheat sheets to analyze
  academic misconduct (Pullen et. al.)

91
QuaLitative Measures

• Think Aloud Protocols
• Studied how users navigate a library web site
  (Cockrell Jayne)
• Usability testing
• Examined students mental models of online
  tutorials (Veldof Beavers)

92
QuaNtitative measures
p lt .05

• CompareThings
• Count Things
• Survey People About Things

93
QuaNtitative measures
p lt .05

• Comparison studies
• Experimental and control groups
• Instructional methodologies (Colaric Cudiner
  Harmon)
• Program assessment using before/after analysis of
  research papers(Emmons Martin)

94
QuaNtitative measures
p lt .05

• Pre Post Tests (Van Scoyoc)
• Measures Scales
• Bosticks Library Anxiety Scale (Onwuegbuzie
  Jiao Van Scoyoc)
• Procrastination Assessment Scale (Onwuegbuzie
  Jiao)

95
QuaNtitative measures
p lt .05

• Numeric Studies
• Citation Analysis?Bibliometrics (Dellavalle)
• Webometrics (Bar-Ilian)

96
Ready Made Data Sets

• National Survey of Student Engagement (Whitmire)
• College Student Experiences Questionnaire (Kuh
  and Gonyea)
• The Web
• Internet Archive (Ryan, Field Olfman)
• Electronic journals (Dellavalle)
• Library server logs

97
Common Pitfalls

• Problems with population
• Sampling?
• Representativeness?
• Self-selection?

98
Research Problem 1 A study assessing student
learning outcomes in 2 broad categories
(concepts, techniques) by examining student
research journals in 1 section of an elective
information literacy course in fall semester.
99
Research Problem 4 A 2004 article on a library
use and services satisfaction study that used as
its measurement tool a survey given to every nth
person entering the library building on 40
randomly selected days throughout the school
year.
100
Research Problem 5 An outcomes assessment
research project of a 5 year old IL program in
which all incoming freshmen must participate.
Total student population on campus is divided
between 32 freshmen to senior (or 4 year) and
68 transfer students.
101
Common Pitfalls

• Problems with operationalization
• Defining of what is measured

102
Research Problem 2 An experimental study that
proposes a fund allocation formula for academic
library collections based on the following
average of overall book price average of
overall serial prices degree level (10 for
undergraduate to 30 for doctorate) / the number
of students enrolled in degree program as majors
the total number of faculty in the department
three total number of students in program.
(OAB OAS) D/(Sn (Fn3))Sn N.B. Not a
standard formula
103
Research Problem 3 A newspaper article you read
just the other day stated that in a recently
published study done at a major U.S. university,
researchers found that domestic violence affects
1 in every 4 women.
104
Research Problem 4 A 2004 article on a library
use and services satisfaction study that used as
its measurement tool a survey given to every nth
person entering the library building on 40
randomly selected days throughout the school
year.
105
Research Problem 5 Over a one year period,
researchers studied the occurrence of turn-aways
in a virtual reference service and noted that the
significantly high occurrence of turn-aways
indicates increased need for virtual reference
service.
106
Common Pitfalls

• Problems with generalizability
• False conclusions
• Transformations

107
Research Problem 1 A study assessing student
learning outcomes in 2 broad categories
(concepts, techniques) by examining student
research journals in 1 section of an elective
information literacy course in fall semester.
108
Research Problem 7 A survey of faculty found
that the majority of those interviewed interacted
most with librarians at the reference desk. The
researchers concluded that most faculty view
librarians in a servile role.
109
Keep In Mind That

• No study is perfect
• All data is dirty is some way or another
  research is what you do with that dirty data
  (Manuel)
• Measurement involves making choices

110
Be Critical About Numbers (Best 2001)

• Every statistic is a way of summarizing complex
  information into relatively simple numbers.
  (Best)
• How did the researchers arrive at these numbers?
• Who produced the numbers and what is their bias?
• How can key terms be defined in how many
  different ways?

111
Be Critical About Numbers

• How was the choice for the measurement made?
• What type of sample was gathered how does that
  affect result?
• Is the statistical result interpreted correctly?
• If comparisons are made, are they appropriate?
• Are there competing statistics?

112
Getting Started

• Read to learn read to analyze
• About research methodology
• Studies on similar topics
• Interesting studies
• Non-library studies

113
Getting Started

• Finding a topic neednt be traumatic
• Work projects? Research studies
• PT overhaul
• Library GO Bond Proposal Project
• Library workshop trends
• User repair strategies

114
Getting Started

• Data collection involves agreement consent
• Forge partnerships
• At some point you will need to leave the comfort
  zone of reading and literature gathering and

115
Just get out and do it!
116
Questions?
117
(No Transcript)
118
Research methodology

• Quantitative Methods
• Qualitative procedures

119
Quantitative Methods

• A definition
• A survey or experiment that provides as output a
  quantitative or numeric description of some
  fraction of the population, called the sample.

120
Components of a survey method

• The survey design
• The population and sample
• The instrumentation
• Variables in the study
• Data analysis

121
The survey design

• Purpose of the survey
• The research question
• Type of survey
• Cross sectional
• Longitudinal
• Form of data collection

122
The population and sample

• Description of the population
• Sampling design
• Single stage
• Multistage
• Stratified
• Sample selection

123
The instrumentation

• The instrument (tool)
• Existing
• New
• Rating scale
• Likert scale Rating the Items. 1-to-5 rating
  scale where
• strongly unfavorable to the concept
• somewhat unfavorable to the concept
• undecided
• somewhat favorable to the concept
• strongly favorable to the concept
• Pilot
• Administration
• Postal survey
• email

124
Variables and analysis

• The research question
• Variable in the research
• E.g. Number of years of academic study
• The questions in the instrument
• E.g. How many years of study in a University
• As an undergraduate?
• As a postgraduate?
• Data analysis
• Steps
• Bias in the data
• Non-response
• Statistics, e.g. mean, standard deviation etc.

125
Components of an experimental method

• Subjects
• Instruments and materials
• The experimental design

126
Subjects

• Selection
• Conveniently
• Random (RCT)
• Group assignment
• Random
• Matched. E.g. Ability, Age
• Size
• Variables
• Dependent
• Independent

127
Randomized Controlled Trial (RCT)

• A true experiment, in which the researcher
  randomly assigns some patients to at least one
  maneuver (treatment) and other patients to a
  placebo, or usual treatment. Key features the
  classic way to evaluate effectiveness of drugs
  (or exercise, diet, counseling). Patients are
  followed over time (Prospective). If properly
  done, an RCT can be used to determine cause and
  effect

128
Instrumentation and Materials

• Description
• Validation
• Pilot
• Content validity
• Prediction validity
• Materials

129
The experimental design

• Type
• Pre-experimental
• No control group
• Quasi-experimental
• Control group, but not randomly assigned
• Single subject design (over time)
• Pure experiment
• Repeated measures
• Change groups

130
Overview of Qualitative Research Design

• Historical routes in anthropology
• Generates new understanding by naming and
  framing concepts and themes
• Removes bias by questioning preconceived
  assumptions of the social group under study
• Promotes neutrality through adoption by the
  researcher of naïve stance or critical
  discussion, challenges pre-conceived assumptions
  of both the researcher and the social group under
  study
• Produces new understanding about the world,
  changes the way power, culture and social
  interaction are understood

131
Data Collection in Qualitative Research

• Observation (Videoed, non-participant,
  semi-participant and participant observation,
  field notes)
• Interviews (individual and group - known as focus
  groups, tape recorded and transcribed, field
  notes)
• Secondary data analysis (using written material
  collected for purposes other than research)
• Questionnaires (unstructured, postal, interviews)
• A mixture of all four

132
Questions in Qualitative Research
In qualitative research questions are open-ended.
Sometimes a check list or topic guide will be
used by the researcher to ensure all the relevant
areas are covered. This is known as
semi-structured data collection. It is used in
all four methods of data collection Sometimes the
only guide is the topic itself and the researcher
collects verbatim or naturally occurring data.
This is known as unstructured data collection. It
is used in all four methods of data collection
133
Sampling in Qualitative Research
The sampling method of choice is theoretical
sampling (queuing behaviour) However, often this
is not possible and people resort to convenience
sampling (students) and snowball sampling
(mental health in black and ethnic minority
communities) Neither of the latter two methods
are considered strong but maybe all that can be
achieved. Research must be viable.
134
Data Analysis in Qualitative Research

• Read and re-read data, become engrossed in it.
• Identify themes common, conflicting, minority
• Test themes across the data set, where are they
  common, under what circumstances are they found,
  not found. This sets the parameters on the
  interpretation and generalisation of data
• Get more than one person to analyse the data
  independently then together
• Demonstrate trustworthiness in data analysis
• Examples
• Biographical continuity
• Nursing routines as a method of managing a
  transient workforce

135
Qualitative research

• Interpretative research
• Process orientated
• Researcher(s) are the primary data collection
  instrument
• Descriptive research
• Outputs are an inductive process

136
References

• MSc project web pages
• http//www.comp.glam.ac.uk/gis/start.asp?whatfile
  gis/gisrc/msc-proj.htm
• Creswell, J. W. (1994) Research design
  qualitative and quantitative approaches. -
  Thousand Oaks, Calif. London Sage
  Publications, ISBN 0803952546