Research Concepts

1
Research Concepts

• SE-690
• Chris Jones
• Dr. Xiaoping Jia

2
Agenda

• What is Research in Computer Science?
• Research Basics
• What research is and is not
• Where research comes from
• Research deliverables
• Methodologies
• Research process
• Quantitative versus qualitative research
• Questions

3
Research in Computer Science

• The Oxford Concise dictionary defines research
  as
• research. 1.a. the systematic investigation into
  and study of materials, sources, etc, in order to
  establish facts and reach new conclusions. b. an
  endeavour to discover new or collate old facts
  etc by the scientific study of a subject or by a
  course of critical investigation.

4
Research in Computer Science

• Research strategies (to help one better
  understand what research is)
• Deductive logic
• Inductive reasoning
• Scientific method

5
Research in Computer Science

• Deductive logic begins with assumptions/premises/t
  ruths and proceeds logically from these premises
  toward conclusions that must also be true.

6
Research in Computer Science

• Deductive Example
• If all tulips are plants (premise 1) and
• if all plants produce energy through
  photosynthesis (premise 2),
• then all tulips must produce energy through
  photosynthesis (conclusion)

7
Research in Computer Science

• Inductive Reasoning uses specific instances or
  occurrences to draw conclusions about entire
  classes of objects or events.

8
Research in Computer Science

• Inductive Reasoning Example
• An anthropologist may draw conclusions about a
  certain cluture after studying a particular
  community within that culture.

9
Research in Computer Science

• Scientific method means the method that searches
  after knowledge. It is a means whereby insight
  into the unknown is sought by (1) identifying a
  problem that defines the goal of ones quest, (2)
  posing a hypothesis, (3) gathering data relevant
  to the hypothesis, and (4) analyzing and
  interpreting the data to see if the data support
  the hypothesis.

10
Research in Computer Science

• Scientific method (cont.)
• Formulating an hypothesis generally comes from
  deductive and/or inductive reasoning.

11
Research Basics

• What research is and isnt
• Research characteristics
• Research projects and pitfalls
• Sources of research projects
• Elements of research proposals
• Literature reviews

12
What Research Is Not

• Research isnt information gathering
• Gathering information from resources such books
  or magazines isnt research.
• No contribution to new knowledge.
• Research isnt the transportation of facts
• Merely transporting facts from one resource to
  another doesnt constitute research.
• No contribution to new knowledge although this
  might make existing knowledge more accessible.

13
What Research Is

• Research is

the systematic process of collecting and
analyzing information (data) in order to increase
our understanding of the phenomenon about which
we are concerned or interested.1
14
Research Characteristics

• Originates with a question or problem.
• Requires clear articulation of a goal.
• Follows a specific plan or procedure.
• Often divides main problem into subproblems.
• Guided by specific problem, question, or
  hypothesis.
• Accepts certain critical assumptions.
• Requires collection and interpretation of data.
• Cyclical (helical) in nature.

15
Research Projects

• Research begins with a problem.
• This problem need not be Earth-shaking.
• Identifying this problem can actually be the
  hardest part of research.
• In general, good research projects should
• Address an important question.
• Advance knowledge.

16
Research Project Pitfalls

• The following kinds of projects usually dont
  make for good research
• Self-enlightenment.
• Comparing data sets.
• Correlating data sets.
• Problems with yes / no answers.

17
High-Quality Research(1 of 2)

• Good research requires
• The scope and limitations of the work to be
  clearly defined.
• The process to be clearly explained so that it
  can be reproduced and verified by other
  researchers.
• A thoroughly planned design that is as objective
  as possible.

18
High-Quality Research(2 of 2)

• Good research requires
• Highly ethical standards be applied.
• All limitations be documented.
• Data be adequately analyzed and explained.
• All findings be presented unambiguously and all
  conclusions be justified by sufficient evidence.

19
Sources of Research Problems

• Observation.
• Literature reviews.
• Professional conferences.
• Experts.

20
Stating the Research Problem

• Once youve identified a research problem
• State that problem clearly and completely.
• Determine the feasibility of the research.
• Identify subproblems
• Completely researchable units.
• Small in number.
• Add up to the total problem.
• Must be clearly tied to the interpretation of the
  data.

21
Hypotheses

• Hypotheses are tentative, intelligent guesses as
  to the solution of the problem.
• There is often a 1-1 correspondence between a
  subproblem and a hypothesis.
• Hypotheses can direct later research activities
  since they can help determine the nature of the
  research and methods applied.

22
Delimitations

• All research has limitations and thus certain
  work that will not be performed.
• The work that will not be undertaken is described
  as the delimitations of the research.

23
Definitions

• Define each technical term as it is used in
  relation to your research project.
• This helps remove significant ambiguity from the
  research itself by ensuring that reviewers, while
  they may not agree with your definitions, at
  least know what youre talking about.

24
Assumptions

• Assumptions are those things that the researcher
  is taking for granted.
• For example a given test instrument accurately
  and consistently measures the phenomenon in
  question.
• As a general rule youre better off documenting
  an assumption than ignoring it.
• Overlooked assumptions provide a prime source of
  debate about a research projects results.

25
Importance of the Study

• Many research problems have a kind of theoretical
  feel about them. Such projects often need to be
  justified
• What is the research projects practical value?
• Without this justification, it will prove
  difficult to convince others that the problem in
  question is worth study.

26
Research Proposals

• Research proposals are documents that describe
  the intended research including
• Problem and subproblems.
• Hypotheses.
• Delimitations.
• Definitions.
• Assumptions.
• Importance.
• Literature review.

27
Literature Review

• A literature review is a necessity.
• Without this step, you wont know if your problem
  has been solved or what related research is
  already underway.
• When performing the review
• Start searching professional journals.
• Begin with the most recent articles you can find.
• Keep track of relevant articles in a
  bibliography.
• Dont be discouraged if work on the topic is
  already underway.

28
Literature Review Pitfalls(1 of 2)

• Be very careful to check your sources when doing
  your literature review.
• Many trade magazines are not peer reviewed.
• Professional conferences and journals often have
  each article reviewed by multiple people before
  it is even recommended for publication.
• The IEEE and ACM digital libraries are good
  places to start looking for legitimate research.

29
Literature Review Pitfalls(2 of 2)

• The Internet can be a good source of information.
  It is also full of pseudo-science and poor
  research.
• Make sure you verify the claims of any
  documentation that has not been peer reviewed by
  other professionals in the computing industry.

30
Processes Methodologies

• Research Process.
• Common Methodologies.
• Methodology Comparison.

31
Research Process

• Research is an extremely cyclic process.
• Later stages might necessitate a review of
  earlier work.
• This isnt a weakness of the process but is part
  of the built-in error correction machinery.
• Because of the cyclic nature of research, it can
  be difficult to determine where to start and when
  to stop.

32
Step 1 A Question Is Raised

• A question occurs to or is posed to the
  researcher for which that researcher has no
  answer.
• This doesnt mean that someone else doesnt
  already have an answer.
• The question needs to be converted to an
  appropriate problem statement like that
  documented in a research proposal.

33
Step 2 Suggest Hypotheses

• The researcher generates intermediate hypotheses
  to describe a solution to the problem.
• This is at best a temporary solution since there
  is as yet no evidence to support either the
  acceptance or rejection of these hypotheses.

34
Step 3 Literature Review

• The available literature is reviewed to determine
  if there is already a solution to the problem.
• Existing solutions do not always explain new
  observations.
• The existing solution might require some revision
  or even be discarded.

35
Step 4 Literature Evaluation

• Its possible that the literature review has
  yielded a solution to the proposed problem.
• This means that you havent really done research.
• On the other hand, if the literature review turns
  up nothing, then additional research activities
  are justified.

36
Step 5 Acquire Data

• The researcher now begins to gather data relating
  to the research problem.
• The means of data acquisition will often change
  based on the type of the research problem.
• This might entail only data gathering, but it
  could also require the creation of new
  measurement instruments.

37
Step 6 Data Analysis

• The data that were gathered in the previous step
  are analyzed as a first step in ascertaining
  their meaning.
• As before, the analysis of the data does not
  constitute research.
• This is basic number crunching.

38
Step 7 Data Interpretation

• The researcher interprets the newly analyzed data
  and suggests a conclusion.
• This can be difficult.
• Keep in mind that data analysis that suggests a
  correlation between two variables cant
  automatically be interpreted as suggesting
  causality between those variables.

39
Step 8 Hypothesis Support

• The data will either support the hypotheses or
  they wont.
• This may lead the researcher to cycle back to an
  earlier step in the process and begin again with
  a new hypothesis.
• This is one of the self-correcting mechanisms
  associated with the scientific method.

40
Common Methodologies

• Methodologies are high-level approaches to
  conducting research.
• The individual steps within the methodology might
  vary based on the research being performed.
• Two commonly used research methodologies
• Quantitative.
• Qualitative.

41
Quantitative and Qualitative Research
Methodologies

• Before defining each, an comparative example is
  given.
• Suppose two researchers are interested in
  investigating the effectiveness of the
  case-based method of teaching business management
  practices.

42
Quantitative and Qualitative Research
Methodologies (cont)

• The first researcher asks how effective is
  case-based instruction in comparison with
  lecture-based instruction?. The researcher
  recruits five classes for the case-based
  instruction and five classes for the
  lecture-based instruction.

43
Quantitative and Qualitative Research
Methodologies (cont)

• During and at the end of the semester,
  achievement tests are given to all classes.
  Using statistical analyses, scores of students
  are compared to determine if one group is
  significantly higher than that of the other group.

44
Quantitative and Qualitative Research
Methodologies (cont)

• (Qualitative) The second researcher asks the
  question, what makes case-based instruction
  effective or ineffective? To answer this
  question, the researcher assumes the role of a
  participant observer in a case-based taught
  course for an entire year.

45
Quantitative and Qualitative Research
Methodologies (cont)

• The researcher spends time talking with the
  instructor and students trying to understand
  case-based instruction from different
  perspectives. The researcher reviews the data
  for patterns and information to form an in-depth
  description and interpretation of what has been
  observed.

46
Quantitative Research Methodology

• Quantitative research is the systematic
  scientific investigation of quantitative
  properties and phenomena and their relationships.
  Quantitative research is widely used in both the
  natural and social sciences, including physics,
  biology, psychology, sociology, geology,
  education, and journalism. The term quantitative
  research is most often used in the social
  sciences in contrast to qualitative research.

47
Quantitative Research Methodology

• The objective of quantitative research is to
  develop and employ mathematical models, theories
  and hypotheses pertaining to natural phenomena.
  The process of measurement is central to
  quantitative research because it provides the
  fundamental connection between empirical
  observation and mathematical expression of
  quantitative relationships.

48
Quantitative Research Methodology

• Is concerned with the tabulation or numeric
  relevance of various kinds of behaviour.
  "Quantitative" is primary concerned with "
  Why"metro.newsmedianet.com.au/home/Glossary.jsp

49
Quantitative Research Methodology (cont)

• Research that examines phenomenon through the
  numerical representation of observations and
  statistical analysis.www.ojp.usdoj.gov/BJA/evalua
  tion/glossary/glossary_q.htm

50
Quantitative Research Methodology (cont)

• a systematic attempt to define, measure, and
  report on the relationships between various
  elements.www.crede.org/tools/glossary.html

51
Quantitative Research Methodology (cont)

• Scientific investigations in which numbers are
  used to measure variables such as
  characteristics, concepts, or things (CINAHL
  Subject Headings, 1990)healthlinks.washington.edu
  /howto/measurement/glossary/

52
Quantitative Research Methodology (cont)

• A research method based on collecting statistical
  data through questionnaires or surveys to help
  researchers determine the public
  opinion.www.communication.gc.ca/glossary.html

53
Quantitative Research Methodology (cont)

• Research which seeks to make measurements as
  distinct from qualitative research.www.mrs.org.uk
  /mrindustry/glossary.htm

54
Quantitative Research Methodology (cont)

• A study that aims to quantify attitudes or
  behaviours, measure variables on which they
  hinge, compare, and point out correlations. It is
  most often conducted via a survey on a sampling
  that must be representative so that the results
  can be extrapolated to the entire population
  studied. It requires the development of
  standardised and codifiable measurement
  instruments (structured questionnaires).www.ipsos
  .com/how/Glossary.aspx

55
Quantitative Research Methodology (cont)

• Information about markets and viewers that
  focuses on the relative size of
  audiences.www.shwcase.com/cas_shell/glossary.html
  

56
Quantitative Research Methodology (cont)

• type of research which uses numbers for data and
  statistics to analyze resultswww.conquerchiari.or
  g/Glossary.htm

57
Quantitative Research Methodology (cont)

• A research technique in which scientific,
  concrete, and projectable numerical data that can
  be statistically analyzed is gathered, often from
  large samples also called hard
  data.www.glencoe.com/sec/busadmin/marketing/dp/mk
  tg_resrch/gloss.shtml

58
Quantitative Research Methodology (cont)

• Deals with facts, figures and measurements, and
  produces data which can be readily analysed.
  Measurable data is gathered from a wide range of
  sources, and it is the analysis and
  interpretation of the relationships across this
  data that gives the information researchers are
  looking for. These data are collected using
  numbers, perhaps through answers to
  questionnaires. The numbers are then examined
  using statistical tests to see if the results
  have happened by chance.reuma.rediris.es/omeract/
  docs/OMERACT20Glossary.htm

59
Quantitative Research Methodology (cont)

• The approach advocated by the Positivist School.
  This approach measures social phenomena and
  obtains numerical values which can be analyzed
  statistically. Surveys using structured
  questionnaires and IQ tests are both examples of
  quantitative research.www.sachru.sa.gov.au/pew/gl
  ossary.htm

60
Quantitative Research Methodology (cont)

• Audience research based on anonymous data with
  samples constructed to represent larger
  populations of viewers, listeners and
  readers.freespace.virgin.net/brendan.richards/glo
  ssary/glossary.htm

61
Quantitative Research Methodology (cont)

• the collection of quantified data, for example
  sales figures, demographic data, purchase
  frequency, etc., that can be subjected to
  statistical analysis.wps.pearsoned.co.uk/wps/medi
  a/objects/1452/1487687/glossary/glossary.html

62
Quantitative Research Methodology (cont)

• Research in which answers to questions are
  processed to produce numbers or percentage of
  people in different categories, according to the
  make-up of the sample and structure of the
  questionnaire, giving particular
  answer.www.indiainfoline.com/bisc/jama/jmmq.html

63
Quantitative Research Methodology (cont)

• The data can be collected through structured
  interviews, experiments, or surveys and are
  reported numerically.www.ied.edu.hk/csnsie/ar/cha
  p1/1_glossary.htm

64
Quantitative Research Methodology (cont)

• Quantitative research is the numerical
  representation and manipulation of observations
  for the purpose of describing and explaining the
  phenomena that those observations reflect. It is
  used in a wide variety of natural and social
  sciences, including physics, biology, psychology,
  sociology and geology. en.wikipedia.org/wiki/Quan
  titative_research

65
Qualitative Research Methodology

• The term qualitative research has different
  meanings in different fields, with the social
  science usage the most well-known. In the social
  sciences, qualitative research is often a broad
  term that describes research that focuses on how
  individuals and groups view and understand the
  world and construct meaning out of their
  experiences. It essentially is narrative-oriented
  and uses content analysis methods on selected
  levels of communication content. Other
  researchers consider it simply to be research
  whose goal is not to estimate statistical
  parameters but to generate hypotheses to be
  tested quantitatively.

66
Qualitative Research Methodology

• In statistics, qualitative analysis consists of
  procedures that use categorical data, that is
  data that concern classifications. An important
  subfield is the analysis of dichotomous data
  that is, data which can take only the values 0
  (zero) and 1 (one). These techniques are suitable
  where events or entities can only be counted or
  classified rather than measured on a higher
  level. The techniques themselves are, of course,
  numerically based.

67
Qualitative Research Methodology

• In climate research, qualitative reconstructions
  of past temperatures rely on records of events
  such as frost fairs which indicate periods of
  cold or warmth, but give little or no information
  as to the degree of temperature variation. Other
  indicators dates of harvest, first flowering of
  plants produce information somewhere between
  qualitative and quantitative.

68
Qualitative Research Methodology

• Is concerned with understanding the processes,
  which underlie various behavioural patterns.
  "Qualitative" is primarily concerned with
  "Why"metro.newsmedianet.com.au/home/Glossary.jsp

69
Qualitative Research Methodology (cont)

• Research involving detailed, verbal descriptions
  of characteristics, cases, and settings.
  Qualitative research typically uses observation,
  interviewing, and document review to collect
  data.www.ojp.usdoj.gov/BJA/evaluation/glossary/gl
  ossary_q.htm

70
Qualitative Research Methodology (cont)

• a way to study people or systems by interacting
  with and observing the subjects
  regularly.www.crede.org/tools/glossary.html

71
Qualitative Research Methodology (cont)

• Research that derives data from observation,
  interviews, or verbal interactions and focuses on
  the meanings and interpretations of the
  participants (Holloway and Wheeler,
  1995)healthlinks.washington.edu/howto/measurement
  /glossary/

72
Qualitative Research Methodology (cont)

• A research method that measures information based
  on opinions and values as opposed to statistical
  data.www.communication.gc.ca/glossary.html

73
Qualitative Research Methodology (cont)

• A body of research techniques which seeks
  insights through loosely structured, mainly
  verbal data rather than measurements. Analysis is
  interpretative, subjective, impressionistic and
  diagnostic.www.mrs.org.uk/mrindustry/glossary.htm

74
Qualitative Research Methodology (cont)

• An exploratory study (to explore an unknown
  sector, identify the main dimensions of a
  problem, draw assumptions, understand
  motivations) or operational study based on
  in-depth analysis of interviewee responses (in a
  group or individually), typically in what's known
  as "focus groups." It most often deals with a
  restricted sample of individuals that does not
  necessarily need to be representative. It may be
  the preliminary phase of a quantitative study or
  stand alone research.www.ipsos.com/how/Glossary.a
  spx

75
Qualitative Research Methodology (cont)

• Information about markets and viewers that
  focuses on attitudes, behaviors and demographic
  composition.www.shwcase.com/cas_shell/glossary.ht
  ml

76
Qualitative Research Methodology (cont)

• Research that gathers information, which is
  varied, in-depth and rich. The information sought
  is about how something is experienced and not
  specifically about facts and figures. Information
  from qualitative research is often more difficult
  to interpret, partly because it cannot be
  measured'. The emphasis is on the quality and
  depth of information. These data might be
  collected in the form of in-depth interviews with
  patients. ...reuma.rediris.es/omeract/docs/OMERAC
  T20Glossary.htm

77
Qualitative Research Methodology (cont)

• Audience research based on discussion groups or
  one-to-one interviews with interaction between
  researcher and subject.freespace.virgin.net/brend
  an.richards/glossary/glossary.htm

78
Qualitative Research Methodology (cont)

• Data can be collected through open-ended
  interview, review of documents and artifacts,
  participant observations, or practice.www.ied.edu
  .hk/csnsie/ar/chap1/1_glossary.htm

79
Qualitative Research Methodology (cont)

• Qualitative research involves an indepth
  understanding of human behaviour and the reasons
  that govern human behaviour. Unlike quantitative
  research, qualitative research relies on reasons
  behind various aspects of behaviour. Simply put,
  it investigates the WHY and HOW of decision
  making, as compared to WHAT, WHERE AND WHEN of
  quantitative research. Hence, the need is for
  smaller but focussed samples rather than large
  and random samples. http//en.wikipedia.org/wiki/Q
  ualitative_research

80
Methodology Comparison

• Quantitative
• Explanation, prediction
• Test theories
• Known variables
• Large sample
• Standardized instruments
• Deductive

• Qualitative
• Explanation, description
• Build theories
• Unknown variables
• Small sample
• Observations, interviews
• Inductive

81
Methodology Comparison

• Following are the differences between qualitative
  and quantitative research
• 1. Qualitative develops theory whereas
  quantitative tests theory 2. Qualitative
  describes meaning or discovery whereas
  quantitative establishes relationship or
  causation 3. In Qualitative, researcher is a part
  of the data gathering process whereas in
  quantitative, the researcher is an independent
  entity 4. Qualitative uses communication and
  observation whereas quantitative uses instruments
  5. Qualitative means unstructured data collection
  whereas quantitative means structured data
  collection
• http//en.wikipedia.org/wiki/Qualitative_research

82
References

• Leedy P. D. and Ormrod J. E., Practical Research
  Planning and Design, 7th Edition. 2001.

83
Useful Websites

• http//computer.org
• http//www.acm.org

84
Questions?