Research in the real world: the users dilemma

1
Research in the real world the users dilemma

• Dr Gill Green

2
Overview of the Lecture

• Context for the examination of research
  approaches
• Examine aspects of the Qualitative Research
  Defining, Attributes, Features Types
• Examine aspects of the Quantitative Research
  Defining, Attributes, Features Types
• Reflect and summarise on each approach

3
The users dilemma

• How do users get what they want?
• Traditional view of software development
• Analysis specification-development-implement-sig
  noff
• Happy users
• So why do researchers report that 70 of system
  implementations fail
• How do users know what they want?
• The Marco Polo effect
• How do you describe something you have never seen
  before
• The gambler effect
• How do you speculate how you may like to do
  things in the future
• The tigger effect
• How easy is it to ask for the wrong thing

4
Some hard words

• Ontology What is
• epistemology what it means to know
• Why is this important?
• You need to know for yourself how you interpret
  your world?

5
Theoretical perspectives and what they teach us

• Positivist
• Reality consists of what is available to the
  senses
• Inquiry should be based upon scientific
  observation and empirical action
• Principles are shared between Natural and human
  sciences and deal with facts not values
• Interpretivist
• Reality is a shifting state culturally derived
  and historically situated
• Inquiry deals with the actions of individuals in
  social settings
• Principles suggest the emergence of unique
  individual qualitative aspects

6
What is Qualitative Research

• Qualitative research is a process of enquiry
  that draws data from the context in which events
  occur, in an attempt to describe these
  occurrences, as a means of determining the
  process in which events are embedded and the
  perspectives of those participating in the
  events, using induction to derive possible
  explanations based on observed phenomena.
• (Gorman Clayton, 1997)

7
What happens in Qualitative Research?

• Data taken from context in which events occur
• Data collection first hand
• Attempt to describe occurrences
• Focus on process not snapshot
• Immersion leading to insight
• Induction

8
Qualitative Research Induction

• Use of bottom-up approach to analyse and
  interpret data
• Research based on observed data
• Grounded theory
• that is based on established theories

9
Qualitative Research Attributes 1

• Assumptions
• social construction of reality
• primacy of subject matter
• complexity of variables
• difficulty in measuring variables
• Purpose
• contextualisation
• interpretation
• understanding participant perspectives

10
Qualitative Research Attributes 2

• Approach
• Theory generalising
• Emergence and portrayal
• Researcher as instrument
• Naturalistic
• Inductive
• Pattern Seeking
• Looking for pluralism and complexity
• Descriptive
• Researcher Role
• personal involvement and partiality
• empathetic understanding

11
Key features of Qualitative Research (Hittleman
Simon)

• Data is collected within its natural setting.
  Main data collection instruments are the
  researchers themselves
• Data are not numerical
• Focus on the process of an activity, not just its
  outcomes
• Data analysed in non-numerical manner. Outcomes
  generate debate rather than verifying a predicted
  outcome

12
Qualitative Research Why is it important in IT

• Many of techniques and methods can be applied to
  the requirements engineering process
• Helps to place user at centre of design process
• Enables triangulation with quantitative methods

13
Doing Qualitative Research

• Many ways of collecting and analysing data
• Historical
• Correlational
• Developmental
• Descriptive
• ...

14
Qualitative Research Overview of Techniques

• Observation
• Interviewing
• Questionnaires
• Group Discussion
• Historical Study
• Content Analysis
• Ethnographical Research

15
Qualitative Research Summary

• Increased knowledge of qualitative research
• Awareness of qualitative approaches relevance to
  computing

16
Quantitative Research What is it?

• The aim of quantitative research is not simply to
  state that something has a relationship with
  something else, but to state causality

17
Quantitative Research

• Focuses on numerical and statistical data
• Positivist approach
• Recognising only positive/measurable facts and
  observable phenomena
• Empirical scientific approach
• Relying on experimentation and not untested
  theory
• Searches for causality and effect

18
Quantitative Research Deduction

• Top-down approach
• The inferring of particular instances from a
  general law
• Working something out from something else -
  Sherlock Holmes style

19
Attributes of Quantitative Research 1

• Assumptions
• objective reality of social facts
• primacy of method
• possible to identify variables
• possible to measure variables
• Purpose
• generalisation
• prediction
• causal explanation

20
Attributes of Quantitative Enquiry 2

• Approach
• Hypothesis based
• Manipulation and Control
• Uses formal instruments
• Experimentation
• Deductive
• Component analysis
• Seeking norms and consensus
• Reducing data to numerical indices
• Researcher Role
• detachment and impartiality
• objective portrayal

21
Features of Quantitative Research 1

• Tests for cause and effect
• X causes Z to happen
• Y does not cause Z to happen
• Not simply that something has a relationship with
  something else
• Involves empirical studies
• Uses numerical and statistical techniques

22
Features of Quantitative Research 2

• Assume primacy
• Researcher defines the research activity
• Relationships are measured
• Causal explanations are made

23
Quantitative Research Descriptive Statistics

• Allows summaries of large quantities of
  information
• Should be easily comprehensible for reader
• Presentation is vital
• long strings of numbers
• tables, charts, graphs
• numerical techniques
• concise, appropriate text

24
Quantitative Research Inferential Statistics

• Procedures for making generalisations about
  characteristics of a population based on
  information taken from that population
• Powerful
• estimation
• hypothesis testing
• Methods and rules for organising and interpreting
  data

25
Quantitative Research Why is it important in IT

• Establishes metrics
• Report on process and system efficiency concerns
• Predict outcomes from developments
• Improve development and operational processes
• Basis for managing risk
• Analysis of incidents
• Identify causal relationships
• Plan

26
Quantitative Research Summary

• Quantitative research is based on scientific
  inquiry
• Offers numerous techniques for data analysis
• Searching for causality and prediction

27
Some questions to answer for next week

• Can you identify your epistemological stance?
• Have you identified a theoretical perspective
• Is your approach deductive or inductive
• Have you considered research methodology