Design%20Science%20and%20Critical%20Realism%20Some%20Methodological%20Issue

1
Design Science and Critical Realism Some
Methodological Issue

• Greg Hill, Monash University
• CBIDSR Workshop, 20/10/05

2
Overview

• Historical Development and Rationale
• Structure of Design Science Research
• Evaluation in Design Science
• Guidelines for Assessment

3
Overview

• Philosophy of Design Science Research
• Ideas from Critical Realism
• Critical Realist Evaluation
• Implications for Information Systems

4
Historical Development

• Natural versus artificial sciences
• truth, necessity, analysis, descriptive,
  predictive
• utility, contingency, synthesis, prescriptive
• Focus on artefacts, goals and environment
• Simulation and Imitation
• Design distinguishes professions

5
Historical Development

• the natural sciences have almost driven the
  sciences of the artificial from professional
  school curricula ...
• Engineering schools have become schools of
  physics and mathematics
• professional schools hanker after
  respectability (Simon, 1969)
• Engineering, Business and Medicine
• Law, Journalism and Library Science
• (Computer Science and Management Science)

6
Structure of Research

• Process View Vaishnavi, V. and Kuechler, W.
  (2004/5). Design Research in Information
  Systems URL http//www.isworld.org/Researchdesig
  n/drisISworld.htm

7
Structure of Research

• Knowledge View (Hevner et al, MISQ, 2004)

8
Evaluation Methods

• Observational
• case study, field study
• Analytical
• static, architecture, optimisation, dynamic
• Experimental
• controlled experiment, simulation

9
Evaluation Methods

• Testing
• black box, white box
• Descriptive
• informed argument, scenarios

10
Guidelines for Assessment

• Design as an Artefact
• Problem Relevance
• Design Evaluation
• Research Contribution

11
Guidelines for Assessment

• Research Rigour
• Design as Search
• Design as Communication

12
Philosophy of Design Science

• Essentially instrumentalist and pragmatist
• accepts natural sciences as law-generator
• claims to knowledge assessed on usefulness
• But spans socio-technical systems
• aeroplanes, computers, mobile phones ...
• organisations, markets, families ?

13
Philosophy of Design Science

• Key question for design of socio-technical
  systems (like information systems)
• Can the philosophies and methods of logical
  positivism work for social science
• Do reasons have causal powers?
• Does poverty cause crime?

14
Ideas from Critical Realism

• Kicked off by Roy Bhaskar (1975)
• Transcendental Realism - attempts to unify
  positivism and interpretivism
• Naturalism - the idea the social and natural
  phenomena can be studied the same way
• Proposes three ontological domains
• real
• actual
• empirical

15
Ideas from Critical Realism

• Rejects social atomism / methodological
  individualism - a form of social reductionism
• Rejects Durkheim (agency -gt structure)
• Rejects Weber (structure -gt agency)
• Proposes an interplay (structure lt-gt agency)

16
Ideas from Critical Realism

• He found God. Worst of all it was a very down
  market god, nothing more than your common or
  garden New Age variety, the type readily
  available at any incense saturated shop
  frequented by a Shirley MacLaine or Nancy
  Reagan. Gary MacLennan.

17
Ideas from Critical Realism

• Put very simply, a central feature of realism
  is its attempt to preserve a scientific
  attitude towards social analysis at the same time
  as recognising the importance of actors meanings
  and in some way incorporating them in research.
  (Layder, 1993)

18
Ideas from Critical Realism

• Analogy and metaphor
• hypothesis formulation
• Patterns and triggers
• demi-regularities, enablers/disablers
• C-M-O configurations
• Context - Mechanism - Outcomes

19
Ideas from Critical Realism

• outcomes cannot be explained in
  isolation rather, they can only be explained in
  the sense of a mechanism that is introduced to
  effect change in constellation of their
  mechanisms and structure, embedded in the context
  of pre-existing historical, economic, cultural,
  social and other conditions. This process of
  explanation, known as retroduction, enables the
  realist inquirer to investigate the potential
  causal mechanisms and the conditions under which
  certain outcome will or will not be realised.
  (Kazi, 2003)

20
Critical Realist Evaluation in Design Science
Research

• Goal of evaluation phase is to understand the
  C-M-O configurations (explanatory)
• What works, for whom, in what circumstances?
• Scenarios, case studies, simulations
• Three boxes of evaluation
• black, grey and white (clear)
• making the box greyer

21
Implications for IS

• What is the role for IS researchers?
• Business needs?
• Reference disciplines?
• Consultancies?
• Multi-methodology research?

22
Implications for IS

• What is the role for IS theory?
• CSF -gt CMO?
• Design theories?
• Evaluation theories?

23
Implications for IS

• I agree that it works in practice. But how can
  we be certain that it will work in theory? --
  Attributed to a professor at École Normale
  Supériere.