Qualitative Document Analysis

1
Qualitative Document Analysis
In this session, I adopt a rather eclectic view
of document. In addition to typical sources
(e.g. media reports, government papers, minutes
of meetings, company reports), I include
documents that are read as part of the literature
review and also the working documents that become
your thesis. My rationale for this is that
similar issues and skills are involved in the
analysis of all of them.

• Hugh Willmott
• Research Professor in Organizational Analysis
• Cardiff Business School
• Home Page http//dspace.dial.pipex.com/town/clos
  e/hr22/hcwhome

2
Outline of Session

• Analysing secondary sources
• Analysing literature
• Analysing your text

Textual Analysis. Textual analysis involves
mediation between the frame of reference of the
researcher and those who produced the text. The
aim of this dialogue is to move within the
hermeneutic circle in which we comprehend a
text by understanding that frame of reference
from which it was produced, and appreciate that
frame of reference by understanding the text. The
researchers frame of reference becomes the
spring board from which the circle is entered,
and so the circle reaches back to encompass the
dialogue between the researcher and the text (J.
Scott (1990), A Matter of Record Documentary
Sources in Social Research, Cambridge Polity
cited in Mason, 2002, p. 110)
Exercise Consider how reading a journal article
on your research topic illustrates the idea of
mediation and dialogue
3
Forms of Textual Analysis Quantitative and
Qualitative

• Quantitative Researchers try to analyse written
  material in a way that will produce reliable
  evidence about a large sample. Their favoured
  method is content analysis in which the
  researcher establishes a set of categories and
  then counts the number of instances that fall
  into each category. The crucial requirement is
  that the categories are sufficiently precise to
  enable different coders to arrive at the same
  results when the same body of material (e.g.
  newspaper headlines) is examined
• Qualitative Researchers analyse small numbers of
  texts and documents for a very different purpose.
  The aim is to understand the participants
  categories and see how these are used in concrete
  activities like telling stories, assembling files
  or describing family life. The constructionist
  orientation of many qualitative researchers thus
  means that they are more concerned with the
  process through which texts depict reality than
  with whether such texts contain true of false
  statements
• D. Silverman (2005), 2nd ed., Doing Qualitative
  Research A Practical Handbook, London Sage, p
  160

4
Analysing secondary sources
5
Secondary Sources An Alternative to Observation
and Questioning

• Secondary sources are those which already exist,
  as contrasted with primary sources which you
  collect/ generate/ construct
• Typically, secondary sources include media
  reports, company histories and records, minutes
  of meetings, diaries, letters, photographs, etc
  Why exclude books, articles etc from this list?
• Secondary sources are examined in order to gain
  knowledge of, and insight into, phenomenon under
  study. Style as well as content may be analysed
• Secondary sources are particularly valuable when
  situations or events cannot be investigated by
  direct observation or questioning, or where
  reliance upon the public record alone (rather
  than, say, interviewee accounts) is preferred

6
Types of Documents

• Personal (e.g. diaries) v. Official (e.g. press
  releases)
• Restricted access (e.g. minutes) v. open access
  (e.g. company reports)

Organizational documents may include company
reports, memoranda, manuals, policy proposals,
website information, accounting records, strategy
documents, sales brochures, , etc
Government documents may include departmental
reports, parliamentary reports, etc. Much of this
is becoming more readily accessible through
websites and freedom of information act.
7
Characteristics of Documents
Source J.Mason (2002), 2nd ed. Qualitative
Researching, London Sage, Ch 6

• They are constructed in particular contexts, by
  particular people, with particular purposes, and
  with consequences intended and unintended, p.
  110
• The researcher may wish to
• investigate why the documents were prepared, who
  prepared them, under what conditions and
  according to what conventions
• know how the documents have been received, and
  what they have been used for

8
Criteria for Evaluating Documentary Sources (1)
See Alan Thomas, Research Skills for Management
Studies, London Sage, 2004, pp 197 et seq

• Is it authentic?

The contents of the statements of corporate
leaders (e.g. CEOs, Vice-Chancellors) that appear
in annual reports might be analysed as part of a
study of their communication styles. But can it
be assumed that the leader, rather than a member
of the PR office, has drafted the statement? Does
it matter?

• Is it credible?

Is the author of the document an impartial and/or
expert witness? Consider the annual report
example given above. When the statement is
optimistic, is the organization doing well, or is
the statement intended to reassure nervous
investors or current and potential stakeholders
with an attempt to instil confidence? See next
slide
9
An Example Performance Records

• accusations of massaging of performance
  records have been levelled at the police service,
  the health service and elsewhere. Falsification
  of corporate accounts has given rise to some
  well-publicised scandals.Business leaders can
  rarely be considered as disinterested reporters
  of their careers and their organizational
  experiences. The analyst must assess the
  likelihood of their having overdramatized their
  role in the events they recount and must be
  sensitive to the social and political perspective
  from which they view the world
• (Alan Thomas, Research Skills for Management
  Studies, London Sage, 2004, p 200, emphasis
  added)
• On what basis is the assessment of the analyst
  to be made?
• How does the analyst develop sensitivity to the
  social and political perspective?
• What about the social and political
  perspective of the researcher?

10
Criteria for Evaluating Documentary Sources (2)
See Alan Thomas, Research Skills for Management
Studies, London Sage, 2004, pp 197 et seq

• Is it representative?

Has there been selectivity in what is recorded
(e.g. successful ventures) in the documents, and
what is preserved (e.g. minutes of meetings)? How
much confidence can be placed upon the integrity
of the data set? What about corroboration?

• What does it mean?

What does the documents content mean to its
author and/or its intended reader? To what
extent, for example, is it an articulation of
current conventions? Are these questions any less
relevant for other documents, such as journal
articles and thesis chapter drafts? See next slide
11
What Meaning is to be Attributed to Documents?

• it is important to realize that documentary
  reality does not consist of descriptions of the
  social world that can be used directly as
  evidence about it. One certainly cannot assume
  that documentary accounts are accurate
  portrayals in that sense. Rather, they construct
  their own kinds of reality. It is, therefore,
  important to approach them as texts. Texts are
  constructed according to conventions that are
  themselves part of a documentary reality. Hence,
  rather than ask whether an account is true, or
  whether it can be uses as valid evidence about
  a setting, it is more fruitful to ask ourselves
  questions about the form and function of texts
  themselves
• (P. Atkinson and A. Coffey, Analysing
  Documentary Realities in D. Silverman (2004),
  2nd ed., Qualitative Research Theory, Method and
  Practice, p. 73
• See also extract from Atkinson and Coffeys
  article. Click here or see next slide

Exercise Can you summarise Atkinson and
Coffreys stance in your own words? Can you
provide a brief illustration, using a document
that is to hand? Does the same argument apply to
their text?
12
(No Transcript)
13
(No Transcript)
14
Analysing literature
15
Reviewing Literature What is the Point? Three
Views
See Alan Thomas, Research Skills for Management
Studies, London Sage, 2004, pp 72 et seq

• The purpose of a literature review is to
  establish the current state of knowledge in the
  field. It will therefore be a significant
  contribution to the dissertation or thesis and
  will usually be included in it as a prelude to
  the report of the empirical work Thomas, 2004,
  p. 73, emphasis added

The purpose of the literature review is to
demonstrate skills in literature searching to
show command of the subject area and
understanding of the problem to justify the
research topic, design and methodology C. Hart
(1998), Doing a Literature Review Releasing the
Social Science Imagination, London Sage, p13,
emphasis added
I object to the practice of simply backing up
with a truckload of stuff and dumping it on
unsuspecting readers, which seems to me what most
traditional reviews accomplish. That is more
likely to create an obstacle that gets in the
way, rather than paves the way, to reporting what
your have to contributeBy all means, flag
important citations to the work of others. But do
so sparingly and only as the references are
critical to helping you to analyse and to situate
your problem and your research within some
broader context H. Wolcott (2001), 2nd ed.,
Writing Up Qualitative Research, London Sage, p
74 and 75, emphasis added
16
Reviewing Literature Some Key Elements

• Your research may comprise a number of areas. A
  review must pay adequate attention to each of
  these
• Your research methodology forms part of a
  literature that will require some critical
  reviewing
• You should seek to achieve a tight coupling
  between your empirical work and your literature
  review.
• The areas of literature that you review must be
  shown to be directly relevant to the topic of,
  and approach to, your empirical study
• Avoid mere descriptive summarising be focused
  and critical

17
Reviewing the Literature Two Types of Review

• Exploratory Review
• examines the extensiveness (size) of the
  literature
• examines how field has been addressed (scope)
  angles, disciplines, themes, theories, key
  contributions?
• not an exhaustive treatment of sources
• Synoptic Review
• concise and thorough review of all material that
  is assessed to be relevant to your study
• Topic content and methodological approaches
• critical evaluation of established literature in
  terms of how it could be extended both in content
  and method
• identification and exploration of gaps

18
Reviewing Literature Searching and Handling

• Sources most relevant for preparation of reviews
• Review papers or books that survey a topic (may
  indicate a dominant approach rather than what is
  relevant for you)
• Star papers that are repeatedly cited (may
  indicate a dominant approach rather than what is
  relevant for you)
• Model papers (may also be star papers that are
  deemed exemplary within their field and may
  provide a benchmark for your own study)
• Recommended literature from experts in the
  field (likely to be highly selective, so be
  cautious)
• Electronic data bases (watch out, as they can be
  overwhelming and incomplete)

19
Approaching Texts Kinds of Reading

• Rapid scanning form an impression of the
  potential relevance of the paper high, low,
  doubtful or potential
• Selective reading paying attention to only
  those sections, chapters or pages that contain
  relevant material
• Top and tailing reading of abstract,
  introduction and conclusion only, perhaps as a
  basis for deciding whether to undertake rapid
  scanning or close reading of the entire paper
• Notes/ Bibliography reading reading the
  references or the footnotes to gain a quick sense
  of where the author is coming from. Sometimes
  the most revealing and insightful points can be
  tucked away in the references
• Close reading attentive and repeated reading

Tip. It may be beneficial initially to undertake
rapid scanning of all sources, even when your
intention is to undertake a close reading
20
Analysing Your Text
21
Some Pertinent Questions and Approaches to Writing

• How do I make my text accessible and convincing?
• What kinds of arguments can I build from my data?
• Where does theory come into my argument?
• What is the purpose of the argument?

Arguing Evidentially supported by evidence how
do you justify the relevance and credibility of
the evidence presented?Arguing Interpretively
shown to be meaningful and reasonable what
standards of meaningfulness or reasonableness are
you invoking?Arguing Illustratively
demonstrate the case through an appealing or
persuasive exampleArguing Reflexively
acknowledging the problems of argumentation but
commending a particular interpretation on the
basis of the value of its particular contribution
Adapted from J. Mason (2002), 2nd ed.,
Qualitative Researching, London Sage, Ch 9
22
Producing Text Starting and Editing

• To give yourself the best chance of producing
  something worthwhile, create the conditions that
  are most conducive to writing. Exclude or
  minimise possible sources of distraction
• To get you started, a good, focussing sentence to
  open up with is The purpose of this
  chapter/paper/thesisBut, if this beginning is
  too demanding, just start your draft with
  whatever is possible, even if its relevance is
  not immediately apparent. Often introductions are
  written after the substantive sections of a text.
• To avoid excessive redundancy and meandering, it
  may be useful to identify a set of key points and
  arguments that you want to make or even create a
  table of contents that sets out on one page the
  intended structure of the text. It is likely that
  this will be elaborated and revised in the
  process of writing perhaps as it becomes clear
  that the material requires two chapters, not one.
• Review your document regularly to assess whether
  you are still on track, or perhaps to gain a
  clearer sense of the (new) track.
• Be sure to give emphasis and space to key points
  where you make a distinctive contribution

23
Writing as Analysing Text (1)
Source H. Wolcott (2001), 2nd ed., Writing Up
Research, London Sage

• Think of writing as an integral part of the
  research process, not as something that comes
  with writing up

Hear this. You cannot begin writing early
enough. And yes I really mean it. Would that mean
that someone might write a first draft before
venturing into the field to begin observations or
interviews? Absolutely. H. Wolcott (2001), p. 21
Writers who indulge themselves by waiting until
their thoughts are clear runt the risk of never
beginning at allWriting is not only a great way
to discover what we are thinking, it is also a
way to uncover lacunae in our thinking (ibid 22)

• When you are writing your drafts of chapters (or
  conference papers) you will be working on a
  document. In effect, you will be analysing its
  content and style in relation to the (shifting)
  objectives you have ascribed to it
• Writing can be especially anxiety-provoking if
  you are hoping/expecting to get it right first
  time. Just get it down. Worry about getting it
  right later.
• Many people work through numerous drafts. That
  means undertaking recurrent analyses and
  critiques of the text-in-progress

24
Writing as Analysing Text (2) Tightening Up

• Reviewing Content and Style
• content is paramount but is rendered accessible
  by style be mindful of your target audience
• invite others to review your work present it at
  seminars,etc subject your work to peer review
• Revising and Editing
• revision addresses content focus,
  qualification, major cuts to remove diversions,
  expansions to incorporate complexity etc
• Logical sequence systematic development
• editing addresses style economy of expression
  avoiding repetitions and discrepancies
  paraphrasing of quotations restructuring
  sentences use of punctuation, etc
• Unnecessary words, passive voice, overused
  phrases, overused punctuation or other devices

25
Revisiting the Evaluation of Documents (see
earlier slide)

• Is it authentic?
• to what extent do you claim ownership of the
  document (e.g. thesis)?
• Is it credible?
• on what basis do you assert and defend the
  credibility of the document?
• Is it representative?
• in what sense is it representative of a passable
  thesis?
• What does it mean?
• What is the significance of the document, and for
  whom?

26
(For Week 8)Institutions and Technology Using
and Producing Documents for Analysis

• Questions for discussion
• How does the analysis of documents figure in this
  paper?
• What kind of use is made of documents?
• What other kind of textual analysis might have
  been conducted?
• How do you evaluate the authenticity and
  credibility of the documentary sources and the
  article itself as a document?
• In what respects can the article be viewed as
  un/representative of a contribution to an
  academic journal?
• What is the significance or meaning of the
  article as a document, and for whom?

T.Bridgman and H.Willmott (2006), Institutions
and Technology Frameworks for Understanding
Organizational Change The Case of a Major ICT
Outsourcing Contract, The Journal of Applied
Behavioural Science, 42, 1 110-126 Click here
for access to the article.
27
Final Thought In a Fog?

• It is not unusual to begin in a fog. A certain
  amount of wandering around is inevitable before
  it is possible to find ones bearings and gain a
  sense of direction. It is necessary to tolerate
  uncertainty at the beginning of a project,
  starting out with a broad view, scanning for a
  range of possibilities and then narrowing down to
  a specific focus (Thomas, 2004, p. 70)

28
Additional Recommended Reading

• J. Scott (1990) A Matter of Record Documentary
  Sources in Social Research, Cambridge Polity
  Press
• L. Pryor (2003), Using Documents in Social
  Research, London Sage
• A. Fink (1998), Conducting Research Literature
  Reviews From Paper to the Internet, London
  Sage
• C.Hart (1998), Doing a Literature Review
  Releasing the Social Science Imagination, London
  Sage
• C. Hart (2001), Doing a Literature Search A
  Comprehensive Guide to the Social Sciences,
  London Sage
• L.Richardson and E.A. St Pierre (2005), Writing
  A Method of Inquiry in N.K. Denzin and Y.S.
  Lincoln, eds., Handbook of Qualitative Research,
  London Sage