Starting to Audit Documents in the Engineering Domain

1
Starting to Audit Documents in the Engineering
Domain
Peter J Wild, Steve Culley, Chris McMahon, Mansur
Darlington, Shaofeng Liu
Innovative Manufacturing Research
Centre Department of Mechanical Engineering
University of Bath
2
Presentation Overview

• Project Context
• Introduction
• Study Setting and Context
• Interim Findings
• Modelling Framework
• Conclusions and Future Work

3
Project Context

• ICID Information Content in Documents
• Three strand project
• Stream A Document Structures Decomposition
  Schemes
• Stream C Computational Methods Mark-up
  Languages

• Stream B Engineers Exploration Behaviour
  Document Use Patterns
• Document Audits
• Diary Studies
• Observational work
• Leading to evaluation / experiments using
• Document structures (Stream A)
• Scenarios of use (Stream B)
• Tools (Stream C)

4
Document Studies and Auditing

• Interest in
• how people use, generate, save, and search, paper
  and electronic based documents
• auditing information flows
• Attempting to pull this into a coherent approach
  to auditing engineering documents
• Starting questions for the auditing process
• What do they have use and generate

5
Audits should provide data about

• Varieties and types of engineering documents
• Strengths and weakness of document manifestations
  
• Patterns of document use within different types
  of engineering company (e.g. SME, design,
  manufacturing)

6
Case Study Setting and Context

• TrollCo
• Engineering Design and Manufacturing company
  based in Wiltshire
• General engineering work
• e.g. nuclear, railway, highways
• Design and manufacture of their own product line
• Culture of pride in the quality of their work
• We are the Rolls Royce of..
• Pressures of the market
• Trouble is people want Rolls Royce at Ford
  prices
• British Standards Institute (BSI) quality
  certified

7
Research Approach

• Engineers would volunteer stories, scenarios,
  information about how things were carried out
• Lacking formal interviews
• Informal interviews, and scenario gathering
• Pen and paper observations
• Iterative discussions with TrollCos engineers
  have taken place
• Audits of physical computer based documents

8
What Do They Have?

• 1000s Documents
• 250 Types
• Reflect a variety of concerns, manifestations and
  life-spans
• Expected and TrollCo specific
• Cannot be easily wedged into a simple
  classification (e.g. ISO 9001)
• We are exploring classification methods

9
Using and Generating

• Use and generate large number of documents.
• Mostly developed by the quality management system
  
• Many generic to product and engineering services
  division
• Distinction between routine and exception
  documents

10
Product Division Documents (Order)
11
TrollCo Consider Their Document Usage Chaotic!

• Documents have
• needed information
• layout and meaning are comprehensible
• Document chaos appears to stem from
• ambiguity in orders
• dealing with multiple instances of orders
• the variety inherent in the product line and
  customer needs
• Design Documents probably are chaotic

12
An Unused Catalogue Taxonomy

• Files marked with a category and subcategory
• Taxonomy was locally developed and reflected the
  forms of material that the engineers used, and
  still use today
• It was apparent that it was neglected

13
Contemporary storage situation
14
And
15
Why?

• Demise of the ordered catalogue store and
  taxonomy due to
• Time
• Location
• Supplier management strategy

16
Paper vs. Electronic Documents / Files

• Minimal use of bookmarks
• Email client file structure was default
• Naming and hierarchy conventions in files are
  variable and inconsistent between electronic and
  paper manifestations

• Struck by paucity of manifestation, binding and
  container mechanisms in computer based file
  systems

17
Paper vs. Electronic Documents / Files

• In future audits we will investigate further the
  semantics ascribed to document bindings
• More generally we could investigate more
  elaborate binding mechanisms in computer based
  documents (see also Rogers 1996)

18
A Modelling Framework?

• Activity being undertaken with documents is
  essential to understanding why they exist and how
  they are used
• Non-partisan approach to how we model the work
  aspects in analysis of data from document audits
• Analysing the context of documents at the five
  levels of Culture, Flow, Sequence, Artifact, and
  Physical Character have been influential in
  framing our understanding

19
Culture Flow and Sequence
20
Artefact and Physical Properties
21
Conclusions

• Work in progress
• Interim findings
• Have Use and Generate
• Catalogue Taxonomy and Storage
• Electronic and Physical Manifestations
• Focussing on 3 questions of have, use, and
  generate, and five levels of analysis Culture,
  Flow, Sequence, Artifact, and Physical Location
• Are Have Use and Generate always that
  distinguishable?
• Do engineers think well in these terms?

22
Future Work

• Arranging to audit documents in other companies.
  
• Particular focus in future audits is adding
  methodological rigour to the data gathering
• Also using auditing software tools to
  unobtrusively gather data and speed up some
  analysis, and provide more robust statistical
  comparisons between organisations.

23
Questions and Discussion