QA For Metadata: The QA Focus Methodology

1
QA For MetadataThe QA Focus Methodology

• Brian Kelly, UKOLN
• Supported by
• Amanda Closier, UKOLN and
• Gareth Knight, AHDS

This session addresses the importance of quality
assurance for metadata and describes the QA
methodology developed by JISC-funded QA Focus
project.
http//www.ukoln.ac.uk/qa-focus/
2
Background

• QA Focus has sought to
• Help ensure the functionality, wide accessibility
  and interoperability of JISC-funded projects
• Provide advice and support materials to projects
• Recognise the limited resources available to
  projects
• Develop a light-weight methodology which can help
  projects achieve interoperability and compliance
  with appropriate standards and best practices
• Develop a self-assessment methodology which
  recognises the difficulties of external checking

3
Why Do Things Go Wrong?

• Why do project deliverables fail to be
• Functional ? Widely accessible
• Interoperable ? Easily deployed into service
• Reasons include
• Failure to implement standards best practices
  (perhaps due to a lack of understanding)
• Inappropriate standard / best practice used
• Use of an inappropriate architecture (tools,
  workflow, )
• Failure to check that standards best practices
  are being used correctly
• Failure to understand limitations of checking
  tools
• Having over-optimistic intentions

4
QA Focus Methodology

• The QA Focus methodology
• Based on mainstream QA principles
• Projects should develop written policies on
  standards, best practices, etc.
• Projects should deploy systematic procedures for
  ensuring the policies are implemented
• Projects should share experiences (good and bad)
• Note that
• The policies should be achievable
• The procedures may lead to changes in systems,
  workflow, etc. if non-compliance spotted
• Audit trails should be kept to help identify
  trends, provide documented evidence of best
  practices, etc.

5
Policies

• How do you know what you should do if you don't
  have documented polices?

Please note that this is a template you can add
additional headings
6
Applying QA Accessibility (1)

• Policy The Web site will strive to attain WAI A
  guidelines.
• Consistent accessibility shortcuts will be used.
• An accessibility policy will be published.
• Architecture The Web site will be based on XHTML
  templates which comply with WAI A.
• Monitoring New and updated pages will be
  validated using ,bobby. A monthly batch checker
  will be used and audit reports published (to
  enable any trends to be spotted).
• Exceptions A list of permitted exceptions will
  be provided.

QA Focus Methodology
7
Applying QA Accessibility (2)

• Policy The organisation has no accessibility
  policy
• Authors are free to implement their own
  accessibility shortcuts (if at all)
• Architecture No centralised policy covering
  authoring tools or architecture will be provided
• Monitoring No monitoring will be carried out

If you don't have a written policy, the unwritten
policy may well be frightening!
8
Who Checks Compliance?

• QA Focus recommendations
• Projects should implement appropriate QA
  procedures
• Projects may find QA Focus support materials
  helpful (including case studies)
• The QA Focus toolkits may be used to help ensure
  appropriate areas are addressed
• Compliance issues should be addressed within
  projects with the expectation that significant
  discrepancies are brought to attention of funders
• External auditing of compliance is not felt to be
  appropriate for the JISC sector / culture

9
Applications To Metadata

• For metadata there is a need to
• Clarify the purpose of your metadata (embedded
  metadata in Web pages won't put you at the top of
  Google)
• Have an appropriate architecture for creating and
  managing your metadata - metadata rots
• Ensure you have appropriate cataloguing rules
• Address interoperability with others and not just
  your own functionality this is hard
• Provide appropriate training, documentation, etc.
• Have appropriate mechanisms for ensuring above
  work correctly
• Document the above for yourselves, for new
  staff and potentially for 3rd parties (funders,
  service providers,

10
Example 1 ACRONYM Tag

• Simple example which illustrates several points
• Why bother (not supported in IE)?
• Difference between Acronymand Abbreviation
• Policy on content (full stops, apostrophes, US
  vs UK English, )
• Error checking when the metadata is not
  displayed

• Solution
• Simple policy produced.
• Used Tom Heath's acronym harvester to create
  automated glossary as (a) value-added service and
  (b) to check for errors.
• Policy and approach documented

11
Example 2 Web Metadata

• QA Focus Web site
• Uses PHP to process simple metadata (e.g.
  author"Brian Kelly", area "documents",
  keywords"xxx")
• Used for processing (area used to pull in
  stylesheet) as well as resource discovery
• Architecture isn't ideal (backend database
  better) but QA is about managing what you've
  got
• How do we detect errors?

• Solution
• HTML validation detects incorrect PHP syntax
  errors
• Visual inspection for correct area values
• Keywords description copied to hard copy
  document, which allowed for visual checking (and
  there were errors)
• Policy documented (shortly)

12
What Needs To Be Addressed?

• You'll need to address
• Purpose of metadata
• Selection of standards, conventions,
• Implementation architecture(s)
• Work flow
• Project outputs
• Note
• Some aspects will be already chosen or outside of
  you control (e.g. FAIR projects will use OAI, )

13
Help Is Available

• Briefing Documents
• See lthttp//www.ukoln.ac.uk/qa-focus/documents/bri
  efings/metadatagt
• Case Studies
• See lthttp//www.ukoln.ac.uk/qa-focus/documents/cas
  e-studies/metadatagt
• Toolkits
• See lthttp//www.ukoln.ac.uk/qa-focus/toolkit/gt

You can contribute to the case studies brief
documents covering (a) the project (b) area
addressed (c) solution used (d) lessons
learnt. You document experiences for sharing with
community and we promote you (at events and on
Google-friendly Web site) e.g. Healthier Nation
14
Toolkits

• Online and paper based toolkits are available

15
I'm Too Busy!

• Projects may feel they are too busy to take on
  this work. However
• You're probably doing much of the work already!
• QA methodology has been designed to be
  lightweight
• It is needed in order to provide the
  interoperability and wide access for which
  projects are funded
• Documented QA policies should help ensure the
  functionality of your deliverables (e.g. when
  staff leave)
• There is more likelihood of your project
  deliverables been deployed into service by others
  if QA issues have been addressed
• QA Focus recommends that future JISC programmes
  require QA to be addressed
• Projects themselves may prefer the lightweight
  self-assessment approach than an external audit
  could this happen if disasters (such as UKeU)
  occur?

16
Your Feedback

• Feedback on our approaches are welcome
• Please complete evaluation forms

17
Questions

• Any questions?