How Far Have We Come From eLib to NOFdigi and Beyond

1
How Far Have We Come? From eLib to NOF-digi and
Beyond
http//www.ukoln.ac.uk/cultural-heritage/events/ci
lip-scotland-2009/
Acceptable Use Policy Recording of this talk,
taking photos, discussing the content using
email, instant messaging, blogs, SMS, etc. is
permitted providing distractions to others is
minimised.

• Brian Kelly
• UKOLN
• University of Bath
• Bath, UK

Email b.kelly_at_ukoln.ac.uk Twitter http//twitter
.com/briankelly/
Blog http//ukwebfocus.wordpress.com/
Resources bookmarked using cilips09' tag
UKOLN is supported by
This work is licensed under a Attribution-NonComme
rcial-ShareAlike 2.0 licence (but note caveat)
2
Contents
Introduction

• Introduction
• About me
• About this talk
• The National Programmes (providers perspective)
• The technical standards
• The support infrastructure
• What We Learnt
• What succeeded, what failed and what we
  discovered along the way
• What Should We Do In The Future
• What do we do next?

3
About This Talk
Introduction

• Talk is based on experiences of national
  programmes
• eLib
• NOF-digi
• DNER/IE
• Michael
• Common characteristics
• Interoperability through open standards
• Managed view of roadmap
• Issues
• Did we get it right?
• Are there alternative approaches?

4
The eLib Programme

• eLib Programme
• Response to the Follet review, 1993
• Initial budget of 15m over 3 years
• 60 projects funded
• Areas covered
• Document Delivery
• Access To Network Resources
• Training Awareness
• Electronic Journals
• Digitisation / Images
• Electronic Short Loan Collections
• On Demand Publishing
• Pre-Prints and Grey Literature
• Supporting Studies

5
eLib Standards Guidelines

• eLib Standards Guidelines
• Provides recommendations for selection use of
  standards
• Strongly encouraged where relevant
• Covered
• Data communications
• Data interchange
• Metadata
• Search retrieve
• Security, authentication payment services

6
eLib Standards
Editors Chris Rusbridge Lorcan Dempsey, Ann
Mumford (myself as a contributor)

• Things we dont care about (?)
• Email (SMTP, not X.400),
• Areas we correctly hedged our bets
• GIF is OK, keep eye on PNG
• Areas we were evasive about
• PostScript PDF
• Areas we got wrong
• It is anticipated that SGML will be a key
  standard ... Projects are encouraged to .. agree
  or, where necessary, develop document type
  definitions
• projects should supply a URL for public
  services, and be prepared to adopt URNs when they
  are stabilised
• Standards which seem to have disappeared
• CGM

7
eLib Standards version 2

• eLib Standards version 2
• Published in 1998 (2 years after v 1)
• Introduced a template for descriptions
• Relevant standards
• Comments
• Consensus
• Templates on recommended standards complemented
  by technical summaries and speculation e.g.
• HTTP-NG should support more secure
  authentication and encryption

8
Beyond The Standards

• Cross-Searching The Vision
• whois lightweight distributed cross-searching
  protocol
• ROADS Open source software used by most eLib
  subject gateways (e.g. SOSIG)
• Z39.50 More heavy-weight solution used in
  library context
• eLib SBIGs/RDN implementation in a distributed
  environment (with departmental providers)
• Cross-Searching Todays Reality
• Intute Centralised database, distributed data
  collection. Cross-searching interfaces, but how
  widely used?

Note dangers of using standards outside of DL
programmes JISC Web site upgrade mandated
Z39.50 support ?
9
Later, in NOF-digitise

• NOF-Digitise programme
• Ran from August 1999 to December 2004
• 50 million funding to put information that
  supports lifelong learning into digitised form.
• Brought together wide range of partnerships
  organisations
• NOF Technical Advisory Service (NOF TAS)
  provided
• Informed support and advocacy of Technical
  Standards and Guidelines
• Assistance in achieving standards compliance
• Detailed and project-specific advice
• Repository of standard and generic advice
• Provide by UKOLN and AHDS

10
Development Culture

• Different culture to HE digital library
  development community
• Tell us the standards which we must mandate
• Caused problems
• NOF-digi project Web sites must have 24x7
  availability very expensive! Requirement was
  availability at weekends!
• Recommended standards werent mature (e.g. SMIL)
  whereas proprietary solutions (Flash) provided
  compelling user services
• Providing pragmatic solutions
• NOF-digi project Web sites should seek to
  maximise their uptime
• Quarterly reporting template provided a get-out
  clause You must (a) describe the areas in
  which compliance will not be achieved (b)
  explain why compliance will not be achieved
  (including research on appropriate open
  standards) (c) describe your migration
  strategies to ensure compliance in the future and
  (d) how the migration may be funded

11
Minerva Experience

• Minerva technical guidelines
• EU-funded
• Built on eLib/NOF/JISC IE resources
• Initially edited by UKOLN

• Continued to promote plausible
• Standards
• Best practices
• which failed to take off

12
Compliance Issues

• What does must mean?
• You must comply with HTML standards
• What if I don't?
• What if nobody does?
• What if I use PDF?
• You must clear rights on all resources you
  digitise
• You must provide properly audited accounts
• What if I don't?

JISC 5/99 programme 80 of project home pages
were not HTML compliant
There is a need to clarify the meaning of must
and for an understandable, realistic and
reasonable compliance regime
13
QA Focus

• QA Focus
• JISC-funded project provided by UKOLN and
  (initially) TASI, then AHDS from 2002-2004
• What QA regime should JISC provide for its
  development programmes? What actions should be
  taken if standards not conformed with?
• Recommendations
• Self-assessment, not external validation
  (projects explained complexities of
  standards-compliance)
• Build on culture of sharing and openness
• Have a pragmatic view of open standards
• Understand complexities of non-conformance /
  failure
• The standard failed, not the project
• The standard may be too expensive to deploy
• Alternatives may become available

14
Why Open Standards?

• JISC's development programmes
• Traditionally based on use of open standards to
• Support interoperability
• Maximise accessibility
• Avoid vendor lock-in
• Provide architectural integrity
• Help ensure long-term preservation
• But (thinking the unthinkable)
• Do open standards deliver?
• What happens if open standards fail?
• What is an open standard?
• Is the only alternative to open standards use of
  proprietary solutions?

15
But Don't Always Work ?

• There's a need for flexibility
• Learning the lesson from OSI networking protocols
• Today
• Is the Web (for example) becoming over-complex
• "Web service considered harmful"
• The lowercase semantic web / Microformats
• Lighter-weight alternatives being developed
• Responses from the commercial world

• Other key issues
• What is an open standard?
• What are the resource implications of using them?
• Sometimes proprietary solutions work (and users
  like them). Is it politically incorrect to
  mention this!?

16
What is An Open Standard?

• Which of the following are open standards today
  (and were open standards in 2006)?
• XHTML 1 ? PDF ? Flash
• Java ? MS Word ? RSS (1.0/2.0)

17
Is RSS An Open Standard?
RSS Example

• Is RSS an open standard ("are RSSs open
  standards")?
• RSS 1.0 (RDF Site Summary)
• XML application using RDF model
• Developed by Aaron Schwarz
• RSS 2.0 (Really Simple Syndication)
• XML application using simpler model
• Developed by Davey Winer
• Note that RSS is a widely used and popular
  application with usage growing through its role
  in podcasts

• Issues
• Are these open standards?
• Are they reliable and robust enough to build
  mission-critical services on?
• Is there a clear roadmap for the future?

18
RSS Governance Issues
RSS Example

• Governance Issues
• RSS 1.0 spec maintained by Aaron Schwartz
• "Aaron Swartz is a teenage writer, hacker, and
  activist. He was a finalist for the ArsDigita
  Prize for excellence in building non-commercial
  web sites at the age of 13. At 14 he co-authored
  the RSS 1.0 specification, now used by thousands
  of sites to notify their readers of updates."
• RSS 2.0 specification developed by Dave Winer
• "Winer is known as one of the more polarizing
  figures in the blogging community. However ..
  there are many people and organizations who seem
  unable to maintain a good working relationship
  with Dave."

19
RSS Summary
RSS Example

• To summarise
• We thought RSS was a great lightweight
  syndication technology
• It was but competing alternatives were
  developed
• No clear winner (RSS 1.0's extensibility W3C's
  support versus RSS 2.0's simplicity and take-up
  in podcasting, iTunes, etc)

• Conclusions
• Life can be complex, even with simple standards
• Technical merit is never enough market
  acceptance can change things
• RSS can still be useful, and interoperability can
  be provided by RSS libraries supporting multiple
  formats
• Need for a more sophisticated approach such as
  model in A Contextual Framework For Standards,
  WWW 2006

20
The Context

• There will be a context to use of standards
• The intended use
• Mainstream ? Innovative / research
• Key middleware component ? Small-scale
  deliverable
• Organisational culture
• HE vs FE ? Teaching vs Research
• Service vs Development ?
• Available Funding Resources
• Significant funding training to use new
  standards
• Minimal funding - current skills should be used

Contextual Issues
An open standards culture is being developed,
which is supportive of use of open standards, but
which recognises the complexities and can avoid
mistakes made in the past
21
The Layered Standards Model
Owner
JISC
3rd Parties
Context Compliance
JISC / project
External
Self assessment
Penalties
Learning
JISC's layered standards model, developed by
UKOLN. Note that one size doesn't always fit all
22
The Standards Catalogue

• The information provided aims to be simple and
  succinct (but document will still be large when
  printed!)

• Standard Dublin Core
• About the Standard Dublin Core is a metadata
  standard made up
• Version New terms are regularly added to
• Maturity Dublin Core has its origins in
  workshops held
• Risk Assessment Dublin Core plays a key role .
  It is an important standard within the context of
  JISC development programmes.
• Further Information
• DCMI, lthttp//dublincore.org/gt
• Author Pete Johnston, UKOLN
• Contributor
• Date Created 04 Oct 2005
• Update History Initial version.

Example
Note that as the standards catalogue is intended
for wide use the contents will need to be fairly
general
The Standards Catalogue is deemed important but
theres a still lack of understanding of the
contextual model
23
What If Web 2.0 Changes Everything?

• Web 2.0 Changes Everything what if this is
  true? The Higher Education in a Web 2.0 World
  report suggests that Senior Managers in HE may
  feel this to be the case
• Network is the platform / The Cloud
• Web infrastructure becomes the infrastructure (HE
  follows, no longer leads)
• Growing importance of informal learning
• Growing importance of informal networking
• Growing reluctance to travel (travelling to
  CILIP-S on par with dodgy MPs expenses claims?)

24
The DNER/IE Diagram

• Web 2.0 in the context of Andy Powells famous
  IE diagram (early version shown)

..which later was developed further
25
My Take

• My Vision
• In 2001 I suggested that application services
  could be provided out there (in The Cloud).
• I speculated about the JISC Spellchecker and JISC
  delicious services

What I Missed! What I thought about but failed to
articulate (it seemed (a) Thatcherite
out-sourcing (b) too complex) was commercial
provision of the services large-scale apps e.g.
Google Docs
26
Services In The Cloud

• Will Web 2.0 services in The Cloud make
  national initiatives irrelevant?
• Or will there be a mix of institutional, national
  and global providers of solutions?
• Or will institutional national services make
  use of infrastructure in The Cloud?

27
What About The Developers?

• In the old days
• Development was slow and required significant
  levels of funding
• Funders and budget holders could manage
  development process
• Today
• Web infrastructure more mature (standards,
  services, APIs, )
• Light-weight is cool
• Developers dont want 3 year projects (and
  associated bureaucracy) but food drink are
  good!

28
What Can Be In A Weekend?

• Tony Hursts visualisations of MPs expenses claims

29
Community Matters

• Importance of developer community now being
  appreciated
• In JISC Circles (cf dev8d week Mashed Library
  events Rapid Innovation Call)
• In Museums sector (cf. Mashed Museum events)
• In commercial sector (cf barcamps)
• In government circles (cf. Government barcamps)

http//dev8d.jiscinvolve.org/2009/03/30/
Dev8D Developer Happiness Days event sponsored
by JISC. See lthttp//dev8d.jiscinvolve.org/gt
30
Risk Assessment Model (c. 2006)

• SSf(SB, S, U, En, ..)
• Selection of appropriate standard (SS) is
  function of
• Standards Body (SB) Maturity, stability, status,
  openness, responsivity,
• Standard (S) Functionality, complexity /
  ease-of-use,
• Users (U) Appropriateness for, benefits to
  adoption by
• Environment (En) Institutional, community,
  sectoral,
• Other factors
• Market acceptance do vendors support it (beyond
  proof-of-concept open source examples)
• Risks (am I betting the company of the standard)
• Exit options (can I easily change my mind)
• Advocacy (is the world campaigning for it) and
  threats (is the world criticising for it)

31
Deployment Strategies

• Interested in using Web 2.0 in your organisation?
• Worried about corporate inertia, power struggles,
  etc?
• Theres a need for a deployment strategy
• Addressing business needs
• Low-hanging fruits
• Encouraging the enthusiasts
• Gain experience of the browser tools and see
  what youre missing!
• Staff training development
• Impact assessment and measurement
• Risk and opportunity management strategy
• Critical Friends and friendly critics
• Culture of sharing

32
Risk Management
Strategies

• JISC infoNet Risk Management infoKit
• In education, as in any other environment, you
  cant decide not to take risks that simply isnt
  an option in todays world. All of us take risks
  and its a question of which risks we take

• Examples of people who are likely to be adverse
  stakeholders
• People .. required to commit resources to the
  project
• People who fear loss of control over a function
  or resources
• People who will have to do their job in a
  different way
• People who will have to carry out new functions
• People who will have to use a new technology
• ..

33
Critical Friends

• JISC UI programme is encouraging establishment
  of Critical Friends

See lthttp//critical-friends.org/gt
34
Towards a Framework

• Time To Stop Doing and Start Thinking A
  Framework For Exploiting Web 2.0 Services,
  Museums the Web 2009 conference

35
Using The Framework
Note personal biases!

• Use of approach in two scenarios use of Twitter
  Facebook

36
Using The Framework (2)
Semantic Web
No standard

• Use of approach with standards doing nothing
  (today) might be an option!

37
The Assumptions

• Standards
• Interoperability through open standards
• Avoidance of proprietary lock-in other benefits
• All we need is to identify the correct open
  standards
• This will save us time, money deliver rich
  functionality and usable useful services
• Development
• The developers can then simply use the standards
• This will also provide seamless evolution to new
  standards

38
Challenging The Assumptions

• Maybe we want
• To challenge the unthinking assumptions in
  national development programmes using evidence
  rather than assertions
• The benefits promised (but not necessarily
  delivered) by open standards
• An understanding that its not a binary open
  standards vs proprietary world
• The world may choose good enough, whilst we want
  to provide the best
• To develop user-focussed services which the
  commercial sector seems to be better at
• To recognise the importance of the developers
  perspective

And we should also challenge these views!
39
Questions

• Any questions, comments, ?

• Additional Resources
• Papers published on standards and national
  programmes
• What Does Openness Mean To The Museum Community?,
  MW 2008
• Openness in Higher Education Open Source, Open
  Standards, Open Access, elPub 2008
• Addressing The Limitations Of Open Standards, MW
  2007
• A Contextual Framework For Standards, WWW 2006
• A Standards Framework For Digital Library
  Programmes, ichim05
• Interoperability Across Digital Library
  Programmes? We Must Have QA!, ECDL 2004
• Deployment Of Quality Assurance Procedures For
  Digital Library Programmes, IADIS 2003
• Developing A Quality Culture For Digital Library
  Programmes, EUNIS 2003
• Ideology Or Pragmatism? Open Standards And
  Cultural Heritage Web Sites, ichim03
• See lthttp//www.ukoln.ac.uk/web-focus/papers/stan
  dardsgt