Incorporating Special Characters into STI

1
Incorporating Special Characters into STI (with
the Power of UTF-8)
Jannean Elliott Harvesting Manager
U.S. Department of Energy
2
Categories of Characters

• Mathematical operators/symbols (such as the three
  dimensional angle used by Euclid)
• Super and Subscripts
• Symbols for chemical elements
• Diacritics and combining diacritics
• Foreign alphabet characters (Greek, Asian,
  Arabic)
• Iconic symbols that have become icons in
  scientific literature (such as infinity)

3
The Golden Vision
But look out for what lies beneath
4

• Lances presentation will be introduced here.

5
So, whats the situation at OSTI ?

• UTF-8 being implemented throughout
• Character errors noted
• Teams formed phases defined

6
Phase 1

• Focus on fixing problems happening now that are
  caused by disconnect between old and new
  character encoding.
• Upgrade all systems ASAP.
• Identify records with corruption and fix
• Catch/stop potential new errors upfront

7
Phase 2
Focus on determining realistic interim
goals Learn what others are doing and
how. Determine whats technically feasible for
OSTI Coordinate with STIP community Develop
policy to support interim goals. Could be
limited to intake, storage, display only Could be
limited to specific fields only Could be limited
to specific subsets of special characters
8
Phase 3
Focus on moving toward the golden vision
Implement the goals from Phase 2 Ride the tide

9
For all Phases

• Will definitely be a staged approach in terms of
  timetable
• May also be staged approach in terms of the work
  process
• Input
• Storage in repository
• Display in databases
• Retrieval capabilities

10
What to do first when you get home

• If you use the web forms to submit
• If you submit in batch mode
• If you harvest

11
Next Steps

• May include a special page for updates on STIP
  web site
• Could include a working team
• Will involve OSTI working individually with
  non-UTF-8 harvesting sites
• Needs all of us to learn more about our own
  site/organizations situation

12
Just hang on. We WILL get there together !