SMARTGS Project: a tool searching, marking up and linking historical documents

1
SMART-GS Project a toolsearching, marking up
and linking historical documents

• Susumu Hayashi, Kazuaki Kobayashi), Kazuhiro
  Kobayashi, Yuuta Hashimoto, Tsukushi Shimizu,
  Yuuki Tamura,
• Humanistic Informatics, Kyoto University
• Kengo Teraswa, Hajime Imura
• Meme Media Laboratory, Hokkaido University
• ) present address Goldman Sachs Japan Holdings,
  Ltd. Technology Division

2
Book pages annotated by underlines marginal
notes, etc.
Underline
Bookmark by PostIt Flag
Marginal note
3
Linked annotations relationship of annotations
A region marked up by the brace
Brace
A line linking the region and the note
Marginal note
4
Annotations for historical documents
My text analysis of the diary of 19-20th century
German Mathematician David Hilbert, who is said
the father of 20th century mathematics.
5
What are we doing?

• Annotating images of book pages and historical
  documents by
• adding markups, bookmarks and texts (e.g.
  marginal notes)
• relating them each other by links
• and, further relating them to resources outside
  books, papers Web resources, e.g. references to
  papers , books and documents in archives.

6
For the cases of Web and PDF documents

• Cyber documents, e.g. PDF files and HTML files,
  can be digitally annotated in similar ways.
• Furthermore, words in such digital documents are
  searchable.
• Imagine that historical documents can be
  digitally annotated and are searchable. Examples
  of such documents
• handwriting documents, e.g. Hilberts diary.
• Image-base digital archives of books, journals
  and other documents, now widely available on Web
  thanks to some libraries

7
An example of image-base digital archives

• ??????????????????? (National Diet Library,
  Digital library from the Meiji Era)
• 143,000 books from Meiji and Taisyo eras are open
  to the public.

From NDL, Digital library from the Meiji Era Y.
Fukuzawa, Gakushya Anshin-ron
8
The goal of SMART-GS project

• Providing platforms for marking up, linking and
  searching historical documents on PC and on the
  network.
• Looks as a dream?
• No, its not a dream!
• We have already built a JAVA application
  SMART-GS with the full functions required except
  the network features.

9
Demo of SMART-GS applied to David Hilberts diary

• The document The first book of D. Hilberts
  diary German handwritings from 1885-1892.
• Searching a word in the diary
• The word searched is Kronecker another German
  mathematicians family name.
• Marking up and linking images and texts
• One-to-Many Link
• SMART-GS used in real researches
• my research on the diary
• Kazuhiro Kobayashis graduation research

10
Working with SMART-GS onthree wide monitors
A snapshot of my desktop. SMART-GS, an online
digital archive, wikipedia, and a digital version
of early 20th century dictionary
11
Caution!

• Search is not free!
• You have to specify lines of a document to
  search it.
• It can be almost automatically done for printed
  documents and well-formed documents such as
  notes written on ruled papers.
• For the worst cases as Hilberts diary, one must
  specify lines by hands like this

12
Specifying lines by hands
13
Towards tools on the Net

• Historians with similar interests are often
  scattered through the globe. They wish to share
  knowledge for every-day researches, but it is not
  so easy.
• SMART-GS technology will provide an excellent
  help for world-wide collaborations of a team of
  historians.
• It may also provide a dramatic improvement on
  image-base online digital archives of handwritten
  and/or printed historical documents.
• The key of such network version is the way the
  annotation was attached to images of historical
  documents Gs-file!

14
Gs-file an XML file storing annotations and
others
Annotations are converted into the XML-format
and stored in gs-file
15
Inside of a gs-file
16
Different gs-files provide different views on the
same historical text
17
A model of network version-archive model-
historian

• Archive

Annotation by gs-file
Gs-file
Annotation by gs-file
Images
historian

• Historian publish their own gs-files to annotate
  document images in a public archive.
• They can annotate also gs-files of other
  historians.

Gs-file
reader
A reader can browse views of historians, if a
historian makes his view open to the public.
18
Future network version can markup digital archives
19
Another model of a network version-online
collaboration model-

• Historian publish their own document images and
  gs-files (views). They comment to views of other
  researchers.

Imagesgs-file
historian
historian
Imagesgs-file
historian
Imagesgs-file
reader
20
Conclusion

• SMART-GS system has been already applied to a
  study of a very important diary in the history of
  mathematics and proved its usefulness.
• There is also a plan to use SMART-GS in a project
  to transcribe a gigantic diary of Yuuzaburo
  Kuratomi (1853-1948), a chairman of the privy
  council.
• The network versions in the future will be useful
  for research collaborations and using digital
  archives.

21
Conclusion (continued)

• For technical and financial reasons, historians
  must often use secondary sources rather than
  primary sources. However, the secondary sources
  are products of interpretations of their
  authors!
• SMART-GS technology together with online digital
  archives will provide historians with an easy and
  reasonable way to use primary sources for their
  studies.

22
The history of the project

• I am a historian of mathematics and logic. I was
  working on Hilberts diary.
• It is so difficult to read. Many notes were
  scribbled. Hilbert wrote no dates, although
  historians call it Hilberts mathematical
  diary.
• The contents are even more nightmarish! Some
  statements were simply unbelievable from the
  conventional knowledge on the history of 19-20th
  century mathematics.
• Thus, even natives cannot read sometimes. Reading
  the diary was an extremely complicated task for a
  Japanese historian who is not really good at
  German language.
• In 2006, I decided to build a tool which helps my
  task by converting a software engineering tool
  SMART, using an image search engine by Terasawa
  and Imura of Hokkaido University.
• SMART was developed in the Faculty of
  engineering, Kobe university by me and my
  students.

23
The origin of the project (cont.)

• My student Kazuaki Kobayashi built SMART-GS
  (SMART für Geschichte Studie) as his master
  thesis study. The first version of SMART-GS was
  finished in the last March.
• I was using the first version for studying
  Hilberts diary and for preparing a lecture
  course on Hilberts physics studies. It proved
  that the tool is sometimes very useful.
• Since the end of summer, a team of my students
  Kazuhiro Kobayashi and others started to improve
  the tool and it is now nearly at the level of
  realistic applications to history researches.
• Along the development of SMART-GS, I envisioned a
  network application of SMART-GS technology.
• A team of historians and computer scientists are
  planning to develop realistic plat forms for
  history studies based on the technology.