CARPE

1
CARPE
CAPTURE, ARCHIVAL AND RETRIEVAL OF PERSONAL
EXPERIENCES
2
Academic
CAPTURE, ARCHIVAL AND RETRIEVAL OF PERSONAL
EXPERIENCES
3
What is CARPE?

• CARPE stands for the Capture, Archival and
  Retrieval of Personal Experiences
• It is the personal storage of all one's media
  throughout a lifetime2, an electronic record of
  experiences a diary which is not written by
  hand, but by software
• while media capture has typically been sparse
  throughout a lifetime, we can now consider
  continuous archival and retrieval of all media
  relating to personal experiences.2

4
What examples are there?

• A limited example of CARP functionality is photo
  organising software, which allows the user to
  view their photos by the dates which they were
  taken on in a time line, annotate them, describe
  them and search through them
• Another is desktop search software, which allows
  the user to search their computer desktop
  experiences, such as e-mail, web history, chat
  history and documents
• A more comprehensive implementation of CARPE is
  the Microsoft Research MyLifeBits project, which
  includes full-text search, text audio
  annotations, and hyperlinks1

5
What examples are there?

• Microsoft researcher Gordon Bell has captured a
  lifetime's worth of articles, books, cards, CDs,
  letters, memos, papers, photos, pictures,
  presentations, home movies, videotaped lectures,
  and voice recordings and stored them digitally.
  He is now paperless, and is beginning to capture
  phone calls, IM transcripts, television, and
  radio.
• Gordon extends the sparse2 capture of media,
  continuously capturing meetings, using a voice
  recorder and a SenseCam, which is a wearable
  camera
• CARPE software enables the sometimes continuous
  capture, archival and retrieval of different
  forms of media

6
Why is CARPE useful?

• CARPE allows an accurate record of captured
  experiences to be collected and later retrieved
  by searching
• Navigation and annotation are also possible
• This is very useful as a memory aid to the
  general population, but is also especially useful
  to those with memory problems, who may have no
  memories at all of daily events
• For these people, wearable devices can allow for
  the retrieval of information based on what is
  being captured, for example software is able to
  place names to faces using face recognition
  algorithms and stored photographs

7
Who first proposed CARPE?

• Vannevar Bush published As We May Think in
  1945, proposing a Memex device in which an
  individual stores all his books, records, and
  communications, and which is mechanized so that
  it may be consulted with exceeding speed and
  flexibility.2
• His vision included the ability to search and
  annotate stored information, with hyperlink
  navigation

8
What experiences can be collected?

• There are many experiences which can be
  collected, however some are more technically or
  legally difficult than others
• Communications
• E-mail
• Instant messaging
• Letters
• Telephone calls
• Personal media
• Photographs
• Home video
• Documents/writing

• Media experiences
• Web browsing
• Radio
• Television
• Books
• Magazine/newspaper articles
• Academic papers
• Physical experiences
• Conversations/meetings
• Lectures
• GPS Travel data

9
Practical
CAPTURE, ARCHIVAL AND RETRIEVAL OF PERSONAL
EXPERIENCES
10
What will be developed?

• The CARPE research area is very broad, so for a
  manageable sized project I must focus on a
  limited area
• There are a wide variety of different experiences
  which may be dealt with by CARPE systems, and so
  the project must focus on the capture and
  retrieval of information from a limited number of
  sources of information
• The project must also be possible without
  specialised hardware or software

11
What will be developed?

• I have chosen the capture and retrieval of
  information from lectures as the practical
  component for my project
• The purpose of the purposed project is to develop
  an application which allows students to integrate
  audio recordings of lectures with PowerPoint
  files of these lectures
• While the attendance of lectures is quite a
  specific experience which is limited to students,
  the Microsoft PowerPoint format, which is the
  standard used for presentations, is commonly used
  in business and education, and so any software
  developed could be applied to a wide range of
  personal experiences

12
What will be developed?
Human/machine transcription
Alignment of slides with transcription
Addition of transcription/ narration to PPT file
Lecture recording
Storage indexing
Application
Lecture slides
Search transcriptions of stored lectures
Lecture slides
View lectures
13
Who needs it and why?

• At present, many students rely on being able to
  and download lecture slides in PowerPoint
  format from VLEs
• These files do not always contain all of the
  information orally conveyed during the lecture
• Lecturer may orally elaborate
• Slides may be out of date
• With the absence of more detailed handouts or
  narrated PowerPoint files, this means that
  students, or at least those without perfect
  memories, may need to make some form of recording
  of lectures in order to be able to refer to them
  later, for
• Revision
• Assignments
• Set work
• For example, lecturers often give additional
  advice and guidance during lectures which could
  be valuable to students, and lectures may be more
  useful in order to revise for university-set
  exams than textbooks

14
Who needs it and why?

• Alternatives are not satisfactory
• Writing handwritten notes
• There is a limit to how much information most
  students are able to write by hand
• Searching handwritten notes is not possible
• Handwriting may be difficult to read
• Using a laptop computer
• Weight may be a problem
• Battery life may not last a day
• There is still a limit as to how much students
  are able to write without skills such as touch
  typing

15
Who needs it and why?

• Camcorder
• Allows for navigation between slides
• Requires a lot of physical or virtual storage
  space, and is therefore not often practical for
  students
• Battery requirements are problematic

16
Who needs it and why?

• Voice recorder
• Speech may be hard to distinguish
• Listening to lectures is a time consuming, linear
  activity
• There is no way of searching audio recordings for
  information, or of navigating to the narration of
  a specific slide in a lecture other than
  listening to an entire recording
• There is also sometimes no way of determining the
  date when a recording was made
• This is not very practical except for revision of
  an entire topic

17
Conclusion

• The storage of a lifetime of personal experiences
  is now technically and economically possible
• The practical aim of the project is to develop an
  application which allows students to integrate
  audio recordings of lectures with PowerPoint
  files of these lectures using alignment
  techniques, and also search through and organise
  their collection

18
Links

• 1. MyLifeBits
• http//research.microsoft.com/barc/MediaPresence/M
  yLifeBits.aspx
• 2. SIGMM CARPE Research Area
• http//www.sigmm.org/Members/jgemmell/CARPE

19
End
CAPTURE, ARCHIVAL AND RETRIEVAL OF PERSONAL
EXPERIENCES