Living Collections, Ambient Data

1
Living Collections, Ambient Data

• Micah Altman
• Harvard University
• Archival Director, Henry A. Murray Research
  Archive
• Associate Director, Harvard-MIT Data Center
• Senior Research Scientist, Institute for
  Quantitative Social Sciences
• E micah_altman_at_harvard.eduW http//maltman.hmdc
  .harvard.edu/

2
Considerations for Preserving Collections Curated
by Third Parties

• Virtual Archives, Living Collections, Data
  Sharing, Community Data data is not curated by
  single organization with single set of standards
• What are the external curators collection
  development goals? How do they align with yours?
• Who is responsible for adding, deleting, or
  modifying the content? Can they delegate this to
  others?
• How will your archive be notified when content is
  added or modified?
• What is the procedure for review of new content?
  Is it ongoing, or retrospective?
• What agreements are in-place to support archiving
  this content?
• Can and does the current depositor make
  commitments with respect to future content?
• Does archive have clear rights to preserve and
  disseminate third-party curated content?
• Are rights to content permanent? Can archive
  preserve content that curator has removed from
  their collection?
• Is agreement itself persistent, subject to
  renewal, etc?
• See Appraisal and Acquisition of Actively
  Curated Collections

3
Evolution at the Murray Archive

• Paper -gt digital
• Ad-hoc metadata -gt Federated Catalog
• Idiosyncratic usage terms -gt standard terms
  standard extensions click-through
• Curator-focused acquisitions -gt depositor focused
  
• Single copies -gt distributed replication

4
Federation and Virtual Hosting
5
http//dvn.iq.harvard.edu/dvn
6
Shifting Evidence Base?

• Examples
• Harvesting and analysis of blogs for virtual
  political opinion surveys
• Continuous collection of CSPAN, real-time subject
  coding, continuous dissemination
• Cell phone data stream collection movement,
  proximity to others, social network analysis
• Participative goals-based redistricting
• Agent-based models of emerging institutions
• FMRI analyses of reaction to political and social
  scenarios
• Modal Features
• Analyses emerge through exploration and
  interactions
• Data collection from non-experimental, non
  instrumental, sources
• Increasing scale of data
• Compute limited
• Data confidentiality
• High-level analysis tools
• Remote collaboration is part of projects

Collective holdings of all U.S. numeric social
science data in all major data archives,
government repositories (estimated) 50 TB lt
Ambient data increasingly becoming subject of
social science research (2002 data) Web
(surface) 167 TB Radio 3,500
TB Television 69,000 TB Web (deep) 92000
TB Email (originals) 441,000 TB Telephone
18,000,000 TB
7
Discussion Questions

• Are we placing too much responsibility on the
  content creators to deposit data? Should the
  Partnerships shoulder even more of the burden to
  preserve content?
• What advice do you have regarding sustainability
  of projects after NDIIPP funding comes to an end?
  In particular, how do we continue to invest in
  collection development activities?
• How is your business model for collection and
  preservation likely to change in the next five
  years?
• What can we do to ensure that growth does not
  outpace the capabilities of young preservation
  services?

8
More Information

• Contact me
• http//maltman.hmdc.harvard.edu/
• ltMicah_Altman_at_harvard.edugt
• Data-PASS Alliance
• http//www.icpsr.umich.edu/DATAPASS/