Blue Ribbon Panel on Cyberinfrastructure Introduction, Context

1
Blue Ribbon Panel on CyberinfrastructureIntroduc
tion, Context Charge

• Dan Atkins, Chair
• atkins_at_umich.edu
• University of Michigan
• April 19, 2002

2
Panel Members

• Daniel E. Atkins, Chair, Univ. of Michigan, EECS
  and SI, atkins_at_umich.edu
• Kelvin K. Droegemeier, Center for Analysis and
  Prediction of Storms, University of Oklahoma,
  kkd_at_ou.edu
• Stuart I. Feldman, IBM Research, sif_at_us.ibm.com
• Hector Garcia-Molina, CS Dept., Stanford
  University, hector_at_cs.standford.edu
• Michael Klein, Center for Molecular Modeling,
  University of Pennsylvania, klein_at_lrsm.upenn.edu
• Paul Messina, Cal Tech, messina_at_cacr.caltech.edu
• David G. Messerschmitt, UC-Berkeley, EECS SIMS,
  messer_at_eecs.berkeley.edu
• Jeremiah P. Ostriker, Princeton University,
  jpo_at_astro.princeton.
• Margaret H. Wright, Computer Science Department,
  Courant Institute of Mathematical Sciences, New
  York University, mhw_at_cs.nyu.edu

3
Meeting Agenda April 19, 2002,NSF, 1-4 pm

• 1. Review of status of the panel's activities and
  goals for this meeting.
• 2. Reports from the authoring sub-committees.
• 3. Review and discussion of the working draft of
  the report.
• 4. Discussion of primary recommendations.
• 5. Stewardship and additional use of the material
  gathered by the Panel.
• 6. Summary of additional activities to create
  final version of report.
• 7. Matters arising.

4
Historical Schematic
CSE research elsewhere in NSF
Support for an array of small, medium, and large
CISE basic research projects
1995
CISE Directorate
OUR REPORT
Computational Science init. Expanded equip.
program.
Hayes Report
Provision of advanced scientific computing
5 Supercomputer Centers, NSFnet,
PACI NCSA NPACI
1984
Terascale Computing Initiatives
Lax -gtCurtis/Bardon Reports
1993 BRPDesktop to Teraflop
5
Charge
OUR REPORT
WRT meeting needs of the scientific and
engineering research community
A) Evaluate the current PACI programs.
B) Recommend new areas of emphasis for CISE
Directorate,
Cyber-infrastructure
C) Recommend an implementation plan to enact
recommended changes.
6
Process

• Web survey
• Hearings
• Reviewing prior reports
• Random input
• Knowledge and expertise of the Panel members.

7
Epigraph

• Cyberinfrastructure is the sine qua non for true
  progress in much of the mathematical and physical
  sciences And progress in CI is often driven by
  real-world problems.
• Robert Eisenstein, AD for MPS, 11/30/01

8
Revolutionizing Science and Engineering through
CyberinfrastructureTable of Contents

• 1. The Vision
• 2. Background and Charge
• 3. Challenges and Opportunities for the
  Scientific Research Community
• 4. The New Cyberinfrastructure What Changed in
  Computing
• 5. The Landscape of Related Activities
• 6. Partnerships for Advanced Computational
  Infrastructure Past and Future Roles
• 7. Achieving the Vision
• 8. Scope and Budget Estimates

9
Draft Report Available in pdf at worktools.si.umic
h.edu/workspaces/datkins/001.nsf
Please send comments by May 1, 2002 to
NSF-PCI-all_at_umich.edu
10
Revolutionizing Science and Engineering through
CyberinfrastructureTable of Contents

• 1. The Vision 2. Feldman
• 2. Background and Charge 1. Atkins
• 3. Challenges and Opportunities for the
  Scientific Research Community 3. Droegemeier
• 4. The New Cyberinfrastructure What Changed in
  Computing 2. Feldman
• 5. The Landscape of Related Activities 2. Feldman
• 6. Partnerships for Advanced Computational
  Infrastructure Past and Future Roles 6. Wright
• 7. Achieving the Vision 4. Messerschmidt
• 8. Scope and Budget Estimates 5. Messina
• Summary and Discussion - Atkins

11
Blue Ribbon Panel on CyberinfrastructureVision

• Stuart I. Feldman
• IBM
• April 19, 2002

12
Recommendations

• New INITIATIVE to revolutionize science and
  engineering research at NSF and worldwide to
  capitalize on new computing and communications
  opportunities 21st Century Cyberinfrastructure
  includes supercomputing, but also massive
  storage, networking, software, collaboration,
  visualization, and human resources
• Current centers (NCSA, SDSC, PSC) are a key
  resource for the INITIATIVE
• Budget estimate incremental 650 M/year
  (continuing)
• An INITIATIVE OFFICE with a highly placed,
  credible leader empowered to
• Initiate competitive, discipline-driven
  path-breaking applications within NSF of
  cyberinfrastructure which contribute to the
  shared goals of the INITIATIVE
• Coordinate policy and allocations across fields
  and projects. Participants across NSF
  directorates, Federal agencies, and international
  e-science
• Develop high quality middleware and other
  software that is essential and special to
  scientific research
• Manage individual computational, storage, and
  networking resources at least 100x larger than
  individual projects or universities can provide.

13
Science and Engineering Research Depends on
Computing and Communications

• Online fast publication (and archives too)
• New collections accessible
• Raw data and digital libraries
• Collaboration (Collaboratories, Access Grid,
  etc.)
• In silico science

14
Furthering the Revolution

• Saving raw data
• Cross-disciplinary collections
• Richer publications
• Grander simulations (cells and organisms entire
  earth system)
• Breadth and depth of collaborations, routinely
  international

15
Thresholds and Opportunities

• Internet and Web use almost universal
• Activity would stop without e-mail and WWW
• Expectations rising with generations and for all
  disciplines
• Supercomputers and terabytes in the lab
• Simulation required to do new science
• Standardized formats, software

16
Risks and Costs

• Inconsistent formats across fields and sites
• Data loss
• Field boundaries
• Duplicative moderate quality software
• Falling behind on computing technologies

17
Proposals for the INITIATIVE

• Large incremental budget
• Drive applications that revolutionize the way
  that research is done
• Fund competitive discipline-driven projects
• With cyberinfrastructure contribution and
  standards and participation by computing experts
• Supply shared resources
• Supercomputers and data farms that provide
  100-1000x what can be found locally
• New shared middleware, content standards, basic
  applications
• New research (emphasizing computation, social
  science,
• New education and outreach
• Central organization with authority

18
The New Cyberinfrastructure
19
Hardware Trends

• Processor speeds and memory increasing with
  Moores Law
• Cluster sizes now 1000s, soon even larger
• Largest sites at 10TF, moving toward PF
• Disk capacity increasing with areal density
  (60-100/year)
• Terabytes typical, petabytes coming
• Wide area networking moving to Gb/s
• Large and high-resolution displays

20
Software

• Information networking applications, messages,
  self-describing content, not just bit streams
• The Grids
• Content management metadata, searches,
  persistence
• Collaboration
• Middleware

21
Ecology of Scientific Computing

• Computing industry
• Commercial requirements drive basic hardware and
  software
• Important additional needs for scientific
  computing
• Computing Research
• Other sciences
• Other federal agencies
• Non-US activities

22
Blue Ribbon Panel on Cyber InfrastructureScience
Engineering Community Needs and Challenges

• Kelvin K. Droegemeier
• University of Oklahoma
• April 19, 2002

23
Goals

• Engage the broadest elements of the science and
  engineering communities as a means for critically
  assessing needs and challenges
• Scientific
• Technological
• Sociological
• Identify barriers and opportunities

24
The Communities

• Domestic and International
• Academia
• Private Industry
• Government Agencies
• Laboratories
• State, Regional, and National Centers

25
Methodology

• Community-wide web survey
• Widely publicized
• gt700 responses
• Quantitative comparisons with the Hayes Report
• Oral public testimony (3 sessions)
• 62 participants selected from research
  scientists and engineers computer and
  computational scientists center directors
  agency and corporate leaders system
  administrators educators students and young
  scientists technicians and consultants
• Emphasis given to traditionally underrepresented
  groups and the physically challenged
• Written transcripts and A/V materials assembled
• Existing reports and planning documents
• Ad hoc communications
• Personal experiences and expertise

26
Analysis

• Results from all 5 methodologies have been
  synthesized
• Remarkable consistency among individual responses
  and within and among disciplines
• No prioritization of findings all summary
  issues are viewed as critically important
• Categorization
• Philosophy and Process
• Current Resources
• Future Infrastructure
• Emerging Paradigms and Activities

27
Philosophy and Process

• Cyber infrastructure lies at the heart of
  revolutionary science and engineering
• NSF should take the lead in charting a national
  course for cyber infrastructure
• NSF should consider human capital and software as
  co-equals with traditional physical
  infrastructure
• Cyber infrastructure requires continuity,
  consistency, and sufficient funding NSF should
  consider the consequences of periodic full
  re-competition of CI centers

28
Philosophy and Process

• NSF needs to
• Provide a framework, motivation, and clear
  direction for building and sustaining linkages
  between academia and industry
• Give attention to the sociological, economic, and
  cultural issues associated with cyber
  infrastructure
• Continue supporting open source software
  strategies

29
Current Resources

• The entry barrier into high performance computing
  continues to be high
• Effective use of parallel computers is becoming
  increasingly complex
• Greater investments are needed in
• Software development
• Training and support

30
Current Resources

• The PACI centers have successfully
• brought high performance computing to the masses
  
• broadened the spectrum of users and
• responded to dramatic changes in the user base,
  technology, and applications
• However, the PACI centers remain a largely batch
  oriented environment and are not configured or
  funded to deliver significant resources in novel
  ways (dedicated, on-demand) to large numbers of
  users

31
Current Resources

• The NRAC allocation process no longer is
  effective
• Double jeopardy
• Yearly resource allocations not congruent with
  multi-year agency grants
• Proposal development process is time-consuming
• Reviewer base insufficiently broad
• Need flexibility to accommodate future resources
  (e.g., data repositories)

32
Current Resources

• The PACI centers have been highly successful in
  developing visionary, innovative technologies and
  prototype tools
• However, insufficient funding and the lack of
  selective investment has hampered transition to
  full deployment

33
Future Infrastructure

• The last mile problem continues and is
  especially serious for HBCUs, Tribal Colleges and
  Universities, and Hispanic institutions
• Research-group and departmental-scale facilities
  (100 to 1000x less powerful than national
  centers) are becoming increasingly important
  thus, national centers need to be a factor of 100
  to 1000x more capable
• High speed networks with high quality of service
  continue to be foundational to research and
  education at all levels
• On-demand (not pre-scheduled) and instantaneous
  access is becoming increasingly important
  (computers, data bases, networks)

34
Future Infrastructure

• Comprehensive environments are needed for linking
  models from multiple disciplines and for
  synthesizing results in interoperable frameworks
• The Grid represents an important opportunity for
  the future and should receive high priority for
  support
• Inexpensive and reliable tools are needed to
  support distance collaborations
• Higher levels of security are needed

35
Emerging Paradigms and Activities

• Cyber infrastructure is becoming the essential
  lynchpin for research at the boundaries among
  disciplines and should be driven by user needs
• The need for a new information technology
  professional is emerging
• Expertise in one or more disciplines plus
  computer science
• They will develop, maintain, and integrate
  complex hardware and software systems
• They are an important bridge to users
• Educational institutions must develop strategies
  for creating this computational science workforce

36
Emerging Paradigms and Activities

• Scientific and engineering applications are
  becoming more multi-scale (both space and time)
  and compute-intensive thus, the need for
  high-end resources continues to grow. However,
  cyber infrastructure research needs to span the
  spectrum from small grants to large centers

37
Emerging Paradigms and Activities

• Significant need exists for access to long-term,
  distributed, stable data and meta data
  repositories and digital libraries
• Legacy data likewise are important and must be
  digitized and preserved

38
Knowledge Frontiers

• Several new projects provide a glimpse of the
  future

39
Blue Ribbon Panel on CyberinfrastructureOrganiza
tion

• David G. Messerschmitt
• University of California at Berkeley
• April 19, 2002

40
Layered structure of the INITIATIVE
Conduct of science and engineering research
Applications of information technology to science
and engineering research
Cyberinfrastructure supporting applications
Core technologies incorporated into
cyberinfrastructure
41
Some roles of cyberinfrastructure

• Processing, storage, connectivity
• Performance, sharing, integration, etc
• Make it easy to develop and deploy new
  applications
• Tools, services, application commonality
• Interoperability enables future collaboration
  across disciplines
• Best practices, assistance, expertise
• Greatest need is software and experienced people

42
Classes of activities
Applications of information technology to science
and engineering research
Cyberinfrastructure supporting applications
Core technologies incorporated into
cyberinfrastructure
43
Defining applications

• Only domain science and engineering researchers
  can create a vision and implement the methodology
  and process changes
• Information technologists need to be deeply
  involved
• What technology can be, not what it is
• Conduct research to advance the supporting
  technologies and systems
• Applications inform research
• Shared responsibility

44
Mapping onto disciplines
All science (natural and social) and engineering
disciplines
Technological (CISE) and social systems (CISE,
SBE)
Core information technologies (CISE, E)
45
Who delivers
Commercial suppliers, development centers,
community development, integrators
End-user staff support, operational centers,
service providers
Long-term and applied researchers (applications,
systems, core technologies)
Research in technologies, systems, and
applications
Operations in support of end users
Development or acquisition
46
Evaluation and assessment
Plans impact and use
Users impact and satisfaction
Ideas outcomes
Research in technologies, systems, and
applications
Operations in support of end users
Development or acquisition
47
Responsibility for applications
48
Responsibility for cyberinfrastructure
All science (natural and social) and engineering
disciplines
Other Directorates
Applications (discipline specific)
Applications (multi-disciplinary)
CISE
49
OFFICE for the INITIATIVE

• Headed by a leader with experience, credibility,
  commitment, persuasiveness, accountability
• Complex matrix organization spaning all
  Directorates needs central direction
• Vision and coordination
• Manage INITIATIVE budget (competitive and
  community input)
• Outreach to agencies, international

50
Blue Ribbon Panel on CyberinfrastructureScope
and Budget

• Paul Messina
• California Institute of Technology
• April 19, 2002

51
To achieve its goals, the INITIATIVE should
include funding for software and people

• Long-term research in IT and CI
• Applied research in IT and CI, with deep
  involvement by applications projects
• Developing new applications enabled by IT and CI
• Enhancing existing applications to take advantage
  of the new facilities and capabilities
• Transforming research software into robust
  products

52
To achieve its goals, the INITIATIVE should
include funding for data

• Creating and operating data repositories in many
  disciplines
• taking existing data collections and making them
  conveniently accessible
• Establishing discipline-specific coordination
  centers to guide and coordinate software and data
  format choices for the repositories
• Establishing STCs for addressing common issues
  that arise in creation and use of data
  collections, especially across disciplines

53
To achieve its goals, the INITIATIVE should
include funding for physical infrastructure and
its operation

• Acquiring and operating high-end computers,
  visualization facilities, data archives, and
  networks of much greater power and in
  substantially greater quantity
• in particular, multiple computers that are among
  the worlds most powerful
• Establishing production data libraries

54
Basis for budget estimates

• Our estimates are based on
• current and previous NSF activities
• testimonies
• other agencies programs in related areas
• activities in other countries

55
Preliminary Budget Overview(Incremental)
56
Is this enough to support a revolution?

• Not by itself
• However, there are activities in CISE, in other
  parts of NSF, and in the world at large that will
  complement the funding we recommend for this
  INITIATIVE

57
Ongoing NSF CISE-funded activities that would be
folded into the INITIATIVE
58
There are other NSF activities that would
contribute to and benefit from the INITIATIVE

• NCAR
• Network for Earthquake Engineering Simulation
  (NEES)
• National Ecological Observatory Network (NEON)
• and others

59
Related activities supported by other
governmental entities

• NASA IPG
• NIH BIRN
• DOE Science Grid
• DOE SciDAC
• DOE/NNSA ASCI
• UK e-Science
• EU Grid projects (9)
• All of the above (and others) support Research,
  Development, and Deployment activities that will
  bolster the NSF INITIATIVE

60
And the private sector is also making investments

• Most high-end computer manufacturers have
  announced substantial efforts in grid software
• and are participating in Global Grid Forum
• Twelve companies announced support of Globus last
  November
• End-user companies in aerospace, pharmaceuticals
  are using or investigating grid approaches

61
Open issues

• Is the funding level high enough for the system
  software and tools RD?
• Taking into consideration the number of people
  who could and would engage in those activities
• Is the funding level high enough for the
  development of production-quality software?
• With same consideration, but note that work not
  necessarily done in universities
• Funding level for production digital libraries

62
Blue Ribbon Panel on CyberinfrastructurePACIs
Past and Future Roles

• Margaret H. Wright
• New York University
• April 19, 2002

63
The PAST

• NSF Supercomputer Centers (1986-87)
• Multiple reports (Branscomb, Brooks-Sutherland,
  Hayes) ? PACI program (1997)
• Two PACI partnerships (NCSA, NPACI)

64
The PRESENT

• Multiple functions within PACI program
• Provision of high-end resources (cycles,
  networking, data, )
• Discipline-specific codes and infrastructure
• Generic tools and infrastructure for users of
  high-end computing
• Education, outreach, and training

65
Part A of our chargeAssessment of PACI program

• Our interpretation the potential roles for the
  PACIs and PSC in a GREATLY expanded context
• Annual evaluations of PACIs positive overall
• Repeated concerns effectiveness of enabling and
  application technology projects in serving the
  science, engineering, and computer science
  communities who use high-end computing

66
Rationale for the Future

• Insatiable demand for highest-end cycles,
  networking, data (quantity, speed)
• Need for sustained work on industrial-strength
  discipline-specific codes and infrastructure,
  generic software tools and infrastructure
• Effort at least one order of magnitude greater
  than high-quality prototypes

67
Within the INITIATIVE

• Disaggregation of PACI functions
• Augmented centralized high-end resources
• Enabling/application infrastructure projects
  peer-reviewed
• Expanded, peer-reviewed education, outreach, and
  training

68
Future of PACI within the INITIATIVE

• Two-year extension of current PACI program
  requested
• Until 2007, PACIs and PSC should receive stable
  funding to provide high-end resources and
  associated operations
• 2004 INITIATIVE funding begins
• Important to retain skilled PACI staff and
  successful collaborations
• PACIs can compete for all aspects of the larger
  INITIATIVE funding
• Separate peer-reviewed enabling and application
  infrastructure projects

69
Blue Ribbon Panel on CyberinfrastructureSummary
recommendations

• April 19, 2002

70
Recommendations

• New INITIATIVE to revolutionize science and
  engineering research at NSF and worldwide to
  capitalize on new computing and communications
  opportunities
• 21st Century Cyberinfrastructure includes
  supercomputing, but also massive storage,
  networking, software, collaboration,
  visualization, and human resources
• Current centers (NCSA, SDSC, PSC) are a key
  resource
• Budget estimate incremental 650M/year
  (continuing)
• INITIATIVE OFFICE with a highly placed, credible
  leader empowered to
• Initiate competitive, discipline-driven
  path-breaking applications within NSF of
  cyberinfrastructure which contribute to the
  shared goals of the INITIATIVE
• Coordinate policy and allocations across fields
  and projects. Participants across NSF
  directorates, Federal agencies, and international
  e-science
• Develop high quality middleware and other
  software that is essential and special to
  scientific research
• Manage individual computational, storage, and
  networking resources at least 100x larger than
  individual projects or universities can provide.