ARION Workshop on FSSE

1
ARION Workshop on FSSE

• Poseidon
• A Distributed System for Interdisciplinary Ocean
  Forecasting with Adaptive Modeling and Sampling
• PIs N.M. Patrikalakis, J.J. McCarthy, A.R.
  Robinson, H. Schmidt
• Staff C. Evangelinos, P.J. Haley Jr., P.F.J.
  Lermusieux, R. Tian
• http//czms.mit.edu/poseidon

2
Ocean Science and Data Assimilation

• Field and remote observations
• Models
• Dynamical
• Measurement
• Error
• Assimilation schemes
• Sampling strategies
• State and parameter estimates
• A Dynamic Data-Driven Application System (DDDAS)

3
Applications

• Ocean Modeling applications
• Better understanding of the ocean's biochemical
  and physical processes
• Ocean Forecasting applications
• Pollution control
• Spills (oil, toxic, etc.)
• Harmful algal blooms
• Resource exploitation and management
• Fisheries
• Maritime and naval operations, law enforcement
• Sea currents

4
Pollution ControlHarmful Algal Blooms
5
Resource Exploitation Fisheries
6
Maritime and Naval OperationsEgypt Air crash
(10/31/1999)
Egypt Air Flight 990 - Floating Debris
Dispersion southern (left) vs. northern (right)
impact point
7
"Poseidon" ITR/APIM scope

• Forecasting, data assimilation, error estimation
  and observations
• Operational real-time Error Subspace Statistical
  Estimation (ESSE) physics-biology-acoustics
• Error metrics for quality control, adaptive
  modeling and sampling
• Adaptive sampling (guided, automated)
• Advanced Models
• Adaptivity
• Biological Oceanography dynamical models
  parameters
• Physical Oceanography parameters
• New interactions
• Physical Oceanography with Acoustics
• Acoustics with Biological Oceanography
• Distributed Computing Infrastructure and Web User
  Interface

8
Physical-Biological Oceanography in HOPS

• Primitive Equation (PE) dynamical model
• Multiple biological models
• Optical dynamical model
• Interface with ocean acoustics
• "Legacy" F77 FD code, multiple if make structure
• NetCDF used throughout
• Portable

9
Error Subspace Statistical Estimation
Improved forecasts with a dynamic error
subspace Ensemble-style approach (w/ full
nonlinear model) Not tied to specific model but
integrated with HOPS
10
Software Solutions

• Exploit parallelism opportunities
• Balance performance and changes for the user
• Direct MPI coupling for strong interactions
  (code)
• Automate file I/O based workflows for weak
  couplings
• Work to the maximum extent possible at the binary
  level
• Metadata for software use (and installation) in
  XML
• Use Grid technologies
• For user, compute and data access solutions
• Drive forecasting, visualization workflows on the
  Grid
• Present results to user's web browser

11
Interdisciplinary Interactions
12
Adaptive Biological Models
13
Modeling issues

• New multi-species adaptive model Variable of
  species, predefined types
• Multiscale coupling across species Within same
  type, across types
• Physics driven, can feedback to physics
• Implementation (compromising between modularity
  and performance)
• Being implemented as separate MPI parallel
  programs, strongly coupled
• Use of a coupling toolkit (eg. MCT) as a later
  development
• Mixed language programming (using function
  pointers) for code adaptivity
• Issues for the future
• Mismatch in space (grid) and/or time (timestep)
• Non-HOPS physical model
• Different modes of adaptive use
• (Many) One-to-one or one-to-many in an ESSE
  ensemble

14
Concurrently executed models
15
Time-adaptive coupled models
. . .
(Nphy)
(Nbio)
16
Initialization Scheme
17
Criteria for adaptivity

• Forecasted results enter regimes where modeling
  assumptions change
• Forecast mismatches
• Observation-forecast well outside forecast
  uncertainty
• Adapt at (adjusted) measurement times
• Choose best model at end
• But uncertainty forecasts assume completed ESSE!

• We can rerun ESSE with new models, discarding or
  adjusting with old results
• Better still to adapt within an ESSE workflow
• Use (adjusted) forecasted uncertainties from
  yesterday
• Estimate uncertainties using a mini-ensemble
  ESSE at frequent time intervals. Discard
  forecasts, increase ensemble size as necessary.

18
Adaptive Sampling

• DDDAS (in the other direction)
• From forecasting to observations
• Use forecasts and their uncertainties to alter
  the observational system in space
  (locations/paths) and time (frequencies) for
  physics, biology and acoustics.
• Locate regions of interest, based on uncertainty
  values and interesting physical/biological
  phenomena.
• Plan observations under operational, time and
  cost constraints to maximize the impact on
  uncertainty (at final time or over the
  observation time interval).

19
MultiSensor Adaptive Sampling
20
Legacy code

• HOPS as well as the acoustics codes used in
  Poseidon are all relatively large complex Fortran
  codes. Source code is available but development
  approach has inertia.
• Complicated makefile options, interdependent/confl
  icting
• hundreds of build and runtime parameters
• For other (future) codes no source code may be
  available
• Classic encapsulation techniques that
  compartmentize the code into subroutines, called
  from wrappers require constant reworking if not
  adopted by the developers
• Encapsulation done at the binary level instead
  but the approach to the problem is more generic

21
HOPS pipeline
22
XML encapsulation for "legacy" binaries

• Encapsulation at the binary instead of
  function/subroutine level (software metadata)
• File and command line I/O replace call arguments
  for the purposes of encapsulation
• Binaries can be built on demand per specification
  from generated makefiles
• Descriptions of I/O files, runtime parameters,
  stdin and command line arguments, makefile
  parameters, requirements and conflicts for
  options, invocation mechanism are needed
• Essentially a computer readable install and user
  guide
• XML description provides software use and build
  metadata
• Design of appropriate hierarchical XML Schemas
  (evolutionary)
• Data file metadata is also usable (NcML for
  NetCDF)
• Provides the constraints for the generation of
  workflows (file I/O based)
• Developers need to keep XML description
  up-to-date with their code (incremental effort)
  w/out switching to more elaborate approaches
• Concept is generally applicable, directly useful
  with other ocean models
• Java-XML based tool generates graphical front end
  for an application using its metadata
• The tool ensures that supplied parameters are
  valid (error checking for new users)

23
Metadata for handling legacy software

• Hierarchical structure for describing code (can
  handle binary only case)
• Basic assumptions
• No GUI, all runtime control from the command line
  and input/stdin files
• All build-time settings can be done by altering
  the makefile and select values (parameters) in
  included files.
• (Currently) cannot handle intelligent stdin
  parsing
• Datatypes and relevant ranges for each parameter
  to ensure validity

24
XML description
lt?xml version'1.0' encoding'UTF-8'?gt ltgroup
endText"99 END of input datagt ltset
info" CARD 1 Various Intialization
parameters"gt ltordergt1lt/ordergt ltvargt
ltnamegtNTSOUTlt/namegt
ltdescriptiongtnumber of timesteps between output
of data.lt/descriptiongt ltconstraints
type"integer"gt ltintegergt
ltvaluegt48lt/valuegt
ltrangegt0POSITIVE_INFINITYlt/rangegt
lt/integergt lt/constraintsgt lt/vargt
... ... lt/setgt ...
... lt/groupgt
25
Java-based GUI for legacy binaries

• Prototype GUI, accepts generic set of description
  files and generates user interface for building
  and running the binary
• Validates user choices, generates relevant
  scripts
• Integral part of the Grid-portal for Poseidon
• Future directions include
• Unit conversion
• Workflow composition Employing the descriptions
  of the binaries and their input/output files as
  constraints. Currently using predefined
  workflows.
• Context mediation When connections mismatch

26
GUI runtime parameters
27
GUI validity checking
28
Generated stdin file
1 CARD 1 Various Intialization
parameters 1 672 8919.0 92 48 1 10 0 0 2
CARD 2 Time-Stepping Parameters 900 900
900 3 CARD 3 Horizontal Mixing Scheme
Switches 1 1 1 4 CARD 4 Shapiro Filter
parameters for momentum, tracers, vorticity 2
1 1 4 5 1 2 1 1 4 1 0 5 CARD 5 Laplacian
horizontal mixing parameters (cm2/s) 1.0E9
2.0E7 6 CARD 6 Vertical mixing parameters
1.0 5.0 1000.0 1000.0 50.0 0.5 1000.0 1000.0 7
CARD 7 Mixed Layer Depth parameters 0
10000.0 10000.0 10000.0 0.7 1.25 0.0 4.0E-4 8
CARD 8 Over-relaxation parameters 300
1.0 1.0E-4 0.5 9 CARD 9 Bottom Mixed
Layer. 0.0025 ... ...
... 30 CARD 30 string with a
maximum of eighty characters. press_bias.nc 31
CARD 31 string with a maximum of eighty
characters. bioAnder.in 32 CARD 32
string with a maximum of eighty characters.
tsource.dat 33 CARD 33 string with a
maximum of eighty characters. grids.nc 99
END of input data
29
User view of the architecture
30
Backup slides

• ESSE equations
• Detailed ESSE workflow
• Example of ESSE bio-physical and acoustical use
• Different modes of parallelism
• Feature extraction

31
(No Transcript)
32
(No Transcript)
33
Use of ESSE Assimilation inCoupled Simulations
Aug.-Sep. 1998 Mass. Bay experiment ESSE
assimilated simulation results compared to
unassimilated forecast and persistence results.
34
Acoustic Data Assimilation via ESSE

• Enhanced acoustic DA for any acoustics model
• ESSE physical ocean realizations lead to ensemble
  of acoustic propagation solns.
• Coupling between ocean acoustics and phys.
  Oceanography is loose because of timescales.

35
Parallelism

• Functional Parallelism
• 1-1
• 1 adaptive physical solver
• 1 adaptive bio-solver
• 1-many
• 1 adaptive physical solver
• Many bio-solvers
• Data Parallelism
• Moderate of procs
• Domain decomposition
• Multi-block or column/row
• Block/cyclic(eg. for SVD)
• Nesting

36
Nested Parallelism
37
Feature Extractionfor Adaptive Sampling

• We need an automated procedure to identify
  features of interest in the flow upwelling,
  eddies gyres, jets and fronts etc.
• Output is used to visually guide the user as well
  as to optimize sensor locations and paths and
  observation patterns.
• Used in combination with uncertainty information