Dynamic Data-Driven Application Simulation (DDDAS)

1
Dynamic Data-Driven Application Simulation (DDDAS)
Clay Harris Jay Hatcher Cindy Burklow
2
General Simulation

• Calculations are predefined
• Boundary conditions are predefined
• Initial data is given
• Time step is predefined
• Additional data input at predetermined times
• Results are recorded and often studied later

3
DDDAS

• Calculations may change depending upon the
  incoming data
• Boundary conditions may be updated during the
  simulation
• Initial data is given, but may be corrected at a
  later time
• Time step may change depending upon incoming data
  values
• Additional data comes in anytime and out of order
• Frequently the results are monitored in real time

4
What is DDDAS?

• A DDDAS is one where data is fed into an
  executing application either as the data is
  collected or from a data archive 1, p. 662.
• The data is then used to influence the
  measurements for additional data the simulation
  may require.

1 Frederica Darema. Dynamic Data Driven
Applications Systems A New Paradigm for
Application Simulations and Measurements.
International Conference on Computational
Science. 662-669. 2004
5
Dynamic Predictions

• Wildfire Forecasting
• Tsunami Forecasting
• Traffic Jam Forecasting
• Weather Forecasting
• Global Warming El Nino
• Ocean Modeling
• Cyclone Movement Prediction
• Threat Management in Urban Water Supplies
• Fault Diagnosis of Wind Turbine System
• Operational Control for Manufacturing
• Brain Machine Interface
• Landscape Biophysical Change

6
Keep in mind with DDDAS

• Typically approximating a nonlinear time
  dependent partial differential equation
  nontraditional convergence
• Perturbations from incoming data
• Inaccurate data
• Propagation of error
• Boundary conditions are rarely known

7
Traditional Simulation Infrastructure
CPU
Graphical Output
Initial Conditions
Initial Algorithm
8
DDDAS Infrastructure
CPU
Real-Time Sensors
Graphical Output
Initial Algorithm
9
What is DDDAS
Simulations (Math.Modeling Phenomenology Observati
on Modeling Design)
Theory (First Principles)
Simulations (Math.Modeling Phenomenology)
Theory (First Principles)
Experiment Measurements Field-Data User
Experiment Measurements Field-Data User
Dynamic Feedback Control Loop
Challenges Application Simulations
Development Algorithms Computing Systems Support
Frederica Darema, NSF
10
A DDDAS Model(Dynamic, Data-Driven Application
Systems)
Discover, Ingest, Interact
Models
Discover, Ingest, Interact
Computations
Loads a behavior into the infrastructure
sensors actuators sensors actuators
sensors actuators
Humans 3 Hz.
Cosmological 10e-20 Hz.
Subatomic 10e20 Hz.
Computational Infrastructure (grids, perhaps?)
Spectrum of Physical Systems
Craig Lee, IPDPS panel, 2003
11
DDDAS Research

• Data injection methods
• 2-way communication with sensors
• Quick methods for static simulation conversion to
  DDDAS
• Infrastructure support for dynamic methods
  including communications support, data driven
  technologies, and OS software

12
Data Determines Everything

• The algorithm used
• Additional data collected
• Simulation restart (cold or warm)
• Output correction
• Communications with people
• The Result!

13
Dynamic Work Flows

• Flexible event handling system notifies
  appropriate recipients of relevant events
• Dynamic workflow handling system coordinates and
  schedules actions in response to known events

14
Dynamic Work Flows

• Events delivered using a publish/subscribe model
  or based on content
• Decision makers receive event notification and
  make an appropriate response decision
• Responses are executed by a workflow engine that
  schedules data transfer and process execution

15
Dynamic Work Flows

• Besides events causing an initial response,
  subsequent events may alter an existing workflow
• Current amount of workflow completed must be
  determined
• Current tasks on the leading edge of the
  workflow must be terminated or allowed to
  complete
• Status and disposition of data referenced by data
  handles must be determined
• Storage management issues
• Dangling references to no data or stale data
• Inaccessible data referenced by no one

16
Data Driven Design Optimization Methodology
(DDDOM)

• DDDOM uses DDDAS to find an optimal solution to
  an engineering design problem
• Used in Multi-criteria Design Optimization (MDO)
  problems
• Uses Rapid Prototyping, Grid Computing, and other
  advanced technologies to perform simultaneous
  experimentation and simulation to achieve optimal
  designs

17
(No Transcript)
18
DDDOM Architecture
19
DDDOM Application Cooling of Electronic
Components

• Optimize design for cooling system
• Maximize heat transfer and minimize pressure drop
• Increasing heat transfer also increases pressure
  drop, so there is no specific solution, but
  rather a set of good solutions
• Problem is a MDO problem

20
DDDOM Application Cooling of Electronic
Components

1. Select 25 sampling points from design space
2. Perform computations at these 25 points on
   supercomputers
3. Get experiment data from experiments
4. Combine experiment and simulation data together
   to build Surrogate Model
5. Optimize SM and obtain the Pareto Set

21
DDDOM Application Cooling of Electronic
Components

• Two Optimization methods used
• Epsilon constraint method
• Multi-Objective Switching Genetic Algorithm
  (OSGA)
• Results are comparable, with OSGA giving more
  data points

22
DDDOM Application Cooling of Electronic
Components
23
OSOAP

• A web services framework for DDDAS applications.
• Geographically distributed set of application
  components
• Reduces the effort required to develop DDDAS
  applications

24
OSOAP Advantages over traditional monolithic
applications

• Developer only needs to implement a program
  component on a single local platform
• Loosely-coupled nature of the components
  facilitates reuse for new simulations
• Allows simultaneous use for multiple research
  projects

25
OSOAP

• Current web service technologies are inadequate
  for DDDAS applications
• They are generally geared for more interactive
  applications
• Often have a learning curve that is steep enough
  to discourage computational scientists from
  experimenting with a remote DDDAS system

26
OSOAP

• DDDAS developers must consider
• Generating Interface Documentation
• Data management concerns
• Asynchronous Interactions
• Authentication, Authorization and Accounting
  (AAA)
• Job Scheduling
• Performance

27
OSOAP

• Current web service technologies present the
  developer with a blank slate
• For a novice, developing a web serviced DDDAS
  application is a difficult undertaking
• OSOAP provides a framework for designing DDDAS
  applications with minimal interface code and
  developer effort

28
OSOAP - Implementation

• Applications deployed as distributed components
• Services automatically documented with WSDL
• Asynchronous communication supported by sending a
  job ID for the remote application back to the
  client, which periodically checks the remote
  applications status

29
OSOAP - Implementation

• Supports small and large data sizes
• If data size is small or programmer requests the
  data is included in a SOAP envelope as XML (pass
  by value)
• If data is large or programmer requests a URL is
  sent in the envelope pointing to the data (pass
  by reference)

30
OSOAP Implementation

• Performance measured with the Pipe Problem
• Simulates an idealized segment of rocket engine
  modeled after one of NASAs experimental rocket
  designs
• Three different sizes of the Pipe Problem used to
  evaluate how performance scales

31
OSOAP Implementation
32
Some Characteristics of DDDAS Projects

• Managing complex scenarios
• Predicting high risk areas safety
• Effects large population of people
• Involves natural environment
• Impact on the overall economy
• Needs multi-disciplined team
• Real-time analysis is critical

33
Threat Management in Urban Water Distribution
Systems

• Situation
• Highly interconnected water transport system
• Frequent flow fluctuations
• Highly dynamic transport paths
• Single point of contamination can quickly spread

34
Contamination Threat Management of drinking H20
involves.

• Real-time characterization of contaminant source
  plume
• Identification of control strategies
• Design of incremental data sampling schedules.

35
Why use DDDAS

• Requires dynamic integration of time-varying
  measurements of flow, pressure and contaminant
  concentration
• Uses analytical modules are highly
  compute-intensive, requiring multi-level parallel
  processing via computer clusters

36
Projects DDDAS infrastructure

• Develop cyber-infrastructure system that will
  both adapt to and control changing needs in data,
  models, computer resources and management choices
  facilitated by a dynamic workflow design
• Virtual Simulations
• Field Studies

37
Fault Diagnosis ofWind Turbine Systems

• Current Situation
• Current practices are non-dynamic non-robust
  for modeling, data collection, processing
  strategies
• Clean wind energy cannot compete with traditional
  energy source
• High financial cost compared to other energy
  sources
• High maintenance cost
• Low confidence in the diagnosis technology
• Need for enabling a cost-effective generation of
  wind electricity

38
Involves

• Development of diagnosis system for wind
  turbines
• Fault diagnosis of blades and gearboxes
• Utilizes historical online signals
• Employs novel de-noising sensor anomaly removal
  algorithms

39
Why use DDDAS

• Involves collaborative research that is
  multidisciplinary
• Benefits a larger range of industries such as
  power generation, automobile, aerospace, and
  engine industries.
• Effects the overall general population with clean
  air issues
• Effects energy economic costs

40
Projects DDDAS infrastructure

• 2 robust data pre-processing modules for
  highlighting fault features and removing sensor
  anomaly
• 3 interrelated, multi-level models that describe
  different details of the system behaviors
• 1 dynamic strategy for the robust local
  interrogation that allows for measurements to be
  adaptively taken according to specific physical
  conditions and the associated risk level.
• Overall incorporates both historical data and
  on-line signals into the system modeling

41
Production Planning Operational Control for
Distributed Enterprise

• Society depends upon many interacting large-scale
  dynamic systems
• Too complex for mathematical analysis
• Behavior of system networks depends on their
  linkages and the environment

42
Involves

• Focus on hierarchical production
• Logistics planning
• Control in highly capitalized discrete
  manufacturing system networks

43
Why use DDDAS

• Requires complex simulations
• Needs dynamic reaction to various situations
• Utilizes centralized control
• High cost financial risk involved

44
Projects DDDAS infrastructure

• Multi-scale federation of interwoven simulations
• Decisions models for planning
• Control with capability for dynamic updating
  through sensors
• Capacity to use off-line performance testing
• Integrated architecture for distributed computing
• Utilizes sensors, transducers, and actuators
• Web service technology

45
Brain-Machine Interfaces (BMI)

• Brain receives uses sensory feedback to learn
  generate signals to produce purposeful motion.

• Address chief problemin current BMI research
  paraplegics cannot train their own network models
  because they cannot move their limbs.

46
Involves

• Cognitive brain modeling from experiments with
  live subjects
• Design of brain-inspired assistive systems to
  help human beings with severe motor behavior
  limitations (e.g. paraplegics) through
  brain-machine Interfaces (BMIs).
• BMI uses brain signals to directly control
  devices such as computers and robots.

47
Why use DDDAS

• Complexity of relationship between the brain
  nervous system
• Learning occurs simultaneously for the subject
  and the control models in a synergistic manner
• Selective use of many computational models
• Interdisciplinary team

48
Projects DDDAS infrastructure

• Develop models
• Implement algorithms
• Deploy computational architecture

All the above will utilize recently proposed
advanced brain models of motor control.
49
Sensor Networks Enabling Measurement, Modeling
Prediction of Biophysical Change in a Landscape

• Collecting environmental data is challenging
• Deployed in remote locations
• No access to infrastructure (e.g. power)
• Wide range of sampling time variables

50
Involves

• Understanding how biodiversity carbon storage
  are influenced by global change
• Wireless sensor network
• Models of tree growth resource allocation
• Adaptive sampling across diverse time space
  scales

51
Why use DDDAS

• Integrates sensors with modeling in adaptive
  framework
• Requires network controls that must be dynamic
• Driven by models capable of learning adapting
  to both environment network

52
Projects DDDAS Infrastructure

• Network of wireless sensors on trees
• Environmental models that provide real-time
  approximate answers.
• In-network controls that schedule new
  measurements
• Communication system to transfer data to server

53
Coast Environment Modeling Applications

• Urgent scientific ecological problems Ocean
  circulation, storm surge, and wave generation
• Coastal Modeling of Louisianas coastal
  Mississippi Delta region

54
Involves

• Modeling ecological, hydrodynamic, sediment
  transport in the Delta
• Develop new infrastructure algorithms to
  address issues for ocean circulation, storm surge
  wave generation
• Collect data via external wireless sensors from
  both water wind

55
Why use DDDAS

• Real-time coupled with data input complex
  workflows
• Complex simulations
• Huge impact on human animal quality of life
• Economical environmental devastation of
  Hurricanes Coastal Flooding

56
Projects DDDAS infrastructure

• Develop system called DynaCode.
• Utilize emerging standards for Cactus, Triana,
  and Grid Services
• Wrapping legacy codes
• Integrating framework for new advanced code
• Running multi-scale simulations

57
Reactive Observing Systems (ROS)
What is ROS?

• A class of observing systems that are
• Embedded into the environment
• Consist of stationary mobile sensors
• React to collected observations

58
Goals of ROS

• Verify or falsify hypotheses with samples taken
  via sensor devices
• Analyze data autonomously to detect trends or to
  alert problematic conditions

59
Applications

• Resource Management
• Environmental Protection
• Public Health
• Any area that requires close environmental
  monitoring would benefit from ROS.

60
Current NSF Grant for ROS

• Focus on marine biology application monitoring
  the concentration of algae micro-organisms
• Stationary mobile sensors
• Wireless wired links
• Collects data in real time

61
Harmful Algae Bloom
62
Why considered DDDAS.

• Develop approach to optimized control sample
  sets of all possible relevant data
• Secure sample at any time while taking in account
  apps objectives resource constraints
• Automatic validation adaptation
• Includes distributed support mechanism for
  locating relevant data of interest

63
Other current DDDAS projects

• Integrated Wireless Phone Based Emergency
  Response System (WIPER)
• Integrating Real-Time Data and Intervention
  During Image Guided Therapy
• Real-Time Order Promising and Fulfillment for
  Global Make-to-Order Supply Chains
• Optimal interlaced distributed control and
  distributed measurement with networked mobile
  actuators and sensors
• Dynamic, Simulation-Based Management of Surface
  Transportation Systems
• Interactive Data-driven Flow-Simulation Parameter
  Refinement for Understanding the Evolution of Bat
  Flight
• Planet-in-a-Bottle A Numerical Fluid-Laboratory
  System
• Integrating Multipath Measurements with Site
  Specific RF Propagation Simulations
• Auto-Steered Information-Decision Processes for
  Electric System Asset Management
• Measuring and Controlling Turbulence and Particle
  Populations
• Robustness and Performance in Data-Driven Revenue

64
Useful Links

• http//www.dddas.org
• http//www.nsf.gov/cise/cns/dddas/index.jsp
• http//www.iccs-meeting.org
• http//www.teragrid.org

Reinforcement Learning in Robotics
http//www.fe.dis.titech.ac.jp/gen/robot/robodem
o.html
65
References

• Frederica Darema. Dynamic Data Driven
  Applications Systems A New Paradigm for
  Application Simulations and Measurements.
  International Conference on Computational
  Science. 662-669. 2004
• Craig Lee, IPDPS panel, 2003 http//www710.univ-ly
  on1.fr/cpham/GDT/DOC/EventDrivenWorkflows_Lyon_09
  04.ppt
• http//www.dddas.org/projects.html
• http//www.cse.nd.edu/news/news.php?id762
• https//www.cs.duke.edu/ari/millywatt/funding.html
  
• http//www.engr.uconn.edu/jtang/research.htm
• http//www.acis.ufl.edu/index.php?l44
• http//www.darpa.mil/baa/baa01-42mod1.htm
• http//forestry.about.com/library/tree/blredwd.htm
  ?pid2820cobhome
• http//www.whoi.edu/science/B/redtide/whathabs/wha
  thabs.html
• http//www.whoi.edu/science/B/redtide/rtphotos
• http//www.baldridge.unizh.ch/nsf/ITR_RTIGNS/
• http//splweb.bwh.harvard.edu8000/
• http//citeseer.ist.psu.edu/656858.html
• http//www.cs.cornell.edu/stodghil/paper...iccs04.
  pdf
• http//coewww.rutgers.edu/knight/dddom/main/deopt.
  php
• http//www.iccs-meeting.org/iccs2006/
• http//phsi.mgmt.purdue.edu/dddas/project.html
• http//www.teragrid.org