PULSE: Populating the Urban Landscape with Simulated Entities

1
PULSE Populating the Urban Landscape with
Simulated Entities

• Norman I. Badler
• Nuria Pelechano
• Jan M. Allbeck

2
Goals

• Creation of heterogeneous population (i.e. human
  texture) in an urban environment.
• Link human characteristics and high level
  behaviors to graphical depictions.
• Tools to enable the creation of specific
  populations for large scale simulations.

3
Simulation Components

• Environment, objects, characters and
  Parameterized Actions.
• Create and bring together graphical, semantic,
  and functional elements such that objects can be
  reasoned about and interacted with.
• Behaviors described in concert with objective of
  simulation.
• Animations created and linked to the higher level
  behaviors and ultimately the characteristics of
  the characters what and how.

4
Parameterized Actions

• A representation to link agent and entity roles
  and activities to behavior descriptions
• Our Parameterized Action Representation (PAR)
• Actionary database.
• PAR fills in details for action execution.
• PAR fields allow simple or complex activities,
  hierarchical description, conditionals, and
  insertion of local planners (manner, route,
  pursuit, etc.).

5
Current Limitations (1)

• Limited by computational, real-time, or
  simulation design parameters.
• No adequate demonstration of the creation of a
  small city of individuals.
• Primarily individual scripted or rule-driven
  agents.
• Behaviors restricted to traveling from one
  location to another and executing a small number
  of predefined behaviors.
• Military focus on close combat tactical training
  or one-on-one language or decision-making systems
  (e.g., ICT projects).

6
Current Limitations (2)

• Commercial software for crowd behaviors
• Few individual variations.
• Concentrates on locomotion and collision
  avoidance.
• Essentially oriented particles.
• Fine-grained control of entity behaviors and
  their mutual interactions are
• Managed manually (for games or motion picture
  special effects).
• Vested in customized simulation engines such as
  JSAF or OneSAF.

7
Recent Research Developments

• Opportunistic interactions that accommodate some
  environmental or social relationships among
  entities. (See HiDAC demonstration.)
• Small-scale implementations of characters
  motivated by goals, standards and preferences
  have been demonstrated using PMFserv (requires
  careful crafting of individual character
  personalities and scenarios).
• Large scale graphics simulations starting to
  include more heterogeneous crowds that include a
  few subgroups of agent types.

8
Next Generation

• In order to create next generation city-scale
  simulations, we must include
• Realistic civilian patterns of behavior,
• Semantically meaningful interactions with the
  environment and other agents,
• Scheduled tasks relevant to simulated time of
  day/week, an agents role and goals,
• Sufficient semantic annotation of urban features,
• Real-time user interaction. (Our VR experiments.)
  

9
The PULSE Challenge

• Creating crowds of 3D characters roaming a 3D
  environment is not in itself that difficult, but
  weaving them into a background tapestry (human
  texture) of realistic city inhabitants requires

10
PULSE Needs

• Specifying the characteristics (roles, goals,
  constraints) of individuals or groups including
  their behaviors and how they might differ from
  other individuals.
• Establishing the daily activities of such
  individuals or groups according to their
  occupations and roles.
• Accessing a library of parameterized animated
  behaviors that can be selected contextually,
  varied statistically, applied to agents and
  executed in real-time in a typical simulation
  environment.
• Giving the agents enough perception to react
  (via PAR field updates) to the environment,
  people, and events around them.

11
Additional PULSE Needs

• Not our current focus, but we would want to
  contribute to the specification and design of
  these components!
• Creating semantic annotations of the 3D geometric
  environment to insure proper agent interpretation
  of obstacles, roadways, entrances, supporting
  surfaces, vehicular transport, etc.
• Building human character models that graphically
  resemble the desired population.

12
Microsoft Project

• Utilizing COTS Microsoft Project to create large
  heterogeneous urban populations.
• MS Project is user interface software for project
  management that includes characterizing and
  scheduling tasks and allocating resources.
• Individuals and groups are special types of
  resources.
• Given the specified constraints it can then
  provide schedules for individuals.
• This paradigm supports constructive simulations
  where entities go about their daily activities
  autonomously but in the urban context.
• To include possibilities for local contextual
  variation due to dynamic environment or real user
  interactions, we need to parlay high-level agent
  descriptions into situated and graphically
  realized actions.

13
MS Project Linked to PAR

• In MS Project, tasks (i.e., behaviors) can be
  broken down into sub-tasks and linked together to
  specify required orderings these high level
  behaviors ignore the details of performance.
• PAR fills in details for action execution
  (subject to context and statistical distribution
  of start times, e.g.).
• E.g., all the workers in an office building might
  finish their days shift at 5pm. The agents
  would begin the next task, that of going home,
  e.g., but they do not just start at exactly the
  same time nor do they teleport there. Each one
  has to follow a set of waypoints to reach home,
  but even that cant simply be executed. Resource
  competition, dynamic obstacles, secondary goals
  (e.g., stop at market), and crowding with other
  agents create low level path and task decisions
  at the PAR level that add contextual variation
  and social realism.

14
MS Project Advantages

• Familiar interface that is widely used.
• Existing method for scheduling tasks and
  allocating resources for large numbers of agents.
• Tasks are suggestions to agent, who then has to
  execute task in context.
• Export to XML format (or link directly to MySQL).
• Can download daily plan into each agent, though
  ultimately the actions are executed in context.
• Alternative to rich agents such as those
  created and motivated by PMFserv.

15
PAR Advantages

• Data-driven, parameterized framework promotes
  action reuse, contextual execution, and
  lightweight computational cost.
• Inherent level of detail and hierarchical
  structure facilities linking to both high level
  behavioral descriptions and low level task
  performances.
• Ability to incorporate the how into motion choice
  or execution (e.g., adverbs, EMOTE, state of
  nearby individuals (e.g., panic), wayfinding).
• Human texture activities include pre-created
  social interactions, everyday behaviors, the
  background appearance of a society.

16
Technical Challenges

• Macros and application-specific fields in MS
  Project would enable users to create individuals
  and subgroups for heterogeneous urban populations
  based on a desired distribution of occupant
  roles.
• Construct Actionary for PAR such that cultural or
  geographic fields can be user-accessed and
  updated from appropriate population profiles.
• Determine what these profiles are through
  observation of typical urban human texture of the
  region desired, coupled with statistical
  distributions to provide variety.

17
Milestones Year 1

• Demonstrate MS Project generation of a typical
  small city of about 500 occupants with
  realistic roles, schedules, activities, and
  dependencies.
• Create PARs and Actionary database to support and
  animate the inhabitants activities in context.
• MS project and PAR/Actionary integration.
• Demonstrate real-time graphics output
  (stand-alone system).

18
Milestones Year 2

• Demonstrate reactive and contextual simulation of
  the city occupants.
• Integrate into desired simulation testbed.
• Build spreadsheet customized action tools and
  controls for specific individuals, groups, or
  cultures manipulated by scenario designer.
• Link user input tools to MS Project.

19
Milestones Year 3

• Demonstrate live contextual interaction between
  user and urban inhabitants and customized agents,
  e.g., by showing PULSE human texture behind a
  specific scripted or PMFserv-driven scenario.
• Validate PULSE validity by demonstrating two
  separate cultural human texture backgrounds
  within a common simulation scenario.
• Document the efforts required to build the human
  texture data and assess progress and further
  avenues for research.

20
PULSE Populating the Urban Landscape with
Simulated Entities University of Pennsylvania,
Center for Human Modeling and SimulationDr.
Norman Badler, Dr. Nuria Pelechano, Ms. Jan
Allbeck

• Project Objectives
• Creation of heterogeneous population (i.e. human
  texture) in an urban environment.
• Link human characteristics and high level
  behaviors to graphical depictions.
• Tools to enable the creation of specific
  populations for large scale simulations.
• Technical Approach/Challenges
• Use Microsoft Project scheduling tool.
• Build on our Parameterized Action Representation
  and Actionary.
• Link these two to produce suggested goals for 3D
  animated agents.
• Execute these goals in the given social and urban
  context.
• Interpret and animate the how of an action as
  well as the what.

Animated group exiting room

• Deliverables
• Tools to construct population activities
• User interfaces to Microsoft project, PARs, and
  Actionary
• Simulations of human texture in an urban
  context
• Demonstration and validation of varying the
  cultural texture.
• Integration with other simulation systems

21
Links, Videos, and Books (Penn)

• HiDAC videos
• http//cg.cis.upenn.edu/hms/people/pelechano/MACES
  /MACES.htm
• SCA video (2D forces cal3d figures during
  evacuation drill)
• http//cg.cis.upenn.edu/hms/people/pelechano/video
  s/PelechanoSCA07.avi
• More information
• http//cg.cis.upenn.edu/hms/people/pelechano/
• These are books that we have chapters in. They
  are a good place to find the better known names
  in the various areas of research
• Life-Like Characters Tools, Affective Functions,
  and Applications, Helmut Prendinger (Editor),
  Mitsuru Ishizuka (Editor)
• Embodied Conversational Agents, by Justine
  Cassell (Editor), Joseph Sullivan (Editor), Scott
  Prevost (Editor), Elizabeth Churchill (Editor)
• Agent Culture Human-agent interaction in A
  Multicultural World, Sabine Payr (Editor), Robert
  Trappl (Editor)

22
Other Contacts

• Michael Zyda, USC
• Ruth Aylett, Heriot-Watt University
• Patrick Olivier, University of Newcastle Upon
  Tyne
• Stefan Kopp, University of Bielefeld
• Catherine Pelachaud, University of Paris 8
• Elisabeth André, University of Augsburg
• Kristinn Thórisson, Reykjavik University
• Hannes Vilhjalmsson, Reykjavik University
• Stacy Marsella, ISI/USC
• Jon Gratch, ICT/USC
• Lewis Johnson, ISI/USC
• Daniel Thalmann, EPFL Switzerland

• Amy Baylor, NSF (Florida State University)
• Justine Cassell, Northwestern
• Demetri Terzopoulos, UCLA
• Norm Badler, UPenn
• Marco Gillies, University College London
• Katherine Isbister, RPI
• John Laird, UMich
• Brian Loyall, Zoesis
• Ana Paiva, INESC-ID / Instituto Superior Técnico
• Helmut Prendinger, National Institute of
  Informatics
• Soraia Musse, Universidade do Vale do Rio dos
  Sinos

23
Other Resources

• Intelligent Virtual Agents http//iva07.ntua.gr/
  
• SIGGRAPH http//www.siggraph.org/s2007/
• Symposium on Computer Animation
  http//www.siggraph.org/sca2007/
• IEEE Virtual Reality http//conferences.computer.
  org/vr/2007/
• Eurographics http//www.eg.org/
• Autonomous Agents and Multi-Agent Systems
  http//www.aamas2007.org/