Homeland Security

1
Homeland Security

• Obviously since 9/11, homeland security has been
  brought to the forefront of public concern and
  national research
• in AI, there are numerous applications
• The Dept of HS has identified the following
  problems as critical
• intelligence and warning surveillance,
  monitoring, detection of deception
• border and transportation security traveler
  identification, vehicle identification location
  and tracking
• domestic counterterrorism tying crimes at the
  local/state/federal level to terrorist cells and
  organizations, includes tracking organized crime
• protection of key assets similar to
  transportation security, here the targets are
  fixed, security might be aided through
  camera-based surveillance and recording, this can
  include the Internet and web sites as assets
• defending against catastrophic terrorism
  guarding against weapons of mass destruction
  being brought into the country, tracking such
  events around the world
• emergency preparedness and response includes
  infrastructure to accomplish this such as
  wireless networks, information sources, rescue
  robots

2
The Dark Web

• Goal collect relevant web pages from terrorism
  web sites and make them accessible for specific
  terrorism-related queries and inferences
• Starting from reliable URLs, use a web spider to
  accumulate related web pages
• link analysis and human input are both applied to
  prune irrelevant pages

• Automatically collect the pages from the URLs and
  annotate the pages (including those with
  multimedia and multilingual content)
• Content analysis performed by humans using domain
  specific attributes of interest

3
Clustering on the Dark Web
Clustering and classification algorithms are run
on web site data to date, much of the data is
manually annotated from the web sites before
clustering/classification algorithms are
run Here are some results
Middle East terror organizations sites
Domestic web sites of US hate groups
Clustering performed using statistical
hierarchical clustering, features include those
derived through social analysis, link analysis,
and patterns derived through groups of links and
sites
4
Dark Web Continued

• With web pages annotated, the Dark Web can now be
  used for document retrieval based on search
  queries (including cross-lingual searches)
• SOM mapping
• link analysis
• content analysis
• Activity scale(c, d)
• c cluster
• d attribute of interest
• wi,j 1 if attribute i is in site j, 0 otherwise
• m total number of web sites, n total number
  of attributes

5
Dynamic Interoperability of 1st Responders

• Test implemented on Philadelphia Area Urban
  Wireless Network
• Idea is to test ability of mobile communication
  network and devices (PDA, laptop, tablet
  computers, etc) to register agents and services
  for first responders who might be mobile as well
  to acquire information during an emergency
• Need

• service registration
• service discovery
• service choreography
• Solution use OWL-S service registry for manets
  (mobile ad hoc networks)

Simulated Manet topology Agent x is looking for
a host with service e but x is only aware of
hosts A, B and D
6
Experiment

• Researchers ran two experiments, first on a
  simulation and second on a small portion of a
  network in Philly using mobile devices (in about
  a 2 block area)
• agents had various options of when to consult
  host registries and how to migrate to another
  host (as packets or bundles)
• Findings
• cross-layered design where agents reason about
  network and service dynamics worked well
• using static lists, performing random walks,
  using inertia (following the path of the last
  known location of a service) all had problems
  leading to poor performance from either too many
  hops or not succeeding in locating the desired
  service
• p-early binding agent would consult local
  hosts registry to identify nearest host that
  contained needed service and would migrate to
  that host as a packet
• b-early binding agent would consult local
  hosts registry to identify nearest host that
  contained needed service and would migrate to
  that host as a bundle
• late binding agent would consult a local
  registry at each step as it migrated, always
  selecting the nearest host with the available
  service

7
Ontology for Bioterrorism Surveillance

• This research deals with an ontology and problem
  solving agents to monitor health threats
• the system, BioStorm, includes a central ontology
  of domain-specific and task-specific knowledge
  for reasoning about bioterrorism events and
  two-tier medication to reconcile heterogeneous
  data
• problem solving methods include statistical
  (counting) approaches for low-level analysis,
  knowledge-based approaches for qualitative
  reasoning, disease-specific knowledge reasoners
  and temporal reasoners

8
Surveillance of San Francisco 911 Emergency
Dispatch Data
9
Deception Detection

• Analyze hand and face motion and orientation (via
  video) to determine degree of truth or deception
  during an interview
• Capture video features
• head position, head velocity
• left/right hand position, left/right hand
  velocity
• distance from left/right hand to head, from hand
  to hand
• Use features as input to classifier on four
  classifiers

• trained neural network
• support vector machine (perform non-linear
  classification using linear classification
  techniques)
• alternative decision trees (an ensemble)
• discriminant analysis
• NN and SVM had highest accuracy in experiments

10
Authorship Identification

• Determine who the author was of some Internet web
  forum message in Arabic or English
• Attributes
• lexical word (or character for Arabic) choice
• syntactic choice of grammar
• structural organization layout of the message
• content-specific topic/domain
• Approaches were to perform classification using
  C4.5 and SVMs
• English identification uses 301 features used
  from a test set (87 lexical, 158 syntactic, 45
  structural and 11 content-specific)
• Arabic identification uses 418 features used from
  a test set (79 lexical, 262 syntactic, 62
  structural and 15 content-specific)
• used web spider to collect test set of documents
  from various Internet forums
• Results are given below
• feature sets F1, F2, F3, F4 are the use of
  lexical, lexical, syntactic, structural and
  content-specific respectively

Notice how accuracy improves significantly as
higher level features are added
11
Arabic Feature Set
Elongation words 10 characters are rare in
Arabic, so words that were overly long
were considered to be no more than 10 characters
so that word length distribution was not distorted
12
Wearable AI

• Wearable computer hardware is becoming more
  prevalent in society
• we want to enhance the hardware with software
  that can supply a variety of AI-like services
• The approach is called humanistic intelligence
  (HI)
• HI includes the human in the processing such that
  the human is not the instigator of the process
  but the beneficiary of the results of the process
• HI embodies three operational modes

• Constancy the HI device is always operational
  (no sleep mode) with information always being
  projected (unlike say a wrist watch where you
  have to look at it)
• Augmentation the HI augments the humans
  performance by doing tasks by itself and
  presenting the results to the human
• Mediation the HI encapsulates the human, that
  is, the human becomes part of the apparatus for
  instance by wearing special purpose glasses or
  headphones (but the HI does not enclose the
  human)
• These systems should be unmonopolizing,
  unrestrictive, observable, controllable,
  attentive, communicative

13
HI Applications

• Filtering out unwanted information and alerting
• imagine specialized glasses that hide
  advertisements or replace the content with
  meaningful information (e.g., billboards replaced
  with news)
• alerting a driver of an approaching siren
• Recording perceptions
• example uses are to have the wearable record hand
  motions while the user plays piano or to record
  foot motions while the user dances to capture
  choreography, or for computer animation models
• Military applications
• aiming missiles or making menu selections in an
  airplane so that the pilot doesnt have to move
  his hands from the controls
• reconnaissance by tracking soldiers in the field,
  seeing what they are seeing
• Minimizing distractions
• using on-board computing to determine what a
  distraction might be to you and to prevent it
  from arising or blocking it out
• Helping the disabled
• HI hearing aids, HI glasses for filtering,
  internal HI for medication delivery, reminding
  and monitoring systems for the elderly

14
Beyond AI Wearables

• As the figure below shows, these devices may be
  more intimately wound with the human body
• We are currently attaching ID/GPS mechanisms to
  children and animals
• Machine-based tattoos are currently being
  researched

• What about underneath the skin?
• Nano-technology
• Hardware inside the human body (artificial
  hearts, prosthetic device interfaces, etc)

15
Sensor Interpretation

• Given sensor readings, interpret what they are
  telling you
• come to an understanding of the state of the
  device or environment
• robot/autonomous vehicle, power plant or factory,
  automobile engine, biological weapons sensors,
  etc
• given the readings, what conclusions can be
  drawn?
• this is typical a process of credit assignment
  finding the cause of the sensor readings
• in a simple situation, the reading(s) points to a
  simple conclusion that is, a one-to-one mapping
  from sensor value to conclusion
• but more often, each sensor reading is explained
  by a different hypothesis, and the hypotheses
  have to be combined into a single, coherent and
  consistent explanation
• this can be accomplished through abduction
• many approaches have been tried
  knowledge-based, Bayesian probabilities or
  Bayesian network, hidden Markov model,
  logic-based, case-based, even neural network
  approaches

16
Abduction Example

• Given a set of findings, generate some hypotheses
  that could potential explain the findings
• Evaluate the hypotheses (how plausible are they?)
• Select those hypotheses that best explain the
  findings while maintaining a consistent and
  coherent explanation
• Best includes such factors as maximally
  plausible, simplest (most parsimonious), most
  complete

Edges indicate which data a hypothesis can
explain The dotted line indicates mutually
incompatible hypotheses
17
Sensor Fusion for Autonomous Car

• Fusion coming to a high-level understanding of
  the sensor input values and how this relates to
  (or threatens) goal(s)
• Obstacle detection
• Environmental state (road conditions in this
  case)
• Sensor malfunctions
• Sensors and sensor interpretation are often
  distributed, but sensor fusion is performed by a
  central controller/reasoner

Figure of the Alice Land-based Autonomous Vehicle
18
Automated Highways

• Features
• Provide guidance information for cooperative
  (autonomous) vehicles
• Monitor and detect non-cooperative vehicles and
  obstacles
• Plan optimum traffic flow
• Architecture
• Network of short-range hi-resolution radar
  sensors on elevated poles

• Additional equipment in vehicles (transponders
  for instance for location and identification)
• Sensors on the road for road conditions and on
  the vehicles for traction information
• Sensors for other obstacles (e.g., animals)
• Computer network
• Roadway blocked off from sidewalk and pedestrian
  traffic

19
Example AV Control Architecture
20
Automated Vehicles

• Three different forms
• Remote controlled largely of no interest since
  it involves little or no AI
• Semi-autonomous decision making performed
  remotely (by humans) but the vehicle must still
  plan paths and avoid obstacles
• Autonomous all aspects controlled by computer
  (usually on-board)
• path planning, sensor interpretation, obstacle
  avoidance
• goal/mission directed planning (achieving
  objectives while maintaining safety)
• failure handling, on-board diagnostics
• Sensors depend on the type of vehicle/robot
• Cameras are difficult unless used merely to
  determine obstacle/no obstacle
• Radar, ladar (laser radar), sonar, microphones,
  ultrasound, other
• A fully autonomous vehicle will employ a number
  of AI techniques
• Mathematical modeling for path planning
• Rules or case-based reasoning for failure
  handling and goal planning
• Bayesian probabilities, HMMs or neural networks
  for low level sensor interpretation
• Some form of abduction (possibly Bayesian) for
  sensor interpretation
• Distributed control throughout the robot with a
  centralized controller to issue commands and make
  high level decisions

21
Autonomous Land Vehicles

• Types
• Mobile robots (indoor and level surface robots,
  outdoor/terrain robots)
• Autonomous automobiles
• Indoor robots tend to be easier to implement
• environment is benevolent
• most interaction/obstacles involve walls and
  furniture (static) and humans (the robot is
  usually programmed to stop and let the human
  pass)
• Autonomous cars have an even surface to work on
  but because of other drivers and the speed, they
  are very challenging its easier if we can
  assume all cars are autonomous
• then you dont have unexpected vehicle movements
  and all vehicles can directly communicate to each
  other
• Terrain vehicles have to deal with different
  types of surfaces and grades making the actual
  movement difficult and it requires more complex
  path planning
• terrain vehicles are often used for exploration
  and might be used for warfare in which case they
  also have to worry about enemy fire

22
Failure Handling No Progress Forward
23
Other Autonomous Vehicles

• Water-based (untethered) vehicles
• there are fewer obstacles to worry about so in
  fact these vehicles are easier to construct
  however, navigation is more problematic
• fewer (or no) landmarks, and currents can cause
  the vehicle to move in unanticipated ways
• these vehicles rely on sonar more than cameras
• an underwater (submarine) vehicle deals with
  depth as well as coordinates
• the greater range of mobility could make path
  planning easier in spite of it being more complex
• failure handling could be as simple as surfacing
  and moving in a circle, sending out a signal that
  a problem has arisen
• most forms of UVs (underwater vehicles) are
  remote controlled as there is less call for
  autonomous UVs than land-based vehicles
• Flying vehicles (probably the least researched)
• although they are similar to underwater vehicles
  in that they have current (air current) and depth
  to worry about, but fewer obstacles
• most flying vehicles are remote controlled
  (predator drones) but autonomous helicopters are
  being researched

24
Autonomous Space Probes

• To date, space probes (including the Mars rovers)
  have been semi-autonomous at best
• Mars rovers are partially controlled from Earth,
  but because of the time lag, path planning and
  failure handling is performed on-board
• other probes (Galileo, Cassini) are largely
  remote controlled with only on-board diagnosis
  and other simple tasks done on-board
• for instance, rotating the probe to point in the
  desired direction
• NASA wants more autonomy in their probes
• their first attempt at this was the 2003 Earth
  Observing-1 satellite with onboard continuous
  planning, robust task and goal-based execution,
  and onboard machine learning and pattern
  recognition
• to perform onboard decision-making to detect,
  analyze, and respond to science events, and to
  downlink only the highest value science data
• to manage event-driven processing/low-level
  autonomy, such as housekeeping routines
• to continuously schedule/plan, execute and replan
  for various actions as downlinking

25
Example

• Here is an image from the EO-1 satellite
• As can be seen, the satellites software
  automatically performs a variety of software
  tasks to detect events via image processing and
  feature detection, and plan to take a new image
  from the results
• EO-1s science algorithms perform several
  operations
• analyze image data
• image feature detection including cloud detection
• detect trigger conditions of science events and
  changes relative to previous observations
• on-board image editing

26
Other Uses of AI in Space/NASA

• Planning/scheduling
• Manned mission planning
• Multi-agent planning, distributed and shared
  scheduling, adaptive planning
• Planning for geological surveys (for rovers)
• Scheduling for observations (e.g., telescope
  usage)
• Deliberation vs. reactive control/planning
• Plan recovery (failure handling)
• Conflict resolution for multiple spacecraft
  missions
• Life support monitoring and control
• Simulations of life support systems
• On-board diagnosis and repair
• Safety (for humans, systems, rovers, probes)
• Science
• Weather forecasting and warning, disaster
  assessment, disaster reduction (for floods)
• Feature detection from autonomous probes (e.g.,
  crater detection by satellites)
• Other forms of visual recognition and discovery

27
Smart Environments

• Sometimes referred to as smart rooms
• although these can be any environment (autonomous
  highway for instance)
• Components collection of computer(s), sensors,
  networks, AI and other software, actuators (or
  robots) to control devices
• Goal the environment can modify itself based on
  user preferences or goals and safety concerns
• A smart building might monitor for break-ins,
  fire, flood, alert people to problems, control
  traffic (such as elevator usage), etc
• A smart house might alter the A/C when people are
  away, adjust lighting, volume, perform household
  chores (starting/stopping the oven, turn on the
  dishwasher), determine when (or if) to run the
  sprinkler system for the lawn
• A smart restaurant might seat people
  automatically, have robot waiters, automatically
  order food stock as items are getting low (but
  not actually cook anything!)

28
Environment 1 Smart City Block
29
Environment 2 Smart Highway
30
Environment 3 Smart Room