EmView: The EmStar Visualizer

1
EmView The EmStar Visualizer

• LECS Meeting
• Sept 11, 2003

2
EmStar

• EmStar is a run-time environment over Linux
• Designed for distributed, embedded systems
• Seamlessly supports same code in
• Distributed, embedded nodes
• Distributed nodes with debugging backchannel
• Centralized emulation with real embedded channel
• Centralized emulation with simulated channel
• EmView EmProxy
• Extensible visualization engine Remote state
  retrieval mechanism
• Seamlessly supports all EmStar usage modes

3
Visualization of Distributed Systems

• Distributed algorithms and protocols
• Operate concurrently
• On many physically distinct nodes, or
• On centralized emulation of many distinct nodes
• Interaction between nodes needs to be understood
• Real time dynamics in algorithm states
• Real time dynamics of traffic patterns and
  resource load
• Real time condition and exception reporting

4
Visualization of Embedded Systems

• Not completely addressed by EmView
• Designed with fast debug backchannel in mind
• Fundamental difficulty
• If debug traffic must share channel, it must be
  carefully designed to avoid contention
• Central collection of debug traffic requires
  routing and other infrastructure to be part of
  system being investigated
• If we assume an existing debug framework,
• The data can be proxied to EmView from multiple
  exit points onto fast backchannel

5
Example
6
How EmView Works

• Instrument your application
• Add Status Devices that report internal state
• Use the Status Reflector
• Implement App-specific EmView module(s)
• Parse the data your application outputs
• Represent it to the EmView core for display
• Then
• EmView requests data from your app via EmProxy
• Data floods back and is rendered on the display

App-specific EmView Modules
EmView
Status interfaces
APP
EmProxy
EmProxy
EmProxy
7
The EmProxy Protocol

• UDP-based
• Unreliable but timely data reporting
• Favors real time dynamics over reliability
• Soft state approach to guarantee recovery from
  losses
• Supports broadcast requests
• Simplifies configuration
• Enables transparent operation among
• Collections of real nodes
• Single or clustered emulation servers

8
The EmProxy Protocol

• Text request format
• Header line followed by zero or more device spec
  lines
• Must be refreshed every 10 seconds

• Header fields
• session a random session nonce that
  distinguishes a restarted client that is assigned
  the same port number
• type REQUIRED command or request. Command mode
  is not yet supported, will support broadcast
  shell commands

• Device specifier fields
• dev REQUIRED unmangled device name
• id REQUIRED tag to associate returning data
• ascii will read as ascii (binary is default)
• arg specifies device argument, will be written
  to device after opening
• reread periodic reread of device in ms, combats
  loss of response data
• ratelimit refractory period after read in ms,
  default 1000
• node optional target node, if set only this node
  will respond. Use -1 for sim itself

session323994typerequest dev/dev/link/0/status
id12reread5000 dev/dev/link/1/statusid13re
read5000
9
The EmProxy Protocol

• Responses arrive asynchronously
• UDP message, with binary header
• Specifies
• Source node ID
• Data ID (i.e. which request line)
• Time of receipt at the node (in nodes local
  time)
• Length, and binary data

typedef struct emproxy_reply_hdr node_id_t
node_id int data_id struct timeval
report_time size_t data_length char
data0 emproxy_reply_hdr_t
10
Using EmProxy

• Client library libemproxy
• Header file emproxy/emproxy.h
• Client program rechocat

rechocat -w -g ltgroup_idgt -a ltdest_addrgt
-b ltbcast_ifgt ltemproxy-command-stringgt Use
-w option to 'watch' -g group Sets the
group ID. A group ID is required if
you are watching an emulator. If the
GROUP_ID env var is set, it will
be used as a default. -b bcast_if Sets dest
addr as broadcast for specified interface.
The default is broadcast for eth0. -a
addr Sets the destination address (can be
broadcast). If no address is set,
broadcast will be used.
11
Status Reflector

• Q How can I integrate non-EmStar code with
  EmProxy?
• A Status Reflector (emproxy/status_reflect)
• Creates a device /dev/emproxy/status_reflect
• Echo in name of a new status device you wish to
  create
• A new status device of specified name will be
  created
• To pass new data to EmProxy
• Open the new status device
• Write a buffer to that new device in a single
  write() call
• Buffer is a null-term string followed by binary
  data

12
Structure of EmView

• Core
• Main program, runs gui, talks to network.
• Accepts module registrations, enables modules
• Renders viewing window
• Component
• Responsible for drawing a part of a node widget
• A few different component types (node, box, strip
  chart)
• Module
• Responsible for requesting and parsing
  application data
• Users of EmView often write a module to go with
  the application they are debugging.
• Very abstract interface

13
Dataflow in EmView
Core
Modules parse responses
Request strings aggregated
Broadcast request to EmProxy(s)
2
1
Module
4
Module
Asynch Response from proxies
Demux on data ID
3
Module
5
Render
6
Data sources updated
push abstract data to core for rendering
14
EmView Node Icons Components

• What is a Component?
• Part of a node icon display
• Component defines fields that render data
• A Node icon always contains a Node component
• Node component is named EMVIEW_NODE_COMPONENT
• Modules create other components, name them, make
  visible

Box component
Node component
ltboxgt/EMVIEW_BOX_ABOVE
Node ID (21)
ltboxgt/EMVIEW_BOX_FLAG123
EMVIEW_NODE_COMPONENT/ emview_node_flag_name(EMVIE
W_COLOR_BLUE)
ltboxgt/EMVIEW_BOX_COLOR
EMVIEW_NODE_COMPONENT/ EMVIEW_NODE_BELOW
ltboxgt/EMVIEW_BOX_INSIDE
15
EmView Module API Objects

• Module
• Logical container for all EmView mechanism
  pertaining to a specific application or EmStar
  component
• Device
• Object that handles input from an EmStar device
  file
• Component
• Object representing a sub-widget of a node widget
• Some modules hook up to another modules
  component (e.g. radio_box) others define their
  own component
• Source
• A logical data source provided to the core to be
  rendered
• Simple data source (numeric/string, per node)
• Link source (enumerate pairs of nodes)

16
API Use of string names

• Objects are typically named by strings
• Easier to integrate to GUI, debug
• Often more user-friendly
• Conventions
• Slash delimited hierarchy, e.g. radio/0/channel
• Use of functions and defines in module headers
• radio_box_name(int index) constructs RadioBoxd
• define RADIO_SOURCE_SLEEP "SleepMode
• For efficiency, glib quarks sometimes used
• Map strings to ints
• API calls that take quarks are XXX_q()

17
API Controlling Rendering

• Modules push data to core via source
• When pushed, no specification for how to render
• So how will a data source be rendered?
• Config-time API call assigns a specific source
  to a specific field of a specific component
• For example, to assign the source MySource to
  the Box component MyBox, in the Above field

int emview_assign_source(node_id_t target, const
char component_name,
int field_name, const char source_name)
emview_assign_source(EMVIEW_NODE_GLOBAL, MyBox,
EMVIEW_BOX_ABOVE, MySource)

• Color fields apply numeric source value to
  colormap Use emview_assign_color() to map a
  numeric value to valid color
• Flags display when numeric source value non-zero
• Above and Below display a stack of sources,
  not just a single one

Specify specific node or GLOBAL for all nodes
18
API Module main()

• Register the module, store pointer to module
• Register the device to watch, store pointer to
  device

emview_module_opts_t mod_opts name
ROUTING_VOPP_MODNAME, private_data state
state.mod emview_register(mod_opts)
Screen name for module
define VOPP_DEVNAME sim_path(/dev/diffusion/tree
) char buf255 emview_device_opts_t opts
name VOPP_DEVNAME, private_data state,
proxy_string buf, data_handler
vopp_handle_data, node_timeout vopp_timeout,
parent state.mod sprintf(buf,
"devsreread5000", VOPP_DEVNAME) state.dev
emview_register_device(opts)
Callback that parses the data
Screen name for device
Pass in your module
Called when node times out, e.g. clear state
Define request to emProxy
19
API Module main() continued

• Register a component
• Register a simple source
• Register a link source
• Define default linkage
• Default linkage is applied when the module is
  selected from the GUI
• Simple sources assign source to component and
  field
• Link sources Specify default color and default
  component to link to

Screen name for component
Type of component to create
emview_register_component(radio_box_name(i),
EMVIEW_COMP_TYPE_BOX) emview_component_set_visibl
e(EMVIEW_NODE_GLOBAL, radio_box_name(i), 1)
Visible for which nodes?
emview_register_source(state.dev,
ROUTING_VOPP_SINK, NULL)
Screen name for new source
Source needs device
emview_register_link_source(state.dev,
ROUTING_VOPP_LINKS)
emview_default_assign_source(state.mod,
radio_box_name(0),
EMVIEW_BOX_COLOR, ROUTING_VOPP_SINK)
emview_default_assign_link_source(state.mod,
radio_box_name(0),
0, ROUTING_VOPP_LINKS)
20
API Device Handler Callback

• Called when new data arrives from a device
• Handler responsible for parsing new data and
  updating any data source values or link sources
  that depend on that new data.
• Called with a emproxy reply and a node struct
• The reply contains the data to be parsed.
• The node struct is a convenient place to store
  per-node, per-device state, e.g. the last data
  received

static int vopp_handle_data(emview_device_t dev,
emproxy_reply_hdr_t reply,
emview_dev_node_t node)
define EMVIEW_DEV_NODE_MAX 5 struct
emview_dev_node void data size_t size
void spaceEMVIEW_DEV_NODE_MAX
/ copy the vopp struct data / if (node-gtdata)
free(node-gtdata) node-gtdata g_new0(struct
opp_emview_msg, 1) memmove(node-gtdata,
reply-gtdata, sizeof(struct opp_emview_msg)) msg
(struct opp_emview_msg )node-gtdata
Save the new data in the node struct
21
API Updating a Link Source

• Next, handler parses data and updates source
• Update the next-hop link source.
• First, clear any old links from this node
• Then, add new link(s)

Accessor for node struct
/ clear links from this any component of this
node / emview_clear_links_to_node_q(state.links,
emview_dn_get_node_id(node), 0,
EMVIEW_LINKS_FROM)
Quark for speed
/ add a link? / if (msg-gtnext_hop)
link_attr_t attr arrow 1, color
emview_assign_color(ROUTING_VOPP_SINK,
msg-gtclosest_sink) emview_update_link_q(st
ate.links, emview_dn_get_node_id(node), 0,
msg-gtnext_hop, 0, NULL, attr)
Specify which sources colormap to use
Draw arrow
Set link color to match sink
Add to link scheme
From node
To node
With style
Link to/from default component (can optionally
name specific components here)
22
API Updating a Simple Source

• Continuing handler, update ROUTING_VOPP_SINK
• ROUTING_VOPP_SINK is a sink indicator.
• If the node is a sink, the sink indicator is the
  color of the sink
• Otherwise, it is transparent (EMVIEW_COLOR_NONE
  0)
• By assigning ROUTING_VOPP_SINK to
  radio_box_name(0), field EMVIEW_BOX_COLOR, the
  sink indicator will set the color of the nodes
  radio box.

Assign color from our sources colormap
Update the sink indicator value for this node
/ update sink indicator / int sink_color
0 if (msg-gtclosest_sink msg-gtnode)
sink_color emview_assign_color(ROUTING_VOPP_SINK
, msg-gtnode) emview_update_source_int(ROUTING_VO
PP_SINK, emview_dn_get_node_id(node),
sink_color)
23
Emview CVS Structure

• emview/include/mod/
• Module-specific header files
• Define
• Names of sources and components
• Functions needed to parse or construct names
• emview/modules/
• Hierarchy of module code arranged by function
• mod.c call all module main() functions, etc.
• emview/core/
• The core implementation and component
  implementations (components may move later)

24
Emview Demos

• Neighbor Discovery, Linkstats (Alberto)
• Voronoi Diffusion Sinks, 1-phase-pull (Henri)
• Global GPS Timesync DAG (Jeremy)
• Running emview
• emview g ltgroup idgt -a ltsim addressgt -b
  ltbcast ifgt
• If group not set, uses value of SIM_GROUP env-var
• If neither a or b is set, will broadcast from
  eth0

25
Neighbor Discovery
26
Voronoi Diffusion Sink Trees
27
Global GPS Timesync DAG