Introduction to Data Explorer DX

1
Visualization - Part 1 Introduction to Data
Explorer
Karen G. Haines Brian Oetiker Paul Alsing Andrew
Pineda
2
Introduction to Data Explorer

• What is Data Explorer
• Data Explorer at the AHPCC
• Data Explorer Resources
• Starting Data Explorer
• Loading tutorials and sample programs
• Image or Scene View Control
• Creating /Connecting modules
• Importing data into Data Explorer

3
Aim of Seminar

• Provide enough understanding to
• start using DX to visualize results of your own
  research
• continue a self-paced DX education
• Introduce some basic concepts on Visual
  Programming
• Cover issues on importing data

4
History of DX

• 1988 - IBM Research wanted to develop a complete
  visualization solution using cutting edge
  technology

Hardware Power Visualization System (PVS) 32 node
Shared memory processor
Software Data Explorer
5
What is Data Explorer (DX)?

• Data visualization programming environment
• It employs a data-flow driven client-server
  execution model
• It provides a graphical program editor that
  allows the user to create a visualization using a
  point and click interface
• It comes with a complete set of standard
  visualization tools (modules) for looking at data
• Has automated fine grain parallelism for shared
  memory programming (SMP)

6
Advantages of Using DX

• Allows users to efficiently visualize results
  from an independent program in an interactive
  environment
• Useful in analyzing large amounts of dimensions
  of data and detecting program errors
• Can be used to display a time line or time slice
  of data
• Supported for a variety of platforms

7
DX Components Reviewed

• Visual Programming Editor (VPE)
• Graphical interface used to create visual
  programs
• Data Importer
• Graphical interface used to import data from
  different data output formats

8
DX Components Not Reviewed

• Module builder
• Scripting language
• DX link

9
Starting Data Explorer Windows Environment

• Upon logging onto the appropriate computer in
  the workshop are configured with Windows (PC side
  of things)
• Select
• Start-gtPrograms-gtData Explorer-gtDX Startup menu
• The Startup Data Explorer window will appear

10
Starting DX From UNIXExercise 1

• Log onto onyx1 and execute DX
• Login as usual
• At the command line type dx

11
Data Explorer Startup Window

• From the startup window you can
• run the DX tutorial (Tutorial)
• run a sample program (VPE)
• run previously written visual programs (VPE)
• import data (Data Importer)
• create or edit visual program (VPE)

12
DX Start Up Window
13
DX Resources On Line Tutorial

• Three ways to access the on line tutorial
• From the DX Startup window click on Run
  Tutorial
• Start DX in UNIX by issuing the command dx
  -tutor
• Choose Tutorial in the Help menu of the Visual
  Program Editor (VPE)
• Tutorial is an on-line overview of DX Quickstart
  Guide

14
On Line Tutorial Exercise 2

• We will now gain some experience with some of the
  more commonly used DX commands and features
• Exercise
• Start the DX tutorial
• Review the section of Tutorial 1
• Opening and executing a visual program
• (at the bottom of the opening tutorial page)

15
Opening a Visual Program

• A visual program must be opened before it can be
  executed
• Default extension for DX visual programs is .net

Exercise

• From the DX startup window select Edit Visual
  Program
• File selection dialog will appear
• Open the file
• HOME/AHPCC_Vis/dx/day1/example1.net (UNIX)
• C/dx/samples/tutorial/example1.net (WINDOWS)
• Visual programming editor will appear with
  assorted modules and connections

16
Visual Program Editor (VPE)
17
Executing a Visual Program

• After the visual program editor window appears,
  click on Execute in the VPE menu.
• This will execute the program

Exercise

• Select Execute once in the pull-down menu
• Notice that when execution begins, the word
  Execute in menu bar is highlighted
• In the VPE canvas, modules that currently
  executing are highlighted
• After executing, an image will be displayed

18
Image Displayed
19
What is Rendering?

• Rendering is the process of generating images
  from some representation of an object
• Once an object is rendered, the object data is
  lost in that data flow path
• You cannot change the camera view (angle,
  lighting, etc.) on a rendered image because the
  data needed to create the new view is not
  contained in the rendered image
• Given a camera view in the 3D world, rendering
  creates a 2D image depicting this view

20
Overview of Imaging in DX

• The Image module...
• Requires only the object as an input
• Renders the object and displays it on the screen
• Has built-in interactive controls for viewing the
  image. That is, you can use Image to control...
• size (size of image)
• view (direction, rotation)
• field of view (panning, zooming)

21
Controlling the Size of an Image

• To change the size of the image drag a corner to
  shrink window vertically and/or horizontally
• Note that the size of the object is automatically
  scaled to fit the new window size
• We will discuss later, how to change the size of
  an object in relation to the window size...

Exercise

• Experiment re-sizing the image window
• Re-execute program with resized window (WINDOWS)

22
Controlling the View of an Object Setting the
View Direction
Direction of the view can be altered by setting
the view in the View Control dialog box (alters
image automatically)
Exercise

• Access view control Dialog box from image window
  menu
• Options-gtView Control
• Experiment with the different views available
• Set View -gt top, bottom, etc
• Restore the original view of the window
• Options-gtReset

23
View Control Dialog Box
24
Controlling the View of an Object Rotation
An object can be rotated in 2D or 3D
Exercise

• In the Control View dialog, set the mode to
  rotate
• Mode-gtRotate
• Experiment with rotating the object in 2D
• 2D rotation uses right mouse button
• Experiment with rotating the object in 3D
• 3D rotation uses left mouse button

25
Controlling the View of an Object Continuous
Rotation
An object can be rotated in 2D or 3D continuously
Exercise

• Set the mode of the View to rotate
• Mode-gtRotate
• Set the Execute mode to execute upon change
• Execute-gtExecute on change
• Experiment with rotating the object in 2D and 3D

26
Controlling the View of an ObjectZooming/Panning

• Zooming enlarges an object to make it appear
  closer or reduces an object to make it appear
  more distant
• This is a way to control the size of the object
  without re-sizing the window
• Pan/Zoom one can change the center of focus while
  zooming
• Notice that you are no longer restricted to
  zooming in on the center of the object

27
Zooming/Panning(cont.)

• Exercise
• Select Zoom in the Mode option list and practice
  zooming in (left mouse button) and zooming out
  (right mouse button)
• Mode-gtZoom
• Select Zoom/Panning in the Mode option list and
  practice zooming and panning
• Mode-gtZoom/Pan
• Cancel recent effects by undoing and restore
  image to original view

28
Controlling the View of an ObjectAuto Axes
Configuration

• We will now explore a means of generating a set
  of axes for an object in the Image window and how
  to specify some of its parameters

Exercise

• Bring up the Auto Axes Configuration dialog
• Options-gtAuto Axes
• Experiment with different configuration options
  available (enter names for the axis labels, turn
  on frame and grid, etc.)
• Rotate the image to see different views of the
  object and axes

29
Auto Axes Configuration Dialog
30
Sequence Control Panel

• Allows the user to view animations of a visual
  image (if you have time series data)
• Controls are similar to a Video Cassette tape
  (forward, backward, stop, pause, etc)

Exercise

• Initialize the sequencer control panel from the
  VPE
• Execute-gtSequencer
• OR double click on the sequencer module in the
  VPE
• Experiment with different running options
  available

31
Sequencer Control Panel
STEP
PALINDROME
EDIT SEQ
LOOP
BACKWARD
FORWARD
STOP
PAUSE
32
More on the Image Module...Important Points

• The Image module
• renders an object and displays it as an image
• functions much like a combination of Autocamera
  and Display modules
• activates direct-interaction features not
  available with the Autocamera and Display modules
• Zoom, Pan
• outputs an image, and camera position

33
Display ModuleImportant Points

• Display module serves two purposes
• If an image is input, Display will display the
  image to the screen
• Given object and camera inputs, Display will
  render the object and display the image to the
  screen as specified by the camera input
• Display is most useful when the object is already
  rendered (that is, when you have an image) and
  you simply want to display it
• Display must be used when scripting is involved

34
Creating a DX Program from Scratch Overview of DX
Modules

• DX comes with a package of modules to visualize
  most types of data
• Modules are listed in the VPE
• Tool Category -gt tools
• To select a tool
• click on a tool to highlight it
• move cursor to canvas (note L cursor)
• move cursor to position on canvas and click

35
Module Icons

• Each module has input (top) and output (bottom)
  tabs
• Inputs/Outputs can be connected directly on the
  canvas by connecting I/O tabs
• Can also specify values in the configuration
  dialog box
• The dialog box is opened by double clicking on
  the module

36
Input/Output Tab Connections
37
Module Dialog Box
38
Selecting and Connecting ModulesExercise 3a

• Create a new program in the VPE
• File-gtnew
• Place the following 3 modules on the VPE canvas
• Import and export -gtImport
• Annotations - gt Plot
• Rendering - gt Image
• Connect the modules as follows
• Import -gt Plot -gt Image
• Try running the program
• (dont worry that the program does not run
  properly we did this on purpose!)

39
Simple Program 3
40
Message Window

• When an error occurs, the message window appears
  by default
• In Windows, need to close window from the menu
  bar due to Hummingbird
• Module icon name changes color if there is an
  error
• This is useful because it gives a good starting
  point for de-bugging the program...

41
Data Explorer ResourcesContext Sensitive Help

• Context-sensitive help
• available in the Help pull-down menu of the
  Visual Program Editor
• Selecting this item creates a cursor in the shape
  of ?
• Place ? on a tool icon or other feature in the
  window
• Clicking the mouse button invokes the appropriate
  Help window

42
Context Sensitive Help Exercise 3b

• Using current program, invoke the context
  sensitive help on the import tool
• Help-gtcontext sensitive
• Experiment using the help page to get information
  about acceptable data input formats
• Save the current program as
• HOME/AHPCC_Vis/dx/day1/example3.net

43
Data ImporterImporting Data into DX

• Data importer specifically designed for importing
  data in General Array format
• General Array format uses a simple header file to
  describe data characteristics
• To be able to import data using the data
  importer, you must be able to describe the data
  in a specific manner

44
Know your data

• Before importing data, you must know...
• What are the dependent and independent variables?
• for topographical data, the position (latitude
  and longitude) would be independent, while
  altitude would be dependent
• for a plot of wind velocity versus altitude in
  the earths atmosphere, the wind velocity would
  be the dependent variable, while altitude would
  be the independent variable

45
Know your data (cont.)

• What is the dimensionality of each data
  component?
• In the first example, the positional data was
  2-dimensional, while the altitude was
  1-dimensional scalar data
• In the second example, the positional data
  (altitude) was 1-dimensional, while the wind
  velocity was 3-dimensional vector data

46
Know your data (cont.)

• How is the data for the independent variable to
  be described?
• The data for the altitude may have been gathered
  on a regular grid of latitude and longitude
• The wind velocity data may have been collected at
  logarithmic intervals, or in an irregular fashion
• The positional (independent) data may be included
  in the data file as an explicit set of coordinates

47
Know your data (cont.)

• What is the nature of the connections between the
  positional data (how should the data be placed on
  a grid and interpolated)?
• Regular regular positions, regular connections
  (the standard grid)

48
What is the nature of the connections...(cont.)

• Deformed Regular Irregular positions, regular
  connections
• On this surface, regular connections are
  specified by regular (flat) planes
• Interpolation between any two adjacent points is
  linear

49
What is the nature of the connections...(cont.)

• Irregular Irregular positions, irregular
  connections
• On this surface, irregular connections are
  specified by triangular planes
• The planes are no longer regular (flat) - so the
  connections are not linear, but vary in an
  irregular way
• Can only be done in the DX native file format

50
Know your data (cont.)

• What is the format of the data?
• ASCII, Binary
• Floating point, integer, unsigned byte, signed
  byte, double, string, etc.

51
Know your data (cont.)

• Is the data position or connection dependent?
• For position dependent data, the data values are
  associated one-to-one with the positions.
• values between the positions are interpolated
  along the connection element
• A One-Dimensional Example

Values between positions are inter- polated
52
Is the data position or connection dependent
(cont.)?

• For connection dependent data, the data values
  are associated with the connections between
  positions
• Values between positions are assumed constant
  over the connection element
• A One-Dimensional Example

Values between positions are constant
x
0
53
Know Your Data (cont.)

• Is the data series data?
• The data may represent a series of frames of data
  which depends on time (time series data) or
  another parameter .
• used to create a time series animation of the
  data...
• The data may represent only one frame of data to
  be visualized.

54
Know Your Data (cont.)

• Is the data file in record (block) or
  spreadsheet (column) format?
• Record data is arranged in horizontal blocks of
  data
• Spreadsheet data is arranged in vertical columns
  of data.

Record style data
Spreadsheet style data
A0 B0 C0 A1 B1 C1 A2 B2 C2 A3 B3
C3
A0 A1 A2 A3 A4 ... B0 B1 B2 B3 B4 ... C0 C1 C2
C3 C4 ...
C
Fortran
55
Know Your Data (cont.)

• If the data is to be assigned to positions on a
  grid, is it ordered as row or column majority?
• For row majority data the last index varies
  fastest.(A11, A12, A13, ... , A21, A22, A23, ...)
• For column majority data the first index varies
  fastest.(A11, A21, A31, ... , A12, A22, A32, ...)

56
Row versus Column (cont.)

• Example If we wish to import a data file
  consisting of numbers 1,2,3,4,5, ... , 20 into a
  5x4 grid, how would column majority differ from
  row majority?

Column Majority Grid
Row Majority Grid
A111, A212, A313, ... , A126, A227, A328 ...
A111, A122, A133, ... , A215, A226, A327 ...
4
12
8
16
20
16
18
17
19
20
7
3
11
15
19
12
11
13
15
14
14
10
6
2
18
10
9
8
7
6
1
17
13
9
5
1
5
4
3
2
57
Know your data (cont.)

• Is there any embedded text in your data you would
  like to skip when the data is read?
• Header information
• Columns or rows of data to be skipped
• Comments throughout the data

58
Exercise 4

• Independent variable time
• Dependent variables student and professor
  populations
• All variables are 1-dimensional
• Independent variable is described in the data
  file
• But since its a regular series of numbers, well
  import it on a 1-D grid
• Positions and connections are regular
• Data is ASCII
• Data is time dependent

• Data values represent a single frame of data
• Data file is columnar in organization, with 100
  rows of data
• Data file has no header information
• Data file has no text interspersed within the
  data values
• File
• HOME/AHPCC_Vis/dx/day1/2DPlots/
• 2D.data

59
Importing DataExercise 4

• Invoke the Data Importer
• click on Data Importer in DX Startup window
• Select the Grid or Scattered data option

60
Data PrompterInitial Dialog
61
Data PrompterGeneral Array Description Dialog
62
Grid Data DescriptionExercise 4

• Set the appropriate variables to initially
  describe the data
• Regular grid type
• Number of variables 3
• Single time step
• Columnar organization
• Click the Describe Data button
• Another, more detailed, Data Prompter window will
  appear

63
Data PrompterData Description Dialog
64
Describing the DataExercise 4

• Set the appropriate data values
• File HOME/AHPCC_Vis/dx/day1/2DPlots/2D.data
• Grid size 100
• Data format ASCII
• Data order columnar
• Modify Field list
• Field 0 time (type int)
• Field 1 students (type int)
• Field 2 professors (type int)
• Save imported data to a file
• HOME/AHPCC_Vis/dx/day1/ex4
• will save as HOME/AHPCC_Vis/dx/day1/ex4.general

65
Other Data Viewing Optionsfrom the Data Prompter

• As a starting point, you can click on the
  Visualize data button in the data prompter
• DX will generate a program, which may or may not
  display the data as you wish to visualize it
• Tends to work best on 3-d data
• Also can browse the data to make sure it is
  being processed correctly
• Will bring up a window which is used to view the
  data file

66
Two Dimensional PlotExercise 5

• Reopen the program saved in exercise 3
• Open the Import module and enter the file name of
  the .general file created in example 4
• after entering the file name close the Import
  dialog window
• Execute the program
• should be a 2Dplot of two variables
• Invoke the context sensitive help to find out
  more about the plot module

67
Plotting Results Exercise 5
68
Data Explorer ResourcesAvailable Documentation

• DX documentation (do not print out - very large
  files)
• located in /usr/lpp/dx/doc (located on nambe)
• Programmer guide - (proguide.ps)
• Quick start guide - (qikguide.ps)
• Users reference - (refguid.ps)
• Users guide - (usrguide.ps)
• Note the Users Reference is available in html
  format at
• /usr/lpp/dx/html/index.htm

69
DX Resources

• On line tutorial
• Application comments
• Sample programs
• Context-sensitive help
• Documentation
• Web page information
• sample programs, modules, etc.

70
Helpful DX Web Pages

• http//www.man.ac.uk/MVC/training
• http//www.dartmouth.edu/comp/ac/unix/services/dx.
  html
• http//www.tc.cornell.edu80/DX
• http//www.almaden.ibm.com/DX