Data Explorer

1
Data Explorer

• Karen G. Haines
• Brian Oetiker
• Paul Alsing
• Andrew Pineda

2
Importing Data in DX

• Using the Data Explorer
• Native File Format, .dx
• Brian Oetiker
• Karen G. Haines
• Paul Alsing
• Andy Pineda

3
Topics Discussed

• DX Native File Format
• comprehensive and flexible
• can represent any of the Objects created in DX
• a data file in this format can be imported into a
  DX session by specifying dx as the value of the
  format parameter for the Import module.
• Examples shown
• Visualizing molecules, and a time series of
  molecules
• Using data, positions, connections creating user
  defined object classes
• data contained in or external to dx format file

4
Overview of DX Native File Format

• A DX (native) file consists of
• a header section followed by an optional data
  section.
• the header section consists of a textual
  description of Objects
• the data section contains the Array Object data
  either as text (ascii) or binary, and is referred
  to by the header section
• the header section can refer to Objects and data
  either in the current file or in other Objects
• See the file
• /u/demo/AHPCC_Vis/dx/day3/regular.dx
• /usr/lpp/dx/samples/data/regular.dx

5
Fields

• Field Objects are the fundamental Objects in the
  DX data model
• a field represents a mapping from some domain to
  some data space
• the domain is specified by a set of positions and
  (generally) a set of connections that allow
  interpolation of data values for points in the
  domain between specified postions
• data is either dependent on the location of the
  sample points (position dependent) or on the
  connections between the sample points
  (cell-centered data)

6
Field (cont)

• A field has componets.
• Each component has a value, that is an Object
• in general components are Array Objects
• positions is an array specifying positions
• connections is an array specifying the
  connections
• data is an array specifying the data values
• components also have attributes

7
A Simple example involving a dx native file
format

• Visualize data and connections on a
• regular 4 x 2 x 3 grid
• illustrate basic Objects of the data model
• imbed data as text in the header section
• Modules
• Import, ShowConnections, AutoColor, Tube, Image
• See the VPE files
• /u/demo/AHPCC_Vis/dx/day3/regular_maui.net
• See the data files
• /u/demo/AHPCC_Vis/dx/day3/regular.dx
• See Example 1, Appendix B, Users Guide, Version
  3, Release 1, Modification 4

8
A dx native file format regular.dx
This example describes a regular grid
object 1 are the regular positions. The grid is
4x2x3. The origin is at 0 0 0, and the
deltas are 1 in the first and third
dimensions, and 2 in the second
dimension object 1 class gridpositions counts 4
2 3 origin 0 0
0 delta 1 0
0 delta 0 2
0 delta 0 0
1 object 2 are the regular connections
Note when using the keyword gridconnections, it
is not necessary to specify the element type
or ref attribute, as these will be
automatically be set for you object 2 class
gridconnections counts 4 2 3 attribute element
type string cubes attribute ref string
positions
9
A dx native file format regular.dx
object 3 are the data, which are in a
one-to-one correspondence with the positions
("dep" on positions). The positions increment
in the order "last index varies fastest", i.e.
(x0, y0, z0) (x0, y0, z1), (x0, y0, z2),(x0,
y1, z0), etc. object 3 class array type float
rank 0 items 24 data follows 1
3.4 5 2
3.4 5.1 0.3 4.5
1 2.3 4.1
2.1 6 8 9.1
2.3 4.5 5
3.0 4.3 1.2 1.2
3.0 3.2 attribute "dep" string
"positions" A field is created with three
components "positions","connections", and
"data object "regular positions regular
connections" class field component "positions"
value 1 component "connections" value
2 component "data" value 3 end
10
Standard Field Components
Component Type Meaning data arbitrary users
data (dependent variable) positions floatn n-s
pace sample points invalid positions char which
sample points are invalid connections intk int
erpolation elements invalid connections char whic
h interp elmts are invalid colors float3 surf
ace or volume colors colors char color index
(see color map) color maps float3 color map
indexed by color comp faces int faces as a
collection of loops normals float3 curve or
surface normals ...
11
Attributes
Component Meaning dep component that this
component depend on ref component that this
component refers to der component that this
component is derived from element
type interpolation method for connections
components Class group, series, multigrid,
field, array, gridpoints, mesharray, camera,
... Attributes
12
/u/demo/AHPCC_Vis/ dx/day3/regular_maui.net
13
Exercise regular_maui.net
/u/demo/AHPCC_Vis/ dx/day3/regular_maui.net

• Invoke the VPE (new program)
• Add the modules
• FileSelector-gtImport-gtShowConnections-gtAutoColor-gt
  Tube-gtImage
• Using the FileSelector, input the file
• /u/demo/AHPCC_Vis/dx/day3/regular.dx or
• /usr/lpp/dx/samples/data/regular.dx
• Open Tube, set diameter to 0.08 (for e.g.)
• Execute the program

14
HOME/AHPCC_Vis/ dx/day3/regular_maui.net
15
Visualizing Water Molecules

• Modules
• ShowConnections, Glyph, AutoColor, Collect
• Mark, Unmark
• Sequencer, Replace
• See the VPE files
• /u/demo/AHPCC_Vis/dx/day3/h2o_maui_1.net
• /u/demo/AHPCC_Vis/dx/day3/h2o_maui_2.net
• /u/demo/AHPCC_Vis/dx/day3/h2o_maui_3.net
• See the data files
• /u/demo/AHPCC_Vis/dx/day3/h2o_001.dx, ,
  h2o_010.dx
• setenv DXDATA /u/demo/AHPCC_Vis/dx/day3

16
A dx native file format h2o_001.dx
Data Set for H20 Data Component Atomic
Numbers O H H e e OBJECT 1 CLASS array TYPE
float RANK 0 ITEMS 10 DATA FOLLOWS 8.0
1.0 1.0 -1.0 -1.0 8.0
1.0 1.0 -1.0 -1.0 Positions
Component Coords OBJECT 2 CLASS array TYPE float
RANK 1 SHAPE 3 ITEMS 10 DATA 0 Connections
Component Bonds OBJECT 3 CLASS array TYPE int
RANK 1 SHAPE 2 ITEMS 8 DATA FOLLOWS 0
1 0 2 0 3 0 4 5 6 5 7
5 8 5 9 ATTRIBUTE "element type" STRING
"lines
17
h2o_001.dx
Data-Specific Comp Pt Charges OBJECT 4 CLASS
array TYPE float RANK 0 ITEMS 10 DATA FOLLOWS
1.0 0.5 0.5 0.0 0.0 1.0 0.5 0.5 0.0
0.0 ATTRIBUTE "dep" STRING "positions" Definiti
on of the field OBJECT "H2O" CLASS field
COMPONENT "data" VALUE 1 COMPONENT
"positions" VALUE 2 COMPONENT
"connections" VALUE 3 COMPONENT "charges"
VALUE 4 END -.00371 .00259 -.00473
.00371 1.00259 -.00473 1.05193 .21342
.17034 1.05193 1.21342 .17034 -.35787
-.71558 .73548 -.35787 0.285
.73548 -.12978 -.41363 -1.00361 -.12978
0.587 -1.00361 -.57703 .92500
.08118 -.57703 1.92500 .08118
18
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_1.net
19
Exercise h2o_maui_1.net
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_1.net

• Invoke the VPE (new program)
• Add the modules (-(left)-gt begin left
  branch,
  -(right)-gt begin right branch)
• FileSelector-gtImport-(left)-gtShowConnections-gt
  Tube-gt
• 
  -(right)-gtGlyph -gt AutoColor -gt
• -(join branches)-gt Collect -gtImage
• Using the FileSelector, input the file
• /u/demo/AHPCC_Vis/dx/day3/h2o_001.dx
• Open Tube, set diameter to 0.025
• In Glyph, click set scale to 0.02, ratio to 0.3
• Execute the program Save Program as
  h2o_maui_1.net

20
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_1.net
21
Exercise h2o_maui_2.netColoring the spheres
according to the component charges in the dx file
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_2.net

• NOTE that the radii of the spheres are set by
  data (the atomic ) since the data is fed
  directly into Glyph
• We want to color the spheres according to the
  component charges in the dx data file
  /u/demo/AHPCC_Vis/dx/day3/h2o_001.dx
• Use Mark and Unmark to swap the component charges
  with the component data

22
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_2.net
23
Exercise h2o_maui_2.net
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_2.net

• On h2o_maui_1.net, Save Program As
  h2o_maui_2.net.
• Modify the right (Glyph) branch by adding modules
• Glyph--gtMark-gtAutoColor-gtUnmark--gtCollect
• NOTE that the radii of the spheres are set by
  data (the atomic ) since the data is fed
  directly into Glyph
• Mark and Unmark swaps the component charges for
  the component data, so that the spheres from
  Glyph can be colored by the charges (with color
  scheme 0.0 -gt 0.5 -gt 1.0)
• Double Click Mark, click name and set to
  charges
• Double Click Unmark, click name and set to
  charges
• Execute the program Save Program as
  h2o_maui_2.net
• NOTE On the Image Window, you may have to Reset
  Server under the Connection tab, when a data file
  is modified

24
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_2.net
25
Exercise h2o_maui_3.netTime series of
imagesEach frame of positions in a separate dx
file
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_3.net

• For a time series of data, the atomic radii (data
  component), the connections and the charges need
  only be read in once.
• Subsequent files need only specify the new
  positions of the elements of the molecule (the
  positions component)
• Use Import to read in the first file and a
  Sequencer to read in all subsequent files
• Use Replace to swap the positions component of
  each new dx file with the original dx file
  /u/demo/AHPCC_Vis/dx/day3/h2o_001.dx

26
h2o_002.dx, h2o_003.dx, ...
Data Set for H20 Data Component Atomic
Numbers Positions Component Coords Note the
0 at the end of the next line is the 0-byte
offset of the data in the line after
END OBJECT 2 CLASS array TYPE float RANK 1 SHAPE
3 ITEMS 10 DATA 0 Definition of the
field OBJECT "H2O" CLASS field COMPONENT
"positions" VALUE 2 END 1.0 .00259
-.00473 -.00371 2.00259 -.00473 2.05193
.21342 .17034 1.05193 2.21342
.17034 .647 -.71558 .73548 -.35787
1.285 .73548 .87 -.41363 -1.00361
-.12978 1.587 -1.00361 .43 .92500
.08118 -.57703 2.92500 .08118
27
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_3.net
28
Exercise h2o_maui_3.net
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_3.net

• On h2o_maui_2.net, Save Program As
  h2o_maui_3.net.
• Between Import and the left and right branch
  split (to ShowConnections and Glyph) add the
  module Replace
• connect Import into the second tab of Replace
• using the FileSelector for this branch, open file
  h2o_001.dx
• in Replace, click and set srcname and dstname to
  positions
• Into the first tab of Replace add the sequence of
  modules
• Sequencer -gt Format -gt Import -(first tab
  of)-gtReplace
• double click Format and set the template to
• /u/demo/AHPCC_Vis/dx/day3/h2o_.3d.dx
• Double Click on Sequencer
• click ellipsoids set Start1, End10, Min1,
  Max10 re-click ellipsoids
• depress loop (upper left), and press forward (gt)

29
Exercise h2o_maui_3.net (cont.)
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_3.net

• Replace
• replaces a component in a field or inserts a
  value list as a component
• syntax outReplace(srcfield, dstfield, srcname,
  dstname)
• in Replace the file in the second tab is read
  into cache first
• by clicking and setting srcname and dstname to
  positions in Replace, the positions component
  from subsequent files (h2o_002.dx -gt h2o_010.dx)
  are read in via the Sequencer (into the first tab
  of Replace) according the the format string (an
  sprintf) in Format
• Save Program as h2o_maui_3.net
• NOTE On the Image Window, you may have to Reset
  Server under the Connection tab, when a data file
  is modified

30
/u/demo/AHPCC_Vis/ dx/day3/h2o_maui_3.net
31
Exercise h2o_quaternion_3.netPrecessing
MoleculeAll position data in a single file
/u/demo/AHPCC_Vis/ dx/day3/precess/ h2o_quaternion
_3.net

• On h2o_maui_3.net, Save Program As
  h2o_quaternion_3.net.
• Substitute Display for Image.
• Instance Camera, connect output of Camera into
  second tab of Display.
• Double click Camera, set to to 0 0 0, set width
  to 5.0
• On File Selector and Format, change /day3 to
  /day3/precess
• See another way to do this using series and
  members in Example 7, Appendix B, Users Guide,
  Version 3, Release 1, Modification 4, p257ff
• See a similar way to do this using the Data
  Prompter and a general file format, in Example 4
  5, Columnar Style, QuickStart Guide, Version 3,
  Release 1, Modification 4, p82ff

32
/u/demo/AHPCC_Vis/ dx/day3/precess/ h2o_quaternion
_3.net
33
Exercise h2o_quaternion_3.net Precessing
MoleculeAll position data in a single file
/u/demo/AHPCC_Vis/ dx/day3/precess/ h2o_quaternion
_3.net

• Use h20_quaternion_3.net
• setenv DXDATA /u/demo/AHPCC_Vis/dx/day3/precess
• Under File Selector, set file to
• /u/demo/AHPCC_Vis/dx/day3/precess/h2o_qu
  aternion_000.dx
• double click Format and set the template to
• /u/demo/AHPCC_Vis/dx/day3/precess/h2o_quatern
  ion_.3d.dx
• Click on the ellipses of the Sequencer and set
  Max to 50 (lt 200)
• Execute Program, by pressing play (gt) on the
  Sequencer

34
/u/demo/AHPCC_Vis/ dx/day3/precess/ h2o_quaternion
_3.net
35
h2o_quaternion_002.dx, h2o_quaternion_003.dx,
...
h2o_quaternion_001.dx OBJECT 2 CLASS array TYPE
float RANK 1 SHAPE 3 ITEMS 5
DATA FILE h2o_all.dat, 250 OBJECT
"H2O" CLASS field COMPONENT "positions"
VALUE 2 END
h2o_quaternion_002.dx OBJECT 2 CLASS array TYPE
float RANK 1 SHAPE 3 ITEMS 5
DATA FILE h2o_all.dat, 500 OBJECT
"H2O" CLASS field COMPONENT "positions"
VALUE END
36
Exercise h2o_quaternion_3.netTime series of
imagesEach frame of positions in a separate dx
file
/u/demo/AHPCC_Vis/ dx/day3/precess/ h2o_quaternion
_3.net

• Data was written in fixed record length format to
  a single file h2o_all.dat in ascii format
• In Fortran 90
• open(1,fileh2o_all.dat,formformatted,access
  direct,recl50)
• write(1,( 3(F12.5,2x) ), reci) x, y, z
  !icurrent record being written to file
• 5 lines (records) constitutes on time frame of
  the molecules position.
• Thus the byte offset into each file is
  250frame_number where frame_number is the
  current number of the time series frame which is
  also attached to the file, e.g. frame_number2 gt
• write byte offset into file h2o_quaternion_002.dx
• Note dx native file format uses byte offsets
  only (no skip lines available)