SAGE Development

1
SAGE Development
2
Architecture

• Free Space manager provides central control
  between apps, UI and system
• SAGE applications send their streams directly to
  the display nodes

3
Writing/Porting SAGE applications

• Modify an existing application.
• Source code should be open.
• Source should be in C.
• If source code is not easily modifiable we can
  try using shared memory. Requires a separate C
  program to read the shared memory and send pixels
  to SAGE
• Write an application from scratch
• Cleanest solution

4
Serial vs Distributed Apps

• Simple SAGE apps run on a single machine. Have
  one frame buffer
• E.g. atlantis
• Distributed SAGE apps can run across multiple
  machines (cluster). Have many frame-buffers
  across multiple machines.
• E.g. MagicCarpet

5
Distributed Apps

• Use MPI or simple ssh to spawn instances across
  machines
• Responsible for internal syncing before doing a
  collective sageSwapBuffer()
• MagicCarpet
• JuxtaView

6
OpenGL Code SAGEified
int main(int argc, char argv)      //
application code       glPixelStorei(
GL_UNPACK_ALIGNMENT, 1 )          sailConfig
scfg       scfg.cfgFile "sage.conf"      
scfg.appName "render"       scfg.rank
0       scfg.ip argv2       scfg.resX
400  scfg.resY 400         sageRect
renderImageMap       renderImageMap.left
0.0       renderImageMap.right 1.0      
renderImageMap.bottom 0.0      
renderImageMap.top 1.0         scfg.imageMap
renderImageMap       scfg.colorDepth 24      
scfg.pixFmt TVPIXFMT_888       scfg.rowOrd
BOTTOM_TO_TOP                          
sageInf.init(scfg)       cout ltlt "sail
initialized " ltlt endl       glutMainLoop()

• // headers for SAGE
• include "sail.h"
• include "misc.h"
•  
• void reshape(int width, int height)
•        // blah blah
•  
• //display function
• void redraw(void)
•        // draw code
•        glReadPixels(0, 0, winWidth, winHeight,
  GL_RGB,
•        GL_UNSIGNED_BYTE, rgbBuffer)
•        sageInf.swapBuffer((void )rgbBuffer)
•  
•        glutSwapBuffers()

7
A Native SAGE App

• // for SAGE
• include ltsail.hgt
• include ltmisc.hgt
•  
• // SAGE Stuff
• int winWidth, winHeight
• sail sageInf // sail object
•  
• // initialize SAGE
• sailConfig scfg
• scfg.cfgFile "sage.conf"
• scfg.appName myApp" 
• scfg.rank rank
• scfg.ip NULL
•  
• scfg.resX (extent.w / extent.zoom)
• scfg.resY (extent.h / extent.zoom)
•  

scfg.imageMap renderImageMap scfg.colorDepth
24 scfg.pixFmt TVPIXFMT_888 scfg.rowOrd
TOP_TO_BOTTOM sageInf.init(scfg)     // Main
while loop while (!quit) // Generate
pixels and copy into buffer      ....    
   // swap buffer        sageInf.swapBuffer(buffer
)
8
TCP vs UDP

• Protocol to be used can be specified in sage.conf
• No need to recompile code
• TCP works well locally (within the cluster
  backplane)
• Suffers from performance hits over long fat
  networks
• UDP is fast but missing packets lead to missing
  pixels.
• New experimental protocols are used for testing

9
Troubleshooting And Debugging

• Failure to Init ports
• fsManager already running in the background
• Failure of SAGE UI to start
• Make sure that fsManager is running. Then make
  sure that sageui.conf contains the correct IP
  address and port of the machine that runs
  fsManager.

10
Troubleshooting and Debugging

• A socket error appears in the console window
• It is likely that something caused fsManager to
  stop working. Close SAGE UI. And kill sage. Wait
  for a a few seconds before trying to restart
  SAGE.  
• Noticeable delay in SAGE UI actions and updates
  on the SAGE display
• The machine running fsManager may be somewhat
  loaded and not be as responsive to SAGE UI
  requests as one would like it to be.

11
The New SAGE

• SAGE 2.0
• The main differences
• Pixel block streaming
• Every app has its own configuration
• UI remains same. User experience is same
• Looks better and more stable ?

PLAY sage-vga.mov
12
What is pixel block streaming?

• SAGE 2.x supports pixel blocks
• Frame is broken into tiny blocks
• Sets of blocks sent to appropriate tiles
• Efficient since the application can send only the
  updated blocks and not the entire frame
• Good for overlapped windows

13
Developing SAGE application on Windows
14
The Goal
15
Bandwidth (BW) Requirementin Megabits per sec.
(Mbps)

• Calculate bandwidth requirements for streaming
  apps.
• Pixels x bpp (bits per pixel) x fps (frames per
  sec.)
• Image Width 640, Image Height 480 307200
  pixels
• Image format RGB (Red, Green, Blue 3 colors)
• Color Depth 8 bits per color 24 bits per pixel
• Image frame rate 30 fps
• Bandwidth 307200 x 24 x 30 221Mbps
• If BW gt 1Gbps, expect slower frame rate.

16
DirectShow SAGE App

• Windows application
• Uses Win32 version of SAIL (sail.lib)
• Streams any video source seen by DirectX to SAGE
• Web cams
• Files (avi, dv, wmv .. Etc)
• HD cameras
• Frame grabbers

17
DirectShow SAIL/SAGE AppDSSAGE

• What is DirectShow?
• What is COM?
• Graph Editor for visual creation and test video
  pipeline
• Easy and intuitive for programming
• Not user friendly
• Needs a better UI
• Just for development

18
DSSAGE The SAIL/SAGE COM Object

• Standard query-able interface
• Most Multimedia devices (including files) come as
  a COM objects under MS Windows
• Can communicate with a device without prior
  knowledge of video format. Interface is exposed
  and agreed upon during initialization

19
DSAGE Main Code Structure

• DSSAGE SetMediaType()
• // Agree with Media type (video format)
• // e.g Greylevel, RGB8, RGB555, YUV422, etc.
• // Populate the SAIL config structure
• .
• // Do SAIL/SAGE init with this input media type
• .
•  
• DSSAGE send(buffer)
• // Get frame buffer pointer from input DShow COM
• pImediaSample-gtGetPointer(buffer)
•     // Using SAIL swap buffer to SAGE
• sageInf.swapBuffer(buffer)
•                    

20
Problems Faced

• SAGE is developed for Linux and there is an alpha
  port for Windows
• Linux assumes the program runs in a terminal
• Debugging under Windows is not intuitive.
• Not enough feedback from SAIL
• No Console to see messages unless redirected
• Console messages not clear
• Configuration files have to be adjusted for every
  application added to SAGE. No central registry
  support.

21
Problem Faced (cont.)

• SAGE 1.4b is unstable
• Looking at SAGE 2.x integration with ROCKS
• SAIL 3.0 promises to handle error messages in a
  better way
• Still is developed mainly for Linux then ported
  to Windows.

22
Online SAGE resources

• For documentation, papers, latest stable
  releases
• http//www.evl.uic.edu/cavern/sage/

23

• Q and A ?