GapiDraw Fast Graphics on Handheld Computers

1
GapiDrawFast Graphics on Handheld Computers

• Johan Sanneblad
• Future Applications Lab
• Viktoria Institute

2
Johan Sanneblad (me)

• Education
• M.Sc. Informatics, PhD - Feb 2005
• Sun Certified JavaProgrammer Developer
• Software Development
• Consultant Cap Gemini (Systems Designer for
  Volvo, Saab, Ericsson, Bilia)
• Mobile Platform Development (ProxyLady,
  GapiTools, GapiDraw, OpenTrek)
• CEO Develant AB (www.develant.com)

3
Before we begin

• Can everyone go to
• www.develant.com... and download GapiDraw 3.50
  beta 2
• We will have some common programming later in
  this session, so everyone is recommended to
  download and install GapiDraw before then

4
Recommended Reading

• Thinking in C 2nd edition Volume 1
• Bruce Eckel
• Download www.mindview.net/Books/DownloadSites
• Cost 0 kr

5
Myth Creatinggames is difficult

• Myth Creating games is difficult
• It was difficult in the 1980s
• Recommendation Read The birth of a
  Paradroidhttp//www.zzap64.co.uk/zzap3/para_birt
  h01.html
• Early problems (In the 1980s)
• Lack of storage space (one floppy disk was not
  able to store an entire game)
• Lack of internal memory (64 kilobytes)
• No debugging possibilities (was first introduced
  in the late 80s)
• Code was written in machine code on paper or
  using custom developed assembler compilers
• Extreme hardware limitations (8 sprites/raster
  row etc.)

Andrew Braybrook, now a client/server- database-de
veloper inthe insurance business
6
Myth Creatinggames takes time

• Myth Creating games takes time
• It can if you need to fill 2-3 CDs
• It can if you do not have the proper tools
• 2D requires a platform such as GapiDraw
• 3D requires a platform such as Unreal
• Custom-written tools (e.g. level design) are
  required in all cases
• It can take time if you need to create
  high-resolution graphics
• This is not a problem on mobile devices
• It can if you do not have anyone to ask

7
Highlight of threeGapiDraw games
8
Highlight of threeGapiDraw games

• For two years, GapiDraw was given to 30 M. Sc.
  students for a 5p project
• The 30 students each year created 12 different
  games in three weeks - all multiplayer for PDAs
• I will highlight three PacMan must Die!, Earth
  Defenders, Wekew

9
PacMan Must Die!

• Video!
• A game created by 3 students in 3 weeks
• Reversed PacMan - players are ghosts!
• Created with the GapiDraw SDK
• Highlights
• Simple session initialization
• Shared displays (made possible using a
  low-latency and high performance wireless
  connection)
• High-performance graphics

10
PacMan Must Die!

• Number of players 2-10
• Dots are distributed across the displays of all
  players
• Players can join the game dynamically - dots are
  then added as needed!
• Ten levels, all with different layouts

11
Earth Defenders

• Number of players 2-5
• Player roles Combat Strategy (1), Star Fighter
  (2-3), Orbital Defence (4), Opposition Commander
  (5)

12
The Quest of Wekew

• Number of players 2
• Players must collaborate to succeed
• The small character can go to places the large
  one cannot (and jump on the large)
• Players collect letters to advance

13
Common Questionson PDA Development

• Why not use Java?
• Why use Pocket PCs (and not Palm / Symbian
  devices)?

14
Java

• Performance-wise, Java is closing in on C / C
• By knowing how the JIT-compiler works, the
  difference can be close to 10-30
• Java will soon get generics (templates)
• Unfortunately, Java on PDAs is not Java
• Java performance on PDAs is really bad
• The Java version on PDAs is usually 1.1, in best
  cases 1.2
• Java on PDAs may get generics in 2-3 yrs

15
Java

• Performance comparisons between C / Java are
  however not fair!
• C template meta programming is never used in
  comparisons (this is used for several hundred
  percent performance boosts in C in some cases)
• Using inline assembler often gives more
  performance boost than you might think (memory
  copy 30 faster than memcpy)
• Most tests compare processor performance, not
  memory access

16
Java

• Finally
• You cant create cool games in Java
• But I would love to be proven wrong!

17
Pocket PCs

• The IPAQ 4150 4350 Pocket PCs
• Bluetooth, Wireless LAN, 140 grams

18
Pocket PCs

• The first PDA (Personal Digital Assistant) from
  Microsoft was introduced in 1999
• The Casio Cassiopeia E-115 (introduced in 1999)
  was the first PDA with color display
• Operating System Windows CE 2.11 for Palm Sized
  computers

19
Pocket PCs

• Windows CE Three Components
• Win32 API (e.g. networking, window and message
  handling)
• MFC - Microsoft Foundation Classes
• The Windows GUI components
• Developing for Windows CE is similar to
  developing for Stationary PCs (CE Compact
  Edition)
• In 2003, Windows CE was renamed to Windows Mobile
  2003 (WM2003)

20
Pocket PCs - WinCE 2.11

• WinCE 2.11 - Too much Windows?
• A typical Windows CE 2.11 application (left) and
  ProxyLady (right)

21
Pocket PC - WinCE 3.0

• Starting with Windows CE 3.0 - Microsoft divided
  the OS into 2 parts
• One version for PDAs got a new GUI and was
  referred to as Pocket PC. Pocket PC is simply
  Windows CE 3.0 with a new GUI
• One version for handheld and tablet computers
  kept the old interface (called Handheld PC)

22
Pocket PC 2002 - WM 2003

• Pocket PC 2002
• Minor changes in the GUI (new today screen etc)
• Some minor internal improvements of Windows CE
  3.0
• Better support for real-time applications
• Windows Mobile 2003
• New core based on Windows CE 4.1
• Built-in support for the .NET platform
• Same old Pocket PC GUI

23
Why use Pocket PCs?

• Some Pocket PC top 5 advantages
• Lots of choices for development (Java, C,
  Visual Basic, C)
• Many devices with built-in Wireless LAN (Symbian
  has none, Palm has two)
• Learn how to program a Pocket PC - and you will
  know how to develop on Stationary PCs as well
• Built for real-time applications such as games
  (excellent timer granularity)
• Complete network IP stack

24
GapiDraw historyGapiTools and Gapi
25
GapiDraw

• When I held the course MAN in 2001, my initial
  plan was to use GDI
• Graphic Device Interface (GDI)
• Is used by Windows to draw the interface
  (rectangles, lines, and icons)
• Event driven architecture (user does something
  and the program reacts)
• For animations you set a timer call me in 100ms
  so I can redraw the display
• Can be used for simpler graphics and animations
  (less than 5 FPS)

26
GapiDraw

• Advantages with GDI
• GDI can be used to easily create form-based
  applications (the user enters data, clicks on
  process data, and another form is displayed
  with new data)
• GDI must be used if the application is to be
  WM2003 certified

27
GapiDraw

• Drawbacks with GDI 1
• Graphics performance is extremely limited (BitBlt
  always copies and transforms each pixel
  individually - GapiDraw block copies 8
  simultaneously)
• Timer performance is poor (not suited for
  animations - jerky playback)
• The event driven architecture is not suited for
  applications such as games
• You cannot take full control of the PDA (such as
  buttons and stylus input)

28
GapiDraw

• Drawbacks with GDI 2
• You cannot easily use landscape modes
• The GDI architecture (with DIBs) depends on a
  hardware 2D accelerator to operate at acceptable
  speed - few PDAs have this
• On stationary PCs, no-one really uses GDI anymore
  (some use of GDI which has some MMX-accelerated
  code)
• Next version of Windows will be based on the .NET
  Framework - GDI will be supported but not
  encouraged

29
GapiDraw

• Decision Do not use GDI
• Four weeks after the 10p course had started I
  decided not to use GDI

30
Your Mission, Jim...

• The Facts
• A course in mobile application development was
  started four weeks ago. The students know GDI
  programming, but should not use GDI. Their final
  5p project will begin in one week.
• The Solution
• Investigare how difficult it is to create real
  time graphics on mobile devices. Assemble this
  information and teach it to the students.

31
Your Mission, Jim...

• After two days of browsing the web
• I found out that real-time graphics on mobile
  devices is really difficult
• Microsoft has something called the Game API
  (GAPI) that sounded promising
• There were some GAPI tutorials scattered across
  web sited - all had significant flaws and errors
• Easy CE (messy layer built on top of GAPI full
  with errors and non-optimized gfx)
• Back to GDI?

32
Handheld Computer Graphics

• A layered solution
• The display is stored as bytes on a certain
  memory address. Each pixel is represented by 2
  bytes (16 bits)
• Using the Microsoft Game API (GAPI), it is
  possible to write directly to this memory address
• To set a single pixel
• Use GAPI and get the address to where in memory
  the display is located
• Write directly to the display bits

33
Handheld Computer Graphics - GAPI

• GAPI (gx.dll) a thin C DLL
• Gapi offers two things
• A pointer to where in memory the display is
  located
• The possibility to lock the hardware keys (such
  as the calendar button)
• To fill the entire display with a white color,
  you must set all pixels on the display to 0xFFFF

34
Handheld Computer Graphics - GAPI

• To set a single pixel on the display using GAPI
  you use code like
• WORD pDisplay (WORD) GXBeginDraw()pDisplay
  0x0000 // Clear the first pixel
• Learning how to program with GAPI takes time and
  patience
• Assuming you want more than simply setting /
  clearing pixels on the display you need more
  graphic operations
• You also need a message handler

35
GAPI A Simple Hello WorldApplication, Step 1

• To give an idea of how simple it is to draw
  graphics directly to the display I have some
  excerpts from a tutorial at microsoft.com A
  Simple Hello World Application
• http//msdn.microsoft.com/library/default.asp?url
  /library/en-us/dnppc2k/html/ppc_api.asp

36
GAPI A Simple Hello WorldApplication, Step 1

• Add the following lines of source-code at the
  end of the InitInstance functionUpdateWindow(hW
  nd) // Try opening the Display for Full screen
  accessif (GXOpenDisplay(hWnd, GX_FULLSCREEN)
  0) return FALSE // we won't be able to
  draw.// Initialize the Hardware
  ButtonsGXOpenInput()// Get the Display
  propertiesg_gxdp GXGetDisplayProperties()//
  Get information about the Hardware Keys and
  fills// the g_gxkl structureg_gxkl
  GXGetDefaultKeys(GX_NORMALKEYS)return TRUE

37
GAPI A Simple Hello WorldApplication, Step 2

• Before we can start coding here we have to dig
  into a little theory. Depending on your Pocket PC
  hardware you can have either a 16-bit color
  display (The Casio, HP, or Compaq) or an 8-bit
  display (black-and-white or color, like the
  Compaq Aero 1550). To determine which display
  your Pocket PC has, you can evaluate the member
  cBPP of the g_gxdp structure. It will tell you
  exactly how many bits you have to deal with.
• Before directly accessing the display memory for
  16-bit displays, you have to know how the display
  controller interprets the 16-bit color mask.
  There are two possible flag settings KfDirect555
  and kfDirect565. The 555 translates into a color
  mask XRRRRRGG.GGGBBBBB for Red-Green-Blue in a
  short variable, while 565 translates into
  RRRRRGG.GGGXBBBBB

38
GAPI A Simple Hello WorldApplication, Step 2

• Add the following function to the top of your
  FirstGX.cpp source file bool ClearScreen(int
  colRed,int colGreen, int colBlue) // 16
  bit per pixel code. Note 2 different pixel
  formats. switch (g_gxdp.cBPP)
  case 16 unsigned short
  PixelCol 0 if
  (g_gxdp.ffFormat kfDirect565)
  PixelCol (unsigned short)
  ((colRed 0xff) (colGreen
  0xff) (colBlue 0xff)) else if
  (g_gxdp.ffFormat kfDirect555)
  PixelCol (unsigned short)
  ((colRed 0xff) (colGreen
  0xff) (colBlue 0xff)) ...

39
Issues with GAPI and PDAs

• Displays are physically aligned different on
  units -if this is not considered performance loss
  is severe
• Displays use different pixel formats (444, 555,
  and 565 formats)
• Some games assume that you can rotate the PDA 90
  degrees
• No support for image loading
• Limited cache memory - must be considered in all
  cases

40
Performance

• Displays are physically aligned different on
  units
• You have to traverse the display in the correct
  order - one of the following loops is three times
  faster than the other depending on the PDA
• for (int x0 xy
• for (int y0 yx

41
To conclude

• If not using GDI, students would have to create
  their own images loading routines and draw to the
  display
• Students would have to consider the hardware
  architecture to get acceptable performance
• If the display is rotated, students have to use
  pixel transformations and pre-rotate the images
  for the cache
• The project begins in five days

42
Solution

• GDI should not be used (!)
• The only acceptable solution Create a new
  graphics toolkit upon GAPI.Development time
  five days
• Result after first night
• Game API
• Read image from file
• Draw image to display
• Color mask support

43
2001
Networking Platform - BluePortal
2000
Nov 2001
Graphics Platform - GapiTools
Mobile Applications and Networking
3p
2p
5p projekt
44
GapiTools

• From www.gapitools.com
• Drawing in both vertical and two horizontal
  (landscape) modes
• Support for all current and future 16-bit Pocket
  PC screen displays
• Smart buffering system with support for multiple
  pixel surfaces
• Switchable verification of all coordinates.
• Support for loading 2, 4, 8 and 24-bit
  RLE-encoded bitmaps
• Bitmaps can be loaded from a resource, file or
  memory block
• Fast block blits to screen / back buffer
• Fast masked blits (sprites) to screen / back
  buffer
• Fast alpha blended blits to screen / back buffer
• Sprite blend effects such as lighten, darken and
  transparency
• Fast sprite-to-sprite and sprite-to-mask
  collision detection
• Transparent, alpha blended solid rectangles
• Fast pixel, line and rectangle drawing
• An easy way to save a "screen capture"
• Full bitmapped text support including multiple
  fonts
• much more...

45
Some Facts

• Development on the graphics platform was
  initiated one week before project start, and was
  finished two weeks later, four weeks before
  deadline
• The documentation for the graphics platform was
  finished two weeks before project deadline
• The network platform had been in development
  since 2000. It had not been tested. It was
  finished four days before deadline.
• The graphics and network platforms only worked on
  handheld computers. It was not possible to
  develop on stationary PCs and then recompile.
• It was not possible to use a debugger
• Still, the students created impressive games!

46
Game Modules

• A total of 12 games was created
• A centralized database storage was created to
  manage player data
• Many games were multiplayer, where players could
  challenge each other

47
More Modules
48
Game Module TrekFighter

• Fixed point 1616 sub pixel coordinate support.
• Smooth 10-degree ship rotation, avoiding the
  classic 90-degree left/right turns of traditional
  snake games.
• Support for up to four players. Players are
  evenly distributed across the corners.
• Support for game maps larger than the display.
  The display will always focus on the local ship
  and smoothly scroll the background if necessary.
• Support for bitmapped tails, constructed as
  sprite objects. Collision detection is performed
  using OpenTrek collision surfaces.
• Support for tokens such as speed increase, speed
  decrease and tail boosters. Tokens can be picked
  up by moving the ship on top of them

49
Game Module TetTrek

• Multiplayer support. Players can view both their
  own map and the map of their opponent on the same
  screen.
• Bombs. Each time a player successfully clears a
  row of bricks, one square in one of the bricks on
  the screen transforms into a colored letter.
  Green letters are good (e.g. clear row, free all
  rows), and red letters are bad (e.g. clear bomb
  list, create wall, rotate brick, drop brick, add
  row). Both bomb types can be stored using a local
  bomb list located in the center of the game
  area, and both bomb types can be applied to both
  the local and the remote player area.
• An Object Oriented architecture, where bricks are
  defined internally as classes. New brick types
  can be created easily by simply adding a new sub
  class to the game engine.

50
6 months later

• GapiTools was released on the web but got serious
  critique
• The GapiTools interface used MFC - something
  avoided by most developers since it is really,
  REALLY slow!
• GapiTools only worked on PDAs - you could not
  develop on stationary PCs
• GapiTools is a proprietary standard (porting
  existing game was difficult)
• GapiTools architecture was Win32 based
• GapiTools only supported BMPs

51
GapiTools problem

• Solution
• Rebuild everything from ground up
• Development time two weeks
• Create the new architecture using template meta
  programming (used by Matrox and Nvidia for their
  drivers)
• Make the platform DirectX compatible
• Add support for all image formats, including BMP,
  PNG, GIF, JPG
• The result GapiDraw

52
The GapiDraw Graphics SDK

• Supports Stationary PCs, PDAs and Smartphones
• Can be used in two modes
• Low level layer (DirectDraw compatible) and high
  level layer (device abstraction)
• Supports 444, 555, 565
• User base Some hundred active developers
• Licensed to WildTangent

Pocket PC Elite (Jon Welch / George Hopper)
53
In Use - Commercial Games

• Some of the over 150 commercial games created
  with the GapiDraw SDK

EverQuest for the Pocket PC (Sony Online
Entertainment)
Warlords II(Pocket PC Studios)
Odyssey(AIM Productions)
54
In Use - Commercial Games

• Some of the over 150 commercial games created
  with the GapiDraw SDK

Jack the Uni-Psychle(Discord Studios)
FirePower - Onrush(Manaisoft)
Tower Mogul(eSoft Interactive)
55
In Use - Business

• StormHawk
• The worlds first mobile, handheld weather
  forecasting and navigation system
• Runs on Windows Mobile Pocket PCs with GPS
  receivers
• Created with GapiDraw 3.0

56
GapiDraw Not Only Games

• Instant Messaging on PDAs and Mobile Phones
• ICQ, MSN Messenger, Jabber
• All require input to be translated into ASCII
• An empty message is 400 bytes (using Mobile
  ICQ)
• Amigo
• Peer-to-peer Instant Messaging using GapiDraw
• Messages are sent as compressed PNG images
  (typically 1kb per message)

57
GapiDraw

• Some technical highlights
• Always operates on multiple pixels simultaneously
  (32-bit reads and writes)
• Extremely optimized blending functions (only one
  multiplication for each pixel for a simple
  Dn(Sna)(Dn(1-a)) alphablend
• Fast Assembler code block copy
• Custom-written image loaders for BMP and PNG with
  zero memory overhead

58
GapiDraw

• Some other cool features
• Clipping support in all operations
• Bitmapped fonts (with kerning tracking)
• Real-time color tints and colorizations
• Real-time rotation and zoom
• Real-time alpha blends
• Existing games can be ported to handhelds in a
  few hours!

59
GapiDraw

• Recent improvements
• Support for Palm OS 5 (March 2004)
• Support for Symbian devices (Jan 2004)(both
  Series 60 and UIQ are supported)
• GapiVFS, the virtual file system (August 2004)
• Current platform base Most current handheld
  devices Pocket PCs, Palms, Symbian devices,
  Stationary PCs, Smartphones
• Upcoming features
• Support for Palm OS 5.4 and 6.0 Cobalt
• Tapwave Zodiac (tapwave.com) hardware support

60
GapiDraw

• Is it difficult to use then?
• NO!
• New games can be created in a few hours

61
GapiDraw - Getting Started
62
Getting Started

• MFC Development
• App / Doc / FrameWnd / ChildWnd (or)
• App / FormView (or)
• App / FrameWindow (or) Dialog based
• Lots of things to consider
• Who really understands what happens? Where is
  WinMain()? What is a DC?
• One global instance - theApp
• The SIP for input SHFullScreen
• Running apps in landscape mode?

63
Getting Started

• GapiDraw Development
• CGapiApplication
• Simple to understand
• Subclass CGapiApplication (e.g.
  CMyApplication).
• Create an instance of your subclass in WinMain
  and call MyApp.Run()
• Configure your subclass with a struct
• If you want to do something when a key is
  pressed, override KeyDown() etc.

64
Getting Started

• Download GapiDraw 3.50 and start the Minimal
  sample application(GapiDraw350\samples\minimal\xp
  \minimal.dsw)
• Open myapplication.cpp
• WinMain
• Creates a config structure
• Creates an instance of our application
• Starts the application with Run()
• Simple!
• Tip Build all new projects on Minimal

65
Getting Started

• Config Structure
• Use the GDAPPCONFIG structure to
• Set screen resolution on Stationary PCs
• Set screen rotation (if the application should
  run in landscape mode for example)
• Choose if the application should be full screen
• Change the window title
• Change how the application should respond to e.g.
  incoming phone calls when run on Smartphones

66
Getting Started

• Some functions that can be overridden
• Constructor / Destructor
• InitInstance / ExitInstance(use these if you
  want to allocate memory and would like the app to
  exit if it fails)
• OnMinimize / OnRestore
• CreateSurfaces
• ProcessNextFrame
• KeyDown / KeyUp
• StylusDown / StylusUp / StylusMove

67
Getting Started

• Functions that can be called
• Run()
• Shutdown()
• Minimize()
• Pause()
• SetDisplayMode()(use this to rotate the display)
• Basically the above functions covers all thats
  needed in most applications!

68
Surfaces and Memory areas
69
Surfaces

• A surface in GapiDraw is an object used to load
  and edit images
• When you create the surface (e.g.
  CreateSurface(0, 320, 240) a memory area of
  320x240 pixels is allocated
• You can also call CreateSurface(filename), where
  a memory area of widthheight is allocated into
  which the image is loaded
• The CGapiSurface object comprises several tools
  for drawing and manipulating images

70
CGapiSurface

• A quick overview
• CGapiSurfaceCreateSurface()(creates surface
  from file or resource)
• CGapiSurfaceFillRect()
• CGapiSurfaceSetPixel()
• CGapiSurfaceBlt
• CGapiSurfaceBltFast
• CGapiSurfaceAlphaBltFast
• CGapiSurfaceSetClipper

71
CGapiSurface

• In GapiDraw, the display is a CGapiSurface
  subclass CGapiDisplay
• Meaning you can draw to the display using e.g.
  CGapiSurfaceFillRect()
• Other subclasses of CGapiSurface are
• CGapiBitmapFont
• CGapiMaskSurface
• And this concludes todays quick overview...

72
Assignment
73
Assignment

• Assignment Today
• Download GapiDraw 3.50
• Modify the Minimal sample application and add
  support for Bouncing Flowers
• Bouncing Flowers
• At least one flower bouncing around inside the
  edges of the display
• Extra Capture Up/Down to add/decrease the
  number of flowers