VideoForge

1
VideoForge

• An image processing system

2
Contents

• Current state of the art
• Introduction
• Tiers
• Base
• Core
• Modules
• User interface
• Interactive mode
• Scripting
• universal interface
• primary implementation COM scripting object
• Implementation templates
• Controllers
• Algorithms

3
Current state of the art

• Libraries
• Loki, Boost
• Direct Show, ACML, IPL(IPP), OpenCV, Microsoft
  Vision SDK, ImageMagick,
• Applications
• GraphEdit (part of the DirectShow)
• Photoshop
• ImageJ
• ImageForge
• Adobe Premiere,

4
Introduction

• System for video and live-stream image processing
• Platform independent
• ANSI C
• Easy-to-do computer vision
• Three tier application
• Base (Serialization, Communication, Logging)
• Core (Module management, Memory management)
• Modules (Devices, Algorithms,)

5
Base

• Serialization
• Binary vs. textual storage
• Automatic vs. manual storage
• Type lists, storing of arrays, classes
• Variable reflection
• offers the possibility of calling the variable
  by name
• m_plugin.FindVarltintgt(varName)
• m_plugin.GetVarNames()

6
User interface

• Communication
• Scripting possibility
• User interface (Interactive mode)
• Types of communication
• Actions non-parametric methods
• Properties like variables, accessor methods
• Methods methods with parameters
• Events (e.g. onClose, )

7
Core

• Module and plug-in management
• Enumeration
• Registration
• Memory management
• Memory (de-)allocation
• Correct memory handling for erroneous modules
• Actions (from plug-ins and core itself)
• User interface handling

8
Modules and plug-ins

• Module file
• Contains plug-in implementations
• compiled-in or represented by DLL
• Plug-ins
• Sources
• Feeds data into chain
• Algorithms
• Transforms data, somehow
• Special plug-ins
• usually compiled persistence, etc.

9
Image Frame

• similar to IPL IMAGE structure
• contains
• resolution
• ROI (rectangle of interest)
• MediaType
• color mode (rgb, YUV, etc.)
• bpp (color depth)
• Frame
• time information (start, stop time, fps)
• previous next frame reference (pointer)

10
Building the graph

• Pipeline building
• Interactive mode vs. non-interactive mode
• Device (Controller Device enumerator)
• Algorithms (Algorithm chain Composite algs)

11
System windows

• Windows
• show graphical information
• part of a particular plugin
• Bitmap windows
• shows contents of an image buffer
• Vector windows
• usually used to show some information in the
  image
• applied as a second layer onto the bitmap window
• Window events
• OnClose,

12
Templates (1/3)

• Application template
• UI application
• Console application
• Module template
• default implementation
• rare modification
• implements a list of plug-ins

13
Templates (2/3)

• Controller template
• Implement source enumeration
• Implement data feed
• Possibly implement UI
• Algorithm template
• Implement data transformation
• Possibly implement UI
• User must implement 1 method

14
Conclusion

• We proposed system for implementation of image
  processing and computer vision tasks
• as platform-independent as possible (ANSI C)
• consists of (lightweight) core and plugins
• offers a user interactive, scriptable user
  interface
• offers various templates for fast implementation
• The system is a part of various grants
• Grant UK
• APVV Grant

15
Future work

• Finalize implementation of the core with the
  basic modules (12/05)
• basic image processing operations
• Implementation of basic modules of 0(1/06)
• working set optimization
• Implementation of special modules (06/06)
• Feature tracking
• Cooperation with avatars
• Statistics classifier training