IPSI Belgrade Ltd.

1
IPSI Belgrade Ltd.
ARTSHOP GALLERY
vm_at_ipsi.co.yuoffice_at_ipsi.co.yuhttp//www.ipsi.co
.yu
2
Authors

Stojanovski Aleksandar
Nikezic Gavro
Marinkovic Ivan
Radakovic Miroslav
Skundric Nikola
Milutinovic Darko
Toskov Ivan
Vujovic Ivana
Milutinovic Veljko
Anucojic Goran
3
Introduction IPSI Bgd
IPSI Belgrade is a company jointly founded by
German and Serbian capital Partners - IPSI
Fraunhofer, Darmstadt, Germany- Telecom Italia
Learning Services, Italy- NYU, School of
Continuous Professional Studies, USA- Instituto
Tecnologico de Durango, Mexico
4
Introduction IPSI Bgd
- Workspaces of the Future- Environments for
Cooperative Working and Learning- Virtual
Information and Knowledge Environments- Mobile
Interactive Media- Open Adaptive Information
Management Systems- Publication Engineering and
Technology- Hardware Design and Operating
Systems- Networks and WWW
5
Introduction IPSI Bgd

• Products
• Advanced Virtual Gallery
• Semantic Web tutorial and book development
• The injection cache, the STS cache, VLSI
  Detection for Internet/Telephony Interfaces,
  Genetic Search with Spatial/Temporal
  Mutations, Browser Acceleration, Technology
  transfer, Testing the Infrastructure for EBI,
  Socratenon Distant Web Educating Machine,
  e-Tourism

6
Problem Statement
- Creating Web based art gallery with look
and feel of the real world exhibitions - Visitor
moves through the gallery by walking with
options
7
Existing Solutions

• Musee national des Arts asiatiques
  http//www.museeguimet.fr/tour-guimet/index.html
• Web Server of the Galleria degli Uffizi in
  Florence http//www.uffizi.firenze.it
• The Distributed Interactive Virtual Environment
  (DIVE) http//www.sics.se/dive/
• The Web3D Repository http//www.web3d.org/vrml
  /artgal.htm

8
Proposed Solution
- Virtual reality gallery - Advanced search
capabilities - Visitors criteria based room
generation
9
Why is it better?
- Dynamically generated gallery - Content based
search engine - User satisfaction
10
Conditions and Assumptions
- PC- Internet connection- Internet Explorer
5.0 or higher - Netscape 7.0 - Cortona VRML
plug-in for IE
11
Analysis and Implementation

• - Application is written in ASP.NET using C as
  code-behind, and ADO.NET for database access.
• - Database server is SQL Server 2000.
• Communication with the database is entirely made
  through XML (using SQLXML3.0 framework).
• Queries are made in XPath, while adding,
  changing and deleting of the records is done
  through UpdateGrams.
• - Application is optimized for Internet Explorer
  5.0 or higher, at the 1024x768 screen
  resolution. Netscape 7.0 or higher is also
  supported.
• - 3D gallery is completely generated on the
  server side (dynamically) using VRML.

12
Track 1
Track Requirements

• To stand as the integrative part for the other
  two tracks
• To provide
• User interaction
• Database connectivity (database independent)
• Search functions (simple and advanced using
  Track3 output)
• Information brokering between artists and buyers
• Administration tools
• Artworks management tools, etc.
• Thin client (3D scene generation on server side)

13
Track 1
Development Tools

• Application server platform
• Windows XP Professional
• IIS 5.1
• MS SQL Server 2000
• Development platform
• ASP.NET
• C as code-behind.
• Communication with the underlying database
• XML XSD using XPath queries (DB independent)
• Currently using SQLXML3.0 add-on for ADO.NET

14
Track 1
Database Design
15
Track 1
Administrator Tools

• Separate entry pointhttp//ltserver_addressgt/arts
  hop/admin

16
Track 1

• Entry pointhttp//ltserver_addressgt/artshop/index
  .htm

Users Exhibitors
17
Track 1
Interesting Details

• Native XML DBMS under development at IPSI
  Fraunhofer
• Practical testing of the XML/XPath database
  access
• Dynamic addition (to the system) of new
  multimedia types
• 3D view of search results

18
Track 1
Interesting Details

• Application that can connect on the fly to any
  DBMS which supports XML/XPath is an interesting
  and possibly useful idea (user just has to set
  one XML file containing local field mapping, and
  one XSD to map the database fields to the
  pre-defined scheme)
• Cons
• XPath queries are lot less powerful then standard
  SQL queries
• Inherently, loss of speed (one complex SQL query
  had to be simulated with couple of XPath queries
  and additional processing in the code).
• For now, SQLXML3.0 does not support complete
  XPath standard.

19
Track 1
Errors Made

• Initially, content analysis, picture processing,
  and adding data to database were completely
  separated (as specified in the contract), with
  the idea of later (partial) integration.
• Turned out to be a bad idea(required a lot of
  interventionfrom the ArtShop system
  administrator when adding artworks).

20
Track 1
Lessons Learned

• Problem solved by complete integration of
  forementioned tasks into the one system process
  which monitors input directory, automatically
  schedules picture processing and content
  analysis, and takes care of updating of all
  necessary fields in all required databases.
• With that, we achieved maximum automation,
  reduced time needed for artwork addition, and
  reduced amount of data transferred through the
  Internet (between the administrators machine
  and the application host).

21
Track 2
Image-Content-Oriented Search
Track requirements

• Images used for extracting objects are artistic
  paintings
• Image analyses
• Extraction of the features
• Create XML file for each image
• Fetch the database with the features

22
Track 2
Algorithm
23
Track 2
Determine histogram
24
Track 2
Regions for processing matrix
25
Creating objects
26
Merge objects into bigger objects
27
Track 2
Tools used in development

• C programming language the chosen tool
• Advantage Includes the best properties from
  other programming languages (C, Java, Visual
  Basic)
• Disadvantage slower processing speed than C,
  which is not necessary in this application
• C - the best alternative tool
• Advantage faster processing speed (unnecessary)
• Disadvantage more complicated code, 50 of all
  bugs due to use of pointers

28
Track 2
Original picture
29
Track 2
Picture after applyinghistogram values
30
Track 2
Picture representedthrough extracted objects
31
Track 2
3D HSL space gt 1D histogram
32
Track 2
3D HSL space gt 1D histogram
33
Track 2
Lessons learned

• It is impossible to extract objects using only
  colors as a criterion
• It is impossible to extract objects, even using
  textures, edges, different transformations as
  criteria
• Semantics should be used in segmentation
• Colors are the most important features in
  artistic paintings

34
Track 3
Track requirements

• Possibility of moving through 3D galleries
• Automatic generation of 3D galleries based on
  users query
• Manual generation of 3D galleries
• User interface for image zooming
• Application for image processing

35
Track 3
Underlying algorithms

• Dynamic creation of gallery
• Creation of static galleries
• Algorithm for picture zooming
• Algorithm for picture processing

36
Track 3
Creation of galleries

• Validation of the created gallery
• Forming VRML files depending on users query
• Determining the number of pictures in the gallery
• Drawing a 2D floorplan based on the 3D gallery
• Forest fire algorithm for filling the floorplan
  with color

37
Track 3
Picture processing

• Loading image into memory
• Clone image into different-size copies
• Filtering of copies
• Parting of copies

38
Track 3
Development tools

• C in .NET Framework for programming image
  processing
• Macromedia Dreamweaver for programming zoom tool
• VRML Pad v2.0

39
Track 3
Flowchart
40
Track 3
Creation of galleries

• Making files based on users data
• Putting data on server so it can be available
  for artists
• Artist chooses which gallery he/she will be
  using for exhibition
• User can move through 3D world
• Selecting the textures for gallery
• Selecting the starting position of the user

41
Track 3
NF filter details

• If the new picture is smaller, every pixel is
  one pixel of the old picture.
• If the new picture is bigger, pixels are
  calculated based on the pixels surrounding the
  current.

42
Track 3
Errors made

• Requests were not precise, so there was a gap at
  the end of the project between wanted and done
• Better results could be done with better using of
  ASP and XML

43
Track 3
Lessons learned

• Every member of the team gets a part where his
  experience is dominating
• More planning at the start reduces a lot of work
  later
• Good communication between programmers can save
  a lot of time

44
Demo
45
Future Plans

• Improving the existing 3D dynamic gallery
• Improving search engine capabilities
• Improving feature extraction algorithms and
  objects recognition

46
Future Track 1

• 3D Multimedia Showroom Environment
• Implementing a general Web based 3D
  Multimedia Showroom Environment
• Exhibiting various MM data types images, 3D
  objects, videos, audio, etc.
• Set of MM data types should be extendable

47
Future Track 2

• MM Object Feature Extraction
• Implementing algorithms and software components
  for extracting features from MM data types
  (images, videos, 3D objects), in order to
  enable content based search
• System should be extendable (plug-in)

48
Future Track 3

• Semantic Abstraction of MM Feature Spaces
• Developing methods and SW components which
  derive mapping from extracted features of MM
  objects to semantic concepts
• Using intelligent classification algorithms
  (Neural Networks, Fuzzy Classifier)
• Developing semantic query engine (answering
  questions, which could previously only be
  answered by humans)

49
Usability

• Art Galleries
• Museums
• Exhibition Fairs

50
Instead of Conclusion
IPSI Belgrade, office_at_ipsi.co.yuhttp//www.ipsi.c
o.yu