Title: Understanding SVG Diagrams
1Understanding SVG Diagrams
- Interim Presentation
- Vivek Ramakrishna
- Honours Program, Computer Science
- vivekr_at_csse.monash.edu.au
2Motivation
- Vast amounts of information stored in diagrams
- Information not accessible to all users
- Blind people
- Search engine agents
- Should be able to retrieve information in a
diagram, given a query
3Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
4Diagram Representations and Types
- Rasterised versus vector graphics
- Scalable Vector Graphics (SVG)
- Vectors have types and attributes
- ltrect x0 y 0 width50 height20
colorredgt - Project will investigate bar charts, organisation
hierarchies and flow charts in SVG format.
5A simple bar chart in SVG
6Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
Interpreter Understands the meaning of the
diagram and produces a useful description
7Constraint Multiset Grammars
- In visual domain, spatial relationships more
important than sequential. - P P1, , Pn where C
- Non terminal symbol P can be recognised by
multiset of symbols P1,, Pn whenever the
attributes of the symbol satisfy the constraints
C.
8CMG for a bar
- A bar comprises of a rectangle and a text label.
- The label is aligned centrally below the bar.
- Bbar() Rrectangle, Ttext
- where ( R.mid T.mid
- R.start gt T.start
- )
- We will use CMGs to produce grammars to interpret
our diagrams.
9CIDER
- Toolkit developed at Monash by Tony Jansen.
- Interprets diagrams based on CMG specifications.
- We will use CIDERs powerful compiler to parse
SVG to recognise diagram tokens.
Bar Chart
Bars
Axes
Bar
X axis
Y axis
Rectangle
Text
Line
Line
10Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
Semantics The key issue when extracting
descriptions is the format to specify them.
11RDF A Description Logic
- Resource Description Framework
- Can express knowledge of a domain by first
defining its terminology. - Equipped with formal, logic based semantics.
- Allows complex relationships to be inferred
despite not having been explicitly stated - In an organisation chart, can infer all superiors
of an employee, without stating each relationship - Disadvantage DLs cant handle fuzzy predicates
12Stages in diagram understanding
Kind of diagram
3. Query Answering System
Query
1. Interpreter
2. Semantics (description)
Diagrams (SVG)
Answer (in context)
Context
Query Interface Communicate question to system
and answer to user. Different requirements for
each user. We will use existing interfaces for
our trials.
13Plan
- Considering SVGs of bar charts, organisational
hierarchies and flow charts. - CIDER used to interpret CMG for each diagram.
- Descriptions generated in RDF.
- Evaluation
- Get corpus of sample diagrams
- Evaluate with corpus of sample queries
- Limitations
- Grammars are inherently brittle
- Description Logics are rigid.
- We wish to investigate when and why failure occurs
14Questions?