Towards real-time camera based logos detection

1
Towards real-time camera based logos detection

• Mathieu Delalandre
• Laboratory of Computer Science, RFAI group,
• Tours city, France
• Osaka Prefecture Partnership meeting
• Tours city, France
• Friday 9th of September 2011

2
Towards real-time camera based logos detection

1. Introduction
2. Devices synchronization for 3D frame tagging
3. Frame partitioning and selection

3
Towards real-time camera based logos
detectionIntroduction (1)
Logo detection from video capture using some
handled interactions, to display context based
information (tourist check points, bus stop,
meal, etc.).
This constitutes a hard computer vision
application, due to the complexity of the
recognition task and the real time constraints.

• To support the real time, two basic paths could
  be considered
• To reduce complexity of the algorithms
• To reduce the amount of data

Camera
Selection
Frames
Pattern Recognition
Frames
4
Towards real-time camera based logos
detectionIntroduction (2)
With static objects, one capture (in time and
space) could be enough for recognition, if
recognition is perspective, scale and rotation
invariant and if occlusions neither appear
Static object without motion and appearance
modification
t0
t1
t2
Capture instance could be detected if the
embedded system can track its own positioning,
and if objects are static
Dynamic object with motion then with appearance
modification
Then, self-tracking embedded system can be set
for single capture of static objects. It can
support real-time recognition by reducing the
amount of data to process, without miss-case
(i.e. one capture is here, at least)
5
Towards real-time camera based logos detection

1. Introduction
2. Devices synchronization for 3D frame tagging
3. Frame partitioning and selection

6
Towards real-time camera based logos
detectionDevice synchronization for 3D frame
tagging (1)
The combination of these devices allows to tag
frames in 3D space.
Camera device, to capture images
Accelerometer device, that measures proper
acceleration.
Gyroscope device, for measuring or maintaining
orientation
7
Towards real-time camera based logos
detectionDevices synchronization for 3D frame
tagging (2)
Most of the commercial wearable systems (e.g.
smartphones) can support frame tagging, but the
multimodality is designed in a separate way, not
in the sense of combination of these modalities.
The device synchronization at hardware level is
not done, and must achieved at the operating
system level. How to do it ?
Polling exchange with device (accelerometer,
gyroscope)
DMA exchange with device (camera)

• value depends of the device, considering
• Acquisition delay of the device
• Data transference time on bus
• Execution time of control instruction
• Interrupt execution time
• Etc.
• value is an estimation, it depends of
• Mean access bus rate
• operating scheduling and interrupt queuing
• Etc.

8
Towards real-time camera based logos
detectionDevices synchronization for 3D frame
tagging (3)
tE0
Root Device D0
Device D1
Synchronization will be done using a two timers
framework - The coarse timer will be scheduled
on the root device - The finer timer will be
used within a upstream frame, to be opened
previously to the next coarse timer period. It
will allow to catch events of the device to be
synchronized
tE1
is the root and interrupt based device, every device will synchronize itself with it
The device to be synchronized with the root device
Ti0 The coarse timer, in charge of the root device at level 0
T0 Period of timer 0
Ti1 The finer timer, in charge of the device to synchronize at level 1
T1 Period of timer 1, with L1 is frame length for T1, N the whished synchronization precision, ? the bounded parameter
I0 Is the first interrupt time
e.g. At I0, run Ti0 Every T0, run Ti1
9
Towards real-time camera based logos
detectionDevices synchronization for 3D frame
tagging (4)
General synchronization algorithm of the Ti1
timer k 0 Every T1 period k k1
When Ii occurs
tE0
Root Device D0
Device D1
tE1
is the root and interrupt based device, every device will synchronize itself with it
The device to be synchronized with the root device
Ti0 The coarse timer, in charge of the root device at level 0
T0 Period of timer 0
Ti1 The finer timer, in charge of the device to synchronize at level 1
T1 Period of timer 1, with L1 is frame length for T1, N the whished synchronization precision, ? the bounded parameter
I0 Is the first interrupt time
10
Towards real-time camera based logos detection

1. Introduction
2. Devices synchronization for 3D frame tagging
3. Frame partitioning and selection

11
Towards real-time camera based logos
detectionFrame partitioning and selection (1)
Device synchronization can support 3D image
tagging
The open problems now are how to detect
overlapping between frames, how to achieve the
frame selection in case of overlapping ,and how
to access the obtained partition.
is the set of frame
is the intersection polygon and set of regions, such as ,obtained next to overlapping
Is the selection method
e.g.
12
Towards real-time camera based logos
detectionFrame partitioning and selection (2)
To detect the overlapping, frames can be
projected into a plane D to be computed with line
intersection and closed polygon detection
algorithms at complexity k?O(n?log(n)).
To do it, it is necessary to fix the position of
P in the 3D space and define an updating protocol
P can be obtained by meaning positioning of
frames
P
Updating of positioning is not necessary at any
frame capture, only when important differences
start to appear between the current plan and
recent frame captures.
D3
D1
At t1, D1 is computed from the current frames
F1
F2
D2
At t2, differences between D1 and D2
(corresponding to recent frame captures) is too
important, D1 is shifted to D3
t
t1
t2
13
Towards real-time camera based logos
detectionFrame partitioning and selection (3)
Once overlapping are detected, at every overlap a
region (coming from the overlapping frames) must
be selected using a selection method
Video frame processing is a producer/consumer
synchronization problem, where producer (i.e.
frame capture) are blocked on memory constraint,
and consumer (i.e. image process) are blocked
when the frame stack is empty. Here, we are
working up to the frame with partition object.
Intelligent access must be driven with RAG
(Region Adjacency Graph) structure and graph
coloring techniques.
e.g.
e.g.
This selection can be done using a spatial
criterion
c1, c2 are projected gravity centers of frames
adjacent side
?F1
?F2
to process together
14
Towards real-time camera based logos detection

1. Introduction
2. Devices synchronization for 3D frame tagging
3. Frame partitioning and selection