Computer Vision: Motion

1
?????
2
????????????

• ????????????????
• ?????
• ????????
• ??????????
• ???????? (mosaics)
• 3??????
• ??????

3
Optical flow???
????????????
??1 ??2
4
Optical Flow??
????
??
????
5
Optical flow??
6
Optical flow???
????????????
??1 ??2
7
????? optical flow

• ?? H ???? I ?????????????????

• ??????????????
• H?????????I???????????????????????

• ?????
• ????? H?????I?????????
• ?????????????????????
• ???? ?????????
• ???optical flow?????

8
Optical flow??? (grayscale images)

• ??????????????????

• ?????? Q ?????

• ???? (u ? v ?1????)
• I ???????(Taylor)????

9
Optical flow???

• ????????????

10
Optical flow???
??????t????
?t??????????
????
11
Optical flow????

• Q ???????????????????

???1 ???2
?????????????

• ???????????????????
• ???????????????????

????Aperture problem???
12
Aperture problem (?????????)
13
Aperture problem (???...)
14
Aperture problem (???...)
?????
??
??????
????
?????
????
??????
????
15
Aperture problem??????

• ??????????????????
• ??????? ?????????
• ????????
• ????????????????????????????????????????????????
• ??????
• ???????????????

16
Aperture problem??????

• 5x5??????????????25??????????

17
RGB version

• How to get more equations for a pixel?
• Basic idea impose additional constraints
• most common is to assume that the flow field is
  smooth locally
• one method pretend the pixels neighbors have
  the same (u,v)
• If we use a 5x5 window, that gives us 253
  equations per pixel!

18
Lukas-Kanade flow

• ?? ????????????????

19
????????

• ???Lucas-Kanade???????(u, v)????

• ?????
• ATA ?????????
• ATA ????????
• ATA ??????? l1? l2 ??????
• ATA ???????
• l1/ l2 ?????????(l1 ??????????)

20
ATA???????

• ??????????????????????
• ?????????????
• ???? ????????????????
• ATA??????????????
• N ? ???????????
• N ?0???????????????????
• ATA??????????????????????

(x,y) ?????????????ATA??
21
Edge

• large gradients, all the same
• large l1, small l2

22
Low texture region

• gradients have small magnitude
• small l1, small l2

23
High textured region

• gradients are different, large magnitudes
• large l1, large l2

24
??

• ???????????????????
• ?????????????????????!
• ??????????????????????????????????
• ???????????????...

25
Lukas-Kanade???????

• ?????????????????
• ATA ?????????????
• ??????????????????
• ?????????

• ????????
• ???????????????
• ????????
• ??????????????????
• window size??????
• ???window size??

26
?????

• ?????????????

• ?????????
• ??????????????????????????

??????????????????

• ??????(Newtons method)???????
• Also known as Newton-Raphson method
• Lukas-Kanade ??????????????????????
• ??????????????????????????

27
??????

• ????Lukas-Kanade ??????
• ???????Lucas-Kanade??????????????
• ???????????????????image warping?????H??????????I?
  ??????????
• ??????????(????????)??????

28
Revisiting the small motion assumption

• Is this motion small enough?
• Probably notits much larger than one pixel (2nd
  order terms dominate)
• How might we solve this problem?

29
Reduce the resolution!
30
Coarse-to-fine optical flow estimation
31
Coarse-to-fine optical flow estimation
run iterative L-K
32
Multi-resolution Lucas Kanade Algorithm
33
Optical Flow Results
34
Optical Flow Results