Omnidirectional Vision for Mobile Robots Ch3

Omni-Vision for Mobile Robots : Map matching Image-based localization Observation of Optical Flow Biomimetic Behaviours Integration of Omni-vision with other sensors: Sonar Laser range finder Outdoor Navigation SLAM (Simultaneous Localization And Mapping) Environment reconstruction & 3D mapping Miscellanea Chapter 3: Omni-Vision for Mobile Robots


Navigation/Localization Tricks : Navigation/Localization Tricks Invariance of Azimuth Rotational Invariance Vertical Lines mapped in radial lines Circumferential continuity Periodicity of the image Robustness to occlusion


Invariance of Azimuth : Invariance of Azimuth The azimuth of the object is maintained by the sensor


Rotational Invariance : Rotational Invariance Initial Position Image Counter-Rotated Robot Rotated by 90° P2 P4 P3 P5 P1


Vertical Lines  radial lines : Vertical Lines  radial lines Original Image Edge Detection + Hough Transform New Design to stretch vertical lines


Continuity & Periodicity : Continuity & Periodicity Original Image Panoramic Cylinder Fourier Transform


Robustness to occlusion : Robustness to occlusion Thanks to the wide FOV, usually occluding objects do not change much the image Several similarity measures have been proved to be robust to occlusion Extreme case presented by Jogan & Leonardis Matjaž Jogan, Aleš Leonardis “Robust localization using an omnidirectional appearance-based subspace model of environment” Robotics and Autonomous Systems 45 (2003) 51–72


Applications : Applications Map matching Image-based localization Observation of Optical Flow Biomimetic Behaviours Integration of Omni-vision with other sensors: Sonar Laser range finder Outdoor Navigation SLAM (Simultaneous Localization And Mapping) Environment reconstruction & 3D mapping Miscellanea


Map matching - 1 : Map matching - 1 Yagi used the vertical edges of the objects to find position of the robot on a map Edges tracking Y. Yagi, Y. Nishizawa, M. Yachida, Map-Based Navigation for A Mobile Robot with Omnidirectional Image Sensor COPIS, IEEE Trans. Robotics and Automation,pp.634-648,Vol.11,No.5,1995.10


Map matching - 2 : Map matching - 2 Menegatti et al. used the Chromatic Transitions of Interest to perform scan matching Monte-Carlo Localization Algorithm Almost the same approach used with Laser range Finders E. Menegatti, A. Pretto, A. Scarpa, E. Pagello Omnidirectional vision scan matching for robot localization in dynamic environments IEEE Transactions on Robotics, Vol. 22,  Iss. 3
 June 2006
 pages 523- 535



Image-based navigation - 1 : Image-based navigation - 1 Ishiguro and Menegatti: FFT magnitude for position FFT phase for heading Self-organization of the memory Image-based Localisation Hierarchical Localization Image-Based Monte Carlo Localisation Emanuele Menegatti, M. Zoccarato, E. Pagello, H.Ishiguro, ``Image-based Monte-Carlo Localisation with Omnidirectional images'' Robotics and Autonomous Systems, Elsevier - 2004 Emanuele Menegatti, Takashi Maeda, Hiroshi Ishiguro, ``Hierarchycal Image-based Memory for Robot Navigation,'' Robotics and Autonomous Systems, Elsevier - 2004


Image-based navigation - 2 : Image-based navigation - 2 Kröse et al: Used Principal Component Analysis to extract linear feature Dataset described in term of eigenimages Probabilistic localization B. Kröse, N. Vlassis, R. Bunschoten, and Y. Motomura. “A probabilistic model for appareance-based robot localization” Image and Vision Comp, vol. 19(6):pp. 381–391, April 2001.


Image-based navigation - 3 : Image-based navigation - 3 Gross et al: Used slices of the panoramic cylinder Slices confronted via colour histograms Hybrid map: topological map aumented with metric information T. Wilhelm, H.-J. Böhme, and H.-M. Gross. “A multi-modal system for tracking and analyzing faces on a mobile robot” Robotics and Autonomous Systems, 48:31–40, August 2004.


Observation of Optical Flow : Observation of Optical Flow Hiroshi Ishiguro, Kenji Ueda and Saburo Tsuji, ``Omnidirectional Visual Information for Navigating a Mobile Robot'', IEEE Int. Conf. on Robotics and Automation (ICRA-93), pp. 799-804, 1993. Ishiguro used: Foci of Expansion (FOE) to estimate relative positions No encoder info Svoboda used: Optical flow to discriminate translation and rotations Tomáš Svoboda, Tomáš Pajdla, and Václav Hlavác. “Motion estimation using central panoramic cameras” IEEE Int. Conf. on Intelligent Vehicles, 1998.


Biomimetic Behaviours : Biomimetic Behaviours Argyros, A.A.; Tsakiris, D.P.; Groyer, C. Biomimetic centering behavior Robotics & Automation Magazine, IEEE
Publication Date: Dec. 2004
. Vol.11, Iss. 4 pp.21- 30 M.V. Srinivasan. A new class of mirrors for wide-angle imaging. Proceedings, IEEE Workshop on Omnidirectional Vision and Camera Networks. Madison, Wisconsin, USA., June 2003. G.L. Barrows, J.S. Chahl and M.V. Srinivasan (2003) Biomimetic visual sensing and flight control. The Aeronautical Journal, London: The Royal Aeronautical Society, vol, 107, No. 1069, pp. 159-168.


Integration with other sensors : Integration with other sensors Shin-Chieh Wei, Yasushi Yagi and Masahiko Yachida, “On-line Map Building Based On Ultrasonic and Image Sensor, 1996 IEEE Int. Conf. on Robotics and Automation(ICRA-98) 1998 Yagi used: Sonar to detect free space Fused the sonar, edge, colour information in a occupancy grid Clerentin used: Laser to find range Fused laser and edges A. Clerentin, L. Delahoche, C. Pegard, E. Brassart "A localization method based on two omnidirectional perception systems cooperation “ ICRA'2000, San Francisco, April 2000.


Outdoor Navigation - 1 : Outdoor Navigation - 1 Omnidirectional Vision for Road Following with NN: Road classification Steering angle Z. Zhu, S. Yang, G. Xu, X.Lin, Dingji Shi "Fast road classification and orientation estimation using omni-view images and neural networks," IEEE Transaction on Image Processing, Vol. 7, No.8, August 1998, pp. 1182-1197.


Outdoor Navigation - 2 : Outdoor Navigation - 2 Paul Blaer and Peter Allen “Topological Mobile Robot Localization Using Fast Vision Techniques” Proceedings of the 2002 IEEE International Conference on Robotics & Automation 2002 Image-based navigation: Topological navigation Histogram matching Different localisation accuracies


Outdoor Navigation - 3 : Outdoor Navigation - 3 José-Joel Gonzalez-Barbosa and Simon Lacroix Rover localization in natural environments by indexing panoramic images Proceedings of the 2002 IEEE International Conference on Robotics & Automation 2002 Image-based navigation: Dimension reduction with PCA Histogram matching


SLAM : SLAM Michael Kaess and Frank Dellaert,
 Visual SLAM with a Multi-Camera Rig,
 Georgia Tech Technical Report GIT-GVU-06-06, 2006 Thomas Lemaire, Simon Lacroix. Long Term SLAM with panoramic vision. Submitted to Journal of Fields Robotics special issue on "SLAM in the Fields".


Environment Reconstruction : Environment Reconstruction H.Bakstein, T.Pajdla. Rendering Novel Views from a Set of Omnidirectional Mosaic Images. Workshop on Omnidirectional Vision and Camera Networks 2003, CD ROM, IEEE June 2003. C. Geyer, K. Daniilidis, Mirrors in Motion: Epipolar geometry and motion estimation, Proc. Inter. Conf. on Computer Vision, October, 2003, Nice, France.


Slide22 : Omnidirectional Distributed Vision System (ODVS) Requirements: Robots’ only sensor: omnidirectional vision No use of external computer Every robot shares its measures Every robot fuses all measures received by teammates Measures can refer to different instants in time The aim of the ODVS: to track multiple moving objects in highly dynamic environments by sharing the information gathered by every single robot E. Pagello, A. D’Angelo, E. Menegatti Cooperation Issues and Distributed Sensing for Multi-Robot Systems IEEE Proceedings of IEEE (in press due October 2006)


Slide23 : One Static Vision Agent (omnidirectional camera) Five Static Acustic Agents (steerable microphone arrays) One Mobile Vision Agent (robot with omnidirectional camera) = = Experiment Layout = Audio-Video Surveillance System with Mobile Robot E. Menegatti, M. Cavasin, E. Mumolo, M. Nolich, E. Pagello Combining Audio and Video Surveillance with a Mobile Robot International Journal on Artificial Intelligence Tools (in press)


The End! : The End! Thanks for your attention! My publications and more information can be found in my web page: http://www.dei.unipd.it/~emg


References : References WWW: The page of omnidirectional vision http://www.cis.upenn.edu/~kostas/omni.html ICCV03 Course on Omnidirectional Vision http://www.cis.upenn.edu/~kostas/omni/iccv03.html Book: R. Benosman & S.B. Kang (Eds.) Panoramic Vision Springer 2001


References : References Special issues: K. Daniilidis& N. Papanikolopoulos The Big Picture IEEE Robotics & Automation Magazine Dec. 2004 Hiroshi Ishiguro and Ryad Benosman Special issue on omnidirectional vision and its applications Machine Vision and Applications (2003) Vol 14 Yasushi Yagi and Katsushi Ikeuchi Special Issue on Omni-Directional Research in Japan International Journal of Computer Vision Vol. 58, Num. 3, Springer July 2004 Peter Sturm, Tomas Svoboda and Seth Teller Special issue on Omnidirectional Vision and Camera Networks Computer Vision and Image Understanding Vol. 103, Iss. 3, Sept. 2006