Presentation040265

Slide1: 

VISION AND CONTROL Introduction Vision can be defined as a faculty of sight, or the act or power of sensing with the eyes. When used in the engineering context, is usually associated with the development of systems or process that are capable of understanding images and those used to extract information from the images. The ultimate aims are to allow such systems to make decisions base on the information in the images and perform specific tasks autonomously or with minimal human interactions. Perception and action are two important components in any autonomous system. It is well known that feedback control systems are better than open-loop systems in terms of robustness, accuracy, adaptation and autonomy. Our ultimate goal is to develop systems that will be able to assist people in their daily activities and enhance their life. These could be robots or other devices used in factories, shops, hotels, streets and even homes. These could also be medical equipment and systems used to improve the outcome of surgeries and other medical interventions. Objectives Research work under the Vision and Control programme is towards the development of autonomous systems that are characterised by the synergetic integration of perception, decision and action modules. Research and Development To achieve the above objectives, this research program covers the two important aspects of an autonomous system: visual perception and intelligent control. Therefore, the research activities of this program are focused on the following areas: 1. Vision Systems Vision research has already reached the stage of developing vision systems with improved robustness and real-time performance. New algorithms are being developed for processing noisy images, like those from medical ultrasound machines. At the same time, we will also look into the aspects of active vision (i.e., controlling the parameters of vision system for the efficiency and robustness) as well as task-driven paradigm. As for the issue of real-time performance, we are interested in the application of parallelism and vectorisation techniques to optimise the execution time of vision algorithms. The output of our vision research is oriented towards visual guidance of autonomous modular mechanical systems as well as geometric object representation and recognition through 3D surface modelling. They are also used for medical image processing for image-guided surgery and

Slide2: 

augmented reality applications. Another area of research is the development of embedded and smart vision systems. Image processing is one of the most important parts in computerised medical intervention. It helps in diagnosis, surgical planning and robotic surgery. Currently the on-going projects include: a) Prostate ultrasound image processing, b) Colon ultrasound image processing, and c) Breast ultrasound image processing. The major objectives in the above projects are detecting the boundaries in the ultrasound images, as shown in Figure 1. Figure 1: An ultrasound image of a prostate and its detected outline. 2. Modular Mechanical Systems: The structure of today’s industrial robot is not flexible enough to handle applications that require high dexterity and high reconfigurability. In addition, the inverse kinematics of today’s industrial robot is too complex to facilitate real-time motion control incorporated with on-line visual guidance system. An attractive solution to overcome these shortcomings is the concept of modular mechanism. Our research on modular mechanism is driven by the following three targets: a) to reduce the complexity of inverse kinematics by increasing the degree of mobility of a modular mechanism, b) to maximise reachable space by incorporating a body’s motion to a modular mechanism in order to achieve human-like mobile manipulation, and c) to design and develop a modular mechanism that is scaleable and reconfigurable.

Slide3: 

These modules may be equipped with their own, independent controllers. These modules must be able to communicate with each other and co-ordinate their actions to complete the complex tasks. 3. Locomotion: Autonomous robotic system includes not only wheeled type mobile robot but also other types of robot locomotion systems, for example, legged robots (biped or multi-legged), snake-like or inchworm-like robots, and self-reconfigurable metamorphic robots. These systems are capable of moving themselves on the ground or other medium through the change of internal configurations. Such systems provide human beings the capability to explore territories that are very difficult to reach by conventional methods, for example, uneven terrain, piping systems, or inside the human body. Our research in this area will address the following issues: the principle of robot locomotion in different formats, the control of robot locomotion, and finally, the integration of vision and other types of control to realise a fully functional locomotive system. Figure 2 shows a centipede robot. This robot is made up of ten segments of six-legged robots connected by nine link systems. The first segment acts as the master while the other segments are the slaves. Figure 2: A centipede robot Figure 3: An underwater eel robot The underwater eel robot as shown in Figure 3 is made up of parts permitting many different configurations to be assembled. 4. Integration of Vision and Control: Our strategic researches are in line with emerging industry and societal needs. The ultimate goal of our strategic research program is to develop fully integrated autonomous systems that are capable of performing intelligent behaviours. Issues covered include hand-eye co-ordination, head-eye co-ordination, vision-guided locomotion, and image guided surgery.

Slide4: 

Figure 4 shows the 3D rendering of computed-tomography images for the path planning of a robot for image-guided surgery, while Figure 5 shows the design of the robot. The robot is designed to drill the skull bone at the base of the skull, for access to the lower parts of the brain. The relationships between the patient and the robot are tracked using optical markers. Figure 4: 3D rendering of CT images for path planning of image guided surgery.

Slide5: 

The vision systems can also be integrated into automated quality control systems. Research issues include the development of algorithms in quality control, reliability, non-linear optimisation and geometrical tolerance. Another application is in augmented reality. Augmented reality is a technology whereby a user's view of the real world is enhanced, with the inclusion or the superimposition of computer-generated information to it. The information may be in the form of labels, texts, outline, 3D models, and shading modifications. Augmented reality has applications in areas such as Computer Aided Surgery, where the surgeon may have images of an internal organ overlaid on his view of the patient to help identify tissues without opening up the patient’s body. It also has applications in other areas varying from manufacturing to the entertainment industries, to military operations, to consumer design, to Robotics, to Telerobotics, and to many uses where extra information to the real world is beneficial to the operation or process. Facilities Some of the key facilities available include the following. · Mobile robots and manipulators: Physik Instrumente M-850 Hexapod NORMAN 200 Mobile Robot MRV4 Mobile Robot · Imaging and vision systems: VolumePro 1000 3D real-time volume rendering accelerator DSP C81 Base Imaged Systems with Analogue Video Module Alpha Parallel Imaging Systems Frame grabbers and development libraries Cameras and lenses · Computing and other equipment: Linux and Window NT workstations 2D Kretz Combison 420 ultrasound scanner and probes Optotrak/3020 3D motion measurement system Future Plans In the areas of vision systems, more robust and faster algorithms are being studied. Smart cameras capable of being embed into miniature robots, and incorporating these algorithms will also be studied. Other studies include the development of algorithms that can detect organ boundaries in medical images like ultrasound scans computed tomography images. These boundaries will have to be compared to the real organs to make sure that they represent the real boundaries of the actual organs in the patients. Another research area is the control for multi-segment robots like the centipede robot. Co-ordination of the various segments is important as each segment has its own controller. More

Slide6: 

efficient methods of locomotion for under-water robots are also being studied. Another topic of research is the co-ordination of a group of intelligent, free-ranging robots following a leader and avoiding obstacles at the same time. Research on the application of augmented reality includes for prostate biopsy and total hip replacement surgery. Three issues have to be addressed. These are: 1. 3D imaging and reconstruction, involving the acquisition and pre-processing of ultrasound images, and the generation of 3D models or information suitable for augmentation. 2. The tracking of target objects in the real world, with the use of tracking cameras to generate co-ordinates for the augmentation of the virtual object in the user's view. 3. The display of the virtual objects. For the development of the image-guided surgical robot for skull-base operation, a prototype has been developed. Further improvement of the system includes making the system more robust and stable. The user-interface will also have to be simplified. Experiments are being carried out to check its accuracy. Future improvements include the incorporation of heptic feedback and control for the system. This will provide the surgeons more control over the robot. It can also be extended into a training system for new surgeons. In the area of quality control, algorithm development and applications of statistical process control (SPC) are being studied. Publications The following is a selected list of related publications. Bai S.P., Teo M.Y., Ng W.S., Sim c. WORKSPACE ANALYSIS OF A PARALLEL MANIPULATOR WITH ONE REDUNDANT DOF FOR SKULL-BASE SURGERY, International conference on Intelligent Robotics and Systems (IROS), 29 Oct -- 3 Nov 2001, Outrigger Wailea Resort, Maui, Hawaii, USA. Proceedings of the 2001 IEEE/RSJ International Conference on Intelligent Robot and Systems pp 1684 – 1699. Khoo L.P. and Teo M.Y. A PROTOTYPE FUZZY-BASED CLASSIFIER SYSTEM FOR TRAJECTORY PLANNING, Journal of Network and Computer Applications, Issue 20:2, 1997, pp. 191 - 202. Kwoh C.K., et. al. OUTLINE OF PROSTATE BOUNDARY USING HARMONICS METHOD, Medical & Biological Engineering & Computing (Communication), Vol 36, No. 6, Nov 1998, pp 768-771. Lim SS, Ong EK and Fok SC, APPLICATION OF NEURAL NETWORK IN VISION CONTROL FOR PCBA HANDLING AND INSPECTION, ISMA'97 International Symposium on Microelectronics and Assembly, 1997.

Slide7: 

Liu Y.J., Ng W.S., Teo M.Y. and Lim H.C. COMPUTERISED PROSTATE BOUNDARY ESTIMATION OF ULTRASOUND IMAGES USING RADIAL BAS-RELIEF METHOD, Medical and Biological Engineering and Computing ,Vol 35, September 1997, pp. 445 – 454. Loh Y.C., et. al. SURGICAL PLANNING SYSTEM WITH REAL-TIME VOLUME RENDERING, Medical Imaging and Augmented Reality, 10-12 Jun 2001, The Chinese University of Hong Kong, Hong Kong. Sim K.Y., et. al. IMAGE-GUIDED MANIPULATOR COMPLIANT SURGICAL PLANNING METHODOLOGY FOR ROBOTIC SKULL-BASE SURGERY, Medical Imaging and Augmented Reality, 10-12 Jun 2001, The Chinese University of Hong Kong, Hong Kong. Sim S.K. and Teo M.Y. ENHANCING FLEXIBILITY OF VISION-BASED ROBOTS USING ARTIFICIAL NEURAL NETWORK APPROACH, Integrated Manufacturing Systems, Vol 8, No 1, 1997, pp. 43 - 49. Tang, S.L., Kwoh, C.K., Teo, M.Y., Ng, W.S. and Ling, K.V. AUGMENTED REALITY SYSTEMS IN MEDICAL APPLICATIONS, IEEE Engineering in Medicine and Biology Magazine, Vol 17, n 3, May/June 1998, pp. 49 – 58. Xie M, A SINGLE CAMERA BASED OBJECT TRACKING SYSTEM: MOTION STEREO OR VISUAL SERVOING, 26th International Symposium on Industrial Robot, Singapore, p399-404, Oct. 4-6, 1995. Xie M, GROUND PLANE OBSTACLE DETECTION FROM STEREO PAIR OF IMAGES WITHOUT MATCHING, 2nd Asian Conference on Computer Vision, Singapore, Vol.2, p280-284, Dec. 6-8, 1995. Contact For further information, please contact: A/P Teo Ming Yeong School of Mechanical & Production Engineering Nanyang Technological University 10 Nanyang Avenue Singapore 639798 Tel: (65) 6790 7707 Fax: (65) 6791 1859 (International), 6792 4062 (Local) Email: mmyteo@ntu.edu.sg More information can also be found at http://www.ntu.edu.sg/mpe/Research/Programmes/Vision/