The Role of Sensors in Robotics


Presentation Description

application of sensor


Presentation Transcript

The Role of Sensors in Robotics :

The Role of Sensors in Robotics

Review: Why is robotics hard? :

Review: Why is robotics hard? sensors are: limited inaccurate noisy effectors are: limited crude the state (internal and external, but mostly external) of the robot is partially-observable , at best the environment : often dynamic (changing over time) full of potentially-needed information


Sensors Sensors are one of the key elements as well as limitations in robotics. Sensors constitute the perceptual system of a robot. Sensors do not deliver state! Sensors are physical devices that measure physical quantities, such as: physical property -> technology contact -> bump, switch distance -> ultrasound, radar, infra red light level -> photo cells, cameras sound level -> microphones strain -> strain gauges rotation -> encoders magnetism -> compasses smell -> chemical temperature -> thermal, infra red inclination -> inclinometers, gyroscopes pressure -> pressure gauges altitude -> altimeters and others... Note: the same property can be measured with different sensors

Mobile Robotics Sensors that we used in the past:

Mobile Robotics Sensors that we used in the past contact -> bump, switch distance -> ultrasound, sonar, infrared light level -> photo cells , cameras sound level -> microphones strain -> strain gauges rotation -> encoders magnetism -> compasses (PSUBOT) smell -> chemical (fire detector) temperature -> thermal, infra red inclination -> inclinometers, gyroscopes pressure -> pressure gauges

Simple and Complex Sensors:

Simple and Complex Sensors Sensors range from simple to complex in the amount of information they provide: a switch is a simple on/off sensor a human retina is a complex sensor consisting of more than a hundred million photosensitive elements (rods and cones) Sensors provide raw information , which can be treaded in various ways , i.e., can can be processed to various levels . For example, we can simply react to the sensor output : if the switch is open, stop , if the switch is closed, go . More complex sensors both require and allows to do more complex processing .

Slide 6:

We can ask the following question: "given the sensory reading I am getting, what was the world like to make the sensor give me this reading." This is what is done in computer vision, for example, where: the sensor ( a camera lens ) provides a great deal of information (for example, 512 x 512 pixels = 262,144 pixels of black & white, or gray levels, or color), and we need to compute what those pixels correspond to in the real world (i.e., a chair, a phone ?). Simple and Complex Sensors

Signals -> Symbols(States) :

Signals -> Symbols(States) Sensors do not provide state/symbols, just signals A great deal of computation may be required to convert the signal from a sensor into useful state for the robot. This process bridges the areas of: electronics , signal processing , and computation. Real state measurement Model of real State in robot memory

Levels of Processing:

Levels of Processing Example 1 . just to figure out if a switch is open or closed , you need to measure voltage going through the circuit ; that's electronics Example 2 . now suppose you have a microphone and you want to recognize a voice and separate it from noise; that's signal processing Example 3. now suppose you have a camera, and you want to take the pre-processed image (suppose by some miracle somebody has provided you with all the edges in the image, so you have an "outline" of the objects), and now you need to figure out what those objects are , perhaps by comparing them to a large library of drawings; that's computation

Processing sensory data needs brain:

Processing sensory data needs brain As you can see, sensory data processing is challenging and can be computationally intensive and time consuming . Why does that matter? Because it means that your robot needs a brain to do this processing.

What does the brain have to have to process sensors: :

What does the brain have to have to process sensors: analog or digital processing capabilities (i.e., a computer) wires to connect everything support electronics to go with the computer batteries to provide power for the whole thing Thus perception requires: sensors (power and electronics) computation (more power and electronics) connectors ( to connect it all)

What does the brain have to have to process sensors: :

What does the brain have to have to process sensors: It is not a good idea to separate : what the robot senses, how it senses it, how it processes it, and how it uses it. If we do that, we end up with a large, bulky, and ineffective robot. Historically , perception has been treated poorly: perception in isolation; perception as "king"; perception as reconstruction. Traditionally these approaches came from computer vision , which provides the most complex data. Not a good idea to separate and reconstruct

The best is Sensor Integration Approach:

The best is Sensor Integration Approach Instead, it is best to think about these as a single complete design: the task the robot has to perform the best sensors for the task the best mechanical design that will allow the robot to get the necessary sensory information to perform the task the body shape of the robot, the placement of the sensors, etc.

New and Better Approaches to Perception:

New and Better Approaches to Perception Perception in the context of action and the task Action-oriented perception Expectation-based perception uses knowledge about the world as constraints on sensor interpretation Focus-of-attention methods provide constraints on where to look Perceptual classes partition the world into useful categories

A New and Better Way:

A New and Better Way Nature is very clever in the way it solves perception/sensing problem; it evolves special sensors with special geometric and mechanical properties. Facetted eyes of flies , or polarized light sensors of birds have, or horizon/line sensors of bugs have, or the shape of the ear, etc. All biological sensors are examples of clever mechanical designs These designs maximize the sensor's properties: it's range correctness. Use ideas from Nature for Perception

Proprioception - internal state:

Proprioception - internal state Origin of received sensory information divides perception into Proprioception: sensing internal state (e.g., muscle tension, limb position) Exteroception: sensing external state (e.g., vision, audition, smell, etc.) Examples of proprioception : path integration ( dead-reconning ) balancing all movements...


Affordances Affordances are "potentialities for action inherent in an object or scene" (Gibson 1979, psychology) The focus of affordances is the interaction between the robot and its environment Perception is biased by what needs to be done. Robot thinks – what is my task? I see a chair because I want to sit on it I see a chair as something to avoid. I see a chair as something to throw at my enemy.


Affordances As a robot designer, you may not get the chance to make up new sensors . But you will always have the chance to design interesting ways of using the available sensors to get the job done. It is often not only a chance but a necessity! Utilize the interaction with the world and always keep in mind the task . Food for thought: how would you detect people in an environment?

How to detect people?:

How to detect people? For example, how would you detect people? Some options include: temperature: pyroelectric sensors detect special temperature ranges movement: if everything else is static shape: now you need to do complex vision processing color: if people are unique colored in your environment Let's think about something even more simple: how would you measure distance: ultrasound sensors give you distance directly (time of flight) infra red through return signal intensity two cameras (i.e., stereo) can give you distance/depth a camera can compute it from perspective use a laser and a fixed camera, triangulate structured light ; overlying grid patterns on the world frequency and phase modulation interferometry

Sensor Fusion:

Sensor Fusion Another clever thing to do is to combine multiple sensors on a robot to get better information about the world. This is called sensor fusion . Sensor fusion is not simple: Different sensors give different types, accuracy and complexity of information; processing is necessary to put them together in an intelligent and useful way, and in real-time. The brain processes information from many sensors (vision, touch, smell, hearing, sound). The processing areas are distinct in the brain For vision, they are further subdivided into the " what " and " where " pathways. Much complex processing is involved in combining the information .

Information Filters:

Information Filters Sensory organs act as information filters . Extract only part of the total information available form a representation or physical encoding which facilitates the answers to some questions while making others impossible to answer Simple light sensors function like a set of goal-oriented detectors. For instance, frog eyes are designed to detect motion not interpret static images .


Vision Vision is the process of converting sensory information into the knowledge of: shape, identity or configuration of objects . Other sensors besides light sensors can also provide similar information: bat sonar pit viper heat detector touch

Vision (more):

Vision (more) Current processing of sensory data can be greatly affected by: previous input and its interpretation and pre-wired processing Seeing is the physical recording of the pattern of light energy received from the environment. It consists of: selective gathering in of light projection or focusing of light on a photoreceptive surface conversion of light energy into a pattern of chemical or electrical activity Characteristics of Vision sensor

Costs and Benefits of Sensing:

Costs and Benefits of Sensing A cost of sensing of a system in terms of: 1. energy , 2. organizational complexity and 3. the possibility of malfunction . The nature of useful information is related to organism’s needs and goals . For example, plants only need information on light direction. Their system compares the light energy received on differently oriented surfaces.

Receptors in biological organisms:

Receptors in biological organisms Sensitivity to environmental influences is a general characteristic of living cells. In addition to general sensitivity, most animals develop a range of specialized receptor cells These often form parts of multi cellular sense organs. Types of senses are called sensory modalities . Tell the story of bacteria foraging

Sensory Modality:

Sensory Modality Classifications of sensors 1 . Exteroceptors - sensitive to external influences 2 . Interoceptors - respond to internal factors 3 . Proprioceptors - signal movements or positions of muscles, joints, etc. Classification can be based on the physical characteristic of the stimulus concerned, e.g.: light, mechanical, chemical. Phasic receptors respond to changes in the environment . Tonic receptors relate to the absolute level of stimulation . Some receptors are a combination of phasic and tonic. Sensitivity to one modality can be exploited to provide information about another.

Sensory Modality:

Sensory Modality Classifications (more) Receptors sensitive to gravity are called statocysts . These receptors function by using sensory cilia in a vesicle which contains one or more dense bodies to sense the position of these bodies. These organs can also sense acceleration . Note: insects lack these specialized organs, instead, they depend on the information from many sense organs associated with their joints to provide relevant information. Sensory Modality

Specialist and Generalist Receptors:

Specialist and Generalist Receptors 1. Receptors which are specialists respond only to a restricted range of whatever they are sensing. For example, olfactory specialists have a restricted spectrum of response to odors with an acute sensitivity to only a single compound such as a pheromone. 2. Generalist receptors respond to a wide variety of stimuli within the modality: but each generalist has its own pattern of sensitivity so a substance can be recognized by the unique combination of receptors activated.

Intensity Coding in biological sensors:

Intensity Coding in biological sensors Information from sensors is usually not just ON or OFF, but also includes `` how much ''. The range of stimulation intensity to which an organism is sensitive is often a controllable factor . Also different cells can operate across different parts of a wide range.

Sensory Processing Example:

Sensory Processing Example In the locust , simple light sensing organs on the top of the head produce a poorly focused image. A massive amount of receptor information (about 1000 receptors) in each organ is funneled through a small number of second-order neurons (25). During flight, the ocelli provide a rapid, overall assessment of the position of the horizon .

Another Example:

Another Example When a male hoverfly has a possible mate in its field of vision , it sets a course to intercept. To plot a course, it needs distance, velocity and course information of target probably not determined from observation. The fly ``assumes'' that the object in the visual field is 1.the size of one of its own kind 2.travelling at approximately the same velocity The size assumption leads to a determination of distance . The direction and speed at which the object moves across the visual field indicate then its course and the intercept can begin!


Convergence Convergence occurs when multiple sources of information are compressed into a much smaller domain. A sensory field is an array of receptors which provide sensory input to a cell or centre in a nervous pathway.


Divergence Divergence is the conveying of information from a single receptor cell, or group of cells, into the nervous system via multiple or parallel pathways. These pathways can be used to extract and segregate different types of information. Divergence also covers the concept of a system responding to a single sensory modality, but providing out to different centers and thus influencing different types of behavior.

Labeled Lines:

Labeled Lines This principle works on the premise that similiar signals from different receptors are handled as if they were ``labeled'' by their origin. An example is the escape response of the cockroac h. The lunging attack of a toad creates a current of air which is detected by sensory hairs on the anal cerci of the insect, The hairs are arranged in a number of columns which are sensitive to wind from different directions. The different columns form distinct combinations of connections with processing neurons so that the insect is aware of the location of the threat . The combinations of sensory input trigger appropriate movements .

The Photopine:

The Photopine Sensors distributed over vehicle body As the sensor is touched, the reflex response is immediate and it determines the area of contact.


Sources A. Ferworn Maja Mataric Fred Martin

Questions for students:

Questions for students List all robotics sensors that you are familiar with. List some inexpensive sensors that one can purchase in Radio Shack or Home Depot and how they can be used in robotics. What is the concept of sensor fusion. Write a simple program to fuse 2 types of sensors. What is proprioception and exteroception? What are affordances, give examples for our robot theatre. Why affordances are so important. Discuss simple vision system for a humanoid robot, what can it do? Give examples of complex and simple sensors and their role. Give examples of perception system of some simple animal and how this system can be modeled in our robots, both stationary and mobile robots. What kind of experiments would you implement with this system? Give examples of divergence and convergence of sensors. How would you implement a vision system for a simple mobile robot that would act according to three principles: (1) if a moving item is small, it is a potential food, attack it. (2) if a moving item is large, it is a potential enemy, escape. (3) if a moving target is your size, it is a potential mating partner, mate.

authorStream Live Help