Ubiquitous ComputingContext Awareness23/01/2006: Ubiquitous Computing Context Awareness 23/01/2006 Hans Gellersen


Slide2: Context-aware systems Sensing Perception Inference Uncertainty A Case Study


Slide3: Context Aware Systems


Context-aware systems: Context-aware systems We normally think of computers as being dependent on what we tell them to do User input as only external stimulus Systems response depending only on input parameters like functions, I/O systems, etc. Context-aware systems are different System response influenced by context: ‘surrounding information’ like use of global variables in functions Context may also trigger computation (possibly replacing user input) System Input Output System Input Output Context


Context-awareness: what’s new?: Context-awareness: what’s new? Context-awareness has been around for a while Natural language processing Things said in the past are context in a dialogue Intelligent user interfaces User models as context for adaptive services Internet shops Analysis of customer behaviour to contextualize individual shopping experience So what’s new then ? Context-awareness in ubiquitous computing Using “real world context”: What’s going on outside the computer ?


Pioneered at PARC: Pioneered at PARC “if a computer merely knows what room it is in, it can adapt its behavior in significant ways” (Weiser, 1991) “context-aware software adapts according to the location of use, the collection of nearby people, hosts, and accessible devices, as well as to changes to such things over time.” (Schilit et al, 1994)


What might we want to be aware of?: What might we want to be aware of? Anything that helps a system understand it’s environment Where it is Who is using it Who and what is nearby What are they doing What devices are available Location Identity Proximity Activity Physical Conditions …


How can we get such context ?: How can we get such context ? What does it take to make systems context-aware ? Sensors to observe the world Perception to extract meaning from sensors Reasoning about observations to infer more information


Slide9: Sensing


Embedded systems: Embedded systems Most computer systems actually already contain sensors … Embedded systems are fairly ubiquitous Home appliances, consumer electronics, in-car systems, security systems, … They are invisible computers: Function not explicitly visible to user No user interface Instead, they interact with their environment (their host device) through sensors and actuators


Embedded systems: Embedded systems Microcontrollers Microcomputer with on-board I/O Sense some aspect of their environment Perform some action to affect the environment e.g. controlling a valve Why we probably wouldn’t call this a context-aware system Targeted at a very narrow application Sensing a particular physical phenomenon Very specific function: typically don’t get re-programmed


Sensor technology: Sensor technology Adoption of embedded sensors for context-aware systems Sensors directly attached to more general purpose devices e.g. Motion/orientation sensor in portable devices More general sensor packages Sensor boards Wireless sensor nodes, e.g. Berkeley Motes When designing context-aware systems we need to be aware of the capabilities and limitations of sensor technology


Sensor technology: Sensor technology General-Purpose Devices with embedded sensor Programmable Wireless Sensor Packages UC Berkely Motes Microcontroller, Wireless Radio Connectors for spec. sensor boards TinyOS, event-based programming


Sensor Characteristics : Sensor Characteristics Sensor systems differ in their characteristics They aren’t all the same, but involve trade-offs Important characteristics Accuracy – how close is the sensor value to the true value Precision – how often does the sensor get it wrong (because of variance in measurement) Frequency – how often does it measure the value Reliability – how often does it fail completely


Accuracy vs Precision: Accuracy vs Precision Accuracy: degree of veracity, how true the data is Precision: degree of reproducibility, how noisy the data is High accuracy, low precision High precision, low accuracy


Accuracy vs Precision: Accuracy vs Precision For example, for characterisation of location systems: Accuracy specified for a certain percentage of measurements The percentage reflects precision of the system e.g. “10cm accuracy for 95th percentile” means 95% of the readings deviate not more than 10cm from ground truth Example: characterisation of an ultrasonic location system comparing good vs poor line-of-sight conditions


Sensor Characteristics: Sensor Characteristics Example: Location using triangulation Accuracy: Distance estimates based on RF signals are inherently inaccurate Person can be anywhere within certain radius around estimated position Precision: Occasionally, a signal may be lost or completely distorted May generate ‘ghost’ reading: a position that is completely off the mark Timeliness: How fast are positions updated? A ghost remains when the real position changes as person moves


Implications: Implications In most applications you can assume that the information you have is accurate and timely You can’t assume that for sensor information Inaccuracy: lack of correctness in the information “Hans is currently in C39” [not far off when I am in meeting room C38, but not correct] this may lead to inferences beyond what’s really the case: “Hans is getting a coffee and it’s a good time to give him a call” [C39 is the kitchen] Imprecision: noise and lack of detail in the information “Hans is on C floor” [not enough detail to decide whether to call] Untimely: sensor information can also be outdated you may work on an old state while the world has moved on “Hans is in the car park” [where GPS last saw me before I entered the building and was no longer traceable]


Inherent Limitations: Inherent Limitations “The physical world is a partially observable dynamic system ...” “... sensors are physical devices have inherent accuracy and precision limitations” source: D. Estrin et al, IEEE Pervasive Vol 1 No 1


Sensor Principles: Sensor Principles Sensors take physical phenomena and turn them into information The limits of physics apply All sensing is noisy and lossy All sensing uses energy Some phenomena can’t be sensed directly A sensor gives you a partial view of reality A partial picture can be misleading! You generally need more to do anything useful More sensors, or different sensors, or background information Source: adapted from S. Dobson


Sensor Trade-offs: Sensor Trade-offs Size, Energy, Cost A sensor may have the required accuracy and precision … but then it may be too big to fit into the application … or it may require too much energy ... or it may just be too expensive Generally, as size goes down, so do energy consumption and cost … and so do accuracy, precision, and reliability. Accuracy Precision Frequency Reliability Size Energy Cost Source: adapted from S. Dobson


More trade-offs: More trade-offs Where the energy goes Relative energy consumption in wireless sensor devices, generally speaking: Most expensive: wireless communication (sending, receiving, and also just listening) less expensive (by a magnitude): sampling sensors least expensive (again by a magnitude): computation “3000 instructions could be executed for the same energy cost as sending a bit 100m by radio”


Sensing - Summary: Sensing - Summary Sensors take physical phenomena and turn them into information Sensors differ in their characteristics This involves trade-offs in accuracy, precision, frequency, reliability vs. size, energy consumption and cost Sensors are inherently limited, noisy and lossy and provide only a partial view of reality A designer needs to be aware of the capabilities and the limitations of sensor technology to match them to a context-aware application


Location Sensing: Location Sensing Locating something is obviously an important factor People – who’s where Artefacts – where is it, is it where it should be, who’s got it Build location-based services so the system changes based on your location search for a coffee shop -> put closest one at top of the list walk past a shop -> sends a message with latest offers


Location Sensing: Location Sensing Ways of sensing location Communication (also a kind of sensing) e.g. Cell Phone Network GPS RFID Tags IR Motion Sensors Cameras


Communication systems: Communication systems Communication is also a kind of sensing Especially at short range RF Systems use radio signal to actively signal presence/identity talk to an antenna – used for identification and inference of proximity (cell-ID or signal strength) Trade-offs Requires a lot of energy, especially over larger distances (inverse square law of signal propagation) Difficult to interpret – signals go through walls, reflection, attenuation


Communication systems: Communication systems Infrared systems use infrared light beacons line-of-sight: good for modelling, not so good for reliability for identity, cell-based proximity, not really for ranging Example: Active Badge system ORL Cambridge, 1990 Badge emits ID every 15s Environment instrumented with sensors Sensor Sensor Sensor Sensor Badge


Cell Phones: Cell Phones As mobile phones move they communicate with they network Antennas define cells Cells can vary in size from a few hundred metres to a couple of kilometres, with some overlap The network tracks a phone to within a block of nine cells at worst, and a single cell at best If we know the footprint of a cell, we know where the phone is If we know who has the phone, we know where the person is


Cell Phones: Cell Phones The trade-offs Accuracy – not great, cells are quite big Precision – good if you only use cell-id; no ghost readings Frequency – good, the network samples the phone quite often Reliability – not bad, but networks have black spots Size – small for a person, but too large for many artefacts Power – a week or so on standby Of course you’re locating the phone, not the person The person is implied – and the inference may not be correct Another possible source of error


GPS: GPS Global Positioning System A network of satellites broadcasting to receivers accuracy of a few metres best in open fields, precision reduced in cities


RFID Tags: RFID Tags Radio-frequency tags store small amounts of information that is transmitted when they are activated inductively very small: can be attached to anything Tag must be moving to be detected, and we need an antenna Range is small – a few metres at most Identification technology purpose is to discriminate one entity from another But identifying something effectively locates it But unless you have antennas everywhere you won’t see the tag for most of the time


IR Motion Sensor: IR Motion Sensor Passive Infrared Sensors A very simple and widely used sensor Detects changes in the background heat profile in its field of view ‘forgets’ about what it’s seeing when it doesn’t move Binary – something is moving somewhere in field of view, or not No discrimination – reacts to anything warm very cheap way of sensing whether somebody is in a certain location (e.g. for alarm systems)


Cameras: Cameras Like a motion sensor: watch for something moving but a camera can be a lot more discriminating recognise particular objects, not every moving thing extract further information more accurate: map location in image to point in space however, very expensive in terms of processing (depending on what you want to extract from the images)


Slide34: Perception and Inference


Sensing and Perception: Sensing and Perception The gap between sensors and applications Sensors observe physical phenomena in the world Computer systems and applications operate on ‘higher-level’ models of the world Perception: associating sensor observations with meaning


How a computer system sees the world: How a computer system sees the world System’s view of physical world at the lowest level the world seen as collection of sensors sensors generate values for observable variables can be symbolic or numeric can be synchronous data streams or async. events sensor values are associated with meta-data e.g. time, location, confidence, etc.


Perception Model: Perception Model Basic perception component Events Data Control Transformation Events Data Transforming observed events/data to “higher level” events/data System can control the transformation


Example: Example Active Badge Sensor transforming badge sightings to location events Badge ID Sensor ID Control Active badge sensor Location Event (ID, Location, Time) Timestamp sensor data “observable variable” meta data


Example: Example Active Badge Sensor Interpretation of sensor information straightforward Lookup of badge IDs to associate with a user (note this may be a source of error) Sensors have fixed positions and are readily associated with a meaningful location In other cases, interpretation of sensor information may require much more processing and intelligence (AI techniques, machine perception)


Perceptual Components : Perceptual Components Associating observations with entities grouping of observations entity corresponds to a physical object or person easy if sensors are directly associated with an object hard if sensors are external Variable 1 Variable n Control Entity Grouping Entity and Properties ...


Perceptual Components : Perceptual Components Detecting relations Determining relations between entities Easier with sensors external to the entities Harder with embedded sensors Entity E1 Entity En Control Relation Observation Relation (E1, ..., En) ...


Perceptual Components : Perceptual Components Detecting relations: Example Detecting whether objects are carried together This component correlates movement data (acceleration patterns, can’t sense motion directly) ‘Carried together’ is inferred from similar movement: an assumption that may involve risk and increase uncertainty Movement E1 Movement En Control Relation Observation Carried together (E1, ..., En) ...


Context Inference: Context Inference Perception: extracting meaningful facts from raw sensor observations Inference: extract further information, through combination of observed facts, and reasoning


Context Inference: Context Inference Example: Home fitted with IR Motion Sensor only Basic events perceived: Motion (room, t) Examples of simple inferences: Movement from room to room Movement patterns Anybody at home at all? … and more speculative inferences Working too much? Hall Living Room Study PIR PIR PIR


Uncertainty: Uncertainty Computers are notoriously precise The real world is notoriously imprecise Potential problems Over-precision: “the temperature is 20.07464379” An assumption of correctness: “Hans is in his office”


Sources of uncertainty: Sources of uncertainty Inherent in data provided by sensors Limitations in accuracy and precision Through Inferences You’ve observed a fact, and it might be incontrovertible – but what does this let you imply about the world ? As a chain of inference gets longer, it almost always gets less sure Even if you know something accurately and precisely, it may not stay that way for long If you make assumptions beyond what’s actually the case, all your future decisions will be affected


What can we do about uncertainty: What can we do about uncertainty Model with uncertainty Keep track of how well we know what we know Use techniques of uncertain reasoning to manipulate it Statistical models, Bayesian networks, Fuzzy logic Makes context model more complicated but at least we are honest about we know Try to compensate for lack of knowledge Use several ways of looking at the same thing Sensors and models that complement each other


Slide48: Case Study


A Context-aware Table: A Context-aware Table


A Context-aware Table: A Context-aware Table Idea: augment a table to be context-aware Instrument with sensors to detect activity on the table surface Use perception techniques to extract context Use context information to support different applications What we might want to detect Placement and removal of things on the table Movement on the table surface Identity of objects on the table


Sensor Technology: Sensor Technology Pressure and load sensors Generally based on materials that change resistance when deformed under pressure Pressure sensors common in home alarm systems Detect when somebody stands/sits on a point Load sensors provide more accuracy to infer load Trade-offs Accuracy Speed Sampling rate Maximal load Force Control Load Sensor Load This is what you have in your kitchen scales


Perception: Signal Processing: Perception: Signal Processing Raw Signal Control Load Sensor Event


Sensor system for our table: Sensor system for our table Load-sensing surface F1 F2 F3 F4 Control Surface Load Centre of Gravity DF1 DF2 DF3 DF4 Control Surface Load Change Position


Extracting context: Extracting context Basic event detection Change in load at x,y Increase in load by w: an object of weight w has been placed at x,y Reduction in load: object removed from x,y Detecting movements Track change in load distribution on surface Continuous change is associated with movement Load Change Control Surface Event Sensor Event Position Load Change Control Surface Movement Tracker Trace on surface Position


Context-aware Table: Video: Context-aware Table: Video


Event Perception: Event Perception cup book touch and move right click move left click move left click and release


Object Identification: Object Identification But how about identification of objects ? The table can detect that objects are placed and removed - but in abstraction of what they are All the table can detect is the weight of an object Could allow for some discrimination And for some tracking: ‘this is the same object that was removed from the other table’ But not sufficient for identification Identification requires additional information Background information: lookup table of weights? – no, inherently limited and not scalable Other sensors: objects communicating their identity


Combining sensors: Combining sensors Objects contain pressure sensors for placement detection Table contains load sensors for object detection Wireless communication Identity Weight


Distributed Event Detection: Distributed Event Detection


Context Model : Context Model Table and other artefacts modelled with Built-in knowledge: identity, physical model Observable events Context that can be inferred collaboratively


Context-Aware Table - Summary: Context-Aware Table - Summary Sensing Extracting Context Using context for different applications More context: combination of evidence Types of context information Background knowledge Observable events Inferred context


Summary: Summary Context-awareness is considered an important feature of ubiquitous computing systems: access to contextual information that defines the environment in which a system operates awareness of a system’s surroundings and users To implement awareness we need: Sensors to observe the world Perception to extract something that is meaningful (in a domain of interest) Inference to extract more information It is critical to understand Limitations and characteristics of sensors Possible pitfalls involved with inferencing