Alexander BrÃndle

Slide1: 

1 Robots Unlimited PHD Summerschool July 2006 Alexander Brändle Marco Combetto

Agenda: 

2 Agenda Who we are? Why robotics? Activity overview Future ideas

Intelligent Environments Team: 

3 Intelligent Environments Team Alexander Braendle Marco Combetto Pierre-Louis Xech Andreas Heil (PHD Student)

Why robotics?: 

4 Why robotics? How will Future Environments look like? Challenges/Needs Personal Devices, Embedded Devices, Sensors, Robots, Appliances? Enabling technologies,Programming paradigms,Interaction paradigms? How will future applications look like? Wide range of user types? Mixing real and virtual world? Robots could form a part of it!

The Robotics Wave: 

5 The Robotics Wave Service and consumer markets just emerging Remote assistance/presence Assistive Facilities maintenance Security Education Entertainment Academic Research Complex topic Moving from 8/16 to 32 bit Lots of hand-coded solutions Education and Hobbyist channel Still mostly 8 bit, starting to shift Strong governmental interest worldwide EURON roadmap: 2015 -> It‘s all about software

Robots!: 

6 Robots! Wide range of applications Commercial use Academic use Personal use How to program them? Challenges Complexity Reusability Reliability Resources Tools Technologies Choice Sharing Transference of skills/experience

Links: 

7 Links European Robotics Network (EURON) European Robotics Platform (EUROP) IEEE RAS Technical Commitee on ‚Programming Environments in Robotics and Automation‘

Robots, too?: 

8 Robots, too? Software Internet Explorer Media Player MS Agent Mobile Phones … Definition: A robot is a device, hard- or software with the capability of sensing and (re-)acting.

Robotics at Microsoft [Research (Cambridge)] Robots in human environments: 

9 Robotics at Microsoft [Research (Cambridge)] Robots in human environments End-user programming Developer Tools (Fischertechnik-Demo, Scatterweb-Demo) Visual Programming (.FUN-Demo) End-user „debugging“ Microsoft Robotics Studio Coding4Fun Enabling new applications Emotional oriented computing Personal robotics Self-reconfigurable structures Learn from interaction with animals

The Fischertechnik ROBO Interface: 

10 The Fischertechnik ROBO Interface The in- and outputs 8 digital inputs 2 digital and analog distance sensors 4 analog sensors for resistance and voltage 4 motors with 8 different speeds The board Serial port, COM, RS232 USB Infrared R/F module available Ethernet for the next hardware revision planned TS%SystemRoot%\System32\mstsc.exe

Adding Control: 

11 Adding Control Now we want to control the robot Let‘s take another off the shelf product Ordinary Joystick +

Coding4Fun: 

12 Coding4Fun http://msdn.microsoft.com/coding4fun/

Integrating Sensors: 

13 Integrating Sensors Sensors are they eyes and ears of robots Increasing Demand for easy robust sensor platform from biology, ecology, civil engineering et al. How to get easy access to Sensor Nodes systems A challenge on it‘s own: Where are sensor node platforms heading? Dealing with vast amounts of real-time data becoming available

Sensor Networks in the real world: 

14 Sensor Networks in the real world Robustness … WSNs have to work 24/7 WSNs have to offer uniform APIs Long-lived, autonomous networks Self-organizing and energy efficient Routing Data aggregation Deployment & Setup …Better development support Realistic simulation tools Heterogeneous testbeds Useful traces Tool integration (in well know platforms) Development for heterogeneous systems ScatterWeb/.NET

Sensors (ScatterWeb) An open and flexible platform for rapid protoyping & implementation of wireless sensor networks: 

15 Sensors (ScatterWeb) An open and flexible platform for rapid protoyping & implementation of wireless sensor networks Nodes with/without sensors Sensors Luminosity, noise detection, vibration, PIR movement detection, Microphone/speaker IR sender/receiver stand-alone/modular Acceleration, humidity, temperature, luminosity, noise detection, vibration, PIR movement detection on demand Stand-by: 7.6µA, 5 years life-time with AA battery and 1% duty-cycle Gateways WLAN, Ethernet, Bluetooth, GPS, GSM/GPRS, USB, RS485 , serial… Software Management, flashing, routing, ns-2 simulation models TCP/IP, web server (http://193.10.67.150/), Contiki, TinyOS, … Basic functions for energy management, routing www.scatterweb.net Jochen Schiller,Free University Berlin

.NET for ScatterWeb: 

16 .NET for ScatterWeb Sensor world available to every developer Easy access to sensor values, events, and functions Understandable namespaces and interfaces Support for IntelliSense and dynamic help Well known programming model (events, methods, properties) Extensibility of the nodes’ logic (Tiny C#) and instant IntelliSense-Update Development, deployment and debugging within Visual Studio Extend the .NET tools and architecture to small devices Jochen Schiller,Free University Berlin

A first step: 

17 A first step Attractive for developers End-users?

.FUN: 

18 .FUN A compelling & engaging programmable environment to play & learn for children (introduce children to Computing in new ways) Make technologies of tomorrow accessible to non technical market (children, nurse, elderly, machine operator) Linking real and virtual world Wider market potential of Robotics (Industrial, Assistive technology, new consumer products, health) Technical University Berlin

Next steps: 

19 Next steps Keep graphical representation Using context data Domain specific language Changing the paradigm …

Integrating a Context Server 1/2: 

20 Integrating a Context Server 1/2 Context Server Application(s) Sensors DB Raw Sensor Data Context Events Context Requests Torben Weis, University of Stuttgart

Integrating a Context Server 2/2: 

21 Integrating a Context Server 2/2 Context Server Application(s) Context Simulator DB Raw Sensor Data Context Events Context Requests 3D Data Torben Weis, University of Stuttgart

Slide22: 

22 Torben Weis, University of Stuttgart

Domain-specific Languages: 

23 Domain-specific Languages Visual N# Domain-specific Graphical language Extension of VRDK C# / VB.NET General purpose programming language N# Domain-specific Textual language Users can switch between both notations Code translator Torben Weis, University of Stuttgart

N# - A Textual Notationfor Visual N#: 

24 N# - A Textual Notation for Visual N# ambient MyAmbient @ Person where filter ($1.Age < 40), filter($1.Location == "Stuttgart") { discover Lichter @ Light where distance(a), filter($1.Color == "red"); process OnLampAdd @ l = Lichter.Added { l.On(); } process OnLampRemove @ l = Lichter.Removed { l.Off(); } a = 100; } Torben Weis, University of Stuttgart

Microsoft Robotics Studio: 

25 Microsoft Robotics Studio Concurrency Services infrastructure Samples and tutorials Robot services Robot models Technology services Microsoft Robotics Studio Simulation Tool Visual Programming Language A development platform for robotics community, supporting a wide variety of users, hardware, and application scenarios. Runtime Authoring Tools Services and Samples Make it easy to manage asynchronous components Avoid need to understand manual threading, semaphores, etc. Provide a scalable programming model Make state observable, easily accessible Provide for reusability and failure Support component discovery and composition Support remote/distributed execution

Microsoft Robotics StudioKey Runtime Features: 

26 Support standalone and distributed processing scenarios Connected operation (remote execution on PC) Disconnected autonomous operation (with optional networked monitoring) Distributed execution (execution across compute units) Microsoft Robotics Studio Key Runtime Features

Microsoft Robotics Studio: 

27 Microsoft Robotics Studio Extensible to a wide variety of hardware

Microsoft Robotics StudioServices and Samples: 

28 Microsoft Robotics Studio Services and Samples Over 15 tutorials VB.Net, C#, JScript Support for LEGO® Mindstorms® RCX LEGO® Mindstorms® NXT fischertechnik® MobileRobots Pioneer P3™

Microsoft Robotics StudioOther University Support: 

29 Microsoft Robotics Studio Other University Support Bryn Mawr College Carnegie Mellon University Cornell University Georgia Tech Massachusetts Institute of Technology Stanford University University of Pennsylvania University of Pisa University of Southern California University of Washington

Imagine a world …: 

30 Imagine a world … where „Paper“ is able to understand, what you are doing …

Robotics at Microsoft [Research (Cambridge)] Robots in human environments: 

31 Robotics at Microsoft [Research (Cambridge)] Robots in human environments End-user programming Developer Tools (Fischertechnik-Demo, Scatterweb-Demo) Visual Programming (.FUN-Demo) End-user „debugging“ Microsoft Robotics Studio Coding4Fun Enabling new applications Emotional oriented computing Personal robotics Self-reconfigurable structures Learn from interaction with animals

Emotion-Oriented Computing: 

32 Emotion-Oriented Computing General Goal: Make interaction between human and machine more natural for the humans Machine should be able to: To register human emotions To convey and comunicate emotion To understand the emotional relevance of the event 1/2/2008 32 Microsoft Internal Only

Human Centred, Affects and Emotions: 

33 Human Centred, Affects and Emotions Enhance communication through compelling, fun and emotional interactions with computing (Mobile devices, Robotics) Seeking to make the process more intuitive, interactive and appealing to a wider group of people. Experimenting, evolving, evaluate emotional models Support privacy, intimacy and different level of information sharing Extending the software as a new medium Affective Media and Mobile media (affective loop) Participatory creation, exchange, authoring and fruition Storing the intimacy, the private value of the things Fusion of different kind of data sensors, embodiments, 1/2/2008 Microsoft Internal Only 33

Human Centred, Affects and Emotions: 

34 Human Centred, Affects and Emotions 1/2/2008 34 Microsoft Internal Only Enhance communication through compelling, fun and emotional interactions with computing. Seeking to make the process more intuitive, interactive and appealing to a wider group of people. Experimenting, evolving, evaluate emotional models Support privacy, intimacy and different level of information sharing Extending the software as a new medium Affective Media and Mobile media (affective loop) Participatory creation, exchange, authoring and fruition Storing the intimacy, the private value of the things Fusion of different kind of data sensors, embodiments,

SensingInteraction with Environments: 

35 Sensing Interaction with Environments Some examples

Affective DiaryDesigning for bodily expressiveness and self-reflection: 

36 Affective Diary Designing for bodily expressiveness and self-reflection Collecting memories – including body memorabilia mingled with mobile materials (SMS, MMS, photographs, music listened to, video,.. Offering a diary medium in which those memories can be mirrored and organised Empowering the user to create meaning and alter those representations Prototype build on TabletPC and Smartphone Sensordata (movement, arousal) 1/2/2008 Microsoft Internal Only 36

BSP: Interactive storytelling in vast location based : 

37 BSP: Interactive storytelling in vast location based Pervasive computing: Location based games provide navigational challenges e.g. Chasing Mobile Media: A flexible media and framework for interaction Interactive storytelling: Balance linearity with user control, believable characters The concept of believable environments, and our implementation, put a research focus on possibilities to: Enrich pervasive games with location dependent narratives Design “the stage set” to improve interactive storytelling 1/2/2008 Microsoft Internal Only 37

SensingInteraction with Robots: 

38 Sensing Interaction with Robots An example

Why Robotics?: 

39 39 Why Robotics? Interest about robots is spreading fast Reinventing Personal Computing New way of interact because the usual ways are not enough (keyboard, mouse, etc.) We look for new interaction modality? Does Human size matter? Social aspects.. Robots as an embodiment How to discover contexts and adapting? Technology Technology seems to be mature enough to produce robots not only for industry Software for controlling social robots should be robust and should adapt to the changing environment It seems that control systems theory doesn’t scale well in this setup

The mind and the body: 

40 40 The mind and the body Neurophysiologists suggest that the body plays a crucial role in our cognitive process The brain experiences the world through the body: The environment is well known to the brain Outside environment is a set of states of inside Agents with body (embodied agents) try to mimic this schema to produce intelligent behavior Dautenhan and Jakobi show that body’s presence influences the behavior of software (anything already heard?)

Component structure: 

41 41 Component structure Knowledge Base Body Map WiFi Roblet MotionRoblet Vision Roblet Sensors Internal perception Network perception Arm Roblet

Robotics4.net: 

42 42 Robotics4.net Realization of a platform for define the body of a robot as a set of agents (Roblets) that acts as intermediaries between the devices and the reasoning software The robot provides a physical body and a set of core services to support the development of highly autonomous agent situated in both cyberspace (e.g Internet, virtual entities) and real world Definition of an Object Model, an API, a set of abstraction that allows: Portability (on different type of platforms) Scalability from small (e.g. Lego) to complex systems Integration with the MS Development Platform and MS OS and Application suites

Functions implemented: 

43 43 Functions implemented The platform provide the following base services to support the software implementing the roblets: General purpose functions (I/O) Basic motion abilities Collision avoidance software Vision software for face recognition Speech and gesturing recognition Positioning and navigation aided by video input Network perception IM interface The Architecture is available as a simple Software Development Kit freely downloadable from http://www.robotics4.net

Perception: 

44 Perception 1/2/2008 Microsoft Internal Only 44

Evolving the framework: 

45 45 Evolving the framework Reduce the programming model of Robotics4.NET to the one adopted by the Robotics Studio CTP Extending the Mind/Body model with self-developing tools, model and methods to verify the behavior of systems imposing very high level constrains and how that can be both formally and programmatically verified Testing interaction in real scenarios

Questions: 

46 Questions