MACS

EU Project MACSMulti-sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances: 

EU Project MACS Multi-sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances Erich Rome Robot Control Architectures Department Cognitive Systems Kick-Off Meeting, Bled, Oct 28–30, 2004

MACS Project Overview: 

MACS Project Overview Sections MACS Facts Affordance-based Robot Control The Vision & the Key Objectives Steps to Achieve the Objectives The Key Milestones Next Steps: Some Details

a. MACS: Facts: 

a. MACS: Facts Project type: STReP Grant no.: FP6-004381 Project start: September 1, 2004 Duration: 3 years Kick-off meeting: Sankt Augustin, September 9–10, 2004 Web site: www.macs-eu.org Consortium: 5 participants

a. MACS: Facts: 

a. MACS: Facts Participants and Competences 1 FhG/AIS (Coord.) Fraunhofer Institute for Autonomous Intelligent Systems, Sankt Augustin, D Erich Rome Robot control architectures, robot & sensors & autonomous systems design, biologically inspired robot vision 2 JR_DIB JOANNEUM RESEARCH Forschungsgesellschaft mbH, Graz, A Lucas Paletta Computer vision, ANN-based sensorimotor learning, mobile mapping 3 LiU-IDA Linköpings Universitet, AI & Integrated Computer Systems Division, S Patrick Doherty Autonomous systems, knowledge representation, AI planning 4 METU-KOVAN Middle East Technical University, Ankara, TR Erol Sahin Evolutionary & swarm robotics, physics-based modelling & simulation, distributed computing environments 5 OFAI Österreichische Studiengesellschaft für Kybernetik, Vienna, A Georg Dorffner Cognitive modelling & neuroscience, symbol grounding, AI learning

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control Some Objectives of the Cognitive Systems Call: To construct embodied systems that can perceive, understand, and interact with their environment while performing goal-directed tasks. Methodologies for the construction of robust and adaptive cognitive systems integrating perception, reasoning, representation and learning.

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control The MACS Claim: The use of affordances in control architectures may link perception, action, learning and reasoning in a new way. Perceiving the world in terms of affordances will provide a paradigmatic change in the architecture of embedded cognitive systems by helping to structure perception and reasoning in both an action-oriented and goal-directed way.

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control Affordances: Notion created by cognitive psychologist J. J. Gibson (1979): „An affordance is a resource or support that the environment offers an agent for action. The agent must possess the capabilities to perceive and act upon it.“

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control Examples of Affordances: throw use as tool hide, climb Place to look for prey

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control Affordances (1): Affordances can be put in terms of abstract properties: throwable -> fist-sized dense object of certain weight range sittable upon -> knee-high flat stable horizontal surface of certain minimum size Affordances depend particularly on the agent’s properties like body size, weight, form, and its perception and action capabilities

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control Affordances (2): Object identity is just another property, not necessarily the most important one – Its importance is goal dependent The abilities to perceive and act upon affordances may be acquired by learning – by experimentation and observation Acting upon affordances may require episodic knowledge: The sequence of actions required to act upon the affordance of a cup of coffee to drink from

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control Affordances (3): Selection of affordances depends on high-level goals Goals influence perception of affordances We do not get flooded by thousands of affordances Affordances comprise a functional view of environment Affordances are suited to structure the perceptual input for action and reasoning Link for perception, action, reasoning and learning

b. MACS: Affordance-based Robot Control: 

b. MACS: Affordance-based Robot Control Some implications for adaptivity: Affordances would allow greater flexibility in manipulation tasks: If a searched object for a manipulation is not available, the robot may look for another one with the same affordances and act upon the substitute instead. Such an ability would be especially helpful in complex environments with significant dynamics.

c. MACS: The Vision: 

c. MACS: The Vision The Vision: Affordance-based control as a new paradigm to better link perception, action, reasoning and learning, suited to advance the further development of embodied cognitive systems.

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives Main objective of MACS: Explore and exploit the concept of affordances for the design and implementation of autonomous mobile robots Develop affordance-based control as a method for robotics Provide a new way for reasoning and learning to connect with reactive robot control

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives 5 scientific & technological objectives plus 1 dissemination objective

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives Scientific and technological objectives: A radically new robot control architecture, implementing affordance-based control Affordance-based control changes deeply the flow of information as well as the required processes Use of affordances in control architectures is no emergent phenomenon, cannot be added on top of an existing control architecture, needs to be considered in the basic design. An affordance-based architecture will be proposed, tested and evaluated. Affordances will be integrated into perception, action and learning.

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives Scientific and technological objectives: Grounded and goal-directed perception of affordances • Affordances spring off perception on a low level, associating salient perceptual features to the representation of what an object affords. • What can or should be perceived and used as an affordance depends on the sensors and actuators that the robot has. • Filtering mechanism to prevent the robot from drowning in affordances needs to be in effect deep down in the process of affordance perception. • It has to be influenced from high-level goal-orientation or attention modes preventing currently irrelevant affordances from distracting the controller.

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives Scientific and technological objectives: Explicit affordance representations for different granularity levels Using affordances for reasoning and symbolic learning requires an explicit representation Representation includes perception and action side of an affordance plus episodic knowledge and expectations about feed-back from the environment when acting upon an affordance.

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives Scientific and technological objectives: Learning affordances by experimentation or by observation Affordances are individual on sensoric, physical, and experience level A natural way of getting at affordances is learning Learning by individual experimentation or by imitation. Suitable learning methods will be developed Extreme option of teaching: programming Will be used for higher-level, complex or abstract affordances

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives Scientific and technological objectives: Integrated demonstrator on an autonomous mobile robot Results will be demonstrated in integrated form Mobile robot: able to navigate and do simple manipulation tasks Wide range of perceptions through multi-modal sensor configuration including vision and a 3D Laser scanner Robot control: plan-based to provide goal-directed behaviour

c. MACS: The Key Objectives: 

c. MACS: The Key Objectives Dissemination objective: Make the different involved scientific communities, possible appliers of the results, and the interested general public aware of the respective MACS achievements

d. MACS: Steps to Achieve the Objectives: 

d. MACS: Steps to Achieve the Objectives Workpackages: WP0 – Management (13+6 PM) WP1 – Infrastructure (39+4 PM) WP2 – Affordance-based Control Architecture (57+6 PM) WP3 – Perception of Affordances (76+2 PM) WP4 – Representation of Affordances (42+4 PM) WP5 – Learning of Affordances (47+4 PM) WP6 – Proof of Concept & Dissemination (38+4 PM)

e. MACS: The Key Milestones: 

e. MACS: The Key Milestones month

f. MACS: Next Steps: 

f. MACS: Next Steps Physical demonstrator KURT2 (with 3D Laser scanner) Commercial platform for research & education Developed initially at FhG/AIS, Produced & distributed by KTO 3D Scanner developed and assembled at FhG/AIS, Simple gripper under construction

f. MACS: Next Steps: 

f. MACS: Next Steps Next steps: Publish evaluation of state of the art in affordance-related research Specify a demonstrator scenario Design missions and tasks that the affordance-based robot should accomplish in the demonstrator scenario Specify requirements for perception, representation and learning of affordances as well as for reasoning about and acting upon affordances Later: Publish benchmark problems suited to demonstrate the power and limitations of the approach

f. MACS: Next Steps: 

f. MACS: Next Steps Next step towards dissemination: Application for a Dagstuhl seminar “Towards Affordance-based Robot Control” International conference and research center for computer science Schloss Dagstuhl in Wadern (close to Saarbrücken, Germany) Dagstuhl-Seminars: gatherings of 35–45 scientists working for a week on a specific computer science related topic with interdisciplinary aspects International participants: established researchers and promising young scientists

f. MACS: Next Steps: 

f. MACS: Next Steps Dagstuhl seminar details: Organizers: Erich Rome, Patrick Doherty, Georg Dorffner, Joachim Hertzberg Deadline: Nov 15, 2004 Seminar: 5 days in 1st half of 2006 Preliminary list of potential participants; may be updated and extended when application succeeds Participation interests can be expressed via email (rome@ais.fraunhofer.de) (preferably non-German female young researchers) Participation fee (150 €) also covers accommodation and food Travel expenses are not covered Young researchers may apply for grants covering travel expenses Information available at: http://www.dagstuhl.de/HLSC/

Slide28: 

Thank you for your attention