IRPPresMay0618

IOWA STATE UNIVERSITYDepartment of Electrical and Computer EngineeringComputer Control of Theater Performance ElectronicsClients:Co-Motion Dance CompanyIowa State Dance: 

IOWA STATE UNIVERSITY Department of Electrical and Computer Engineering Computer Control of Theater Performance Electronics Clients: Co-Motion Dance Company Iowa State Dance SD May06-18 Team Members: Faculty Advisors: Amanda Farniok Dr. Julie Dickerson Sheng Ly Prof. Gerald Sheble* Alex Sills Dr. David Stephenson Tarun Bhatia* *Only on project Fall 2005 April 26, 2006

Presentation Outline: 

Presentation Outline Project Overview Research Activities Hardware Configuration Software Development Implementation Resources and Scheduling Commercialization Future Work Lessons Learned Closing Remarks Questions

Definitions: 

Definitions Isadora - Isadora is a graphic programming environment that provides interactive control over digital media, with special emphasis on the real-time manipulation of digital video. SDK - Software development kit Actor – Module for Isadora software developed as an end product. Scene – A collection of user grouped actors in Isadora. TinyOS - TinyOS operating system designed for the Crossbow transceivers nesC - an extension to the C programming language designed to embody the structuring concepts and execution model of TinyOS. OSC – Open Sound Control

Acknowledgement: 

Acknowledgement Janice Baker – Iowa State Dance Valerie Williams – Co-Motion Dance Company Dr. Julie Dickerson – faculty advisor Dr. Gerald Sheble – faculty advisor Dr. David Stephenson – emeritus faculty Jason Boyd – ECpE departmental technician Daji Qiao – associate professor Wen-Chieh – HCI 572 student

Problem Statement: 

Problem Statement Control dancer environment without the help of stagehands or technicians Need dancer-operated sensor pack which sends input to be received by a computer program called Isadora Isadora can be programmed to control lighting, video projection, audio, and other characteristics of the dancer’s stage environment Similar commercial technologies are not able to differentiate between the inputs

Solution-approach Statement: 

Solution-approach Statement Crossbow Technology Components: Sensor Board, Transceivers, Serial Interface Board Flex sensors (variable resistors) used to monitor movement Used nesC code in TinyOS (Crossbow program) to program components and forward data Used C code to write serial-to-OSC relay

Operating Environment: 

Operating Environment Theatrical dance settings: Betty Toman Dance Studio in Forker Hall, Fischer Theater, Stephens Auditorium, Ames City Auditorium Possibility of ambient electromagnetic noise caused by the multitude of electrical equipment in the immediate vicinity Substantial abuse caused by the motion of the dancers, their impact with the floor or wall, and the sweat and heat that their bodies produce during performance

Intended Users and Uses: 

Intended Users and Uses Users Members of Iowa State Dance Members of Co-Motion Dance Company Directors and colleagues of the companies Uses Practices, performances, and productions of the users Usable in any situation analogous to the conditions of these groups’ activities Transmitter is within the requested range of 60 feet from the receiver Impacts inflicted on the transmitter and sensor by the dancers are minor

Assumptions and Limitations: 

Assumptions and Limitations Assumptions Flex sensor data transmission only No clear line of sight from the transmitter to the receiver Receiver and computer, as well as the visual aids, will run on power supplied from an external source Sufficient power to operate for at least the duration of an ISU/Co-Motion dance production Limitations End-product will be only one transmitter/receiver pair but will allow for expansion for up to four transmitters Small and compactly packaged to allow for full range of motion by the user Distance from the transmitter to the receiver will be at most 60 feet

End-Product and Deliverables: 

End-Product and Deliverables Wireless transmitter with four sensors Receiver connected through Serial/USB adapter to computer Sensor watcher written to communicate between TinyOS and Isadora User Manual for client to recreate and use full functionality of system

Data Flow - Functionality: 

Data Flow - Functionality Hardware Flow Diagram Flex Sensor, Analog Voltage Signal MDA300 Analog Input Board MICAz Transmitter Mote MICAz Receiver Mote MIB510 Serial Interface 2.4 GHz Wireless Transmission HARDWARE CONNECTED TO COMPUTER HARDWARE WORN BY DANCER

Previous Research Project: 

Previous Research Project VrJuggler was previously used on a research project to manipulate input from sensors and display 3D video on projectors. Certain aspects of previous research were taken into consideration Flex sensors Crossbow Technology Inc.

Present Accomplishments: 

Present Accomplishments Transmitter/receiver tested for range and use with sensors, being used in theater Hardware programmed through nesC for correspondence between transmitter and computer Packaging purchased, installed Able to get sensor readings into Isadora

Future Tasks: 

Future Tasks Complete User Manual Purchase Crossbow system for client’s possession

Hardware Design - Research: 

Hardware Design - Research Analog audio wireless transmission method Problems: Modulation/compression issues, capacity/channel issues, possible interference problems

Hardware Design - Research: 

Hardware Design - Research Industrial SCADA-type wireless transmission method Problems: Bulky packaging, insufficient channels/capacity, possible software (Isadora) compatibility issues

Hardware Design - XBow: 

Hardware Design - XBow Crossbow “Mote” Transmission Method Reasons for choice: Compact sizing, 2.4GHz band avoids interference and allows for multiple transmitters, TinyOS accompanying software suited for project goals Possible problems: TinyOS/Isadora communication, line of sight issues

Hardware Design - XBow: 

Hardware Design - XBow

Analog Output Circuit: 

Analog Output Circuit Crossbow motes accept analog inputs Flex sensor variable resistors used in a simple analog circuit to give 1.31 - 1.95 Volt output to mote board

Circuit Diagram - Four Sensor Output: 

Circuit Diagram - Four Sensor Output

MDA300 Sensor Board: 

MDA300 Sensor Board Accepts analog inputs via screw terminals Provides excitation voltages Converts analog to digital signal for wireless transmission

MICAz Transceivers: 

MICAz Transceivers Connects to sensor board and serial interface with 51-pin connector Transmits on 2.4 GHz Zigbee compliant band (good range, multiple channels, minimal interference problems) Powers sensor board, simple AA battery power supply

MIB510 Serial Interface: 

MIB510 Serial Interface Receives signal from transceiver and transmits to computer through TinyOS software Powered by external AC adapter Through TinyOS, programs/configures transceivers and sensor board

Hardware Software Integration: 

Hardware Software Integration TinyOS is a platform bridging the hardware serial input using nesC Provides Libraries for parsing sensor data Environment for the Programming the Wireless Motes. Program for receiving wireless data and putting on the serial port provide by Xbow

Wireless Transmitter Program: 

Wireless Transmitter Program Developed in nesC

Isadora Module Development: 

Isadora Module Development Provides overall design needed for working Isadora Plugin/Actor. Provides data structures and functions that are needed to implement.

General Flow of the Functions: 

General Flow of the Functions

Open Sound Control Solution: 

Open Sound Control Solution C backend solution

Isadora Actor Mockup: 

Isadora Actor Mockup Our Actor Mockup Actor Link Connection

Putting the Pieces TogetherImplementation: 

Putting the Pieces Together Implementation Integration between hardware and software using the research and documentation Implementing in theater with components on loan from the Department of Electrical Engineering

Testing Procedures and Results: 

Testing Procedures and Results Measuring flex sensor voltage 1.31 to 1.95 volts Distance between transmitter/receiver and line of sight issues Work up to 60 feet and without line of sight Reliability issues when in enclosed corners Packaging integrity Currently beta testing by client for use in a performance

Resources: 

Resources

Schedule: 

Schedule

Project Evaluation: 

Project Evaluation Research of Hardware and Software 100% Complete Using findings to implement system Testing 95% Complete Hardware works with software Implementation 98% Complete Software designed and loaded onto hardware, currently in use Documentation 95% Complete All formal reports complete User Manual under revision

Commercialization: 

Commercialization Theatrical companies readily share technological findings Share findings with those that shared information with us Consult Crossbow Technology before any package sales could be done Selling Price: $1,000 Market: High demand in performing arts in near future

Future Recommendations: 

Future Recommendations Switch batteries from AA to 3 V coin Custom design casing for compact yet safe design Connectors for between sensors and wires and stability of sensor joint Expand to six sensors per transmitter Expand to four transmitters Use different sensor input

Lessons LearnedSuccesses: 

Lessons Learned Successes Group teamwork and communication Group supervision by faculty Attainable trial hardware

Lessons LearnedSetbacks: 

Lessons Learned Setbacks Loss of team member Original circuit design not applicable Developing software before hardware was decided

Lessons LearnedExperience Gained: 

Lessons Learned Experience Gained nesC Programming RF communication Multitasking Client relations Communicating different expertise Delegating responsibilities

Lessons LearnedIf we did it all again…: 

Lessons Learned If we did it all again… Hardware team more involved with software development More research on software cross-platform development

Risk and Risk Management: 

Risk and Risk Management Procurement of Materials Hardware borrowed from department for testing and implementation before purchase Loss of team member Encountered and overcome by members gathering completed progress and taking on more tasks Involved other resources – HCI 572 Design and Evaluation of HCI Data Loss All documentation easily transferable between group members Physical Damage Kept components in tackle box for safe keeping

Closing Remarks: 

Closing Remarks Successfully provided means for dancers to control own environment Human computer interaction Leading other companies in theatrical technology Expansion is possible

Questions: 

Questions