CALI final

Your Life as a Movie: 

Your Life as a Movie Sailesh Chutani Director, Worldwide University Relations Microsoft Research

What if..: 

What if.. Everything you ever say, hear, observe, do, and sense is recorded Those recordings are stored permanently and can be accessed easily You could analyze or modify copies of these recordings and share them

Some Questions: 

Some Questions Is this scenario technically feasible? What would such a future look like? What new opportunities and risks will we be confronted with? Elements of such a future are already here but just not uniformly distributed (to paraphrase William Gibson)

Microsoft Research: 

Microsoft Research Founded in 1991 Staff of over 700 in over 55 areas Internationally recognized research teams 200+ WW university relationships Research lab locations : Redmond, Washington San Francisco, California Cambridge, United Kingdom Beijing, People’s Republic of China Mountain View, California

A University CS Department inside Microsoft: 

A University CS Department inside Microsoft University organizational model Flat structure, critical mass groups Open research environment Aggressive publication of results in peer-reviewed literature Frequent visitors, daily seminars Strong ties to University Research Nearly 15% of basic research budget directly invested in Universities Lab grants, research grants, fellowships, etc. Hundreds of interns and visitors

MSR Mission: 

MSR Mission Expand the state of the art in each of the areas in which we do research Rapidly transfer innovative technologies into Microsoft products Ensure that Microsoft products have a future

Expanding the State of the Art: 

Expanding the State of the Art Thousands of peer-reviewed publications 13% of papers at 2001 CHI (UI) 20% of papers at 1996 SIGGRAPH (graphics) 25% of papers at 1999 PLDI (prog languages) 30% of papers at 2001 PLDI More papers at SIGGRAPH 2003, SIGMOD 2004 , SIGIR 2004 and other key conferences than any other organization Community leadership Professional societies Journals Conferences

Segmenting the Problem: 

Segmenting the Problem Information Capture Storage Query and Retrieval Analysis and Processing

SenseCamA Digital Memory for Everyone: 

SenseCam A Digital Memory for Everyone Lyndsay Williams & Ken Wood @ MSRC and the MyLifeBits team @ BARC

SenseCam: 

SenseCam Essentially a “Black Box” data and image recorder for the human body Current wearable prototype can be worn for a day and captures up to 2000 images in that time Image capture triggered by sensors, e.g. motion, light, temperature, people in field of view, ... Sensor data is also recorded for later presentation, analysis, and correlation

Slide11: 

Captures everything regardless of exact camera orientation Ultra Wide-Angle Lens

Image Stabilisation : 

Image Stabilisation Accelerometer measures motion of the device When any sensor indicates image capture, we wait a few tens of milliseconds to capture image when movement is less than 10 degrees/second if possible Greatly reduces the number of blurred images Before After

A Short Walk Through Cambridge: 

A Short Walk Through Cambridge

How does SenseCam work?: 

How does SenseCam work? Accelerometers for motion detection Passive Infrared (PIR) sensors triggered by people or other living beings in front of camera Digital sensors for light level and temperature Images captured automatically on sensor events caused by, e.g. turning around, sitting down, standing up, running, falling over, ... moving between locations (rooms, car, bus, indoor/outdoor, etc.) meeting a friend in the street Simple hand gesture allows intentional image capture Accelerometers also allow image stabilisation

SenseCam Applications: 

SenseCam Applications Personal memory enhancement Where did I leave my umbrella? What was the name of that wine we had last week? What was written on the whiteboard? Tourism Automatic photo journals or blogs of specific trips, e.g. to the Edinburgh Festival Clinical support for the memory-impaired We are working with a Cambridge neuropsychologist on a trial for Alzheimer’s patients

Future: 

Future Now building next-gen SenseCams smaller, better power management, audio trigger, wireless comms

Storage Needs: 

Storage Needs Number of seconds in 80 years {60*60*24*365*80=2,522,880,000} Data recording rate for MPEG-4 is 4Mb/s Storage required for full motion video of 80 years of someone’s life is 10k tera bits or 10 peta bits (10^15) Cost of such a storage in today’s terms is approximately $10m Storage capacity is doubling every year, in 10 years 10 peta bits may cost <$10k

Storage Trends: 

Storage Trends Source: Ed Grochowski, IBM Research Almaden

Moore’s Law: 

Moore’s Law “A threshold event will take place early in the 21st century: the emergence of machines more intelligent than their creators. By 2019, a $1,000 computer will match the processing power of the human brain - about 20 million billion calculations per second. Organizing these resources - the “software” of intelligence - will take us to 2029, by which time your average personal computer will be equivalent to 1,000 human brains.” -- Ray Kurzweil

Human Scale Storage: 

Human Scale Storage Terabyte == $1600 today $400 in 2-3 years Terabyte can hold A lifetime of all your conversations A year of what you can see

Querying: 

Querying When did I meet the Dalai Lama? How long did I live in Lausanne? How many people did I encounter in my life with whom I spend more than 5 minutes in conversation? Where was I on my wife’s birthday during the last 10 years? What were the scariest moments in my mountaineering trips?

Stuff I’ve Seen (SIS): 

Stuff I’ve Seen (SIS) Support information re-use With SIS: Unified index Fast, flexible search Immediate update Rich UI possibilities for your content Today: Many locations Many interfaces

SIS Usage Observations: 

SIS Usage Observations Very short queries (1.6 words), quick filtering/sorting Few searches for “best matching” item Know and use other attributes Age of items opened 50% more than one month old, some up to 8 years Importance of people 25% of queries involve people Importance of time Date by far the most common sort attribute “Useful” date varies by source

Contextualizing Search: 

Contextualizing Search Search is not the end goal … Goal is information management in the context of ongoing work activities Search always available Search from within apps Show results in context of apps

MemoryLens: 

MemoryLens Harness models of human memory to assist with navigating large stores of personal content

Slide27: 

MemoryLens Search UI

Modeling the interface after the way people think & feel: 

Modeling the interface after the way people think & feel Cognition Memory associations vs. facts/details Deep History objects recede vs. disappear Dynamic organization multiple simultaneous categories vs. fixed Personality & Emotion Know what people care about and use to help organize Watch users habits & behavior to customize experience Context Many people, places and events

Slide29: 

MemoryLens LifeBrowser Adjustable threshold for memorability

Walkthroughs: 

Walkthroughs Re-live the past, explore the events as they happened at your own pace Analogous to “action replay” but at a large scale DEMO

WWMX (wwmx.org) Geography for Photography: 

WWMX (wwmx.org) Geography for Photography The World-Wide Media eXchange is a centralized index of the world’s image media tagged by location (and other standard metadata).

Why Location for Photos?: 

Why Location for Photos? Provides immediate context, hints at semantics E.g., photos taken at Disneyland, video shot in Hawaii… Repeatedly appears in user studies regarding organization Family trips strongly associated with location Event = time + location Subject as person doing something somewhere Universal index (like date/time) Independent of ontology/taxonomy, language, culture, etc. Familiar visual browsing UI via maps Scales well Technology to acquire location becoming common GPS, e911 services, radio-tower-based, 802.11-based…

Applications: 

Applications Real estate tours Travel and tourism Shared travelogues Driving directions with photos of landmarks Centralized media depository for community events Etc.

Creating Art: 

Creating Art Recordings can serve as the raw material for creating art DEMO

Meta Questions: 

Meta Questions Who would regulate the recording of this information? Who will own this information? Who would be able to access it? Should copies and modifications be allowed? What would be the legitimate uses of this information?

Meta Questions: 

Meta Questions What would it mean for us as human beings? Will human behavior change in light of the “permanent record”? How will we substitute for “forgetting” if memory becomes perfect? Would we become more empathetic if we can literally see other points of view? Would resolving conflicts be just a matter of comparing recordings? Would we create a class of people who live vicariously? Would life be safer? Is video/experience editing the next big growth industry?

Clues for Answers: 

Clues for Answers Current usage in forms of video surveillance, credit histories, public records policy on picture phones, popularity of reality TV use of web search to get background Exploration in Art, movies that have touched the themes Being John Malkovich, After Life, Slacker

Conclusions: 

Conclusions Scientists and engineers create the intellectual and physical building blocks; Artists imagine the possibilities, but It is the interplay of Society and Culture that shape the actual usage and norms