naaman

Slide1: 

Adventures in Space and Time: Mor Naaman Digital Library Project, Database Group Stanford University Leveraging Geo-Referenced Digital Photographs

Geo-referenced photos: 

Geo-referenced photos

Geo-photography Technology: 

Geo-photography Technology + 1) 2)

Outline: 

Outline Personal Enhancing value of collection “Social” Enable sharing to create more value Personal Enhancing value of collection

Enhancing Personal Collections: 

Enhancing Personal Collections Browse/search for photos by location Detect events Add metadata from other sources

Browsing by Location/Time: 

Browsing by Location/Time Use a map to browse for photos wwmx.org from MSR:

Map issues: 

Map issues Lots of screen space Sparse Not intuitive for some Limited interaction?

Automatically-Created Hierarchies: 

Automatically-Created Hierarchies Location hierarchy Event hierarchy Our algorithm: Creates them simultaneously Such that they inform each other Assigns geo-names using gazetteers and Google

Result of processing: 

Result of processing

Challenges: 

Challenges Locations should be intuitive Events are tricky 3-days trip to NYC The kid’s soccer game, followed by a birthday party Good names are important.

Location Hierarchy: 

Location Hierarchy Photoshop Album (about 4 man-hours) Our PhotoCompas application (about 0 man-seconds)

Location Hierarchy (US): 

Location Hierarchy (US) San Francisco, Berkeley, Sonoma, CA -Stanford, Mountain View, Monterey, CA Monterey (58 miles S of San Jose)  Mountain View (4 miles NW of San Jose)  Stanford; Palo Alto (14 miles W of San Jose) -Colorado (219 miles W of Denver) -Long Beach (35 miles S of Los Angeles, CA) -Philadelphia, PA -Seattle, WA -Sequoia N.P. (153 miles E of Fresno, CA) -South lake Tahoe; Bear Valley, CA -Yosemite N.P.; Yosemite Valley, CA 

Event “Hierarchy”: 

Event “Hierarchy” PhotoCompas (about 0 man-seconds): 4-5 Oct 2003: Sequoia NP 12 Oct 2003: Stanford,CA 27 Nov 2003: Oakland,CA 30 Nov 2003: San Francisco,CA 5 Dec 2003: Stanford,CA 19 Dec 2003: Stanford,CA 24 Dec 2003: Palo Alto,CA … Photoshop Album (about 4 man-hours)

Experiment 1: Processing: 

Experiment 1: Processing Tested on 3 real-world geo-referenced photo collections Location hierarchy accepted by owners High accuracy (precision/recall) for events Names intuitive and accurate

Experiment 2: Map vs. No Map: 

Experiment 2: Map vs. No Map Paid 13 participants to “reference” their photos Loaded to WWMX and to PhotoCompas Most liked the map better, but… Performed the same for search/browse tasks Event notion helps overcome location handicap P.S. Some didn’t touch the map, yet used our location hierarchy. P.S.2 This was on a BIG screen!

Enhancing Personal Collections: 

Enhancing Personal Collections Browse/search for photos by location Detect events Add metadata from other sources

Enhancing Personal Collections: 

Enhancing Personal Collections Sources: web services, camera sensors Local time… easy but important! Day/night status – sunset photos, anyone? Weather – it was freezing!

Demo!: 

Demo!

Outline: 

Outline Personal Enhancing value of collection “Social” Enable sharing to create more value

The Motivation: 

The Motivation

Finally, can share!: 

Finally, can share! Location data is key for sharing

The Vision: few label, many benefit: 

The Vision: few label, many benefit Shared website / application / P2P system

Simple Example: 

Simple Example Hector takes a picture in front of Gates building Hector labels the photo. Label and coordinates recorded on LOCALE server “Gates building”

Simple Example (Cont.): 

Simple Example (Cont.) Mor takes a picture from the same location Too lazy to label. Mor’s computer contacts LOCALE server, gets Hector’s label “Gates building”

Problems: 

Problems May label the photo differently “A typical Stanford Scene” “Two Mating Squirrels”

Problems: 

Problems May be using a different name “CS building”

Problems: 

Problems May have pointed the camera in a different direction “Packard Building”

Problems: 

Problems May be just plain wrong “Packard Building”

LOCALE:Location to Label: 

Users submit labels and locations of photos Services for users: Search (their own photos) Label their photos Using IR techniques LOCALE: Location to Label

LOCALE Submissions: 

LOCALE Server User machine User labels LOCALE Submissions (label, location) pairs (label1,loc1) (label2,loc2) … No need for the photos!

LOCALE Labeling: 

LOCALE Labeling LOCALE Server User machine Lat,Lon Proposed Labels (label1,loc1) (label2,loc2) …

Label: 

Lat1,Lon1 “Mom and Dad”, Lat1, Lon1 “Empire State Building”, Lat2, Lon2 Lat2,Lon2 Label

What will it Look Like?: 

What will it Look Like?

Stanford Campus Experiment: 

Stanford Campus Experiment Gave Cameras and GPSs to visitors After tour, participants labeled their photos “Enjoying the beautious weather” “Hoover Tower & Fountain” “Crowds of Stanford students” “Fountain in Action” “Non-Denominational Church”

The Numbers: 

The Numbers 52 participants, 1100 photos (~21 photos each) 37 participants submitted 760 labels “Enjoying the beautious weather” “Hoover Tower & Fountain” “Crowds of Stanford students” “Fountain in Action” “Non-Denominational Church”

Example Data – “????” labels: 

Example Data – “????” labels Example Data – “Fountain” labels

Example of Suggested Labels: 

Example of Suggested Labels

A Note About Privacy: 

A Note About Privacy Notice previous page had no “personal” data “General” terms float, others sink. “IDF” can help identify stop words Anonymous submissions

Search: 

Search Lat1,Lon1 “Mom and Dad”, Lat1, Lon1 Lat2,Lon2 “Empire State Building”, Lat2, Lon2

Search Demo: 

Search Demo

Example Of Search: 

Example Of Search Ranked results Find my “Hoover tower” photos

Future Work: 

Future Work Add time to detect “events”

Outline: 

Outline Personal Enhancing value of collection “Social” Enable sharing to create more value

Thank You!: 

Thank You! Advised: Andreas Paepcke, Hector Garcia-Molina Did the work: Susumu Harada, Yee Jiun Song, QianYing Wang Gets the credit: mor@cs.stanford.edu http://www-db.stanford.edu/~mor/