nsgic partnerships

USGS-Microsoft TerraServer Cooperative Research and Development Agreement: 

USGS-Microsoft TerraServer Cooperative Research and Development Agreement

Today’s Agenda: 

Today’s Agenda Background on USGS-Microsoft CRADA Who is TerraServer? What is TerraServer? Why did we do it? How we did it? TerraServer today What’s next?

TerraServer-USA: 

TerraServer-USA A Geo-Spatial Raster Store Built on SQL Server Bay Area Research Center Tom Barclay & Jim Gray

TerraServer History: 

TerraServer History MS Research project commissioned by MS SQL Server Team (April 1996) Goal: Test & Demo scalability of SQL 7.0 Image Server chosen as test/demo vehicle Lot of cheap bytes, no translation, interesting Partners between 1996 and 2004: UCSB, USGS, SPIN-2, DEC, Compaq, HP, ADIC, Veritas, Legato, Encarta, Home Advisor, MapPoint, .NET Framework

What is TerraServer?: 

What is TerraServer? A remote sensing image store Images are “shredded” into small, uniform sized tiles Multiple input images are mosaicked into a single “scene” An HTML based web application Navigate by point-n-click, gazetteer or coordinates Pan-n-zoom by clicking on simple navigation buttons (No fancy java, activex, flash controls) An XML based Web Service Meta-data and tiles accessible via W3C web service API An OpenGIS compliant Web Map Server Tiles merged into single “map” image Re-projection from UTM to “geographic” supported

Design Objectives: 

Design Objectives Public: Access to remote sensing data with no GIS expertise required Ubiquitous: No special hw/sw required by client Delivery: All OnLine/Internet Based, no tape or CD distribution Simple: Designed to be used by a “6th grade geography student” Scale-up: creating multi-TB PC Server Availability: Test large MS Cluster system in a 24x7 situation Lights out: all operations & maintenance occurs remotely Easy: Minimal ops and dev staff Programmable: Meta & Imagery data accessible as a “web service” User/App Goals Technology Goals

More Concepts: 

More Concepts Three data classifications: Imagery: Stores Tiles and Tile metadata Partitioned across one or more databases / servers Redundant (multiple databases/server with same content) Meta-Data: Stores Tile meta-data, Source Meta-data, Gazetteer NOT partitioned Redundant (multiple databases/server) Administration: Stores Tile meta-data, Source Meta-data, Gazetter, Tile imagery temporarily NOT partitioned, NOT Redundant Contains “links” to all Imagery and Meta-data servers

User Interface: 

User Interface User is presented with “seamless” mosaic of low-to-high resolution imagery User navigation: Pan by ½ screen in any direction Zoom in/out by 2x Switch to overlapping imagery Search by: Place Name Address Longitude / Latitude

http://terraserver-usa.com: 

http://terraserver-usa.com

USGS Image Data: 

USGS Image Data Digital OrthoQuads 18 TB, 305,000 files uncompressed Digitized aerial imagery 98% coverage conterminous US 1 meter resolution < 15 years old Digital Raster Graphics 1 TB compressed TIFF, 65,000 files Scanned topo maps 100% U.S. coverage 1:24,000, 1:100,000 and 1:250,000 scale maps Maps vary in age Urban Area 1 foot resolution Natural Color 133 major U.S. cities 45 (4 TB) received as of September 2004 2001 or later Produced by NIMA for Homeland Security

Image Coverage: 

Image Coverage 100% U.S., Topo Maps [DRG] (light green) 2m to 1024m resolution 98% 48 Conterminous States, Ortho B&W [DOQ], (medium green) 1m to 1024m 45 U.S. Urban Areas, Ortho Natural Color (dark green) .25m to 1024m resolution Urban Area Coverage Seattle, Portland, Stockton, Modesto, Fresno, Sacramento, Chicago, Orlando, Atlanta, Amarillo, Houston, Lubbock, Springfield, Birmingham, Dallas, Albuquerque, Oklahoma City, El Paso, Lincoln, Lexington, Tampa, Washington DC, Mobile, Ft Wayne, Colorado Springs, Baton Rouge, San Francisco, Los Angeles, Denver, Knoxville, Montgomery, Shreveport, Tucson, Detroit

NASA Data Sets: 

NASA Data Sets Blue Marble 1km / pixel resolution natural color geographic projection Full Earth LandSat-5 30m / pixel resolution psuedo-color Geographic projection -80° to +80° latitude Shuttle Relief Topology Mission (SRTM) 3 arc second (90m), world wide coverage 1 arc second (30m), U.S. coverage 16-bit values Compressed BIL format

TerraServer Users: 

TerraServer Users Hobbyists: Hikers, Campers, Hunters, Kayakers, etc. “Amateur Anthropologists” – one user traced the Santa Fe “ruts” (trail) Engineers: Civil Engineers City/county planners Web Map Server / TerraService Applications: NASA World Wind (3D Viewer) USCG Container Tracking USDA NRCS USGS National Map USC ISI – research on conflation… UC Berkeley Map Server Variety of small “shareware” apps – Pocket Terra, Golf-Terra, etc.

TerraServer Fast Facts: 

TerraServer Fast Facts Daily Usage: 75k – 120k visitors 1.5million to 2.2 million page views 10 to 20 million “tiles” 20 to 40 mega-bits per second peak network bandwidth Usage rising (new apps like NASA World Wind) Database Statistics: 5.3 TB of compressed (jpeg/gif) imagery 525 million image “tiles” 1 billion rows (Meta & Imagery)

Last 7 Months: 

Last 7 Months 50% increase over 2003! 100% increase In last 6 months! 300% increase over Last year

Average Daily Usage: 

Average Daily Usage

TerraServer Today: 

TerraServer Today TerraServer Today: 100k visitors per day, 25+ million hits: 1.5 to 2 million page views, 15-20 million “tiles” per day, 750k Web Maps, 1 million web service calls 5+ TB of compressed imagery [and growing] Programmable Web Service and OpenGIS Web Map Server used by USDA, EPA, NASA, USGS (National Map), USCG, NASA World Wind and others A component of MSN Local Search Transferred to MapPoint BU in April 2005 Imagery source MSN Virtual Earth Support will continue for SOAP/XML and OGC Web Map Server access to U.S. Public Domain imagery products (USGS DOQQ, DRG, UrbanArea; NASA Blue Marble, LandSat-5, SRTM)

Application Goals: 

Application Goals BIG — 1 TB of data including catalog, temporary space, etc. PUBLIC — available on the world wide web INTERESTING — to a wide audience ACCESSIBLE — using standard browsers (IE, Netscape) REAL — a LOB application (users can buy imagery) FREE — cannot require NDA or money to a user to access FAST — usable on low-speed (56kbps) and high speeds(T-1+) EASY — we do not want a large group to develop, deploy, or maintain the application 5 Available – Always, 24x7x52 99.99% of the time Programmable – .NET applications can integrate TerraServer data into their apps

NextGen Imagery Issues: 

NextGen Imagery Issues Service availability Seamless national coverage with distributed data from multiple partners Integration with licensed data provided by commercial sector Data redundancy Archive requirements and standards Access to historic data

2005 Image Schedule“ 10 Terabytes or bust”: 

2005 Image Schedule “ 10 Terabytes or bust” USGS: More Urban Area data (Boston, Newark, Las Vegas, Philadelphia, etc.) NASA: SRTM – 30m U.S. and 90m world DEM LandSat-7 – colorized 30m, world wide Blue Marble – low resolution, natural color world wide New Partnerships: Private/commercial vendors Other federal agencies, e.g. USDA NAIP State and local agencies, e.g. NY State GIS, Florida

Seeking New Partners: 

Seeking New Partners State partners, rich and poor Other federal agencies

TerraServer ArchitectureDistributed Deployment: 

TerraServer Architecture Distributed Deployment TerraServer-USA TerraServer-UK TerraServer-Europe TerraServer-India TerraServer-Japan

What’s in it for you?: 

What’s in it for you? Backup for your data during major event Free image web browser to your data by the general public which off loads demand Free OGC-compliant Web Map Services Agency accreditation as source of data Public domain data will remain in public domain

How are partnerships chosen?(Successful partnerships have had these properties): 

How are partnerships chosen? (Successful partnerships have had these properties) “Win-Win” for both parties Usually something new [and interesting] for both parties – an experiment or trial Too “something” – costly, risky, technically or administratively unknown – for either party to do alone Public domain data No monies trade hands

USGS Technical Assistance Agreement: 

USGS Technical Assistance Agreement Serve orthoimagery of National Map partners Be a compliant National Map raster node Be a backup to the primary USGS raster server Cost-free Web access Web server access to distributed data Archive capability

Partnership Discussions: 

Partnership Discussions USDA-FSA USFS Florida New York

Who to contact?: 

Who to contact? USGS State Liaisons, see http://geography.usgs.gov/www/partners/crreps.html George Lee, 650-329-4255, gylee@usgs.gov Tom Barclay, 425-707-1090, tbarclay@microsoft.com