CEOS FALL IDN

CEOS WGISS 24 - IDN Task Team: 

CEOS WGISS 24 - IDN Task Team Task leader: Lola Olsen, represented by Tyler Stevens

Agenda: 

Agenda 1. GCMD/IDN Science User Working Group 2. Review of Minutes and Action Items 3. Update on Population and Usage Statistics 4. Development Plans for MD9.8 - MD10 5. GEOSS Activities and other Collaborations 6. Node Reports 7. Evolution of Controlled Vocabularies 8. Re-emergence of the Interop and Communication 9. The CEOS IDN Newsletter

Briefing on the GCMD IDN Science User Working Group Meeting, June 19-20 2007 by Dr. Wyn Cudlip: 

Briefing on the GCMD IDN Science User Working Group Meeting, June 19-20 2007 by Dr. Wyn Cudlip

Key Topics from WGISS 23: 

Key Topics from WGISS 23 New IDN Features in MD 9.7 Evolution of the DIF Future Development Information System and Data Center (ISDC) for Geo-scientific Data User Working Group (UWG) Meeting IDN Contributions to Interoperability Handbook

WGISS Action Items: 

WGISS Action Items No. 22-13: When JAXA is ready, add a link to JAXA multilingual prototype to the GCMD page. No. 23-8: Arturo to report back to WGISS-All on the awareness of the GEO task teams of the new IDN portals that have been created for those GEO tasks, and clarify what reviews are currently underway for the new GEO portals. Status: 23 June 2007: Lola and Arturo distributed a memo describing the portals that are currently in place, and they have been register in GEO registry. CLOSED No. 23-10: Arturo (IDN Team) to assess the differences between the platform and instrument descriptions in the IDN and those contained in the CEOS database (CEOS Earth Observation Handbook). No. 23-22: The IDN team (Lola Olsen) to perform a detailed analysis of user query patterns and present a summary of usage patterns and lessons learned at WGISS-24.

Characteristics of IDN Usersthrough Population and Usage Statistics : 

Characteristics of IDN Users through Population and Usage Statistics

Most Popular Keyword Search: 

Most Popular Keyword Search December 2005 - October 2007

DIF Population by Earth Science Topic: 

DIF Population by Earth Science Topic

Total Searches: 

Total Searches

Most Popular Full-Text Searches : 

Most Popular Full-Text Searches December 2005 - October 2007

Searches by Ancillary Keywords: 

Searches by Ancillary Keywords Platforms 12% Instruments 14%

DIF Population Growth: 

DIF Population Growth Deletion and eventual replacement of Antarctic DIFs

DIF Population by IDN Node: 

DIF Population by IDN Node

New, Revised, and Deleted DIFs - 2007: 

New, Revised, and Deleted DIFs - 2007

Web Page Hits: 

Web Page Hits

Slide16: 

Users by Continent North America Europe Asia Australia South America Africa Other January 2007 - October 2007

Slide17: 

Users by European Country

How the Users Access the IDN: 

Google (50%) GCMD Direct Access (20%) Yahoo (6%) NASA (4%) Others (20%) wikipedia.org noaa.gov usgs.gov The IDN allows robots to crawl the site to help broaden user access. How the Users Access the IDN

Slide19: 

When Users Access the IDN Lets us know best time to schedule server maintenance. January 2007 - October 2007

Slide20: 

Most Popular Browsers and Platforms Important for cross-platform and browser support of IDN software. January 2007 - October 2007

Metadata Contributions January-September 2007: 

Metadata Contributions January-September 2007 From docBUILDER 45% 55 % Improved translation technology has increased contributions from sources using other “mandated” standards.

Summary of Usage Patterns: 

Summary of Usage Patterns Keyword Interface (Science Keywords, Ancillary Keywords) Accounts for 20% of website usage U.S. users high throughout search interfaces Full-Text Interface Accounts for 3% of website usage Interests are different depending on country Words searched in upper & lower case version Google/Cached Interface Accounts for 18% of website usage Top 10 countries searches are lower case

Summary of Usage Patterns continued: 

Summary of Usage Patterns continued Total New Visitors: 209,480 Repeat visitors: 26,733 Average visits per week: 7,048 Average pages viewed per visit: 4 Most Popular Browser: Internet Explorer 6.x Most Popular Platform: Windows XP Most Popular Screen Resolution: 1024 X 768 December 2005 - September 2007

Top 5 Keyword Searches by Country: 

Top 5 Keyword Searches by Country United States Atmosphere, Oceans, Climate Indicators, Land Surface, Agriculture Great Britain Climate Indicators, Oceans, Atmosphere, Land Surface, Agriculture Canada Land Surface, Climate Indicators, Oceans, Agriculture, Atmosphere Germany Atmosphere, Land Surface, Climate Indicators, Oceans, Agriculture Australia Climate Indicators, Oceans, Atmosphere, Land Surface, Agriculture Italy Land Surface |Erosion/Sedimentation, Atmosphere, Solid Earth, Oceans, Land Surface India Agriculture, Atmosphere, Oceans, Land Surface, Climate Indicators France Atmosphere, Oceans, Land Surface, Climate Indicators, Agriculture Spain Oceans, Climate Indicators, Atmosphere, Land Surface, Agriculture China Land Surface, Atmosphere, Oceans, Agriculture, Climate Indicators

Top 5 Free-Text Searches by Country: 

Top 5 Free-Text Searches by Country United States global warming, GES DISC, temperature, AIRS,OMI Canada marwhale, canada, DIF/Project:GO*, obis.marwhale, Woods Hole Oceanographic Institution Great Britain uk, global warming, climate change, UK, globec Australia Australia, heard island, global warming, tide gauge, heard island scott Germany china, germany, africa, argentina, europe Japan Russia, japan, Japan, alos, ndvi Italy mediterranean, italy, DIF/IDN_Node: AMD*, europe Spain amd/es, bermudez, spain, cadmium, Shetland China china, China, gauge, planetary albedo, cn-nadc India india, global warming, India, vegetation, XBT

Lessons Learned from User Stats: 

Lessons Learned from User Stats Concentrate resources on keyword interface because most consistent usage come from there. Consider adding spell-corrector for full-text search to give users more chances of finding data. Improve functionality of docBUILDER and metadata ingestion tools. Capitalize on Google cache and provide additional mechanisms for accessing metadata (metadata in KML). Use semantic web technology to offer the option to search on related words to their original query.

Development Plans forMD 9.8-MD 10: 

Development Plans for MD 9.8-MD 10

MD 9.8 - Migrate from Oracle to “Open Source” Software (MySQL): 

MD 9.8 - Migrate from Oracle to “Open Source” Software (MySQL) Staff evaluated alternative databases: PostgreSQL, Mckoi, MySQL, and XML databases. MySQL chosen as alternative to Oracle. Staff is now in the process of migrating to MySQL.

Oracle Vs. MySQL Database: 

Oracle: 4610 ms. MySQL: 1582 ms. (# of milliseconds to query 150 Atmospheric Keywords) Oracle Vs. MySQL Database

Benchmark Report: 

GCMD Oracle: 4610 ms. GCMD XML DB: 6361 ms. OS X MySQL: 1582 ms. OS X XML DB: 2978 ms. GCMD BDB (Cache): 2378 ms. OS X BDB (Cache): 1589 ms. (# of milliseconds to query 150 Atmospheric Keywords) Benchmark Report

MD 9.9 - Quality Upgrade: 

MD 9.9 - Quality Upgrade To improve user experience To enhance interoperability To reduce maintenance To streamline software testing process

MD 10 - Next Generation IDN: 

MD 10 - Next Generation IDN Empower data providers to manage their “own” metadata (load, query, share, etc.) through a distributed input capability. Represent the GCMD vocabulary as an ontology. Re-engineer server to support rising levels of data sets and increasing demands for services.

GEOSS Activities and Other Activities/Collaboration: 

GEOSS Activities and Other Activities/Collaboration

IDN Registered in GEOSS Registry Publication Portal: 

IDN Registered in GEOSS Registry Publication Portal

GEOSS Portals: 

GEOSS Portals

IDN Architecture Within GEOSS: 

IDN Architecture Within GEOSS Import metadata into GEOSS Portal catalogues Metadata content harvest from FGDC clearinghouse node IDN IDN Components in GEOSS registry

Slide37: 

IPY Data & Services Portals Find IPY DIFs in

Slide38: 

Customized DocBUILDER

New Portals Added Since May 2007: 

New Portals Added Since May 2007 NASA Data THREDDS OBIS-SEAMAP EBA-AMD SCAR Project GEOSS Wildfires GEOSS Wildfires Services GEOSS InSitu Instruments European Space Agency Essential Climate Variables

The “King” Request: 

The “King” Request Mapped IDN Keywords to Essential Climate Variables (ECV)

Slide41: 

Enhanced Platform & Instrument Descriptions

Slide42: 

Expanded database table structures for Instrument and Platform hierarchical keywords 10 Platform Categories 66 Platform Series/Entities 3 Instrument Categories 25 Instrument_Classes 29 Instrument_Types 22 Instrument_Subtypes

Requirements for a Platform/Instrument Search Interface : 

Requirements for a Platform/Instrument Search Interface Capitalize on existing platform and instrument ancillary information. CEOS Earth Observation Handbook NASA Science Reference Handbook IDN Descriptions Utilize new technology to create an intuitive interface. Provide capability to sort platform and instrument information by columns in sequential order. Enhance capability to refine search within interface. Directly link from instrument to platform descriptions and vice-versa, and link directly to associated metadata.

Slide44: 

Left Hand Navigation NASA Earth Observation Platforms Launch Date TERRA Polar Sun-Synchronous 1999-12-18 TERRA METADATA UARS ACRIM CLAES HALOE HRDII MLS PEM SAMS SOLSTICE SUSIM WINDII Polar Non-Sun-Synchronous 1991-09-12 UARS METADATA ASTER CERES MISR MODIS MOPITT Platform Name Orbit Type Associated Instruments Metadata MOCK UP

Slide45: 

Left Hand Navigation NASA Earth Observation Instruments MODIS AQUA TERRA 2002-06-25 MODIS METADATA IIR CALIPSO IIR METADATA Instrument Name Associated Platforms Metadata Start Date Passive Instrument Class Passive 2006-04-28 MOCK UP

docBUILDER Tool for Platforms - Demo: 

docBUILDER Tool for Platforms - Demo

Slide47: 

PLATFORMS COMPARATIVE TABLE IN RESPONSE TO TASK WGISS 23-10 Note: Description information provided in the document task were created by Olsen, L.M., G. Major, S. Ritz, T. Stevens, M. Meaux, A. Restrepo, 2007 . NASA/Global Change Master Directory (GCMD) Earth Science Keywords.

INSTRUMENTS COMPARATIVE TABLE IN RESPONSE TO TASK WGISS 23-10Note: Description information provided in the document task were created by Olsen, L.M., G. Major, S. Ritz, T. Stevens, M. Meaux, A. Restrepo, 2007 . NASA/Global Change Master Directory (GCMD) Earth Science Keywords. : 

INSTRUMENTS COMPARATIVE TABLE IN RESPONSE TO TASK WGISS 23-10 Note: Description information provided in the document task were created by Olsen, L.M., G. Major, S. Ritz, T. Stevens, M. Meaux, A. Restrepo, 2007 . NASA/Global Change Master Directory (GCMD) Earth Science Keywords.

Follow-up: 

Follow-up Evaluate the addition of fields within the IDN Instruments description. Swath width Accuracy Continued addition of Instruments and Platforms from CEOS Handbook. Online Demo to add Platforms and Instruments descriptions – previously viewed. Suggestions and questions from WGISS-24 participants.

Node Reports: 

Node Reports

Bernd Ritschel - GFZ ISDC: 

Bernd Ritschel - GFZ ISDC

AMD: 

AMD

Slide56: 

AMD Search Page Hits Over 10,000 Hits in 2005 ~ 14, 000 Hits in 2006 ~ 40 % increase in 2006 from 2005

Slide57: 

AMD Search Page Hits vs. Retrievals Over 5,500 Retrievals in 2005 Over 6,500 Retrievals in 2006 Over 10,000 Retrievals in 2007 (Jan - Jul.) Launching of IPY 2007-2008? (over 500 unique IPs) XXIX SCAR/JCADM Meeting

Slide58: 

Note: Stats shown only represents 55% of total portal stats from July 2006 – August 2007

Slide59: 

Hits by Country Average visitors per day 19 Total repeat visitors 1,235 Most frequent domain: Australia Visit site between 9pm-10pm Highest volume day of week: Tuesday

Evolution of Controlled Vocabularies: 

Evolution of Controlled Vocabularies

Slide61: 

Controlled Keywords Lists List is dynamic: keywords may be changed, deleted, or added. Changes are implemented according to the Keyword Rules http://gcmd.gsfc.nasa.gov/Resources/valids/rules.html In addition to Science Keywords, the GCMD maintains the following controlled keyword lists: Services Keywords, Data Resolution, Locations, Instruments, Wavelength Keyword, Platforms, Orbit Type, Launch Site, Projects, Data Centers, URL Content Types, Chronostratigraphic Units (geologic time), IDN Nodes

Slide62: 

Controlled Keywords Recent/Upcoming Changes Keyword hierarchy extended to 5-Levels to assist in refinement of the search. Recent Changes: Atmospheric Chemistry Biological Classification (New Topic) Ecological Dynamics (Revised) Terrestrial Hydrosphere (Renamed) Upcoming Changes: Atmosphere Human Dimensions Oceans Biosphere NEW

Slide63: 

Atmospheric Science Keyword Revision (February - October 2007) “The Long Journey to the GOLDMIND: Steps Towards Developing a Comprehensive Keyword Ontology” 2. Do the Research 3. Collaborate 4. Persevere The GOLDMIND 1. Set a Goal

Slide64: 

1. Set a Goal A Complete Revision of the IDN Atmospheric Science Keyword list in 2008 Science Keyword Revision “The Long Journey to the GOLDMIND”

Slide65: 

2. Do the Research Review DIFs in the GCMD AMS Glossary Online Online publications through “Google Scholar” Consult data providers and experts Review Text Books: Critchfield, “General Climatology” Holton, “An Introduction to Dynamic Meteorology” Wallace/Hobbs, “Atmospheric Science: An Introductory Survey” Science Keyword Revision “The Long Journey to the GOLDMIND”

Slide66: 

3. Collaborate Meet with project manager - Lola Olsen Consult Data Providers and Experts Science Keyword Revision “The Long Journey to the GOLDMIND”

Slide67: 

4. Persevere Be willing to accept small steps towards a larger goal. Be patient when a trail leads to a roadblock and cheerfully backtrack and try a new trail. Never give up! Science Keyword Revision “The Long Journey to the GOLDMIND”

Slide68: 

The Journey is almost over! The GOLDMIND is in sight. Science Keyword Revision “The Long Journey to the GOLDMIND” GCMD Ontology Language for Discovery of Metadata INformation and Data

Reemergence of the Interop and Communications: 

Reemergence of the Interop and Communications

Need for Communication: 

Need for Communication With increased usage, we must communicate: DIF/SERF modifications Keyword changes Software changes User Support/FAQ Technologies to help with Communication: Mailing Lists Forums Wikis Content Management Systems (CMS) RSS

CEOS IDN Standards Interoperability Forum (INTEROP): 

CEOS IDN Standards Interoperability Forum (INTEROP) Moving to ceos-idn-interop@lists.nasa.gov Continue to use GNU Mailman Continue as a moderated list Possible future split of list: ceos-idn-interop (for general questions/discussions) ceos-idn-keywords (for keyword announcements)

Communication Tools: Mailing Lists: 

Communication Tools: Mailing Lists

Communication Tools: RSS Feeds: 

Communication Tools: RSS Feeds The IDN RSS feed allows users to view the most current Data Set and Service descriptions added to the IDN. Once a data set or service description is added, it will show up in your RSS feed.

Communication Tools: Forums/Bulletin Boards: 

Communication Tools: Forums/Bulletin Boards

Communication Tools: Wikis: 

Communication Tools: Wikis

Communication Tools: Content Management Systems/Collaboration: 

Communication Tools: Content Management Systems/Collaboration

Communication Tools:Cyber Infrastructure/Community : 

Communication Tools: Cyber Infrastructure/Community

One Final Communication….: 

One Final Communication…. The IDN/GCMD Strategic Plan

Slide79: 

Thank You Tyler.B.Stevens@nasa.gov GIS/Services Coordinator Lola.M.Olsen@nasa.gov Task Leader Special Thanks to the GCMD staff for their assistance and contributions.