AAAS

Slide1: 

Global Discovery: Turning Vision into Reality Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC Symposium: Global Discovery on the Internet: A Grand Challenge AAAS Annual Meeting 16-20 February 2006 St. Louis, MO

Speeding up the Diffusion of New Scientific Knowledge: 

Speeding up the Diffusion of New Scientific Knowledge What is the goal of Global Discovery? Greatly increase the contact rate between distant communities – through a virtual aggregation or federation of diverse deep web databases. How will we achieve it? Through multiple simultaneous deep web searches with integrated ranking of results. Global Discovery: 2

Slide3: 

Mathematician’s Scientific Discovery Biology Researcher’s Scientific Discovery Physics Scientific Discovery Math Community Biology Community Physics Community Biology Databases: Research Papers Correspondence Conferences Physics Databases: Research Papers Correspondence Conferences Knowledge Diffusion in Action 3

Challenges in Working with Thousands of Data Sources: 

Challenges in Working with Thousands of Data Sources Locate Reliable Sources Categorize Sources by Content Configure Sources for Searching Maintain Sources 4

Challenges in Searching Thousands of Sources: 

Challenges in Searching Thousands of Sources Automatically Select Sources to Search Perform Many Searches in Parallel Analyze and Organize Results Relevance Rank Cluster/ Visualize 5

The State-of-the-art Federated Search Engine Behind Science.gov: 

The State-of-the-art Federated Search Engine Behind Science.gov Scalable, grid-computing based federated search engine Sophisticated Search Conductor Multi-tier relevance ranking Framework accepts integration of advanced linguistic, analyses, and visualization modules Sponsored by DOE and Science.gov Alliance ResearchAssistant 6

Slide7: 

Grid Computing: Distributing the Workload 7

Slide8: 

Search Conductor 8

Multi-tier Relevance Ranking: 

Multi-tier Relevance Ranking QuickRank – Ranks results based on occurrence of search terms in title and snippet MetaRank – Ranks results utilizing custom algorithms applied to metadata DeepRank – Downloads and indexes full-text documents 9

Source Selection Optimizer: 

Source Selection Optimizer 10

Summary: 

Summary It is unknown at this time how many data sources would be searched through a comprehensive Global Discovery portal. Although there are significant challenges, DOE, and the Science.gov alliance with DWT as a partner are well on the road to turning the vision of global discovery into reality 11

Slide12: 

Turning Vision into Reality Abe Lederman 122 Longview Drive Los Alamos, NM 87544 abe@deepwebtech.com www.deepwebtech.com 12