vishik

Simplified Discovery Model: 

Simplified Discovery Model Claire Vishik Sterling Commerce

Introduction: 

Introduction The talk is about a general purpose broadly defined service registry based on automatic indexing of textual artifacts upon submission. Text indexing can be adapted for the environment. Categorization is done automatically, while review and evaluation may be done by auditors. Retrieval interfaces for end users and APIs for applications are part of the system. The talk describes rationale, operations, and potential business models for such a registry.

Agenda: 

Agenda Semantic interoperability Integration and automatic service discovery Minimalist approach to semantics and standards in general Service discovery Economic advantages System advantages & functionality Who may be able to run such a system: possible business models Conclusions

IPD—Interoperability, Discovery, Processing: 

IPD—Interoperability, Discovery, Processing Protocol Matching Pattern Matching Semantics Interoperability Processing Discovery

Shared Semantics -- Vision: 

Shared Semantics -- Vision Reusable apps Richer data Standard automated integration New architectures Adaptability Self healing applications Uncoupling data from applications Expert and reasoning systems Context awareness Universal policies Rapid development Intelligent systems Rules-based systems Reusable, modular content

IPD– what’s really important: 

IPD– what’s really important Protocol Matching Pattern Matching Semantics Interoperability Processing Discovery

Reality: 

Reality Implementation time Functional features Semantics-based Implementation time Pattern-matching Functional features Apps Apps

Define versus Find: 

Define versus Find Some artifacts can be easily standardized; they are not ambiguous Addresses Programming/protocol statements Measurements Other objects are much harder to standardize, but it is possible POs Structured documents But some objects are extremely hard or impossible to define, even if a standard classification system had been created Concepts Place in a classification (cf. call numbers and subject headings in libraries) Narrative

Examples: Address (Define Structure): 

Examples: Address (Define Structure) <address> <street>123 Pickle St.</street> <street>Apt. #12</street> <city>Sourville</city> <state>NX</state> <zip>99999-9999</zip> <country>USA</country> </address> <address> <street>Via Garibaldi, 23</street> <city>Sorrento</city> <postalCode>123 456</postalCode> <province>NA</province> <country>Italy</country> </address> <postalCode> <CodeEntityLevel1> What follows <city> and contains digits is a PostalCode, etc.

Examples: PO (Define Mapping): 

Examples: PO (Define Mapping) <cat:CatalogueItemIdentification> <!-- using catalog to define industry identification codes   -->   <cat:ID>N000001</cat:ID>   </cat:CatalogueItemIdentification> <cat:ReferencedCatalogue>   <cat:CatalogueID>AviationAuthority</cat:CatalogueID>   </cat:ReferencedCatalogue> <cat:BasePrice>   <cat:PriceAmount currencyID="USD">100</cat:PriceAmount>   </cat:BasePrice>   </cat:Item> <cat:DeliveryRequirement>   <cat:ID /> <cat:DeliverToAddress>   <cat:ID>01</cat:ID>   </cat:DeliverToAddress>   <shipTo country="US"> <name>Alice Smith</name> <street>123 Maple Street</street> <city>Cambridge</city> <state>MA</state> <zip>12345</zip> </shipTo> <billTo country="US"> <name>John Bob</name> <street>242 Main Street</street> <city>Beverly Hills</city> <state>CA</state> <zip>90210</zip> </billTo> <items> <item partNum="242-NO"> <productName>Nosferatu - Special Edition (1929)</productName> <quantity>5</quantity> <USPrice>19.99</USPrice> </item>

Example: Message (Define…?): 

Example: Message (Define…?) <SOAP-ENV:Envelope xlmns:SOAP-ENV="http://schema.xmlsoap.org/soap/envelope/">     <SOAP-ENV:Body>         <SOAP-ENV:Fault>                 <SOAP-ENV:faultcode>SOAP-ENV:Client</SOAP-ENV:faultcode>                 <SOAP-ENV:faultstring>Invalid input</SOAP-ENV:faultstring>                 <SOAP-ENV:Detail>                             <INVALID_INPUT:invalidInputDetails xmlns:INVALID_INPUT="http://myfaultcodes.com/faults"/>                             <Message> Invalid category number used </Message>                             <ComputerPart>                                 <Category_Number> 124-342-343</Category_Number>                                 <Description>Keyboard</Description>                                 <Maker>    Dell    </Maker>                                 <Price>    50.32</Price>                              </ComputerPart>                             </INVALID_INPUT:invalidInputDetails>                 </SOAP-ENV:Detail>       </SOAP-ENV:Fault>     </SOAP-ENV:Body> </SOAP-ENV:Envelope <soap:Envelope> <soap:Body> <soap:Fault> <faultcode>SOAP-ENV:Server</faultcode> <faultstring>Exception from service object: maxResults must be 10 or less.</faultstring> <faultactor>/search/beta2</faultactor> <detail> This message is generated because the settings are at variance with the request</detail> ... </soap:Fault> </soap:Body> </soap:Envelope> How can a system understand if the two error messages are connected or have no mutual dependencies?

Semantic Interoperability Has Remained Elusive: 

Semantic Interoperability Has Remained Elusive Each system is built for its own domain There are few simple methods to address semantics Mechanisms of “definition” and “understanding” are still little understood. Consequence: many attempts have been made to address semantic interoperability is a multi-domain, multi-enterprise environment, and they met with limited success.

Towards Semantic Interoperability : 

Towards Semantic Interoperability Towards Semantic Interoperability Organizational Consensus Ontology Taxonomy Use Cases Mapping Tools Syntactic Structural Custom integration Semantic Reasoning Controlled Vocabulary Taxonomy Thesaurus Conceptual Model Ontology XML XCD RDF OWL Semantic Web

Interoperability and Integration: 

Interoperability and Integration In order for the systems to interoperate, they need to be integrated Integration may be easier if the systems in question are using similar standards It can be more difficult if they are using different standards or none But in any event, an effort will be necessary for integration. If standards are comprehensive, they become too complex, and they are not being implemented. If they are too simple and narrow, ad hoc variability occurs during implementation. Two systems using XML or SAML or even a narrower semantic standard (T1M1) are not interoperable out of the box.

Asking the Right Questions: 

Asking the Right Questions Example: a project on grammars for a narrow domain for an agent that would analyze a new story about a fire and then determine if “the fire was problematic.” Complex grammars, extensive processing power needed. Can you answer such a question? Simpler approach: While processing documents about fire incidents, look for certain words Tag stories if these words are found. Optionally, look for combinations of words Additionally tag for “strength.” Is the simple approach precise? Not completely Is it functional? Yes.

Minimalist Approach – Simplistic Reasoning: 

Minimalist Approach – Simplistic Reasoning Minimalists and structuralists Minimizing requirements supports interoperability Versus Maximizing coverage protects uniformity and, therefore, interoperability Asking the right questions is difficult Providing broader answers is easier Example: You believe that when people search for a name in an enterprise context, they are more interested in a directory record than in documents containing the name. Two ways to implement it: Check against a dictionary, and if it is not found there, assume it is a name, search LDAP, display results. Look in LDAP, and if it is found there, assume it is a name and display results.

Economics: Exchange Market: 

Economics: Exchange Market Barter economies suffer from double coincidence of wants – the need for both agents in a transaction to find objects offered in exchange acceptable. Consequence – increased search efforts (m2 for m agents). Needs Has Needs Has

Analogy in Discovery: 

Analogy in Discovery Catalog A offers flowers for remote delivery. Web site B offers information about care for the same types of flowers. The information is complementary, but not compatible If a buyer from Catalog A needs information on how to care for flowers purchased, he/she will need to discover and access in the information from Site B separately from the first transaction (purchase).

Asymmetric Information: 

Asymmetric Information Search efforts are not the only inhibitor of barter markets. Asymmetric information about products offered on exchange can be equally difficult to overcome: One needs to understand and be able to translate the value of various products. One needs to discern the value of different products. Discovery example: You may be able to select the best deal for the purchase of flowers, but how do you know that the flower care information from Web site B is correct? How do you determine if paying for such information may be a significant positive investment in your flowers?

Economics: Monetary Market: 

Economics: Monetary Market Money is a universal equivalent, and its value is recognized Monetary economies decrease search efforts The agent only needs to find the products he/she needs, not a favorable exchange situation, since money is always accepted by the other party.

Discovery Example: 

Discovery Example Each domain has (or may have) its “currency” – translation maps, controlled vocabularies, data models. These artifacts require significant effort to build, maintain, and implement Universal “currency exchange” can be implemented by mapping these diverse domains to each other. An even greater effort of creation and maintenance. Requires significant resources An earlier effort to carry out inter-disciplinary translation of terms at UIUC required pre-processing with super-computers.

Economics: Intermediation: 

Economics: Intermediation Intermediaries are positioned between producers and users, supporting economies of scale and scope Shopping malls (between individual stores and end custmers0 Web portals (between producers and end users or information) Credit card networks (between issuing banks and end users) Roles of Intermediaries: Trusted third parties Aggregators “Eliminators” of friction in markets Promoters of efficiency (e.g. decreased search efforts).

Digital Intermediary: 

Digital Intermediary Processing Tools Trusted repository Trusted Repository Input/Normalization/Aggregation

Discovery Intermediation: 

Discovery Intermediation Processing Submission Retrieval/Discovery

Discovery Intermediation Activities: 

Discovery Intermediation Activities Take the responsibility for distribution off producers [Collect, normalize, prepare for presentation and discovery] Take the effort of discovery off users [Create interfaces, monitor queries, adapt for usage] Mitigate the risks and perceptions of risks [Screen users and producers, usage and submissions, protect users and producers]

Service Discovery & Specialized “Services” Registries: 

Service Discovery & Specialized “Services” Registries Service discovery was the main rationale for building such registries But the current implementations focus on governance and management Some researchers argue that discovery is at variance with existing business models, premature, or not needed. Existing approaches to introduce service discovery are based on including narratives (descriptions) and/or extending registry models with additional attributes. This approach increases deployment and processing efforts and may not be acceptable for general purpose registries. Creative methods to support discovery have been successfully used: E.g., n-gram based indices to quickly process RosettaNet PIPs to discover relevant business processes.

Some UDDI Components: 

Some UDDI Components Some Components businessEntity --description of a business businessService-- a container to group Web services by the commonality of processes bindingTemplate --data with additional support for transactions tModel--metadata “store” for the specifications Some Interfaces Publication API contains save and delete calls associated with the data structures of the UDDI. Inquiry API: FIND_XX provides broad access to registration data; GET_XX allows the requestor to retrieve components when the keys to specific data are already known.

ebXML Registry: Components & Interfaces: 

ebXML Registry: Components & Interfaces Registry Information Model contains the following components: User, Organization, PostalAddress, AuditableEvent, ClassificationNode, Package, External Link, RegistryEntry, and Slot. ebXML registry stores metadata about items that reside in the repository. Items in the repository are created, updated, or removed through requests made to the registry. The repository can house various objects (Documents, DTDs, schemas). Current Registry Service Specification is complex and describes interfaces that allow other applications to use the registry. These interfaces include: RegistryService Interface is used to communicate with the ObjectManager and ObjectQueryManager interfaces. RegistryClient Interface is used to communicate with ObjectManager and ObjectQueryManage interfaces. ObjectManager Interface contains specifications allowing the trading partners to submit schemas, business process documents, etc. in registry/repository; to create, update and delete objects. ObjectQueryManager Interface supports query and retrieval of objects.

Minimalist Approach to General Purpose Registry: 

Minimalist Approach to General Purpose Registry Simplify organization and deployment By automation where possible Reducing process steps Sacrificing precision for coverage Simplify submission By minimizing requirements Taking over repetitive tasks Undertaking verification Simplify discovery By eliminating complex rules By eliminating non-essential parameters

General Purpose Registry --Functions: 

General Purpose Registry --Functions Broad range of topics Minimal number of required fields for manual submission Automated updates for trusted providers Automatic data collection from trusted locations, with no metadata requirements Text indexing “Artifacts” indexing Automatic categorization General repository area for items that are not easily categorizable. Review process Access control Statistics and audit Published interfaces

Operations: 

Operations Submitters Producers Reviewers, Approvers Analysis Payment Application Integration Subscribers

Example Process: Submission to Display: 

Example Process: Submission to Display Submission High Demand? Threshold: some measure of frequency of terms and incidence of other parameters (e.g. URIs, references, etc.)

Example 2: Retrieval & Discovery: 

Example 2: Retrieval & Discovery Query Found? Topic, Domain: predefined categories Used to partition repository for discovery. Found? Yes No Found? Found? Yes Yes No No

Trusted Discovery Intermediaries – possible choices: 

Trusted Discovery Intermediaries – possible choices Several organizations can fulfill responsibilities for intermediation in service discovery: Telephone companies Academic institutions Commercial operators Combination of commercial operators and professional communities/communities of interest

Summary: 

Summary Discovery that was the primary reason for designing registry specifications, took second place, ceding the first to governance and management. If we assume that a general purpose service discovery mechanism is needed, a simple model for a registry is required to encourage submissions and improve automation. A potential solution can be briefly described as follows: Text indexing (or similar fully automatable mechanism for non-textual artifacts) Minimal or zero metadata Simple submission process Trusted intermediary responsible for operations, including technology and governance.

Thank You: 

Thank You Questions?