logging in or signing up Middleware Diarrhea and Other Ailments aSGuest25622 Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: (To copy code, click on the text box) Embed: URL: Thumbnail: WordPress Embed Customize Embed The presentation is successfully added In Your Favorites. Views: 31 Category: Entertainment License: All Rights Reserved Like it (0) Dislike it (0) Added: September 10, 2009 This Presentation is Public Favorites: 0 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript Middleware Diarrhea and Other Ailments : Middleware Diarrhea and Other Ailments Michael Stonebraker Adjunct Professor Massachusetts Institute of Technology (stonebraker@lcs.mit.edu) Slide 2: Outline Too much middleware XML ailments Web services ills Our professional sickness Slide 3: Client-Server Got Replaced by N-Tier Computing The Web Gizmos Scalability and management problems with client server Slide 4: Humility Lesson We all sold client-server hard during the 80’s and even into the 90’s Less than 10 years later it is the worst idea on the planet We should feel really dumb! Slide 5: N-Tier Computing Produced Lots of Middleware App servers EAI/messaging ETL Federators Workflow CMS Portals DBMS Slide 6: Middleware Diarrhea Average enterprise has one (or more) app servers one (or more) EAI packages one (or more) ETL packages one (or more) portal products one (or more) application packages and maybe someday a federated DBMS Slide 7: All of these systems Contain transformation engines And often do function activation (app service) And often have adapters to legacy systems Huge overlap in functionality!! Slide 8: Less Moving Parts Less systems More uniformity Less duplication Slide 9: Less Systems Less system administrators Less training Less manuals Less bugs Less cross system issues Slide 10: More Uniformity Every island has memory management security model threading model Less is better Slide 11: Less Duplication Most of the islands support transformations reasonable chance you will do each one 6 or more times maintenance headache Slide 12: So How To Consolidate…… Converge app server into OR DBMS dumbest OR query is execute function Remember that everything looks like a nail to the guy with the hammer! Pictorially : Pictorially client DBMS DBMS Slide 14: This Requires…. DBMS to send queries to other DBMSs I.e. be a data federator Load balance also requires a federator Slide 15: Best of Breed Federators Support schema heterogeneity by executing OR functions Support materialized views to cache static data Slide 16: Less Moving Parts…. Federators dominate ETL ETL only supports “push” federators do both “push” and “pull” Slide 17: Workflow A collection of rules who’s allowed to buy what and who must approve it Best considered as a boxes and arrows diagram And compiled into components to run on an app server Workflow Framework -- PO’s : Workflow Framework -- PO’s manager PO Big? no Laven yes IT? no yes Data Intensive Workflow Should Move Inside an OR DBMS : Data Intensive Workflow Should Move Inside an OR DBMS GUI for “boxes and arrows” Compiler for the diagram processing steps become components business rules become triggers all data flow inside the DBMS Worked great in Media/360 Why? : Why? Big Big Big performance advantage no polling of the DBMS no data movement easy to change! Watch for Informix product in this area! Slide 21: Nirvana One integrated system that does federation EAI app service With a single transformation system Based on DBMS technology (or something else….) Slide 22: XML Good for content storage and movement Good as “on the wire” format for data movement as long as you don’t need to send a lot of stuff fast Bad for data storage! Slide 23: History Lesson 1960’s IMS and IDMS get traction customers start complaining about rewriting everything when schema changes Slide 24: History Lesson 1970 Codd writes pioneering paper starts a decade long argument between IMS/CODASYL advocates and Codd supporters Slide 25: Net-Net of Argument Putting semantics into data order is bad restricts storage options Hidden meaning bad no self-defining fields Slide 26: Net-Net of Argument Data independence is good schemas change often don’t want to rewrite anything when this happens Slide 27: Net-Net of Argument Complexity is bad high level query languages are good KISS arguments Call these three premises “Codd’s laws” Slide 28: History Lesson 1983(?) Codd wins Turing award acknowledgement for being right Slide 29: XML in This Historical Light Most of the bad features of IMS/Codasyl allows semantics in data order data independence will be a challenge try updates on inverted hierarchies look at IMS LDBs more complex than Codasyl Slide 30: Our Field We look a little silly saying an idea renounced in the 1970’s is back Leading our colleagues to ask “What’s different?” if somebody disproved Codd’s laws; they didn’t tell me….. Slide 31: How to Win the Turing Award Circa 2020 2000’s XML data storage gets traction 2010 dust off Codd’s paper Wait 10 years to be proven right Slide 32: In Any Case In line tags turn 1Tbyte of EMP data into 10 Tbytes of EMP data Won’t store anything big in native XML will use something else…. like what? Slide 33: OR DBMS XML is merely this year’s data type Next year it will be WML or … and there will be a next year…. Slide 34: XMLSchema Contains the kitchen sink Complexity run amok diarrhea from the SGML types Includes lots of known hard stuff e.g. union types Slide 35: Xquery Mostly syntactic sugar on OR SQL // is a user-defined function in Informix OR engine Try to keep the semantics close to OR SQL Slide 36: Another History Lesson Typical enterprise wanted data integration for business analysis badly needed data in a variety of systems in a variety of formats often with no unique ids often with incompatible semantics 2 day delivery means lots of things often dirty Slide 37: ETL Warehouse Projects of the 90’s Well into 8 digits Usually a factor of three behind schedule Delivering a factor of 3 less stuff Everybody dented their pick on semantic heterogeneity which is hard, hard, hard and not solved by the blizzard of 3 letter acronyms from Redmond Slide 38: Web Services Will be a long time coming outside of simple domains (where there is no data integration to deal with) E.g. catalog management Grainger perspiration…. Slide 39: The Depressing State of Affairs ~50-75% of IT projects fail if we built bridges, our profession would be fired and the same mistakes are repeated over and over (excessive ambition, rolling specs, bad design, failure to load a large data set early) Slide 40: What To Do? We typically don’t teach this stuff (and do a serious disservice to our students) probably because we don’t (can’t) spend any time in industry to figure it out Action item: at the very least read a couple of Robert L. Glass’s books Slide 41: The Depressing State of Affairs Hardware “half-life” is 18 months Software half-life is 18 years (or more)! In 25 years we moved from C to Java SQL to Xquery Slide 42: What To Do? Much higher level design environments vis workflow special purpose languages (report writers,…) And stop turning down papers on this stuff Slide 43: Grand Challenge Improve application productivity (probability of success * programmer productivity) by 2 this decade You do not have the permission to view this presentation. In order to view it, please contact the author of the presentation.
Middleware Diarrhea and Other Ailments aSGuest25622 Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINT lite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: (To copy code, click on the text box) Embed: URL: Thumbnail: WordPress Embed Customize Embed The presentation is successfully added In Your Favorites. Views: 31 Category: Entertainment License: All Rights Reserved Like it (0) Dislike it (0) Added: September 10, 2009 This Presentation is Public Favorites: 0 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript Middleware Diarrhea and Other Ailments : Middleware Diarrhea and Other Ailments Michael Stonebraker Adjunct Professor Massachusetts Institute of Technology (stonebraker@lcs.mit.edu) Slide 2: Outline Too much middleware XML ailments Web services ills Our professional sickness Slide 3: Client-Server Got Replaced by N-Tier Computing The Web Gizmos Scalability and management problems with client server Slide 4: Humility Lesson We all sold client-server hard during the 80’s and even into the 90’s Less than 10 years later it is the worst idea on the planet We should feel really dumb! Slide 5: N-Tier Computing Produced Lots of Middleware App servers EAI/messaging ETL Federators Workflow CMS Portals DBMS Slide 6: Middleware Diarrhea Average enterprise has one (or more) app servers one (or more) EAI packages one (or more) ETL packages one (or more) portal products one (or more) application packages and maybe someday a federated DBMS Slide 7: All of these systems Contain transformation engines And often do function activation (app service) And often have adapters to legacy systems Huge overlap in functionality!! Slide 8: Less Moving Parts Less systems More uniformity Less duplication Slide 9: Less Systems Less system administrators Less training Less manuals Less bugs Less cross system issues Slide 10: More Uniformity Every island has memory management security model threading model Less is better Slide 11: Less Duplication Most of the islands support transformations reasonable chance you will do each one 6 or more times maintenance headache Slide 12: So How To Consolidate…… Converge app server into OR DBMS dumbest OR query is execute function Remember that everything looks like a nail to the guy with the hammer! Pictorially : Pictorially client DBMS DBMS Slide 14: This Requires…. DBMS to send queries to other DBMSs I.e. be a data federator Load balance also requires a federator Slide 15: Best of Breed Federators Support schema heterogeneity by executing OR functions Support materialized views to cache static data Slide 16: Less Moving Parts…. Federators dominate ETL ETL only supports “push” federators do both “push” and “pull” Slide 17: Workflow A collection of rules who’s allowed to buy what and who must approve it Best considered as a boxes and arrows diagram And compiled into components to run on an app server Workflow Framework -- PO’s : Workflow Framework -- PO’s manager PO Big? no Laven yes IT? no yes Data Intensive Workflow Should Move Inside an OR DBMS : Data Intensive Workflow Should Move Inside an OR DBMS GUI for “boxes and arrows” Compiler for the diagram processing steps become components business rules become triggers all data flow inside the DBMS Worked great in Media/360 Why? : Why? Big Big Big performance advantage no polling of the DBMS no data movement easy to change! Watch for Informix product in this area! Slide 21: Nirvana One integrated system that does federation EAI app service With a single transformation system Based on DBMS technology (or something else….) Slide 22: XML Good for content storage and movement Good as “on the wire” format for data movement as long as you don’t need to send a lot of stuff fast Bad for data storage! Slide 23: History Lesson 1960’s IMS and IDMS get traction customers start complaining about rewriting everything when schema changes Slide 24: History Lesson 1970 Codd writes pioneering paper starts a decade long argument between IMS/CODASYL advocates and Codd supporters Slide 25: Net-Net of Argument Putting semantics into data order is bad restricts storage options Hidden meaning bad no self-defining fields Slide 26: Net-Net of Argument Data independence is good schemas change often don’t want to rewrite anything when this happens Slide 27: Net-Net of Argument Complexity is bad high level query languages are good KISS arguments Call these three premises “Codd’s laws” Slide 28: History Lesson 1983(?) Codd wins Turing award acknowledgement for being right Slide 29: XML in This Historical Light Most of the bad features of IMS/Codasyl allows semantics in data order data independence will be a challenge try updates on inverted hierarchies look at IMS LDBs more complex than Codasyl Slide 30: Our Field We look a little silly saying an idea renounced in the 1970’s is back Leading our colleagues to ask “What’s different?” if somebody disproved Codd’s laws; they didn’t tell me….. Slide 31: How to Win the Turing Award Circa 2020 2000’s XML data storage gets traction 2010 dust off Codd’s paper Wait 10 years to be proven right Slide 32: In Any Case In line tags turn 1Tbyte of EMP data into 10 Tbytes of EMP data Won’t store anything big in native XML will use something else…. like what? Slide 33: OR DBMS XML is merely this year’s data type Next year it will be WML or … and there will be a next year…. Slide 34: XMLSchema Contains the kitchen sink Complexity run amok diarrhea from the SGML types Includes lots of known hard stuff e.g. union types Slide 35: Xquery Mostly syntactic sugar on OR SQL // is a user-defined function in Informix OR engine Try to keep the semantics close to OR SQL Slide 36: Another History Lesson Typical enterprise wanted data integration for business analysis badly needed data in a variety of systems in a variety of formats often with no unique ids often with incompatible semantics 2 day delivery means lots of things often dirty Slide 37: ETL Warehouse Projects of the 90’s Well into 8 digits Usually a factor of three behind schedule Delivering a factor of 3 less stuff Everybody dented their pick on semantic heterogeneity which is hard, hard, hard and not solved by the blizzard of 3 letter acronyms from Redmond Slide 38: Web Services Will be a long time coming outside of simple domains (where there is no data integration to deal with) E.g. catalog management Grainger perspiration…. Slide 39: The Depressing State of Affairs ~50-75% of IT projects fail if we built bridges, our profession would be fired and the same mistakes are repeated over and over (excessive ambition, rolling specs, bad design, failure to load a large data set early) Slide 40: What To Do? We typically don’t teach this stuff (and do a serious disservice to our students) probably because we don’t (can’t) spend any time in industry to figure it out Action item: at the very least read a couple of Robert L. Glass’s books Slide 41: The Depressing State of Affairs Hardware “half-life” is 18 months Software half-life is 18 years (or more)! In 25 years we moved from C to Java SQL to Xquery Slide 42: What To Do? Much higher level design environments vis workflow special purpose languages (report writers,…) And stop turning down papers on this stuff Slide 43: Grand Challenge Improve application productivity (probability of success * programmer productivity) by 2 this decade