The Complexity ofProgramming Models: The Complexity of Programming Models Grady Booch IBM Fellow


How much software exists in the world?: How much software exists in the world? SLOC is a measure of labor (not of value) Old code never dies (you have to kill it) Some code is DOA Some assumptions 1 SLOC = 1 semicolon Number of software professionals worldwide %of software professionals who cut code SLOC/developer/year $100US/SLOC


Number of software professional worldwide: Number of software professional worldwide


% of software professionals who cut code: % of software professionals who cut code


SLOC/developer/year: SLOC/developer/year


New or modified SLOC/year and cumulative: New or modified SLOC/year and cumulative


Dimensions of software complexity: Dimensions of software complexity Higher technical complexity - Embedded, real-time, distributed, fault-tolerant - Custom, unprecedented, architecture reengineering - High performance Lower technical complexity - Mostly 4GL, or component-based - Application reengineering - Interactive performance Higher management complexity - Large scale - Contractual - Many stake holders - 'Projects' Lower management complexity - Small scale - Informal - Single stakeholder - 'Products' An average software project - 5-10 people - 3-9 month duration - 3-5 external interfaces - Some unknowns andamp; risks Royce


Stakeholders and views: Stakeholders and views A given system is many things to many different stakeholders End user Customer Sys admin Project manager System engineer Developer Architect Maintainer Tester Other systems Multiple realities, multiple views and multiple blueprints exist


Simplicity has different points of view: Simplicity has different points of view User Simple metaphors and gestures End-user programmer Access to significant parts and flexible mechanisms for behavior Architect Common architectural patterns Developer Common design patterns and idioms Tester Access to significant parts


Simplicity has different points of view: Simplicity has different points of view Business analyst Clear separation of rules Database analyst Purity of semantics Security officer Clear and perfectly executed policies Administrator Clear separation of components


Slide11: In the presence of essential complexity, establishing simplicity in one part of a system requires trading off complexity in another


Creating the illusion of simplicity: Creating the illusion of simplicity


Simplicity in languages: Simplicity in languages Tradeoff between primitiveness and convenience Control structures Tradeoff between explicitness and abstraction Java garbage collection Tradeoff between performance of development and performance of execution VisualBasic Smalltalk Tradeoff between packaging for design versus packaging for development versus packaging for deployment Beans Services Aspects


Slide14: A programming model specifies the semantic universe within which the developer labors and is defined by the languages, platforms, tools, and best practices of that constellation


Web-centric programming model: Web-centric programming model Languages HTML CSS XSL XML SQL RSS Java JavaScript PHP Flash UML Platforms Linux Apache MySQL J2EE Best practices Coding Design patterns Deployment User interface Accessibility Internationalization Security Logging Backup Tools Eclipse Dreamweaver Photoshop ClearCase ClearQuest RSA Portfolio Manager RequisitePro Tester


Alternative programming models: Alternative programming models Game developer High performance computing Command and control Artificial intelligence Domain-specific frameworks … Handbook of Software Architecture, http://www.booch.com/architecture


Slide17: A system is shaped by a myriad of design decisions by different stakeholders that work to balance the forces swirling around the system


Forces in civil architecture: Forces in civil architecture Avoiding failure - Safety factors - Redundancy - Equilibrium Kinds of loads - Dead loads - Live loads - Dynamic loads Any time you depart from established practice, make ten times the effort, ten times the investigation. Especially on a very large project. - LeMessuier


Forces on software: Forces on software


Why is software inherently complex?: Why is software inherently complex? Complexity of the problem domain Difficulty of managing the development process Fluidity of software Fundamental challenges of discrete systems


Complexity of the problem domain: Complexity of the problem domain Volume of requirements Presence of competing/contradictory requirements Non-functional requirements that push the limits of software Requirements churn Difficulty of communicating requirements Impedance mismatch among stakeholders Unrestrained external complexity Software drag


The limits of software: The limits of software


Difficulty of managing the development process: Difficulty of managing the development process Software as a team sport Presence of multiple languages, platforms, processes, architectures, and tools Issues of scalability Technology churn


Scalability: Scalability Size up Increasing database size by a factor of x increases query response time by at most a factor of x. Speed up Increasing the capacity of your hardware configuration by a factor of x decreases your query response time by no less than a factor of x. Scale up Increasing the workload on your system by a factor of x while maintaining response time and/or throughput requires increasing your capacity by a factor of no more than x. Scale out Increasing workers by a factor of x requires replicating your capacity by a factor of at most x. http://www.intelligententerprise.com/db_area/archives/1999/991602/scalable.jhtml


Fluidity of software: Fluidity of software Software springs from pure thought and is intrinsically malleable, yet it can be made manifest in our hardware systems, limited only by our vision (and certain immutable laws of physics and software)


Fundamental challenges of discrete systems: Fundamental challenges of discrete systems Non-continuous behavior of discrete systems Combinatorial explosion of states Corruption from unexpected external events Lack of mathematical tools and intellectual capacity to model the behavior of large discrete systems


Essential complexity: Essential complexity 'Einstein argued that there must be simplified explanations of nature, because God is not capricious or arbitrary. No such faith comforts the software engineer. Much of the complexity that he must master is arbitrary complexity.' [Brooks] We may master essential complexity, but we can never make it go away.


Measuring complexity of biological systems (syntactic): Measuring complexity of biological systems (syntactic) Kolmogorov Entropy Mean component size Number of behaviors exhibited Species richness, relative to tolerance to risk Species guilds Energy flow Grammatical complexity Number of feedback loops Cyclomatic measures (arcs, vertices, and components) Graph complexity Hamming distance http://www.carleton.ca/~hmasum/complex.html


Measuring complexity of biological systems (semantic): Measuring complexity of biological systems (semantic) Wordcount of description Minimal description length (Rissanen) Measure of environmental knowledge Evolvability Eigenbasis/measure of survivable environmental states Program complexity http://www.carleton.ca/~hmasum/complex.html


Measuring complexity of software-intensive systems: Measuring complexity of software-intensive systems Kolmogorov Relative size of a program capable of generating a given string Entropy Enumeration of states and transitions http://cscs.umich.edu/~crshalizi/notebooks/complexity-measures.html


Measuring simplicity: Measuring simplicity If we don’t know how to measure complexity, it is reasonable to suggest that we don’t know how to measure simplicity 'I can’t define it, but I know it when I see it.' [Supreme Court Justice Brennan]


Beauty: Beauty Elegance is not an approach to finding a solution to a problem, it is the label we stick on the optimum solution Elegance is doing the most with the least Elegance means simplicity and less new code. An elegant solution solves the whole problem.' [Fisher, p. 37] Fisher andamp; Gipson, 'In Search of Elegance'


Triggers of complexity: Triggers of complexity Significant interactions High number of parts and degrees of freedom Nonlinearity Broken symmetry Nonholonomic constraints Localized transient anarchy Flood, et al, Dealing with Complexity


Attributes of a complex system: Attributes of a complex system 'Frequently, complexity takes the form of a hierarchy, whereby a complex system is composed of interrelated subsystems that have in turn their own subsystems, and so on, until some lowest level of elementary components is reached.' [Courtois] Hierarchic systems are decomposable if they can be divided into identifiable parts; they are nearly decomposable if their parts are not completely independent. [Simon]


Attributes of a complex system: Attributes of a complex system The choice of what components in a system are primitive is relative arbitrary and is largely up to the discretion of the observer of the system. As systems evolve, objects that we once considered complex become the primitive objects upon which more complex systems are built.


Attributes of a complex system: Attributes of a complex system Intracomponent linkages are generally stronger than intercomponent linkages. This fact has the effect of separating the high-frequency dynamics of the components – involving the internal structure of the components – from the low-frequency dynamics - involving interaction among components. [Simon]


Attributes of a complex system: Attributes of a complex system Hierarchic systems are usually composed of only a few different kinds of subsystems in various combinations and arrangements. [Simon]


Decomposible and nearly-decomposible systems: Decomposible and nearly-decomposible systems Vertically, the components of a complex system tend to be organized in increasing layers of abstraction Horizontally, the components of a complex system tend to be clustered according to frequency Both vertically and horizontally, the most resilient systems tend to exhibit loose coupling and tight cohesion among components Simon, The Organization of Complex Systems


Components: Components Loosely-coupled components adapt more easily to change Loosely-coupled components permit time- and space-separation of processing Overall flexibility can be enhanced by limiting the number of different kinds of components in the system (the system’s alphabet) Alphabets are necessary but insufficient Complex systems also require common languages, defining semantics of structural organization of alphabetic elements and interactional behavior among structures Simon, The Organization of Complex Systems


Languages: Languages Must have sufficient variety in its primitive processes so that no meaning is absolutely excluded from expression Must have sufficient flexibility in its rules of combination so that any nuance can be expressed by building up composite structures Simon, The Organization of Complex Systems


Attributes of complex systems: Attributes of complex systems Complex systems will evolve from simple systems much more rapidly if there are stable intermediate forms than if there are not. [Simon] A complex system that works is invariable found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system. [Gall]


Complex adaptive systems: Complex adaptive systems Emergent behavior Attributes Classification of components Identity of objects Non-linearity of behavior Flow of objects Diversity Use of internal models Clustering Holland, Hidden Order


Characteristics of self-organizing systems: Characteristics of self-organizing systems Dynamic, requiring continual interactions among lower-level components to produce and maintain structure Exhibit bifurcation leading to multistable systems Strange attractors Sante Fe Institute


Self-organization in biological systems: Self-organization in biological systems Pattern formation in slime molds Feeding aggregation of bark beetles Synchronized flashing among fireflies Fish schooling Nectar source selection by honey bees Trail formation in ants Swarm raids of army ants Colony thermoregulation in honey bees Comb patterns in honey bee colonies Wall building by ants Termite mount building Construction Aagorithms by wasps Dominance hierarchies in paper wasps Camazine, et al, Self-Organization in Biological Systems


Creating order in biological systems: Creating order in biological systems Self-organization Emergence of patterns at the global level solely from numerous interactions among lower-level components of the system, the rules for which are executed using only local information Imposed organization Direction from a supervisory leader Organic blueprints or recipes Patterns in the environment Camazine, et al, Self-Organization in Biological Systems


Complexity, Functionality, and Understandability: Complexity, Functionality, and Understandability Complexity Functionality Understandability


Slide47: Fundamentals never go out of style


Shearing layers of change : Shearing layers of change Brand, How Buildings Learn


Fundamentals of well-engineered software-intensive systems: Fundamentals of well-engineered software-intensive systems Crisp abstractions Clear separation of concerns Balanced distribution of responsibilities Simplicity via common abstractions and mechanisms


Abstraction: Abstraction All abstractions are context-dependent All non-trivial abstractions are, to some degree, leaky (and leaky abstractions are problematic). [Joel on Software] There is no such thing as a perfect abstraction Perfect is the enemy of good enough


Worse Is Better: Worse Is Better Simplicity is the most important consideration in a design.Both implementation and interface must be simple, though it is more important for the implementation to be simple. The design must be correct in all observable aspects; it is slightly better to be simple than correct. The design must not be overly inconsistent; it is better to drop those parts of the design that deal with less common circumstances than to introduce implementational complexity. The design must cover as many imporatant situations as practical; completeness can be sacrificed in favor of any other quality. Gabrial, 'Worse is Better'


Loose abstractions: Loose abstractions Over-engineering a solution is the most common approach to dealing with complexity, yet it typically leads to total implosion. Software which is flexible, simple, sloppy, tolerant and altogether forgiving turns out to be most resilient. [Bosworth]


Slide53: The entire history of software engineering Is one of rising levels of abstraction Assembly -andgt; Fortran/COBOL -andgt; Simula -andgt; C++ -andgt; Java Naked HW -andgt; BIOS -andgt; OS -andgt; Middleware -andgt; Domain-specific Waterfall -andgt; Spiral -andgt; Iterative -andgt; Agile Procedural -andgt; Object Oriented -andgt; Service Oriented Early tools -andgt; CLE -andgt; IDE -andgt; XDE -andgt; CDE Individual -andgt; Workgroup -andgt; Organization Languages: Platforms: Processes: Architecture: Tools: Enablement:


Attacking complexity: Attacking complexity Fundamentals Crisp abstractions Clear separation of concerns Balanced distribution of responsibilities Simplicity via common abstractions and mechanisms Relax a constraint Make assumptions


Architecture defined: Architecture defined Architecture n (1563) The art or science of building or constructing edifices of any kind for human use The action or process of building Architectural work; structure, building The special method of ‘style’ in accordance with which the details of the structure and ornamentation of a building are arranged Construction or structure generally The conceptual structure and overall logical organization of a computer or computer-based system from the point of view of its use or design; a particular realization of this Oxford English Dictionary, 2nd ed.


Physical systems: Physical systems Mature physical systems have stable architectures Aircraft, cars, and ships Bridges and buildings Such architectures have grown over long periods of time Trial-and-error Reuse and refinement of proven solutions Quantitative evaluation with analytical methods Mature domains are dominated by engineering efforts Analytical engineering methods New materials New manufacturing processes


Software-intensive system : Software-intensive system A system in which software is the dominant, essential, and indispensable element E-commerce system IT (business) system Telephone switch Flight control system Real-time control system (e.g. industrial robot) Sophisticated weapons system Software development tools System software (e.g. operating systems or compilers)


Architecting software is different: Architecting software is different No equivalent laws of physics Transparency Complexity Combinatorial explosion of state space Non-continuous behavior Systemic issues Requirement and technology churn Low replication and distribution costs


Architecture defined: Architecture defined Software architecture is what software architects do Beck


Architecture defined: Architecture defined Perry and Wolf, 1992 A set of architectural (or design) elements that have a particular form Boehm et al., 1995 A software system architecture comprises A collection of software and system components, connections, and constraints A collection of system stakeholders' need statements A rationale which demonstrates that the components, connections, and constraints define a system that, if implemented, would satisfy the collection of system stakeholders' need statements Clements et al., 1997 The software architecture of a program or computing system is the structure or structures of the system, which comprise software components, the externally visible properties of those components, and the relationships among them http://www.sei.edu/architecture/definitions.html


Common elements: Common elements Architecture defines major components Architecture defines component relationships (structures) and interactions Architecture omits content information about components that does not pertain to their interactions Behavior of components is a part of architecture insofar as it can be discerned from the point of view of another component Every system has an architecture (even a system composed of one component) Architecture defines the rationale behind the components and the structure Architecture definitions do not define what a component is Architecture is not a single structure -- no single structure is the architecture


Architecture defined: Architecture defined Architecture establishes the context for design and implementation CODE implementation design architecture Architectural decisions are the most fundamental decisions; changing them will have significant ripple effects.


Architecture defined: Architecture defined IEEE 1471-2000 Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution Software architecture encompasses the set of significant decisions about the organization of a software system Selection of the structural elements and their interfaces by which a system is composed Behavior as specified in collaborations among those elements Composition of these structural and behavioral elements into larger subsystems Architectural style that guides this organization Booch, Kruchten, Reitman, Bittner, and Shaw


Architecture defined : Architecture defined Software architecture also involves Functionality Usability Resilience Performance Reuse Comprehensibility Economic and technology constraints and tradeoffs Aesthetic concerns


Architectural style defined: Architectural style defined Style is the classification of a system’s architecture according to those with similar patterns A pattern is a common solution to a common problem; patterns may be classified as idioms, mechanisms, or frameworks


Model, views, concerns, and stakeholders: Model, views, concerns, and stakeholders A model is a simplification of reality, created in order to better understand the system being created; a semantically closed abstraction of a system A view is a representation of a whole system from the perspective of a related set of concerns A concern is those interests which pertain to the system's development, its operation or any other aspects that are critical or otherwise important to one or more stakeholders A stakeholder is an individual, team, or organization (or classes thereof) with interests in, or concerns relative to, a system


Stakeholders and views: Stakeholders and views Architecture is many things to many different stakeholders End user Customer Sys admin Project manager System engineer Developer Architect Maintainer Tester Other systems Multiple realities, multiple views and multiple blueprints exist


Representing software architecture: Representing software architecture Logical View End-user Functionality Implementation View Programmers Configuration management Process View Deployment View System topology Communication Provisioning System engineering Conceptual Physical Use Case View Clements, et al, Documenting Software Architectures


Adapting views: Adapting views Not all systems require all views Single process (ignore process view) Small program (ignore implementation view) Single processor (ignore deployment view) Some systems require additional views Data view Security view Other aspects


Logical view: Logical view The view of a system’s architecture that encompasses the vocabulary of the problem and solution space, the collaborations that realize the system’s use cases, the subsystems that provide the central layering and decomposition of the system, and the interfaces that are exposed by those subsystems and the system as a whole Focuses on Functionality Key Abstractions Mechanisms Separation of concerns and distribution of responsibilities


Process view: Process view The view of a system’s architecture that encompasses the threads and processes that form the system’s concurrency and synchronization mechanisms Focuses on Performance Scalability Throughput


Implementation view: Implementation view The view of a system's architecture that encompasses the components used to assemble and release the physical system Focuses on Configuration management


Deployment view: Deployment view The view of a system’s architecture that encompasses the nodes that form the system’s hardware topology on which the system executes Focuses on Distribution Communication Provisioning


Use case view: Use case view The view of a system’s architecture that encompasses the use cases that describe the behavior of the system as seen by its end users and other external stakeholders


Relations among views: Relations among views   Logical view Implementation view Process view Deployment view


Architecture metamodel: Architecture metamodel


Architecture metamodel: Architecture metamodel


Architecture metamodel: Architecture metamodel


The architecture of biological systems: The architecture of biological systems Gene Cell component Cell Tissue Organ System


Cross-cutting concerns in biological systems: Cross-cutting concerns in biological systems Gene Reproduction Protein creation Cell component (mitochondria) Metabolism Glutamate-mediated excitotic neurlogical injury Cellular proliferation Regulation of the cellular redox state Heme synthesis Heat production


Cross-cutting concerns in biological systems: Cross-cutting concerns in biological systems Cell Structure Metabolism Reproduction Protein synthesis Transport/container Tissue Structure Work Transport/container


Cross-cutting concerns in biological systems: Cross-cutting concerns in biological systems Organ (liver) Digestion Carbohydrate metabolism Glucoenogenesis Glycogenesis Breakdown of insulin Lipid metabolism Cholesterol synthesis Production of triglycerides Coagulation factors Neutralization of various products Storage of glucose and various vitamins Red cell production for the fetus System (circulatory) Transport Heat regulation Healing mechanism


The reification of concerns: The reification of concerns Concerns are not isomorphic to structure In biological systems, these aspects evolved simultaneously and interdependently at each level of abstraction They existed a priori as reactions to evolutionary forces Post hoc we can name them


Representing software architecture: Representing software architecture Logical View End-user Functionality Implementation View Programmers Configuration management Process View Deployment View System topology Communication Provisioning System engineering Conceptual Physical Use Case View Clements, et al, Documenting Software Architectures


Cross-cutting concerns in software-intensive systems: Cross-cutting concerns in software-intensive systems Some structures and behaviors crosscut components Security Concurrency Caching Persistence Such elements usually appear as small code fragments sprinkled throughout a system Such elements are hard to localize using traditional approaches


The role of aspect-oriented software development: The role of aspect-oriented software development Remember the fundamentals Crisp abstractions Clear separation of concerns Balanced distribution of responsibilities Simplicity via common abstractions and mechanisms The current sweet spot for aspects involves elements of each of these fundamentals Especially with regard to building crisp abstractions and the separation of concerns for roles relative to packaging This impacts primarily the interplay of the logical view and the use case view


What’s missing/what’s next: What’s missing/what’s next Remember the already complex programming model Don’t make it more complex by adding yet another orthogonal mechanism The current pragmatic focus is upon transformation tools that focus on already visible artifacts The harder focus - plus the one that is most disruptive yet most potentially valuable - is upon transformation tools that focus on deep semantic representations and then the creation of these traditional artifacts by reflection I.e. source code as a pretty-printed side-effect, not a central object


Summary: Summary This stuff is fundamentally, wickedly hard And it’s not going to get any better in my lifetime And I plan on having a long life Remember that the world doesn’t need Yet More Technology We need less And ultimately, the best technology is invisible


Bibliography on complexity: Bibliography on complexity Allen, T. andamp; Starr, T., Hierarchy: Perspectives for Ecological Complexity, University of Chicago: 1982. Axelrod, R., The Complexity of Cooperation, Princeton: 1997. Barrow, J., Davies, P., andamp; Harper, C., Science and Ultimate Reality, Cambridge University Press: 2004. Bowker, G. andamp; Star, S., Sorting Things Out: Classification and its Consequences, MIT Press: 1999. Buchanan, M., Nexus, Norton: 2002. Camazine, S., Deneubourg, J., Franks, N., Sneyd, J., Theraulaz, G., andamp; Bonabeau, E., Self-Organization in Biological Systems, Princeton: 2001. Duda, R., Pattern Classification, 2nd ed., Wiley: 2001. Epxtein, J. andamp; Axtell, R., Growing Artificial Societies, MIT Press: 1996. Flood, R. andamp; Carson, E., Dealing With Complexity: An Introduction to the Theory and Application of Systems Science, Plenum Press: 1988. Gleick, J., Chaos: Making a New Science, Penguin Books: 1987. Hollland, J., Hidden Order, Perseus Books: 1995. Johnson, S., Emergence, Scribner: 2001.


Biography on complexity: Biography on complexity Kauffman, S., At Home in the Universe, Oxford University Press: 1995. Kipfer, B., The Order of Things, MJF Books: 2001. Lakoff, G., Women, Fire, and Dangerous Things: What Categories Reveal about the Mind, University of Chicago: 1987. Lakoff, G. andamp; Johnson, M., Metaphors We Live By, University of Chicago: 1980. Markman, E., Categorization and Naming in Children, MIT Press: 2002. Nicolis, G. andamp; Prigogine, I., Exploring Complexity, Freeman: 1989. Pattee, H., Hierarchy Theory: The Challenge of Complex Systems, George Braziller: 1973. Prigogine, I., The End of Certainty, Free Press: 1996. Simon, H., The Sciences of the Artificial, 2nd ed., MIT Press: 1969. Waldrop, M., Complexity: The Emerging Science at the Edge of Order and Chaos, Simon andamp; Schuster: 1992.


Slide91: Thank you!