Creativity Support Tools: A Grand Challenge for Interface DesignersBen Shneiderman (ben@cs.umd.edu)Founding Director (1983-2000), Human-Computer Interaction Laboratory Professor, Department of Computer ScienceMember, Institute for Advanced Computer Studies & Institute for Systems ResearchUniversity of MarylandCollege Park, MD 20742UIST Vancouver November 3, 2003 :Creativity Support Tools: A Grand Challenge for Interface DesignersBen Shneiderman (ben@cs.umd.edu)Founding Director (1983-2000), Human-Computer Interaction Laboratory Professor, Department of Computer ScienceMember, Institute for Advanced Computer Studies & Institute for Systems ResearchUniversity of MarylandCollege Park, MD 20742UIST Vancouver November 3, 2003


Slide 2: Human-Computer Interaction Laboratory Interdisciplinary research community - Computer Science & Psychology - Information Studies & Education www.cs.umd.edu/hcil


User Interface Design Goals :User Interface Design Goals Cognitively comprehensible: Consistent, predictable & controllable Affectively acceptable: Mastery, satisfaction & responsibility Design philosophy: Direct Manipulation NOT: Adaptive, autonomous & anthropomorphic


Scientific Approach (beyond user friendly) :Scientific Approach (beyond user friendly) Specify users and tasks Predict and measure time to learn speed of performance rate of human errors human retention over time Assess subjective satisfaction (Questionnaire for User Interface Satisfaction 7.0) Accommodate individual differences Consider social, organizational & cultural context


Scientific Approach (beyond user friendly) :Scientific Approach (beyond user friendly) Specify users and tasks Predict and measure time to learn speed of performance rate of human errors human retention over time Assess subjective satisfaction (Questionnaire for User Interface Satisfaction 7.0) Accommodate individual differences Consider social, organizational & cultural context www.awl.com/DTUI 4th Edition: April 2004


Library of Congress :Library of Congress Scholars, Journalists, Citizens Teachers, Students


Visible Human Explorer (NLM) :Visible Human Explorer (NLM) Doctors Surgeons Researchers Students


NASA Environmental Data :NASA Environmental Data Scientists Farmers Land planners Students


Bureau of Census :Bureau of Census Economists, Policy makers, Journalists Teachers, Students


NSF Digital Government Initiative :NSF Digital Government Initiative Find what you need Understand what you Find UMd & UNC www.ils.unc.edu/govstat/


Creativity Support Tools :Creativity Support Tools


Creativity Support Tools :Creativity Support Tools


Creativity Support Tools :Creativity Support Tools


Creativity Support Tools: Goals :Creativity Support Tools: Goals More people, more creative, more of the time Revolutionary breakthroughs, paradigm shifts, H-creativity Evolutionary, normal science, music & art, creative knowledge work Impromptu everyday creativity Raised expectations for professionals Tailored solutions Cheaper, faster, better


Structuralists: A plan, method, process :Structuralists: A plan, method, process Polya's four steps in How to Solve It (1957): 1) Understanding the problem 2) Devising a plan 3) Carrying out the plan 4) Looking back Couger (1996) reviews 22 "creative problem solving methodologies" Preparation Incubation Illumination Verification


Structuralists: A plan, method, process :Structuralists: A plan, method, process Atman's design steps: Problem definition – identify need Gather information Generate ideas – brainstorm & list alternatives Modeling – describe how to build Feasibility Analysis Evaluation – compare alternatives Decision – select one solution Communication – write or present to others Implementation (Atman et al., Design Thinking Research Symposium2003)


Inspirationalists: Aha, Aha, Aha! :Inspirationalists: Aha, Aha, Aha! Free associations Brainstorming Thesauri, photo collages Random stimuli, inkblots Breaking set Getting away to different locations Working on other problems Meditating, sleeping, walking Visualization 2-d networks of ideas Sketching


Situationalists: context, community, collaboration :Situationalists: context, community, collaboration Personal history Family history, parents, siblings Challenging teachers, inspirational mentors Supportive peers and partners Consultation Peers and mentors Early, middle and late stages Information and empathic support Motivations Fame, legacy, admiration Competition


Csikszentmihalyi’s book Creativity (1993) :Csikszentmihalyi’s book Creativity (1993) 1) Domain: e.g. mathematics or biology "consists of a set of symbols, rules and procedures” 2) Field: "the individuals who act as gatekeepers to the domain...decide whether a new idea, performance, or product should be included” 3) Individual: creativity is "when a person... has a new idea or sees a new pattern, and when this novelty is selected by the appropriate field for inclusion in the relevant domain"


Genex (Generator of Excellence) Framework :Genex (Generator of Excellence) Framework Collect Learn from previous works in digital libraries, the web, etc. Relate Consult with peers & mentors, early, mid & late stages Create Explore, discover, compose, evaluate possible solutions Donate Disseminate refined results and contribute to the digital libraries, the web, etc. (Codex, memex, genex: The pursuit of transformational technologies IJHCI, 1998)


Genex: Some potential down sides :Genex: Some potential down sides Collect Will knowledge of previous work limit imagination? Relate Could mentors discourage exotic ideas? Could peers rip-off your innovation? Create Will using standard tools limit creativity? Donate Could the desire for intellectual property protection limit dissemination?


Eight Activities in Genex :Eight Activities in Genex 1) Searching & browsing digital libraries 2) Consulting with peers & mentors 3) Visualizing data & processes 4) Thinking by free associations 5) Exploring solutions - What if tools 6) Composing artifacts & performances 7) Reviewing & replaying session histories 8) Disseminating results (Creating creativity: User interfaces for supporting innovation ACM TOCHI, 3/2000)


1) Searching & Browsing Digital Libraries :1) Searching & Browsing Digital Libraries Effective search: Basic Google Search Improved multimedia search Overviews & previews Result set categorization & visualization Multiple session searches (Clarifying Search, CACM 4/98)


1) Advanced Search: Google :1) Advanced Search: Google


1) Search: Overviews & Previews :1) Search: Overviews & Previews Faceted search Preview cardinality of result sets www.epicurious.com www.endeca.com


2) Consulting with Peers & Mentors :2) Consulting with Peers & Mentors Early, middle and late stages Information and empathic support Build trust by negotiated expectations Email, listservs, newsgroups, discussion boards Chat rooms, instant messaging, audio/video conferencing Comprehensive online communities Tele-medicine, tele-meeting, tele-democracy Collaboratories


2) Components of Negotiated Expectations :2) Components of Negotiated Expectations Clearly identify and refine through dialog who I am what I want to do   Declare understandings Why I think you can help How you can help (specific request with time period) How much is in it for you (payment, shared honor, appreciation)


2) Example Request for Consultation :2) Example Request for Consultation Poor: Dear Prof. Shneiderman: Attached is my PhD proposal, please tell me what you think. Better: Dear Prof. Shneiderman: I am a Computer Science PhD student at Imperial College. My advisor, Prof. Spence, knows you and I have read your papers. My work extends your concept of dynamic queries. I’ve attached my 2-page doctoral proposal in the hopes that you can give me your comments before I defend it in two weeks. Your input would be acknowledged and Dr. Spence would be glad to pay for your plane ticket to join the committee meeting.


3) Visualizing Data & Processes :3) Visualizing Data & Processes Visual bandwidth is enormous Human perceptual skills are remarkable Trend, cluster, gap, outlier... Color, size, shape, proximity... Human image storage is fast and vast Opportunities Spatial layouts & window coordination Information visualization Scientific visualization and simulation Telepresence and augmented reality Virtual Environments


3) Visualizing Data & Processes :3) Visualizing Data & Processes Visual bandwidth is enormous Human perceptual skills are remarkable Trend, cluster, gap, outlier... Color, size, shape, proximity... Human image storage is fast and vast Opportunities Spatial layouts & window coordination Information visualization Scientific visualization and simulation Telepresence and augmented reality Virtual Environments


Treemap Implementations :Treemap Implementations


3) TimeSearcher :3) TimeSearcher Time series Stocks Diseases Weather Genes User-specified patterns Rapid search


3) Information Visualization: Mantra :3) Information Visualization: Mantra Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand Overview, zoom & filter, details-on-demand


3) Information Visualization: Data Types :3) Information Visualization: Data Types 1-D Linear Document Lens, SeeSoft, Info Mural, Value Bars 2-D Map GIS, ArcView, PageMaker, Medical imagery 3-D World CAD, Medical, Molecules, Architecture Multi-Dim Parallel Coordinates, Spotfire, XGobi, Visage, Influence Explorer, TableLens, DEVise Temporal Perspective Wall, LifeLines, Lifestreams, Project Managers, DataSpiral Tree Cone/Cam/Hyperbolic, TreeBrowser, Treemap Network Netmap, netViz, SeeNet, Butterfly, Multi-trees (Online Library of Information Visualization Environments) otal.umd.edu/Olive


4) Thinking by Free Associations :4) Thinking by Free Associations Free associations Brainstorming Thesauri, photo collages Random stimuli, inkblots Breaking set Getting away to different locations Working on other problems Meditating, sleeping, walking Visualization 2-d networks of ideas Sketching


Slide 38:An Idea Visualization Tool The human vision is by far the most developed and powerful faculty. The Idea Processor exploits visual attributes such as: color, shape, size, scale, position, depth, link, icon, etc. Visual cues facilitate recall, association, and discovery. Diagrams and pictures help you to represent and solve complex problems. Visualization reinforces your short term memory. Towards Higher Abstractions Ideas and diagrams are the basic abstractions of the Axon Idea Processor. Ideas are shown as graphical objects and its relationship shown as links. You get the big picture at all times, and details can be hidden from view. Stimulate Recall & Creativity The Idea Processor has an integrated Checklist Management System and a library of Checklists on problem-solving strategies, words of wisdom, etc. Checklists are effective means of capturing and transferring knowledge, and it effectively amplifies your intelligence.


5) Exploring Solutions - What If Tools :5) Exploring Solutions - What If Tools State space exploration “Combinationist theory” Spreadsheets Simulation as a third paradigm of science SimCity & Flight Simulator Economic models Weather forecasts


5) Exploring Solutions - What If Tools :5) Exploring Solutions - What If Tools Terry & Mynatt: Previews


6) Composing Artifacts & Performances :6) Composing Artifacts & Performances Initiate a new composition Exemplars Templates Processes Revise at multiple levels Low Middle High Evaluate and refine Feedback about problems Measurement (Composition, Hawaii Int’l Conf. on Systems Science, January 2000)


6) Composing Artifacts & Performances :6) Composing Artifacts & Performances Initiate a new composition


7) Reviewing & Replaying Session Histories :7) Reviewing & Replaying Session Histories Record compact histories Allow users to review & annotate history Disseminate histories (histories as first class objects) Send by email Post to website Consult synchronously & asynchronously Edit, extract, combine, search Replay: slow, fast, reverse Macros to automate exploration (Photoshop) (Learning Histories, Computer Supported Collaborative Learning Conf., December 1999)


7) Imaginary dialysis procedure exploration :7) Imaginary dialysis procedure exploration


7) Adobe Photoshop History tools :7) Adobe Photoshop History tools


8) Disseminating Results :8) Disseminating Results Disseminate refined solution to gatekeepers Facilitate web publishing & focused advertising Reach subscribers & organizational gatekeepers Ensure quality by editors & reviewers Contribute to digital library communities Journals, books, resources, libraries, communities Science fairs, student projects, galleries, performances


Genex: Integrated Framework for Software :Genex: Integrated Framework for Software Modular design to allow components Common file formats to ease data movement Consistent commands to reduce cognitive load Shortcuts for experts & immediate feedback Direct manipulation for rapid learning, powerful actions, low error rates, high retention


Genex Phases and Activities :Genex Phases and Activities


Challenges for Creativity Researchers :Challenges for Creativity Researchers Domain knowledge is vital Creative work may take years Individuals have highly varied approaches Evaluation is difficult Controlled studies are unrealistic Case studies are not replicable Theories are shallow


An Inspirational Muse: Leonardo da Vinci (1452-1519) :An Inspirational Muse: Leonardo da Vinci (1452-1519) Renaissance Man Combined science & art Integrated engineering & esthetics Balanced technology advances & human values Merged visionary & practical (MIT Press, Oct 2002)


Slide 51: Human-Computer Interaction Laboratory 21th Annual Symposium June 3-4, 2004 www.cs.umd.edu/hcil


References :References Shneiderman, Ben and Plaisant, Catherine, Designing the User Interface: Strategies for Effective Human-Computer Interaction: Fourth Edition, Addison Wesley Longman, Boston, MA (2004) Shneiderman, Ben, Relate-Create-Donate: A teaching philosophy for the cyber-generation, Computers & Education 31, 1 (1998), 25-39. Shneiderman, B., Creating Creativity: User interfaces for supporting innovation, ACM Transactions on Computer Human Interaction 7, 1 (2000), 114-138. Shneiderman, B., Supporting creativity with powerful composition tools for artifacts and performances, Proc. 33rd Hawaii International Conference On System Sciences (HICSS) (1/2000). www.cs.umd.edu/hcil