can_req_be_creative

Slide 1: 

“Can Requirements Be Creative? Experiences with an Enhanced Air Space Management System” Authors: Neil Maiden, Cornelius Ncube, Suzanne Robertson Date: May 2007 Source: ACM International Conference on Software Engineering (ICSE)? Presented by: Adriana Ogasawara & Joshua Mahaz

Slide 2: 

Introduction Purpose of the paper is to report a workshop that employed creativity techniques & an engineering process called RESCUE to determine new requirements for EASM EASM (Enhanced Air Space Management) is a software system designed for the UK and the rest of Europe Helps make air space management more efficient and provide more long-term planning capabilities

Introduction : 

Authors state that requirements analysts lack processes to guide their creative thinking Goals of this paper: Do ideas generated from these workshops have an impact on requirements in the final Operational Concept of Use document? Is analogical reasoning a cost effective creativity technique? Do the ideas generated by this technique impact the final specifications? Introduction

Slide 4: 

What is RESCUE? RESCUE is a requirements process in which different modelling and analysis methods take place concurrently Mainly through the use of system modelling to find system boundaries, actor dependencies, and system goals And use case modelling and scenario driven walkthroughs to facilitate communication with stakeholders and the creation of clear, precise, and testable requirements

Slide 5: 

What is RESCUE ? Incorporates creativity workshops “to encourage creative thinking with which to discover and invent requirements” RESCUE uses Sternberg’s definition of creativity “ . . . the ability to produce work that is both novel (ie. original, unexpected) and appropriate (ie. useful and adaptive concerning task restraints).”

What is RESCUE? : 

What is RESCUE? Workshop activities are designed using three established models of creativity Alex Osborn’s Creative Problem Solving (CPS) process Henri Poincaré’s philosophical model of creativity Margaret Boden’s three basic types of creativity

Slide 7: 

How does RESCUE incorporate CPS? Workshop periods support the divergence and convergence towards ideas described in CPS Each workshop period starts with a current agreed upon system model and diverges Divergence generates lots of ideas (facts, problem definitions, evaluation criteria, implementation strategies)? Then convergence towards a new agreed upon system model occurs at the end of the period In convergence only the most promising ideas are selected

Slide 8: 

RESCUE & Poincare’s 4-Stage Model of Creativity Each workshop period is also designed so participants go through the 4 stages of creativity as the period converges and diverges: Preparation: define and analyze the problem Incubation: the subconscious mind works on the problem Illumination: moment of insight when the solution is made conscious Verification: the new solution is tested against the problem, data is checked, etc

RESCUE Creativity Workshop Structure : 

RESCUE Creativity Workshop Structure

Slide 10: 

RESCUE & Boden’s Three Basic Types of Creativity Exploratory (Analogical): The process of searching an area of conceptual space governed by certain rules. RESCUE specifically encourages analogical reasoning to generate new ideas. Analogical reasoning is about seeing similarities between different concepts and making inferences based on those perceived similarities Research has shown that people can exploit analogies if helped to understand them.

RESCUE & Boden’s Three Basic Types of Creativity : 

Combinatorial: The creation of new ideas from a combination and synthesis of existing ideas. Transformational: People change the solution space in a way that things that were considered impossible are now possible For example, by challenging pre-conceived constraints and exploring new solutions to existing problems. RESCUE & Boden’s Three Basic Types of Creativity

The EASM Creativity Workshop : 

Use cases and system models were generated before the workshop by two EASM systems engineers from existing EASM documentation A “fun” atmosphere was encouraged & rules about avoiding criticism and time boxing topics so people could feel free to relax and voice ideas without criticism The EASM Creativity Workshop

The EASM Creativity Workshop : 

Ideas were generated by 2 different external experts and 19 stakeholders Stakeholders were employees of Eurocontrol (nat’l air traffic service), military traffic control, or a major airline The workshop started with a system wide and use case specific brainstorming session before moving into sessions that specifically incorporated one of Boden’s creativity types The EASM Creativity Workshop

Using Exploratory Creativity in EASM : 

Workshop facilitators chose two different analogies: a museum exhibition and a TV program scheduler Museums and air space management both deal with the layout and management of finite 3D space TV program schedulers and air space management both have the same resource scheduling abstraction Using Exploratory Creativity in EASM

Using Exploratory Creativity in EASM : 

Workshop participants then listened to expert presentations on the two analogies Afterwards, they were encouraged to find similarities between agents, objects, actions, constraints, and goals in the two analogies Each mapping was then used to generate new ideas for EASM by transferring knowledge about solutions from the museum exhibition and TV program scheduler analogies Using Exploratory Creativity in EASM

Using Transformational Creativity in EASM : 

Facilitators led brainstorming groups to discover as many constraints as possible on the EASM system design Then each group contemplated the removal of each constraint to generate new EASM ideas based on this removal Also leads stakeholders to consider trade-offs between the satisfaction of competing goals and constraints Using Transformational Creativity in EASM

Using Combinatorial Creativity in EASM : 

Storyboarding was used to elaborate and combine creative ideas in the last period of the workshop Stakeholders worked in groups and produced a storyboard that described the possible combination of requirements and ideas associated with one use case during the first 3 periods of the workshop Using Combinatorial Creativity in EASM

Workshop Results : 

To answer the two research questions previously mentioned . . . Do ideas generated from these workshops have an impact on requirements in the final specifications? Is analogical reasoning a cost effective creativity technique? Do the ideas generated by this technique impact the final specifications? The authors analyzed the number of ideas generated by the RESCUE process and elicited data from the EASM systems engineers to determine whether the ideas were novel and useful to EASM Workshop Results

Workshop Results : 

Over its two day course, the workshop generated 145 ideas 28 new ideas from the “regular” brainstorming session on the first day 94 by identifying & removing constraints (transformational) 15 from the museum exhibition analogy 8 from the TV program scheduler analogy 4 from storyboarding (combinatorial) Workshop Results

Workshop Results : 

Ideas were also rated by how “novel” and “useful” they were according to Sternberg’s definition of creativity “ . . . the ability to produce work that is both novel (ie. original, unexpected) and appropriate (ie. useful and adaptive concerning task restraints).” A simple 3-point scale was used to rate the novelty of each idea Workshop Results

Workshop Results : 

1 indicated that all elements of the idea were novel 2 indicated that some elements of the idea were novel 3 indicated no novelty 2 EASM system engineers ranked each idea independently for its novelty and usefulness using this 3 point scale Workshop Results

Workshop Results : 

Out of the 145 ideas generated by the workshop, the engineers couldn’t agree on a usefulness rating and 4 were thrown out b/c they couldn’t be reviewed adequately Of the remaining 139, only 2 ideas (one from brainstorming and one from constraint removal) were rated as being completely novel 40 of those ideas (10 from brainstorming, 9 from analogies, 21 from constraint removal) were rated as being novel in at least one element Workshop Results

Workshop Results : 

97 of the ideas were rated as not being novel 67 of these were generated during constraint removal Overall approximately 30% of the ideas generated by the creativity workshop were considered novel Workshop Results

Workshop Results : 

76% were rated as having at least some impact (“usefulness”) on the EASM final specifications Out of 106 ideas, 21 were generated from brainstorming, 17 from analogies, 68 from constraint removal Workshop Results

Conclusion : 

EASM stakeholders considered the workshops to be a success However, despite the fact that analogical reasoning occupied 2 out of 4 of the workshop periods, it only generated 15% of the total ideas It also led to fewer novel and useful ideas than other techniques Suggests this strategy is not as effective as hoped for Conclusion

Conclusion : 

On the other hand, data gathered indicates that the workshops did impact the final requirements specification for EASM 106 of 139 ideas with some impact 28 of 139 ideas with major impact Results from the two day workshop held four months into the EASM project carried over into subsequent requirements work for another eight months of the project Conclusion

Thoughts on the Paper : 

Thoughts on the Paper

Pros of RESCUE : 

2 days in a 12 month project – has the potential to be a cost-effective means of discovering requirements Pros of RESCUE

Cons of RESCUE : 

Analogies are not a cost-effective strategy As shown in the study, they do not generate as many ideas as other processes Need to find an analogy that has a sufficient number of ideas People interpret analogies differently & human ability to reason analogically varies by individual Need to find an analogy that maps correctly to your domain ? this takes time and time == money Cons of RESCUE

Cons of RESCUE : 

The 3 point ratings given by the EASM systems seems subjective The authors admit that 8 months passed between the workshop and the end of the rating process over which the engineers’ understanding of EASM increased Also, the authors of this paper were present at the ratings meetings & admit themselves this could have biased the engineers to rate ideas as more useful or novel Cons of RESCUE

Improvements : 

Consult more with stakeholders to find analogies that are appropriate to their domain Good example of a paper that has a decent idea, but is not very well written Typos & errors in math calculations do not facilitate good communication and does not make the reader inclined to take you seriously as a professional in your field of study Improvements

Questions? : 

Questions?

Works Cited : 

Bilton, Chris. Management and Creativity. Blackwell Publishing. 23 January 2008. <http://books.google.com/books?hl=en&id=6esBg4JCbm8C&dq=chris+bilton+management+and+creativity&printsec=frontcover&source=web&ots=2ln5mBgMJr&sig=0MjG-JW4VQYw7NgzatR2kpyKDxg> Schmalhoffer, Franz, et al. Proceedings of Eurocogsci 03. 2003. Lawrence Erlbaum Associates. 25 January 2008. <http://books.google.com/books?id=GpGBZy6QGJ4C&pg=PA157&dq=boden+%2B+analogical+reasoning&lr=&ei=QWuaR7qXKozGyAT0mZxs&sig=9axTIVWrbymaxlVGuXMMgbmJ3vE#PPP1,M1> Works Cited