Slide 1

Robert Heim

Agile Software Development

17.01.2013

I assume: Audience has heard the term „Agile Software Development“ and has heard of practices like „Pair programming“, „Test-Driven-Development“, „Prototyping“, … I will: be abstract on the methods itself (one can easily look them up if further info needed), but be detailed on how to rate methods

Slide 2

Content  

Motivation Agile Methods ◦ ◦ ◦ ◦

 

Current movements: Programs of Agile Alliance Discussion of Adoption ◦ ◦ ◦ ◦



Attributes – What is „Agile“? Problems Evaluation criteria Examples + differences

Software Process Improvement Standards Example: British Telecom Empirical studies

Conclusion & Recommendations ◦ Agile Framework (process) ◦ Architecture Framework for Agile Processes (software)

 

Resources Notes on some methods and additional slides for further explanations 17.01.2013

Slide 3

Motivation 

Problems: Delivery on time, QM, late changes, etc.



Rapidly growing: internet and mobile apps



Need: lighter, faster, more flexible software development process

17.01.2013

1/22

Slide 4

Content  

Motivation Agile Methods ◦ ◦ ◦ ◦

 

Current movements: Programs of Agile Alliance Discussion of Adoption ◦ ◦ ◦ ◦



Attributes – What is „Agile“? Problems Evaluation criteria Examples + differences

Software Process Improvement Standards Example: British Telecom Empirical studies

Conclusion & Recommendations ◦ Agile Framework (process) ◦ Architecture Framework for Agile Processes (software)

 

Resources Notes on some methods and additional slides for further explanations 17.01.2013

Slide 5

Agile Software Development 

„Late 1990 several methodologies began to get increasing public attention. Each had a different combination of old ideas, new ideas, and transmuted old ideas. But they all emphasized close collaboration between the programmer team and business experts; face-to-face communication (as more efficient than written documentation); frequent delivery of new deployable business value; tight, self-organizing teams; and ways to craft the code and the team such that the inevitable requirements churn was not a crisis.” – Agile Alliance4 17.01.2013

deployable business value = less documentation

2/22

Slide 6

For more see: http://agilemanifesto.org/principles.html

Agile Methods - Attributes Incremental (max. 3 month16) rapid dev. cycle cooperative close customer straightforward easy to learn / flexible methods adaptive last minute changes 17.01.2013

3/22

A cycle is e.g.: 1. Functionality needed immediately, 2. Fast implementation 3. Collect feedback 4. react rapidly to business + technology changes [16] Lichter, reference models, SWC department

Slide 7

Agile Methods - Problems 

E.g. Agile Alliance

◦ 2000+ papers since 2002 21 ◦ Agile Conference1 2012: ~200 talks



But still: lack of scientific studies and empirical evidence²



People not aware of approaches/suitability for real-life situations 2, 14, …

?

 How

to rate, choose and benefit from specific agile methods? 17.01.2013

4/22

[2] Technical research center of Finland Abrahamsson… New Directions on Agile Methods : A Comparative Analysis. Science. 2003 [14] Austrian study

Slide 8

Agile Methods – Evaluation Criteria² Software dev. life-cycle  Project management support  Abstract concrete  Universally situational  Empirical evidence 

17.01.2013

Abstract / concrete: principles concrete guidance Universally = fit per se in all agile dev. situations

5/22

Slide 9

Agile Methods – Examples and Differences²

17.01.2013

Inception = project begin/founding of project ASD = Adaptive Software Development AM = Agile Modeling DSDM = Dynamic Systems Development Method FDD = Feature Driven Development ISD = Internet Speed Development PP = Pragmatic Programming

6/22

Slide 10

Agile Methods - Choosing Cannot answer the questions with ease: „Should I use ASD in my upcoming project? And which methods are suitable?“ 



NO STANDARD, yet (even if some ASDs are well known by many people, e.g. XP, Scrum)

17.01.2013

7/22

Slide 11

Content  

Motivation Agile Methods ◦ ◦ ◦ ◦

 

Current movements: Programs of Agile Alliance Discussion of Adoption ◦ ◦ ◦ ◦



Attributes – What is „Agile“? Problems Evaluation criteria Examples + differences

Software Process Improvement Standards Example: British Telecom Empirical studies

Conclusion & Recommendations ◦ Agile Framework (process) ◦ Architecture Framework for Agile Processes (software)

 

Resources Notes on some methods and additional slides for further explanations 17.01.2013

Slide 12

Agile Alliance – Current Programs1 

”Supporting Agile Adoption: It’s about Change”  organizational change (compare Lichter16 )



In 2012: ◦ Characteristics of agile organizations18 ◦ AWG – A sustainable Engine for Enterprise Agile Adoption19 ◦ Project Studies Program: longitudinal studies1 17.01.2013

8/22

Agile Alliance: a nonprofit organization with global membership, committed to advancing Agile development principles and practices Compare: my seminar 10/11 on improvement models: business has to address the culture of the organization AWG = Agile Working Group = Scrum team, with product: enterprise implementation of Agile, evidence of management support

Slide 13

Content  

Motivation Agile Methods ◦ ◦ ◦ ◦

 

Current movements: Programs of Agile Alliance Discussion of Adoption ◦ ◦ ◦ ◦



Attributes – What is „Agile“? Problems Evaluation criteria Examples + differences

Software Process Improvement Standards Example: British Telecom Empirical studies

Conclusion & Recommendations ◦ Agile Framework (process) ◦ Architecture Framework for Agile Processes (software)

 

Resources Notes on some methods and additional slides for further explanations 17.01.2013

Slide 14

Agility and Software Process Improvement (SPI)16 

See [7, 12]

Success factors ◦ People, technology, process ◦ Team needs skills + tools required to apply process



„Never apply process models for their own sake!“16



Fitting Portfolio of Process Models (PM)



Monitor process, to see if it still fits ( Quality Management e.g. software metrics12) 17.01.2013

9/22

Portfolio = Organization choose the mix of old (like CMMI) and new (like XP) models PM Selection criteria: Organization-culture, developer-team, project-constraints, business-goals Needs to be accepted from all involved people (that it supports their daily-work) (compare [14] Austria study) [16] = Lichter Software Processes in an Agile World. Department did many research on Software-Process-Improvement Models, Seminar WS10/11 SCRUM in large organizations works [ 7] = study from Pakistan, Agile Software Development : Impact on Productivity and Quality [12] = software metrics for ASD, University of Magdeburg, Germany

Slide 15

Standards 

2003: XP connectable with current ISO “software life cycle process”11 ◦ Main idea: „working software over comprehensive documentation“ ◦ practical evidence missing?! ◦ Conclusion: project specific standards



Agile Alliance Program (2012): Costs of IT Projects in coop. FASB (founded 1973):

Agile Accounting Standard

◦ “define and standardize internal IT development costs […] ◦ […] for organizations that use an […] agile software development methodology”5



But: no standard for ASD it self  17.01.2013

10/22

Projects often need to fit ISO standard (due to organization, management, regulation) 11: Standards and Agile Software Development.  abstract, XP and ISO norm for "software life cycle" integration, but only abstract. To fit to docu-standards-requirements: in XP as system requirement FASB (financial accounting standards board: develop generally accepted accounting principles (GAAP) within the United States in the public's interest

Slide 16

Example – British Telecom15 

Environment: ◦ 2006, ~8000 IT professionals ◦ Before: waterfall with CMMI, 2 years later: DSDM + Scrum ◦ Aim/Focus: Delivering Business Value



Problems / Risks: ◦ Architecture of IT-Organization + mindsets of people have to be changed ◦ Complexity ◦ Outsourced parts of code, where no tests exists



Practice: ◦ ◦ ◦ ◦ ◦



Before: 12month + cycles Now: 90 days cycle First 3 days cross-functional teams exploring one business problem At day 90 want deployable, fully-tested solution within that 90days use internal agile methods (like 2-4week cycles).

Result: challenging, but early success and they would not turn back 17.01.2013

11/22

CMMI = Capability Maturity Model Integration DSDM = Dynamic systems development method High risk = program has business commitments to make and little scope to take extra risk of adopting new practices  Deliver business value or close program if it fails to deliver value over some cycles

Slide 17

Results of (good) empirical study14  

Austria 2008 Interviewed developer and manager (but answers did not differ as much as expected)

100 companies, 42% acceptance-rate  40,5% large organizations (>250)  Focus: XP, especially Pair-Programming  Results: 

◦ Most know XP (~40%) and Scrum (~30%) ◦ General awareness of ASD exists ◦ But lack of knowledge of concrete practices 17.01.2013

12/22

Slide 18

Results of (good) empirical study14 

Results

◦ People are main obstacle to practical application of ASD, Most important:  Lack of knowledge and time  refusal of management



Concrete Reality: Agile methods used on demand  Test-first:  75% rarely use it (only when needed)

 Pair-programming:  32% only for complex tasks  7,1% for tutoring (or on demand)



 No really systematic approach towards agile development in practice (2008) 17.01.2013

Bad Mindset: “Adoption of some practices is enough for agile dev.” Lack of knowledge (but only 1/3 wanted more information)

13/22

Slide 19

Results of quantitative analysis7 (to be understood carefully) 2010 Pakistan  Online survey, 200 requests, 42 responses (21%), Focus: Impact on Productivity  Results: 

◦ Mainly Scrum, MSF and XP known/used ◦ Fewer but competent/experienced developers deliver outputs in lesser time and better quality ◦ Train people to be up to date with tools and technologies 17.01.2013

MSF = Microsoft Solutions Framework

14/22

Slide 20

Results of quantitative analysis7 (to be understood carefully)  

100% do knowledge sharing as an essential constituent of agile methodologies Techniques used:

17.01.2013

15/22

Regression test = test to ensure that a new change has not spawn new bugs. CI: small incremental changes, developer often commit, each commit = build hole system + ideally tests TDD a little controversial to other study, cause they say test-FIRST rarely used

Slide 21

Results of quantitative analysis7 (to be understood carefully) Increase in productivity / quality

17.01.2013

16/22

Slide 22

Content  

Motivation Agile Methods ◦ ◦ ◦ ◦

 

Current movements: Programs of Agile Alliance Discussion of Adoption ◦ ◦ ◦ ◦



Attributes – What is „Agile“? Problems Evaluation criteria Examples + differences

Software Process Improvement Standards Example: British Telecom Empirical studies

Conclusion & Recommendations ◦ Agile Framework (process) ◦ Architecture Framework for Agile Processes (software)

 

Resources Notes on some methods and additional slides for further explanations 17.01.2013

Slide 23

Conclusion 

Pro‘s

◦ Fast delivery in praxis ◦ Easy to learn (compared to other SPI-methods) ◦ Early success/results in most companies could been shown



Con‘s

◦ Missing evidence  difficult method-decision ◦ Self-organizing teams not always best22 ◦ Customer‘s role stressful and not sustainable over long periods10 ◦ May fail in large organizations 10 (controversial)  High commitment 17.01.2013

17/22

Self-organizing not best: in complex, non-linear problem-solving situations Controversy: experienced developer tend to adopt ASD better, risk higher, see British Telcom [10 ] = 2005 What do we know about Agile Software Development?

Slide 24

Conclusion – State of Research 

2008 - Roadmap for empirical research: Scientific research and experimental evidence needed 22, 10 ◦ Done: XP, Scrum23, especially Pair-Programming ◦ Test-driven: productivity increase less obvious and as seen rarely used in reality



Research should concentrate on: Include criteria of other fields / theories experienced ASD teams and organizations  longitudinal studies  Prioritize management-oriented approaches  



If interested: see problem definition and some practical results (mostly Scrum): 2007 - Exploratory study from Microsoft Research calling for further studies 23 17.01.2013

18/22

Other fields: problem: SE references only SE research, but what is with not directly related to agile methods, e.g. task-technology-fit, social and organizational learning theory, knowledgebased theory of firm, … Longitudinal studies: to see effects on other components of business [22] = 2008, A Preliminary Roadmap for Empirical Research on Agile Software Development [10] = 2009, What do we know about ASD, Frauenhofer Center for Experimental SE [23] = promising Exploratory Study from Microsoft

Slide 25

Conclusion – State of Research 

2008 and Goal for 2015 22

17.01.2013

19/22

Most important: Expertise = competence Research methodology Evidence / impact on industry Cohesiveness = how established ASD is [22] = 2008, A Preliminary Roadmap for Empirical Research on Agile Software Development Additional slide for explanation at end

Slide 26

See [10] for further practical & realistic results

Conclusion - Reality 

Adoption / choosing methods: ◦ ASD needs to be accepted by all participants ◦ Study characteristics of    

Project Organization structure & culture 18, 19 Regulations compare to methods‘ characteristics to find matches

◦ Train people (best in University20)

17.01.2013

20/22

Participants: developers, managers, customers, etc. [10] = 2009, What do we know about ASD, Frauenhofer Center for Experimental SE [18] = 2012 Agile Alliance - Characteristics of Agile Organizations. 2012, [19] = 2012 Agile Alliance - AWG (Agile Working Group) A sustainable Engine for Enterprise Agile Adoption [20] = 2009 Weaving Agile Software Development Techniques into a Traditional Computer Science Curriculum

Slide 27

Content  

Motivation Agile Methods ◦ ◦ ◦ ◦

 

Current movements: Programs of Agile Alliance Discussion of Adoption ◦ ◦ ◦ ◦



Attributes – What is „Agile“? Problems Evaluation criteria Examples + differences

Software Process Improvement Standards Example: British Telecom Empirical studies

Conclusion & Recommendations ◦ Agile Framework (process) ◦ Architecture Framework for Agile Processes (software)

 

Resources Notes on some methods and additional slides for further explanations 17.01.2013

Slide 28

Recommended Agile Framework7 (2010) 

Refactoring to change internal code without affecting external functionality



See [7] for more details

17.01.2013

emphasis flexible design, to be able to integrate late changes

21/22

Slide 29

Recommended Software Architecture for Agile Business Process13 (2010) 

Problem: „ASD is an excuse for developers to implement as they like, coding away without proper planning or design […] and consequently causing suboptimal design decisions.“ 13



Approach 2010 Architecture Framework for Agile processes (AFA) 13



Idea: Decoupling Components:

◦ ◦ ◦ ◦ 

Simple APIs of internet addressable containers Layers with different responsibilities Pro‘s: allowing changes in architecture with minimal effects on other components  less testing  better time-to-market Con‘s/Feature work: benchmark/performance because additional layers

Compare [6] (2010) for more detailed and practical scenarios and decision making, including freezing the architecture and how to decide when to do it, calling for more research 17.01.2013

22/22

Container = collection of entries accessible via basic API [13] = guys from Vienna University of Technology, Austria Coordination: e.g.: Fifo instead of Lifo can be changed, without touching algorithms for data computation [6] = Agility and Architecture: Can they Coexist? 2010.

Slide 30

Literature / Resources   

   

     

[1] Agile Conference 2012 - http://agile2012.agilealliance.org/program/ [2] Abrahamsson P, Warsta J, Siponen MT, Ronkainen J. New Directions on Agile Methods : A Comparative Analysis. Science. 2003. [3] Agile Manifesto - http://agilemanifesto.org/principles.html [4] Agile Alliance - „What is Agile Software Development?“ http://www.agilealliance.org/the-alliance/what-is-agile/ [5] Agile Alliance - „ Agile Accounting Standard Program“ www.agilealliance.org/programs/agile-accounting-standard-program [6] Abrahamsson, Pekka; Babarr, Muhammad Ali; Kruchten P. Agility and Architecture: Can they Coexist? 2010. [7] Ahmed A, Ahmad S, Ehsan N, Mirza E, Sarwar SZ. Agile Software Development : Impact on Productivity and Quality. Development. 2010:287-291. [8] Begel A. Usage and Perceptions of Agile Software Development in an Industrial Context : An Exploratory Study. Time. 2007:255-264. [9] Dingsøyr T, Dybå T, Abrahamsson P. Agile 2008 Conference A Preliminary Roadmap for Empirical Research on Agile Software Development. Most. 2008:83-94. [10] Dybå T, Dingsøyr T. What do we know about Agile Software Development? Small. 2005:0-3. [11] Kourie DG, Watson BW. Standards and Agile Software Development. Computer. 2003:1-11. [12] Kunz M, Dumke RR, Zenker N. Software Metrics for Agile Software Development. Work. 2008:673-678. [13] Mordinyi R, K E, Schatten A. Towards an Architectural Framework for Agile Software Development. 2010;(section IV).

17.01.2013

L1

Slide 31

Literature / Resources 

[14] Schindler C. Agile Software Development Methods and Practices in Austrian IT-Industry : Results of an Empirical Study. Technology. 2008:321-326.



[15] Agile Delivery at British Telecom 2006 http://www.methodsandtools.com/archive/archive.php?id=43



[16] Lichter H. Software Processes in an Agile World. 2012;6(November)



[17] Agile Alliance - Resources http://www.agilealliance.org/resources



[18] Arell R, Coldewey J, Gat I, Hesselberg J. Characteristics of Agile Organizations. Methods. 2012.



[19] Arell R, Coldewey J, Gat I, Hesselberg J. A Sustainable Engine for Enterprise Agile Adoption. 2012.



[20] Maher P, Avenue EL, Louis S. Weaving Agile Software Development Techniques into a Traditional Computer Science Curriculum. 2009:0-1.



[21] Agile Alliance - Conference Papers - http://conferences.agilealliance.org/sessions



[22] 1. Dingsøyr T, Dybå T, Abrahamsson P. A Preliminary Roadmap for Empirical Research on Agile Software Development. Most. 2008:83-94.



[23] Begel A. Usage and Perceptions of Agile Software Development in an Industrial Context : An Exploratory Study. Time. 2007:255-264.

Fee-based 

[24] IGI Global - 2005 - Agile Modeling, Agile Software Development, and Extreme Programming: The State of Research, http://www.igiglobal.com/article/journal-database-management-jdm/3343



[25] AIS - Association for Information Systems - 2011 - Overview and Guidance on Agile Development in Large Organizations http://aisel.aisnet.org/cais/vol29/iss1/2/ 17.01.2013 L2

Slide 32

APPENDIX

17.01.2013

Slide 33

(E)Xtreme Programming (XP) Developed at Chrysler ~1996 – 2000  Collection of concrete best practices  E.g. 

◦ ◦ ◦ ◦ ◦

Short iterations Pair programming Rapid feedback / close customer Communication Continuous refactoring + integration + testing ◦… 17.01.2013

A1

Slide 34

Scrum ~1995  Manage Software development process  Empirical base, focus on 

◦ ◦ ◦ ◦ ◦

Flexibility Adaptability Productivity Frequent management activities + Sprints Developer chooses specific techniques/practices for implementation 17.01.2013

A2

Slide 35

Adaptive Software Development (ASD) ~2000  Adaptive/incremental paradigm instead of waterfall  Constant prototyping  Framework for guidance to prevent chaos, but not suppressing emergence and creativity 

17.01.2013

A3

Slide 36

Agile Modeling (AM) ~2002  Focus: modeling and cultural principles  Modeling using agile philosophy  Advanced models  Aim: amount of models and documentation as low as possible 

17.01.2013

A4

Slide 37

Crystal Family ~1998  Different methods  Select best fitting based on „color-rating“ (size and criticality)  Tailoring methods to fit needs  Integrate other agile methods like XP/Scrum 

17.01.2013

A5

Slide 38

Dynamic Systems Development Method (DSDM) One of the first truly agile methods (~1994)  Normally: fixed functionality + adjust time/resources Here: fix time/resources and adjust functionality 

17.01.2013

A6

Slide 39

Feature Driven Development (FDD) ~2002  Process oriented for business systems  Design and building phases  Iterative  Quality aspects in process  monitoring of progress 

17.01.2013

A7

Slide 40

Internet Speed Development (ISD) ~2001  Need: Fast releases in a chaotic fast moving process  Descriptive, management-oriented framework to handle fast iterations: 

◦ Time-drivers ◦ quality dependencies ◦ good people = less process

17.01.2013

A8

Slide 41

Pragmatic Programming (PP) ~2000  Collection of best practices  ~70 so called „tips“ focusing day-to-day problems  Incremental, iterative development, testing, user-centered design 

17.01.2013

A9

Slide 42

Description of current status and suggested goal for 2015 22

17.01.2013

A10