Agile Software Development T-110.6130 Systems Engineering in Data Communications Software P

29.9.2011 Kristian Rautiainen, Aalto University

Agenda • • • •

Agile Software Development Basics eXtreme Programming (XP) Scrum 10 Ways to Fail when Going Agile

Kristian Rautiainen 29.9.2011 2

Agile Software Development Basics

Kristian Rautiainen 29.9.2011 3

Agile software development •

Structured and disciplined, fast-paced Iterative and Incremental Development (IID) – NOT “build-and-fix” or “hack away”

•

Packages existing good software engineering practices – Nothing new, except the underlying philosophy and the combination of practices

•

Embraces change rather than controls it – Suitable for extremely turbulent environments

•

Focuses on delivering business value in frequent increments – Customer/User involvement paramount

•

Can’t be used in all projects! – E.g. small teams recommended

•

More information: – http://www.agilealliance.org/

Kristian Rautiainen 29.9.2011 4

Agile Manifesto •

Signed by authors of ASD, XP, Scrum, Crystal, FDD, DSDM, and ”pragmatic programming” + some others in Feb 2001

•

Agree at the first level

•

Agree at the second level (Agile Manifesto)

– need to respond to change

– – – –

individuals and interactions over processes and tools working software over comprehensive documentation customer collaboration over contract negotiation responding to change over following a plan

•

Agree at the third level

•

Don’t agree at the fourth level

– 12 more detailed statements (Principles)

– detailed project tactics

Kristian Rautiainen 29.9.2011 5

Principles of the Agile Alliance • • • • • • •

Satisfy the customer through early and continuous delivery of valuable software Agile processes harness change for the customer’s competitive advantage Deliver working software frequently Working software is the primary measure of progress Agile processes promote sustainable development at a constant pace Business people and developers must work together daily Build projects around motivated individuals, give them support and trust them to get the job done

Kristian Rautiainen 29.9.2011 6

•

The most efficient and effective way of conveying information is face-to-face conversation

•

Attention to technical excellence and good design enhances agility

•

Simplicity--the art of maximizing the amount of work NOT done--is essential

•

The best architectures, requirements, and designs emerge from selforganizing teams

•

At regular intervals the team tunes and adjusts its behavior to become more effective

Agile process models • Provide the “project tactics”

– A set of specific values, principles, and practices

• Many models have been proposed – – – – – – – – –

eXtreme Programming (XP) Adaptive Software Development (ASD) Rapid Application Development (RAD) Dynamic Solution Delivery Model (DSDM) Scrum Crystal Feature-Driven Development (FDD) Lean Development (LD) ...

• Most agile process models employ time pacing Kristian Rautiainen 29.9.2011 7

The concept of time pacing • Time pacing refers to – Creating a rhythm for product development by timeboxing development work • The schedule is fixed • The scope is flexible (prioritization needed)

• Time pacing can be done on several time horizons – Time horizon = planning time horizon, i.e. the length of the timebox – Long-term goals are split into short-term sub goals – This provides • visible progress on shorter time horizons by reaching the sub goals • flexibility to reflect on and change plans and goals in the control points based on visible and concrete progress

Kristian Rautiainen 29.9.2011 8

Agile philosophy and time horizons Every idea, feature, bug, etc. considered for the product is kept in

Roles: 1 Product Owner 1 Team 1 Coach

planned scope for release allocated into

Release Backlog

Release Backlog parts chosen and allocated into Iteration Backlog

Iteration Backlog

Iteration Backlog Progress of a Release checked in Iteration Demo

consists of tasks that are needed to implement the allocated backlog items

Progress status of the tasks of an Iteration checked in Daily Heartbeat meetings

Kristian Rautiainen 29.9.2011 9

Progress status

Governance

Product Backlog

Sweet spots of agile software development •

3-8 people in one room – Information moves the fastest face-to-face

•

Onsite expert user – Minimized time for feedback of the solution – Possibility to ask questions immediately  fastest response time

•

One-month (max) iterations – Feedback of the product and the process – Today’s “industry standard” is 2-week iterations

•

Supporting infrastructure/technology in place – Continuous Integration environment with automatic build process – Fully automated regression tests • Confidence to changing and improving the system

Kristian Rautiainen 29.9.2011 10

Comparing process models

Specification

Time

Design

Implementation

Test

Sequential

Iterative & Incremental (IID)

Kristian Rautiainen 29.9.2011 11

eXtreme Programming (XP)

eXtreme Programming (XP)

(Based on the original book by Kent Beck)

Kristian Rautiainen 29.9.2011 12

XP Values (1/2) •

•

Communication – Lack of it is the reason for most problems – Punishment from bad news kills it – XP employs practices that keep programmers, customers and managers communicating

Simplicity – What is the simplest thing that could possibly work? – General vs. simple design • anticipatory design assumes more work today saves work later – YAGNI

• low rate of changes – anticipatory design • high rate of changes – refactoring and continuous design – A simple solution is easier to understand Kristian Rautiainen 29.9.2011 13

XP Values (2/2) •

Feedback – Concrete feedback about the state of the system – Scale of minutes or days • unit tests, quality of stories (requirements), progress of tasks – Scale of weeks or months • acceptance tests, schedule, system in production

•

Courage – Changing the system – Throwing code away – Pair programming

Kristian Rautiainen 29.9.2011 14

XP Principles •

Fundamental principles – rapid feedback • critical for learning – assume simplicity • treat every problem as if it can be solved with simplicity – incremental change • series of smallest changes that make a difference – embracing change • best strategy preserves most options while actually solving your most pressing problem – quality work • excellent quality • nobody likes doing a bad job Kristian Rautiainen 29.9.2011 15

•

Secondary principles – – – – – –

– – – –

teach learning small initial investment play to win concrete experiments open, honest communication work with people’s instincts, not against them accepted responsibility local adaptation travel light honest measurement

XP Practices • Practices – – – – – – – – – – – –

planning game small releases test driven development continuous integration system metaphor simple design refactoring pair programming collective code ownership coding standard whole team (on-site customer) sustainable pace

Kristian Rautiainen 29.9.2011 16

XP Process

Kristian Rautiainen 29.9.2011 17

Scrum

Kristian Rautiainen 29.9.2011 18

Scrum Process Overview

Kristian Rautiainen 29.9.2011 19

Scrum roles – Product Owner (PO) • • •

Represents the interests of everyone with a stake in the project/product Creates the project’s ROI objectives and release plans Manages and controls the Product Backlog (PBL) – – – – – –

•

Ensures that the PBL is visible to everyone Sets priorities for the items in the PBL For PO to succeed, everyone must respect his/her decisions Works with others to estimate items in backlog (for total effort) Participates in Sprint Planning Meeting (and Sprint Review) Participates in re-scoping or re-prioritizing the sprint

PBL – PBL is an evolving, prioritized queue of business and technical functionality that needs to be developed into a system – PBL emerges from many sources • Marketing, Sales, R&D, Support, ...

Kristian Rautiainen 29.9.2011 20

Scrum roles – Scrum Master • Agile coach – Master of the Scrum process and practices – Coaches the team members

• Facilitator/impediment remover – Shields the team from outside interruptions – Helps remove impediments

Kristian Rautiainen 29.9.2011 21

Scrum roles – Development team • Responsible for realizing the PO’s vision for the product – The team makes the technical choices and delivers working software to demo at the end of Sprints • Shared responsibility

• Self-organized and empowered – The team chooses its ways of working – No appointed team leaders, leaders emerge for different situations

Kristian Rautiainen 29.9.2011 22

10 Ways to Fail when going Agile

Kristian Rautiainen 29.9.2011 23

Ceremony Without Understanding • Doing everything as before, but calling it Agile – New terminology, but same routines – E.g. Weekly meeting => weekly Scrum... • ...or Daily Scrum that lasts as long as a weekly meeting

• Fallacy of allowing feature creep during iterations – ”because we’re supposed to be agile”

Kristian Rautiainen 29.9.2011 24

Underestimating Transition Effort and Time • Organisational change or process improvement is always challenging and takes a lot of time – ”This agile thing doesn’t work for us. We tried it for a couple of months, but...” – Fallacy of not risking some time now in order to win a lot of time in the future • Transition to Agile slows down productive work for an unknown period of time – Business representatives tend to be impatient...

Kristian Rautiainen 29.9.2011 25

Throwing Away Existing Good Practices • ”Let’s start from scratch with this Agile thing”

• ”We used to observe users to elicit requirements. Now we invite them to iteration demos instead” • ”Something that works for us is not considered Agile. We have to throw it away.”

Kristian Rautiainen 29.9.2011 26

Cultural Conflicts • Project manager => Scrum Master – You often see this role transition in companies • But the differences in the roles cause trouble and confusion • (technical) project manager => (technical) product owner is actually an easier transformation

• Breaking down ”silos” – Creating truly cross-functional teams is a big challenge... • ...but without them lots of effort for coordination is needed

Kristian Rautiainen 29.9.2011 27

Challenging Roles • Product Owner – Need I say more...?

• Empowered, self-organised team – ”Say what?” – A transition from a ”clear roles and responsibilities + a project manager to tell us what to do” to self-organised is rocket science and a huge step for most people

• A lot of coaching and support is needed for the role transitions (also for top-level managers!) Kristian Rautiainen 29.9.2011 28

Lacking Competence • Getting rid of the difficult practices because they take too much time to learn – ”Test-Driven Development (TDD) was really hard to understand and learn, so we stopped doing it” – You need to understand the trade-offs and underlying assumptions, otherwise you end up in total ad-hocracy

• Working at high velocity requires high discipline and skills + good-quality software – Work on building competence to the people, it will always pay off

Kristian Rautiainen 29.9.2011 29

Minimal Plans = Minimal Planning • Fallacy of planning only in Release or Iteration planning sessions – ”I just saw the demo. Now I need time to figure out what you should do next” – Planning should be made ”continuously” in a rolling wave fashion • Requirements refinement (Agile Product Management)* • 5% workshops (aka story time) have not been around in most companies (not for long, anyhow) *K. Vlaanderen, S. Jansen, and S. Brinkkemper, “The agile requirements refinery: applying scrum principles to software product management,” in Proc. 3rd International Workshop on Software Product Management, 2009.

Kristian Rautiainen 29.9.2011 30

Emerging architecture = no upfront architectural work • Working in short iterations may compromise the architecture – ”The best architectures emerge” • Sure thing, as long as you have the best people

– ”Just enough arcitecture” • How do you know how much is enough?

– ”We have these 10 teams working on the release of this product” • ”Architectural runway” is a must

• Architectural work is challenging no matter what you call the development process! Kristian Rautiainen 29.9.2011 31

Anyone Seen the Onsite Expert User? • Working with a simple backlog may cause losing sight of the ”Big Picture” – Most backlogs only contain vague one-liners, not even proper user stories – If the onsite expert is not there to explain things, the end result might be completely off target • The positive thing is that we know it pretty fast, but it is still waste

– How much explanations should be written?

Kristian Rautiainen 29.9.2011 32

Agile Only Concerns R&D • After all, the Product Owner is part of the team and is responsible for all ”that other stuff” – Still, agile (and lean especially) does change the way everybody should be thinking about developing software

• Agile requirements (= backlogs, stories, ...) mean that Business representatives need to work in new ways too! • Customers still want detailed plans and contracts! – Business/sales representatives force the teams to produce detailed plans upfront – Who trains the customers? Kristian Rautiainen 29.9.2011 33

Questions or comments?

Kristian Rautiainen 29.9.2011 34