Software Testing Days at TUT, August 14–15, 2006

Acceptance Testing In An Agile Product Development Project Juha Inkinen, SW Engineer, Nokia Networks

1

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Context • Project management framework Scrum [6] used in the project • Some Scrum terminology • Product Owner = Customer representative • Scrum Team = A cross-functional development team • Scrum Master = The ”servant-leader” of a Scrum Team, responsible for removing obstacles and promoting Scrum practices in the team. Can also be a team member.

• Practices from Extreme Programming (XP) [1] used in software development • Continuous Integration • Pair-Programming • Common Code Ownership • Test-Driven Development • Acceptance Testing

2

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Acceptance Testing – What? • Defining the requirements of a product with tests • High-level (functional) tests • Acceptance tests are automated per definition • ”Customer Testing” in XP • Testing done by the customer is part of the XP testing strategy • Closely related to the Test-Driven Development ideology

• Acceptance testing is not … • V-model Acceptance Testing • “Plain old” test automation • Manual, as mentioned above

3

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Acceptance Testing – Why? • Digging for requirements [3] • Forms a contract between the Product Owner and the Scrum Team • Controls the development of the product • Provides visibility of the project’s state to stakeholders • Metric for tracking progress: Running Tested Features [4]

RTF

Time

4

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Acceptance Testing – How? • Domain Specific Languages (DSL) • Parser generators (JavaCC, Yacc...) • Ruby programming language

• Specialized testing frameworks • Only build a framework once you have at least three applications... [5] • Build one on top of e.g. JUnit

• Exactor (Java) • Framework for Integrated Tests (Fit)

5

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Acceptance Testing with Fit and FitNesse • Fit is an acceptance testing tool [2] • “Fit is a tool for enhancing collaboration in software development”

• FitNesse is a wiki for using it • User Stories [1] or use cases can be stored in FitNesse with the tests • Traceability • Visibility

• Tests can be run from the wiki

6

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Fit(Nesse) and Fixtures • Fit needs hooks to the software under test; these hooks are called fixtures • Think of a fixture as sort of a view/controller or user interface • Fixture translates the text in a table to function calls in the software Test table 1

: Fixture1

Test table 2

: Fixture2

: System Under Test

7

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Test table 3

: Fixture3

Creating Fixtures • Start designing and writing the software and the fixtures simultaneously • Those designing the fixtures need to understand the software architecture and design decisions just as well as the other developers • In fact, fixtures should be developed by the developers, not testers • Everyone contributes to the design of the software, only the point of view may differ • Fixtures cannot be written in isolation • Although it may be possible, fixtures should avoid accessing functions/functionality that are not normally available to the user interface

8

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Conclusions • Acceptance testing is hard work… • Steep learning curve • Instructions on how to do it are still scarce and scattered • Immature tools (Fit is an exception to this) • Requires a paradigm shift for both the customer and the developers • Tests always need to reflect the current situation

• …but worth the trouble • Better and deeper understanding and consensus of requirements • Accurate knowledge of the project’s progress all the time • Required features are implemented – no effort wasted on predictions and guessing • Up-to-date requirements (acceptance tests) result in up-to-date implementation • Testability and architectural issues are brought to surface early • Supports the notion of Test-Driven Development on a high (feature) level

9

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

Contact Information

Juha Inkinen +358 50 486 8493 [email protected]

10

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI

References [1]

Beck, K., Extreme Programming Explained: Embrace Change

[2]

Cunningham, W., Framework for Integrated Test, http://fit.c2.com/

[3]

Hunt A., Thomas D., The Pragmatic Programmer

[4]

Jeffries, R., A Metric Leading to Agility, http://www.xprogramming.com/xpmag/jatRtsMetric.htm

[5]

Johnson R., Roberts D., Evolving Frameworks

[6]

Schwaber, K., Agile Project Management with Scrum

General References/Good Reading Crispin, L. & House, T., Testing Extreme Programming Cunningham, W. & Mugridge R., Fit for Developing Software

11

© 2006 Nokia

AcceptanceTesting.ppt / 2006-08-15 / JI