MDD and its impact on testing

MDD and its impact on testing... A nanotech case study Bryan Bakker EuroSTAR virtual conference May 2012 Contents     Intro ASD MDD and Test...
2 downloads 3 Views 953KB Size
MDD and its impact on testing... A nanotech case study

Bryan Bakker

EuroSTAR virtual conference May 2012

Contents

   

Intro ASD MDD and Testing Experiences

© Sioux 2012 | Confidential | 2

About Bryan Bakker

    

Test Expert Certifications: ISTQB, TMap, Prince2 Member of ISTQB Expert Level on Test Automation Tutor of several test related courses Domains: medical systems, professional security systems, semi-industry, electron microscopy  Specialties: test automation, integration testing, design for testability, reliability testing

© Sioux 2012 | Confidential | 3

About Sioux

UTRECHT

MOSCOW

EINDHOVEN HERENTALS

NEDERWEERT

© Sioux 2012 | Confidential | 4

About FEI

 World leader in electron microscopes  Light microscope: 1000x  200nm (limited by the wavelength of light)  Electron microscope: 4Mx  0.05 nm  Nm = a billionth of a meter (10-9 meter)

© Sioux 2012 | Confidential | 5

About FEI

Breast cancer cell. Magnification 5.000x

Salmonella bacteria. Magnification 80.000x

Atomic structur of Ge (Germanium). Distance is 0.5 nm

2 nm

0. 5 nm

© Sioux 2012 | Confidential | 6

ASD (Analytical Software Design)    

Model Driven Development technology Component Based Development Models are verified mathematically at design time Extensive model checker:  Deadlock  Live-lock  Starvation  Race conditions, etc

© Sioux 2012 | Confidential | 7

ASD Model ASD Model: - Precise - Complete - Correct

© Sioux 2012 | Confidential | 8

ASD Model ASD Model: - Precise - Complete - Correct

Design errors

Generate formal model

Formal model and verification

© Sioux 2012 | Confidential | 9

ASD Model ASD Model: - Precise - Complete - Correct

Design errors Generate defect free source code from verified model

Generate formal model

Formal model and verification

Guaranteed equivalence

Source code: - Java - MISCRA C - C++ - C#

© Sioux 2012 | Confidential | 10

What is covered?  Interface model:  Specification of interfaces of components  “What”  Design model:  Implementation of interfaces  Internal implementation  “How”  Also the outside interfaces are specified  Interfaces to not-modelled components © Sioux 2012 | Confidential | 11

ASD  Typically used to model behaviour described in:  State machines  Sequence diagrams  Algorithms  What is modelled?  All normal functionality (good weather)  All exceptional functionality (bad weather)  All illegal behaviour

© Sioux 2012 | Confidential | 12

MDD and Testing Wish, need Release

User Requirements

Acceptance Test

System Requirements

System Test

Architecture

Integration Test

Formal methods Design

Component Test

Implementation © Sioux 2012 | Confidential | 13

MDD and Testing Wish, need Release

 Formal methods typically:

User Requirements

Acceptance Test

System Requirements

System Test

Architecture

 Applied at Design level  Only in Software  For part of the software

Integration Test

Formal methods Design

Component Test

Implementation

© Sioux 2012 | Confidential | 14

MDD and Testing Wish, need

 Part of the component tests are not needed anymore (Software Engineer)  Test activities of Test Engineer:  From integration level and up  Are still needed  But:  MDD has impact on quality of deliverables

Release

User Requirements

Acceptance Test

System Requirements

System Test

Architecture

Design

Integration Test

Formal methods Component Test

Implementation

© Sioux 2012 | Confidential | 15

Testing experiences    

Testing is still necessary Integration (with other SW, with HW) System testing Model incorrect  Rubbish in, rubbish out New insights, so model will change  Functionality must still be verified (and errors are found)  Programming errors or integration issues:  In handwritten code  On boundary between generated and handwritten code (hard to analyse)  Not in generated code! © Sioux 2012 | Confidential | 16

Testing experiences  Reliability of generated code: very high!  But crashes can still occur... (but not in generated code)  Reliability tests  HW failures/wear detected much earlier  Integration and system test  not hampered by reliability issues  big improvement, less cycles

© Sioux 2012 | Confidential | 17

 Likelihood

Product Risk Analysis 15 Comp1

9 Comp2

Comp3

3 3

9

15  Impact

© Sioux 2012 | Confidential | 18

 Likelihood

Product Risk Analysis 15Could test

Must test Comp1

III

I

9

IV

Comp2

3 “Won´t test” 3

Comp3

9

II

Should test 15  Impact © Sioux 2012 | Confidential | 19

 Likelihood

Product Risk Analysis 15 Comp1

9 Comp2

Comp3

3 3

9

15  Impact

© Sioux 2012 | Confidential | 20

15

ASD

 Likelihood

Product Risk Analysis

9 Comp1 Comp2

Comp3

3 3

9

15  Impact

© Sioux 2012 | Confidential | 21

Conclusions  Component tests (white box) could be skipped  Internal working is correct  Although functionality can be incorrect  Integration and system testing a lot smoother  Less interface issues  Less reliability issues  Reliability testing  reliability measurement  Integration and system test more efficient (less cycles)  Architecture and design takes more effort  More re-designs with ASD are planned © Sioux 2012 | Confidential | 22

Questions

© Sioux 2012 | Confidential | 23

www.sioux.eu +31 (0)40 26 77 100 [email protected] © Sioux 2012 | Confidential | 24