6-5 Initiation of product development at software level
ISO 26262-6 Product development at Software level
2
6-6 Specification of software safety requirements
6-7 Software architectural design
6-8 Software unit design and implementation
Software testing
Software testing
BTC Embedded Systems AG proprietary · all rights reserved
6-10 Software integration and testing
6-9 Software unit testing
Overview
ISO recommandations for Unit testing •
Req based testing methods •
Functional and non-functional tests
•
One-stop solution for testing code and model with BTC EmbeddedTester
•
Coverage analysis •
Structural coverage model vs. code
•
Requirements coverage
•
Back 2 Back testing methods •
Enhanced Back-to-Back testing with automatic test generation
•
Automated test sequences and extended use cases for Back-2-Back testing with BTC EmbeddedTester
BTC Embedded Systems AG proprietary · all rights reserved
Methods for Unit testing
Methods
ASIL A
ASIL B
ASIL C
ASIL D
BTC Solutions
1a Requirement-based test
++
++
++
++
1b Interface test
++
++
++
++
1c Fault injection test
+
+
+
++
1d Resource usage test
+
+
+
++
1e Back-to-back comparison test between model and code
+
+
++
++
Table 10 — Methods for software unit testing Methods
ASIL A
ASIL B
ASIL C
ASIL D
BTC Solutions
1a Analysis of requirements
++
++
++
++
1b Generation and analysis of equivalence classes
+
++
++
++
1c Analysis of boundary values
+
++
++
++
1d Error guessing
+
+
+
+
Table 11 — Methods for deriving test cases for software unit testing
4
BTC Embedded Systems AG proprietary · all rights reserved
“1a Requirement-based Test” with BTC EmbeddedTester
What is Requirement-based testing? Requirements-based testing, sometimes also called functional testing, is a quality assurance (QA) process that bases its test cases on the specifications of the software unit. What is the goal of requirements-based testing? Understand the functionality and causality of all use cases addressed by the requirements to define and apply tests cases and examine expected behavior How to apply this method using BTC EmbeddedTester Test Vector Editor
5
•
Analyze software requirements
•
Create test cases by •
providing values and/or signals on the Inputs
•
providing a verdict for an automated test assessment
•
Link the test cases to requirements for traceability
•
Automatically run the tests
BTC Embedded Systems AG proprietary · all rights reserved
Elements of a test case in BTC EmbeddedTester
•
•
•
6
Input Values •
Manual value definition
•
Signal generators
•
Import of specific signals
Calibration Values •
Each testcase can have its own set of calibration data
•
Manual value definition or import
Output/Display Values (verdict) •
Important for automation
•
Manual value definition or signal generators
•
“don’t care” values
•
Time- and Value-tolerances
•
Verdict functions
BTC Embedded Systems AG proprietary · all rights reserved
“1a Requirement-based Test” with BTC EmbeddedTester
DOORS/PTC Integrity/SYNECT/Excel/Word
Import Requirements Write Test Results
Debugging:
BTC EmbeddedTester
•
Simulink
•
Test Results
•
TargetLink
•
Requirements Coverage
•
Visual Studio
•
Model Coverage
•
Code Coverage
Execute Test Vectors Import Test Architecture
MIL 7
Reporting:
.c/.h
.obj
SIL
PIL
BTC Embedded Systems AG proprietary · all rights reserved
“1a Requirement-based Test” with BTC EmbeddedTester
BTC EmbeddedTester - Test Vector Editor supports Requirement-based test 1 – Import requirements into test project
2 – Create requirement-based test cases
8
BTC Embedded Systems AG proprietary · all rights reserved
3 – Execute tests, generate test results report and requirement coverage report
DEMO
9
BTC Embedded Systems AG proprietary · all rights reserved
“1b Interface Test” with BTC EmbeddedTester
What is Interface test? At software unit level, Interface tests are conducted by testing the domains of values taken by the interfaces and their boundary values. What is the goal of Interface test? Based on Interfaces specifications, this test will ensure the robustness of interfaces to prevent errors.
How to apply Interface test? A – Apply Interface test using Equivalence Classes B – Apply Interface test using Boundary Value test
10
BTC Embedded Systems AG proprietary · all rights reserved
“1b Interface Test” with BTC EmbeddedTester
A - How to apply Interface test using Equivalence Classes? Traditional manual approach •
Define the equivalence partitions
•
Assign values manually for each partition
•
Measure domain coverage of the Inputs and Outputs
Time consuming
Automatic approach with BTC EmbeddedTester • •
Assisted generation of equivalence classes for Inputs AND Outputs Automatic test vectors generation can help to find appropriate scenarios
Automated and exhaustive Formal proof of unreachable partition
11
BTC Embedded Systems AG proprietary · all rights reserved
“1b Interface Test” with BTC EmbeddedTester
BTC EmbeddedTester – Domain Coverage Observer supports Equivalence Classes 1- Assisted definition of range partitions
2- Automatic generation of test cases
12
3- Analisys of domain coverage report
BTC Embedded Systems AG proprietary · all rights reserved
“1b Interface Test” with BTC EmbeddedTester
B – How to apply Interface test using Boundary Value test? Traditional manual boundary value testing approach • • •
For each Inputs and Calibrations, Apply values crossing the boundary values, With the combination of other Inputs and Calibrations
Test numbers increases exponentially Exhaustively not guaranty Time consuming
Automatic recursive approach with BTC EmbeddedTester Prove with a formal method the non-violation of Outputs to guaranty the robustness of the next unit Inputs • •
Automatic creation of range non-violation goals Formal-analysis of software unit to fulfil the non-violation goals
Pragmatic and automatic approach Formal proof of Interfaces robustness 13
BTC Embedded Systems AG proprietary · all rights reserved
BTC Embedded Systems AG proprietary · all rights reserved
DEMO
15
BTC Embedded Systems AG proprietary · all rights reserved
“1c Fault injection test” with BTC EmbeddedTester
What is Fault injection test? This method is used to voluntary introduce faults in order to test the safety mechanisms implemented in the unit What is the goal of Fault injection test? This test will guaranty the unit behaves correct when errors occur How to apply this methods ?
Fault injection test can be conducted by applying faulty values to the inputs of the SUT The expected output behavior needs to be defined
16
BTC Embedded Systems AG proprietary · all rights reserved
“1c Fault injection test” with BTC EmbeddedTester
BTC EmbeddedTester supports Fault-injection test: Edit fault test cases and analyze the results using Verdict Functions in Test Vector Editor
17
BTC Embedded Systems AG proprietary · all rights reserved
DEMO
18
BTC Embedded Systems AG proprietary · all rights reserved
“1d Resource usage test” with BTC EmbeddedTester
What is Resource usage test? This method is applied when the software unit tests are executed on the target hardware to estimate resource usage
What is the goal of Resource usage test? Measure and make sure the hardware has sufficient resources to run the software units How to apply this methods ? Run all test cases and measure: •
RAM/Stack consumption: Call of all nested functions on same run
•
CPU: Apply decisions that activate the paths that most consuming part of the algorithm
•
ROM: Activate all possible configured part of the software (pre-processor selection) to the measure the biggest size of code and data memory
BTC EmbeddedTester supports the PIL mode simulation to measure Stack and CPU load and generate the report
19
BTC Embedded Systems AG proprietary · all rights reserved
Error guessing method
Methods
ASIL A
ASIL B
ASIL C
ASIL D
BTC Solutions
1a Analysis of requirements
++
++
++
++
1b Generation and analysis of equivalence classes
+
++
++
++
1c Analysis of boundary values
+
++
++
++
1d Error guessing
+
+
+
+
What is Error Guessing for deriving test cases? Collection of use cases based on knowledge and history of the developed system issued from a prior development experience or through a domain-specific expertise Existing test data can be imported into testing environment to ensure the error-proof All BTC EmbeddedTester available testing functionalities can be used to apply these tests
20
BTC Embedded Systems AG proprietary · all rights reserved
