Push Your Functional Testing Further into
Technology and
Security Alan Richardson EvilTester.com @EvilTester
Part the first, wherein we describe Functional Testing in terms of Systems and Models, and expand A model of testing to include Technical Testing.
@EvilTester
2
Functional Testing ●
Testers learn how to test systems –
Requirements
–
'What' a system 'should' do
@EvilTester
3
Systems ●
System under Development
●
System of Development
@EvilTester
4
System Under Development ●
System under Development –
Requirements
–
Architecture
–
Environments
–
etc.
@EvilTester
5
System Of Development ●
Methodological Context
●
Social Constructs
●
Model Different Systems of Testing –
Systems of Feedback
–
Systems of Learning
–
Systems of Questioning
–
...
@EvilTester
6
Requirement Example – RestMud
@EvilTester
7
A Model of Testing ●
Modelling,
●
Observation,
●
Intent,
●
Reflection,
●
Manipulation
@EvilTester
8
We can push our functional testing further ●
“What is it supposed to do?” –
vs “What does it do?”
●
Comparison to other models
●
Is it viable?
●
Precondition analysis
●
Presupposition analysis @EvilTester
9
We can push our functional testing further ●
Explore 'How' the system does what it does
●
Understand the technology used to build the system –
Identify technology risks
–
Identify risks at different levels of the stack
–
Work at different levels of the stack
@EvilTester
10
A Model of Technical Testing ●
Modelling,
●
Observation,
●
Interrogation,
●
Reflection (includes intent),
●
Manipulation
@EvilTester
11
Part the second, wherein two technical models are provided with a discussion of possible technical testing approaches.
@EvilTester
12
Example – a Java App ●
HouseOfTest.se –
/2016/02/testers-contest-crappy-little-datagenerator/
@EvilTester
13
Observation Example – a Java App ●
Double click run – see GUI
●
tail -f DataGeneration.txt
●
Text Editor
●
–
Line endings
–
Refresh
No exceptions shown
@EvilTester
14
Observation Example – a Java App ●
java -jar crappy_little_datagenerator_v_1.0.jar –
Now can see stdout written to command line, exceptions, errors etc.
@EvilTester
15
Interrogation Example – a Java App ●
Decompile e.g. http://jd.benow.ca/
●
Load .jar as a library into a Project
@EvilTester
16
Interrogation Example – a Java App
@EvilTester
17
Observe - Multiple Entry Points
@EvilTester
18
Modelling Example – a Java App
GUI
DataGeneration
CLI
@EvilTester
19
Manipulation Example – a Java App
@EvilTester
20
Manipulate Example – a Java App
@EvilTester
21
Risks ●
Does this test approach add risk? –
Because testing at a lower level in the stack?
–
Because working against a non-deployed version?
–
Because it is not how the user would run it?
@EvilTester
22
Risk ●
Mitigating risk provides one reason for testing
●
No test approach mitigates all risk
●
Multiple test approaches required
@EvilTester
23
Modelling – a Web App Browser
Web Server App Server
HTML GUI Web App
@EvilTester
24
Observe & Interrogate – a Web App HTTP via Proxies
Web Server App Server
Web App HTTP via Dev Tools
HTML & DOM
View Source Dev Tools @EvilTester
25
Manipulate – a Web App HTTP via Proxies
Web Server App Server
Web App
HTML & DOM
Dev Tools @EvilTester
26
Modelling – this Web App Player GUI Web Server App Server
Web App
Rest API Admin GUI
@EvilTester
27
Part the third, wherein the overlap between technical testing and security testing are explored.
@EvilTester
28
Deeper Testing
@EvilTester
29
Technical Testing ●
We will find defects and issues we would otherwise miss –
●
Observing, Interrogating, Manipulating lower
Some defects normally associated with security testing –
I am not a security tester
–
Technically Informed Modelling
@EvilTester
30
Security Overlap Examples ●
Code reviews find hard-coded security issues
●
Form field inspection exposed emails, executable shell code, file names for survey answers
●
HTML Commented URLs to 'secret' parts of the application
●
User HTTP json contains more info than displayed as HTML @EvilTester
31
Security Overlap Examples Because... ●
Code reviews find hard-coded security issues Observed More Deeply
●
Form field inspection exposed emails, executable shell code, file names for survey Interrogated More Deeply answers
●
HTML Commented URLs to 'secret' parts of the Interrogated More Deeply application
●
UserObserved, HTTP json Interrogated, contains more Manipulated info than displayed as HTML More Deeply
Because Modelled More Deeply @EvilTester
32
Any Methodology : Any Tester ●
●
Methodology context does not dictate –
'Process' context might dictate
–
'Social' context might dictate
Any Tester can do this –
Limited by technology knowledge
–
Limited by technical skill
–
Limited by choice
@EvilTester
33
Part the fourth, wherein the steps to increase technical ability are made lay afore the public.
@EvilTester
34
I am fairly Technical ●
Books
●
SeleniumSimplified.com
●
EvilTester.com
●
JavaForTesters.com
●
Online Training Courses
●
Consultancy, work hands on with teams
@EvilTester
35
I grew up with computers...
http://www.retrogamer.net/profiles/hardware/zx-spectrum-hardware-profile/ @EvilTester
36
I grew up reading computer books...
@EvilTester
http://www.usborne.com/catalogue/feature-page/computer-and-coding-books.aspx
37
...but not with the internet
http://www.worldofspectrum.org/hardware/feat24.html @EvilTester
38
I had to learn how to test the web
@EvilTester
39
How to learn to test the web ●
Model What You Know –
●
HTML? HTTP? Browsers?
Increase your ability to Observe at the GUI –
View Source
–
Inspect Element - Dev Tools
–
You will see things you don't understand (add to your model & research)
@EvilTester
40
How to learn to test the web ●
●
Increase your ability to Manipulate at the GUI –
Inspect Element - Dev Tools
–
Amend DOM prior to submitting a form
–
Inspect and manipulate URLs
Cookies, Local Storage –
Inspect
–
Figure out how to manipulate (plugins required?)
@EvilTester
41
Basic Web Challenges ●
View Source and inspect Element of: –
Your favourite web sites ●
●
Any free 'pdf' report that requires 'email' –
●
How do they do 'that'?
find the download without adding your email
Newspapers - 'you have read too many articles today' – how can they tell? Manipulate to bypass? @EvilTester
42
How to learn to test the web ●
Observe HTTP Traffic in Browser –
●
Network tab in Dev Tools
Observe HTTP Traffic outside Browser –
HTTP Proxy
–
Fiddler, Charles, BurpSuite, Owasp ZAP
–
Interrogate and Manipulate Traffic with a Proxy
@EvilTester
43
How to learn to test the web ●
●
Learn features in the browser –
View Source, Users, Dev Tools
–
How can the feature help you test?
Learn features in the proxies –
Replay Message, Fuzzers, Auto Responders
–
How can the feature help you test?
@EvilTester
44
Technology Basics ●
Model the Technology –
Where are the gaps in your understanding?
–
These gaps are risks to your testing.
●
How can I observe X?
●
How can I interrogate X?
●
How can I manipulate X?
●
Repeat
@EvilTester
45
Application Basics ●
Model the application
●
What is it actually doing? –
●
Not just what is it supposed to do
How does it do X? –
Observe, Interrogate, Manipulate
@EvilTester
46
Going Further
@EvilTester
47
Even Pushing Functional Testing Further Go Even Further: Explore and automate systems below the GUI ● GUI as API ● Quickly enter more combinations of input than would otherwise be feasible ● Fuzzers, Setup Test Data ● Test at an API level without specialist tools ● Proxy message creation ●
@EvilTester
48
Do the work! ●
Research
●
Experiment
●
Learn
●
Take Small Steps
@EvilTester
49
Do The Work! ●
Alan Richardson –
@EvilTester
–
www.EvilTester.com
–
www.JavaForTester.com
–
www.SeleniumSimplified.com
–
www.CompendiumDev.co.uk
@EvilTester
50
