This guest blog post is part of an Atlassian blog series raising awareness about testing innovation within the QA community. You can find the other posts in this series under the QA Innovation tag.
Intro
It is becoming more common for software applications to be written using web technologies, and for users to want to access them from anywhere, using an internet connection. Security of browser-based applications is very different from how things work with traditional thick-client architecture. There are far fewer boundaries between different web sites inside the browser than between different pieces of code that run on your computer under the control of the operating system. Security testing is therefore a very important part of testing web applications, which means that these skills are growing in demand for QA teams. Even for an experienced tester, web application security can seem daunting. How do you start building up these skills? Where can you turn to for more information?
In fact, security testing is in many ways similar to functional testing. You identify a risk, define what the expected behaviour should be, and then perform some testing to mitigate that risk by demonstrating that the unexpected does not happen. For example, say the system under test is an internet-facing web application, backed by a database. A risk could be that an attacker somewhere on the internet could use the front-end and gain access to sensitive data stored in the back-end (this is called SQL injection). The expected behaviour in this case is that the application will not let this happen – user input will not be directly pasted into an SQL statement that is executing on the database. To test this, you may try manually entering strings that you suspect might confuse the application into executing your commands, or use an automated tool to do this for you, or perform a code inspection to see how an input string will be treated.
The main difference when security testing is one of mindset. When functional testing, you are trying to prove that a feature works for an end-user – it does what they expect, and does not hinder them from completing their tasks. You would probably prioritise accordingly – focus on features that are used more often, used by more users, are considered the most important, etc. As a security tester, your ‘end-user’ is now an attacker trying to break your application. The goal of your testing is to prove that a specific attack scenario does not succeed, for any attack scenario. A significant difficulty here is that proving that a feature works is much easier than proving that a specific feature cannot be hacked by any method. You could use a similar prioritising approach as with functional testing – test only a set of most likely or simplest or most popular attacks for each feature. Another point to note is that popular developer responses to bug reports such as “a user would never do that” and “won’t fix – feature is hardly ever used” are simply not valid when security issues are involved – a potential attacker can do anything they like to perform a successful attack.
In this post, I will outline some tips for building up team skills in security testing.
Basics
1. Understand your own application
It is important to be familiar with the application you are testing so that you can assess where the risks are. Everything else will assume that you have this knowledge – the technologies used by the application, the profile of different users, the abilities you should and shouldn’t have with different levels of access, and the potential data that is stored by the application. The testing you would do is very different for a website that simply displays pictures of cats over the internet to anonymous visitors, versus one which sells pictures of cats to logged-in users who need to enter their credit card details.
2. Understand security terms and definitions
OWASP is a great source for this. The volume of terms and concepts might be overwhelming at first, so just concentrate on understanding some of the terms, preferably the ones most likely to apply to your application. Examples may be XSS, XSRF, SQL injection and path traversal. The CWE/SANS Top 25 lists the most widespread and critical errors that cause vulnerabilities. For an exhaustive list of all known attack methods check out CAPEC.
Build Knowledge
3. Use online training tools
A great way to start learning is to start testing an application which has known vulnerabilities, where you are provided with guidance on how to find them. My preference is for Google’s Gruyere which has separate lessons to cover each concept. You can look at hints to help you find the vulnerability, and the answers if necessary.
Some other options are OWASP’s WebGoat and Damn Vulnerable Web App.
4. Learn from others
It is likely that among the developers in your company, there will be some with knowledge of security topics. Ask them to pair with you to investigate the application behaviour. They should be able to demonstrate, for example, that a SQL injection string is not executed on the database server, and why it is not. If it is, then that will be educational for you both. They can also explain to you the design of the application and how it is intended to protect from attacks. If there are many people wanting to learn about security, get them to give a presentation.
5. Learn to use an automated vulnerability scanner
A good commercial option is Burp Scanner; there are also free options such as OWASP’s ZAP and Google’s RatProxy. These work by routing the HTTP traffic to and from an application through a proxy, and then resending the requests with various attack attempts replacing the original values. This can be an effective way of finding certain classes of vulnerability in a short amount of time, but it is important to understand (and make sure that your stakeholders understand) that this is not a magic bullet. The tool is naive, and has no knowledge of the applications business logic – it is simply replaying requests and checking the responses. There are many types of vulnerability that can not and will not be found with this strategy, and use of a scanning tool absolutely does not replace the need for manual security testing.
Automated tools, even expensive ones, find only relatively simple vulnerabilities and they usually come up with a lot of “noise”, or false positives. You need to know enough about security vulnerabilities to be able to evaluate each finding of the automated tool. Taking a scanner report and sending it unverified to the developers is the worst possible thing one could do.
Share Knowledge
6. Pass on what you are learning
As you start to build up knowledge, make sure that others also benefit from it. Give a presentation on some of the basic security concepts. Run a class about how to use an automated scanner. Both developers and testers can learn from you, and you will cement your own grasp on the topics.
7. Convince people that security is important
You may work with individuals who don’t know or don’t care about security issues – perhaps they are new graduates, or have previously worked in places where the software was firewall-protected. It is worth raising their awareness – remind them of the backlash against some big-name companies that have lost user-data. When your testing finds a vulnerability in an application, make sure you demo it, along with the potential exploits that can follow. A good tool to demo is BeEF – which shows just how much power a simple XSS vulnerability can give you over another user and their browser.
8. Communicate security issues in context
It is important that you evaluate all security vulnerabilities you discover in the context of your application. A cross site scripting vulnerability that is only exploitable in obscure conditions is much less important that a vulnerability allowing someone to run any code on your web server. You may want to establish a scoring system for vulnerabilities you find. One of popular scoring approaches is CVSS. In addition to scoring, consider the business context – what happens if the attack succeeds? Losing pictures of your cats is of less impact (generally speaking) than someone tampering with company’s business records. If you need to prioritise what should be fixed, prioritising based on impact usually works better.
Maintain
9. Use good default test data
When testing a feature, you will probably be creating test data. Instead of using ‘test1’, ‘test2’, etc. or cartoon character names, get into the habit of using attack strings. This way, you’ll find you come across vulnerabilities almost by accident, just when using a feature. If you have an automated tool or import file providing the test data, do the same thing. You can share such data with other testers and developers, meaning they may come across issues without even knowing they are doing security tests.
10. Practice, practice, practice
Like any skill, you will get better with practice. As you start to find vulnerabilities in an application, you’ll start to get a feel for where they are likely to be in future, and will be able to raise them further in advance. Running regular scans against the code will mean you become more effective at using the scanner. Participate in code reviews and you can start pointing out where vulnerabilities are likely to be before even using the application.
11. Use automation when appropriate
Consider whether automation would help in security testing. You can often reuse existing functional tests for such a purpose.
Improve
12. Read a book
There are a number of good books about web application security. The recent ones are Web Application Hacker Handbook 2nd ed by the creator of Burp scanner Dafydd Stuttard and The Tangled Web: A Guide to Securing Modern Web Applications by Google’s Michal Zalewski.
13. External training
This post covers the basics of getting a team started with security testing. There is plenty more to know – and a wealth of online resources to help. You may decide that more focused training would help, like various courses by providers such as SANS. There are few security training courses specifically for QA people, so look for security courses for web developers instead. So-called “penetration testing” courses tend to focus on network hacking, but they often do have parts dedicated to breaking into web applications, so check the course’s content in advance.
Learn more about software testing and its role in continuous delivery below!