(Andrew Storms, pictured, is vice president of security services at New Context, a San Francisco consultancy that specializes in lean security and helping companies build better software.)
For much of the history of software development, testing was an adjunct process performed within a separate department. Developers wrote code, compiled it, and then turned it over to a separate quality assurance team to test for bugs.
Fixing those bugs was generally put off until another release cycle, which could take weeks, months or years, depending on the pace of development. This approach worked when releasing software involved burning and shipping physical CDs or posting files for download on a website.
However, the Internet created a revolution. Companies realized they could release software early and include updates that fixed the code during the first run of the program. Modern releases send out new versions of an application within minutes of updating a server, thanks to cloud technology. This speedy change requires a more fluid approach to testing, one that supplements the release cycle instead of acting as a bottleneck.
If software testing is part of your business model, here are five strategies to ensure your tests are working for you and your clients, rather than against you.
1. Implement Test-Driven Development
With test-driven development, programmers write functional tests before they write the code. This way, by the time the code is completed, it produces the expected outcome as described.
That methodology, coupled with a continuous integration system, ensures buggy code will be caught sooner in the development cycle and code quality can be measured.
2. Involve Developers
Developers need to take part in writing automated code-level tests for the software they create, but that doesn't mean quality assurance gets a pass. Sometimes, a second set of eyes can catch a problem that isn't obvious to someone more familiar with the software.
In fact, test-driven development was created with developer involvement in mind. The process lets developers work more productively while ensuring that testing and quality remain priorities. Rather than getting in the way, testing becomes complementary to the development process.
3. Test Continuously
Don’t wait until you release a new application version to test it — test against every change you make within the codebase. Putting in place a good continuous integration process is essential to do this effectively.
When test-driven development and continuous integration combine, they generate immediate feedback when a change introduces a new bug or breaks an existing feature. Most continuous integration servers generate triggers on the basis of test results and alert appropriate parties to any issues.
Developers can also use a “feature flag” so that new code or features are exposed on the basis of an administrative choice. Feature flags let developers turn new features on and off at will, as long as an administrative flag is attached to that feature within the code. Companies such as Netflix, Etsy, and Flickr also use it to test new features on a limited number of users or to test code that is not yet complete.
4. Replicate Production
Test on an environment that is as similar to production as possible. If you run tests on a system that differs significantly from your production setup, some bugs may manifest themselves only after deployment.
Virtualization and cloud technologies are excellent tools in your arsenal. Amazon Web Services makes it easy to clone an environment and then perform tests against the replica. And Delphix provides a suite of tools that permit developers to safely pull and mask production data back into development systems. This gives you the confidence that you won’t introduce breaking changes when you upgrade.
Some companies, such as Netflix, take this a step further by actually running tests within their production infrastructure. One famous example is the Chaos Monkey, an open-source tool Netflix uses to test its production infrastructure resiliency. The tool actually shuts down servers and services randomly with the goal of catching non-redundant services early so they can be redeployed in a resilient manner.
5. Plan For Security
Build security into your testing strategy from the beginning. Testing against production data is good, but cleanse any sensitive values before you put the data in a test environment.
Regularly perform automated penetration testing, preferably every time your continuous integration system runs. Many open-source and proprietary tools are available to help, so there’s no need to develop comprehensive testing tools from scratch.
Using these strategies will ensure you can successfully incorporate testing into your development workflow while maintaining an application’s security and preventing bugs from reaching the end user. Tracking the number of bugs fixed and security issues detected will also help you make a case for increased testing in the future.