The Increasing Importance of Software Testing

Quality Assurance (QA) directly affects customer experience. A report from the Consortium for IT Software Quality (CISQ) found that the cost of poor-quality software in the US in 2018 was $2.84 trillion. And over three-fourths of Americans form a negative opinion of a brand if it has an application that doesn’t perform well. It’s obvious in a software industry now becoming subscription-based (SaaS FTW!) that customers have more freedom of choice. A critical element of sticky SaaS products is quality product and quality support. Bugs lead to frustration, and frustration drives churn. Low quality also impacts employee morale — “customer beatings will continue until morale improves” is not fun for teams. Software testing has become essential. And as we’ve outline in our articles about DevOps & CI/CD, it’s becoming more important to shift left — moving testing earlier in the software development process to save time down the road. The importance of software testing cannot be understated. Technology issues can paralyze customers, so having smart processes in place for issue detection and remedy is not optional anymore.

Types of Software Testing

Software testing can be divided into two steps: verification and validation. Verification ensures that software correctly delivers on functionality, and validation ensures that the software meets customer requirements. Tests can be either manual or (preferably) automated. Manual testers act as an end-user, identifying unexpected behaviors or bugs. Testers typically follow test plans to ensure complete testing. Automated testing uses scripts and specialized software, automating manual processes. Automation testing can use those original manual tests and run them quickly and repeatedly. Regression testing ensures that any new changes do not break existing functionality by re-testing everything from the beginning. Automated tests can also verify load, performance, and stress factors.

We can classify software testing using four categories.

Unit Testing

In unit testing, individual components of the system are tested to validate that they perform as designed.

Integration Testing

Integration testing combines individual units and tests them together to find errors with integrated units.

System Testing

System tests evaluate the complete system’s performance with respect to the specifications and requirements.

User Acceptance Testing (UAT)

User acceptance testing validates acceptability for end users, verifying that the system meets business requirements.

Software Testing with DevOps & CI/CD

With the mass adoption of agile methods over the past twenty years and DevOps over the past ten, QA is moving earlier in the development process. CI/CD pipelines require implementing robust testing practices. Shifting left — including QA as early as possible — enables teams to move faster while still maintaining quality. Finding issues early and ensuring there is a bug fix streamlines processes later. About three-fourths of testers work on scripting and automation. Automated testing improves existing manual testing processes by reducing the time & effort. Automated testing allows for scalability — a critical element for software development companies looking to grow.

CI/CD, DevOps, and agile share and work toward the goal of delivering high quality software at velocity. Automated testing enables continuous testing, which ensures that bugs are found early and fixed before end-users experience the impact. In addition, automated testing in CI/CD pipelines is quite flexible, and teams can address a wide array of tests using CI/CD automation:

Unit testing
Functional testing
API testing
Load testing
UI testing
Regression testing
Browser testing

Also, with DevOps, development, testing and business are no longer siloed. Nearly 50% of software companies use software testers for the vast majority of testing. About a third of companies involve non-testers in the software testing process.

When to Start Software Testing

You should start software testing now. If you’re already using a testing process, good for you. Chances are it could be better, though. If you’re not using a testing process, you should start. No matter what, you’ll be reducing technical debt if you test.

Trust us, we’ve worked on applications that were tested only by end users…and it’s a nightmare when new versions release. Not only does a lack of testing frustrate users, but it impacts your team. The help desk gets barraged with complaints, salespeople lose customers (and probably commissions), developers are stressed trying to repair problems (and hearing complaints from sales!), and management looks at churn reports in dismay.

We highly recommend starting with unit tests. Make a test plan. Automate tests where possible. Manual test too. Every test you add makes your job easier on the next release. Iterate. Improve. Repeat…