Maximizing Test Coverage
We’ve written about test tools and processes before, but we were recently reminded of the importance of maximizing test coverage. Ideally, applications are tested from inception, but we all know that is rarely the case. Testing happens when an application matures, in most cases, when there’s traction and the problems that ensue from having real users. We regularly experiment with testing tools and techniques, looking for better ways to deliver high quality modern applications. We share our insights to help you too…
“Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning.”
― Rick Cook, The Wizardry Compiled
Test Coverage 101
Test coverage is “what are we testing and how well are we testing?” It is the monitoring of test quality to locate missing or unvalidated areas of an application. At a basic level is unit testing, or code coverage. Developers should write tests that target all areas of the code at least once. Test coverage should facilitate testing every requirement at least once. Some teams consider a requirement covered if there is at least one test case against it. Others may consider it covered if at least one team member is assigned to it or if all test cases against it are executed. Three areas should be reviewed: product, risk, and requirements. Product coverage determines what parts of the application have been tested. Risk coverage evaluates risks that the application could face when users get their hands on it. And requirements coverage ensures that the business requirements were fully implemented.
The Big Picture
Software testing is critical in software development and maintenance. An application should do what it is supposed to do, right? We’ve worked in organizations where testing was a “luxury”, and it made release days into nightmares (we appeased unhappy customers with frequent rollbacks and patches). Modern application development makes it easier to create comprehensive testing programs and measure testing completeness and effectiveness.
Test coverage is a job for the quality assurance team (not development). Software test plans guide testing activities using an organized process. Testing should validate as much functionality and accuracy as feasible. The goal of test coverage is to determine how well the application is tested in relation to its requirements. A test plan describes the test strategy, objectives, schedule, deliverables, and resources needed to maximize test coverage.
The first step in a good test plan is unit testing. During the development (coding) phase, deveiopers test individual components to validate that the code performs as expected. A unit may be an individual function, method, procedure, module, or object. It is first level of testing done before integration testing.
After unit testing comes integration testing where modules are integrated logically and tested as a group. Since systems typically have modules written by different programmers, integrating and testing can expose defects in the interaction between integrated software modules. Integration testing checks data communication between modules.
Most applications require manual testing before it is possible to automate any testing. A tester manually performs test cases to identify any issues or defects in the application. It requires more effort but helps determine the feasibility of automation.
Cross Browser Testing
Cross Browser Testing verifies that an application works across different browsers. Each browser — Safari, Edge, Chrome, Firefox — interprets display information differently. Some browsers lack the features that an application relies on so it appears broken on that browser.
User Acceptance Testing
The final testing phase before go-live is User Acceptance Testing (UAT). UAT helps determine if the application or module performs as designed according to the organization’s business requirements.
Manual testing requires considerable time and effort to ensure the software performs as expected. Each tester is responsible for verifying the functionality of the software and documenting issues. Testing with an automation tool helps increase test coverage and scalability. Automated testing is best for large projects, projects that require testing the same areas over and over, and projects that have been through manual testing. Automated testing strategy is most beneficial in modern software development.
Maximizing Test Coverage
Ensuring solid test coverage is important if you want to deliver software with a pleasant user experience. Although it’s daunting to implement testing later in a software project’s lifecycle, it’s never too late. You’re probably manually testing something already, so the culture change is systematizing how you do it. Use unit testing for any new development and add unit tests when you update older modules. Use something like Selenium to automate user acceptance tests (write once, run many times!). Over time, your test coverage will improve, and with it, your product.