Early testing saves costs

In the initial phase and sprint planning, the acceptance criteria are defined, i.e. it is worked out what is to be tested in accordance with the requirements. These acceptance criteria form the basis for the tests to be carried out. In each new sprint, the criteria and the tests are supplemented accordingly. Test objectives and possible test ideas are summarised in the test charter.

The sprint then starts and all relevant tests are prepared, i.e. the test data and the test environment are provided and the tests are then carried out. This happens in parallel with the development of the features. In the daily Scrums, test blockers or test obstacles are discussed with the Scrum team. Errors found during the tests are rectified immediately.

The developers create unit tests and execute them on the basis of test-driven development, a core technique of extreme programming. The tests are consistently written before the components that you actually want to test (test-first). By refactoring in good time, the focus is on the essentials and no unnecessary code is created in advance. The code and test cases are only checked in once the local unit tests have been successfully completed, thus ensuring that everything works perfectly before it ends up in a build. All code is integrated into the build and tested together. This happens several times a day in a “daily build”, which is automatically tested in the continuous delivery pipeline before release.

A continuous delivery pipeline comprises continuous integration, delivery and deployment and is often referred to as a CI/CD pipeline. The CI/CD pipeline is part of a larger tool chain that includes automated testing and version management. Continuous delivery pipelines move the application code from the developer to the user, and test automation is the basis of all modern delivery pipelines. After the successful unit tests, the developer checks in the new/updated code and a series of automated tests starts. This should include integration tests as well as system tests with functional and non-functional tests that check, for example, performance, networking, external APIs, dependencies and security. This prevents unstable, risky or insecure code from being published and issued to the user. The automated build process only starts once these tests have been successfully completed. The non-functional tests in particular have a long runtime, so it makes sense not to schedule all tests for every daily build, but to determine a selection of them. All more complex tests can be carried out weekly or before a specific release, for example. The same applies to the system integration test, which should also only be tested when necessary – for example, when connecting a new interface to the system.

In the sprint before a release, the system tests take place in a production environment, possibly with an acceptance test by the specialist department. Once these tests have been successfully accepted, the product is handed over to DevOps (Development Operations) or ITIL (Information Technology Infrastructure Library) for release into the production environment. Central units provide cross-team services with basic tasks for the test, such as the provision of test environments and test data, the maintenance of test tools or the connection to other applications.

Of course, there are also significant differences in test automation in the agile approach compared to the classic approach. In classic projects, which are structured according to the waterfall or V-model, an extensive test phase is usually only required at the end of the project. In agile projects, on the other hand, the software must be subjected to a test phase and quality control after each sprint.

Unlike the sequential approaches mentioned above, the introduction of automated tests is a prerequisite for a successful project in agile projects. As the software continues to grow after each sprint, the number of tests and the testing effort also increases. In order to ensure the greatest possible test coverage of the software over the course of the project, the development of a framework for test automation and a CI/CD pipeline must be planned right from the start. Even if the test scope is smaller at the beginning of the project and the tests are limited to the lower test levels for component and module tests, the framework criteria defined in advance provide the basis for a successful implementation.

The fact that an agile project is not feasible without test automation becomes clear at the system test stage at the latest. In each sprint, not only are the new functions checked in the system test, but all previous functionalities must also be taken into account as a regression test. While developers might still be able to carry out a module test after each sprint, manual system testing would go beyond the scope of an agile project in terms of effort, time and budget. The developed software increases in functionality and complexity with each sprint – but the number and complexity of the tests, especially the system tests, increases to the same extent. This testing effort can be significantly reduced through test automation and a functioning CI/CD pipeline.

Employees usually don’t have enough time for manual system testing after each sprint. At the end of an agile project without test automation, errors could suddenly and unexpectedly occur. It is therefore all the more important in agile projects to implement an automated and structured test framework right from the start, which grows together with the software. And don’t forget: the earlier testing takes place, the earlier possible errors can be rectified and the lower the costs of the entire development process.