Nowadays more and more test activities are being automated, especially when it comes to executing test cases. But what about the development of test cases? Are the test cases still being developed in a structured way with a certain test coverage and based on a well-considered risk? When there is a change in the test basis, are the test cases easy to maintain? If we are mainly focused on automating the test execution then what about the Shift-left testing approach? I believe that with Model Based Testing (MBT) we already have the answer to many of these questions. In this blog I would like to share my vision to you on this.
Model Based Testing
It is well known that Model Based Testing (MBT) has many advantages over other test approaches and that MBT offers a solution to automate the developing of test cases. This is very important because in the present there is no time to manually set up the test cases and work them out using the traditional test specification techniques. Despite the limited time, the goal remains unchanged; develop test cases with a certain test coverage in a structured way for the purpose of test execution.
If we, as testers, want to keep up with the ever-faster development cycles and still want to deliver the same test quality, we also need to automate the preparation of the test cases in the preparation phase. With MBT we can automatically generate the test cases from a model. And when there are any changes to the test basis, the maintenance is minimal. We only have to update the model and the test cases can be automatically generated again and again.
Shift-left test approach
Furthermore, MBT fits within the Shift-left test approach, which means that the test activities must start earlier in the Software Development Life Cycle (SDLC). By applying MBT you prepare a model in the test preparation phase and then derive the test cases from that model. With this you Shift some test activities to the left. By starting testing earlier, we ultimately ensure that we can provide feedback earlier and that we discover any bugs earlier in the process. Ultimately, any bug we find earlier is cheaper to fix, as also described in the well-known “Law of Boehm”.
But the question is if it is early enough? Even with a Shift-left test approach and / or MBT. Especially when we realize that about 60% of all bugs in production can be traced back to the requirements. To avoid these bugs, we will therefore have to start testing the requirements (static test)! Ultimately, the requirements are also the basis for developing and testing the desired software. So we need to make sure that this foundation is complete and clear. So that all stakeholders are on the same line even before we even start writing the first lines of code and testing it. Because before we know it the first bugs will appear pretty soon!
Early Model Based Testing (eMBT)
As described above, MBT fits perfectly within the Shift-left test approach. However, MBT is often started too late in the process or used in a way without the approach of testing the requirements, but purely to generate automated (executable) test cases. We will therefore have to apply MBT in a specific way, at the earliest possible stage. I call this test approach early Model Based Testing, eMBT for short. By applying eMBT you will break down the requirements at the earliest possible stage by modeling them by means of a so-called eMBT-model. Such an eMBT-model has a high level of abstraction and by drawing it up you will quickly encounter ambiguities, contradictions, open ends, questions etc. Which at this point you can then discuss with the stakeholders. When drawing up an eMBT-model, as a tester you will think differently about the requirements and you will put yourself more in the shoes of the final customer. You are currently testing the requirements in an exploratory-like way and already give meaningful feedback, namely feedback on the basis. In addition, an eMBT-model is a user-friendly visual representation of the desired situation and is readable for all stakeholders. So it is defintively not technical or difficult to read the model. The user-friendly visual representation of an eMT-model stimulates the communication between all stakeholders with the aim of achieving a shared understanding of what needs to be built and therefore also what exactly needs to be tested.
The eMBT approach does require a tool that supports this approach. For example, the tool must offer the possibility to draw a model with a high level of abstraction, like the eMBT-model. This not only increases the readability of the model, but by drawing the eMBT-model you, as a tester, are also forced to think about both the ‘happy’and ‘non-happy’ flow. Furthermore, the tool must have the possibility to include the questions and comments you have on the model. The tool which supports this eMBT approach is TestCompass. A Cloud based eMBT tool.
Below an example of a type of eMBT-model from TestCompass, based on the following requirements (figure 1).
Requirements calculator app
If you open the calcuator app, the calcuator standard shows 0.
1+1= with result 3
You can close the calculator app by clicking on the button ‘close’.
As you can see in figure 2, the eMBT-model is based on the principles of a flowchart, but with some specific conditions. This eMBT-model is not only easy to set up, but also clearly readable for everyone. Only three relevant nodes (action / status, decision and result) are used and furthermore, all questions / uncertainties etc. can be included directly in the eMBT-model via a separate node (baloon). As soon as all questions have been answered and all uncertainties have been removed then all stakeholders are on the same page on what needs to be built and the test cases can be generated automatically. These test cases then can be performed manually or being automated for the automated test.
Automation is increasing within the test profession, especially when it comes to test execution. But what about developing the test cases? Are we still doing this in a structured way and should we not automate this in order to keep up with the short development cycles? This is possible with Model Based Testing (MBT). MBT is now a well-known and proven test approach that offers many advantages over other test approaches. In addition, MBT fits within the Shift-left test approach because you can use MBT early in the process. However, we often see that technical models are used to derive the test cases and that the preparation of the model is started too late and therefore not used as a static test on the requirements. The technical model also does not stimulate the communication between the stakeholders, which is precisely so important at the start of the process.
Early Model Based Testing (eMBT) can be the solution for this. By using the right eMBT approach and tooling to start testing the requirements at the right time, unnecessary bugs are discovered at a very early stage. In addition, with this eMBT approach all parties agree on of what needs to be built and tested. And this even before a line of code is written. So when the requirements are clear and complete, then with one click you can automatically generate the test cases.
In figure 3 an addition to the well-known test pyramid with the eMBT approach, in which testing starts with the (automated) testing of the requirements, the bottom layer.
If you would like to know more about the approach of early Model Based Testing (eMBT) or about the tool TestCompass, please let me know.