A Gentle Introduction to Software Testing
Depending on the software you are writing, testing has a crucial role in validating the correctness of your program. Some systems, such as banking systems, are expected to be thoroughly tested. In this post, we will cover the motivation for writing tests and the different types of testing levels that exist today.
The idea behind automated testing is to validate the correctness of our code and uncover underlying bugs that may exist. In most cases, it is impossible to manually test the program after every change. Investing time and effort into writing good automated tests saves countless hours of bug-hunting in the long term.
There are generally three different levels of testing: unit, integration, and end-to-end. We will delve into each one to understand the differences between them. We will use an analogy of testing cars in the following explanations.
Unit Testing
Unit tests are the base layer of testing. The idea is to test the smallest testable components of the program, commonly referred to as "units". A unit is usually a function, and as we know, functions should usually have only one purpose. Unit tests aim to validate the proper behavior of the smallest components in isolation.
Unit testing is similar to testing the smallest individual components in a car. For example, testing the tire, the window glass, the speedometer, etc. Unit testing does not cover a door for example, because the door is made out of several parts (window, handle, window control buttons, speakers.)
Advantages of writing unit tests include early detection of flaws, easy bug tracing, and higher code quality. They are also considerably faster than integration and end-to-end tests. A good way to go about writing unit tests is to define the valid behavior of a function (inputs and outputs), write a few scenarios that cover most of the partitions of the function, and then write the test cases. An example of good partitioning for the absolute value function: a positive input, zero, and a negative input. These inputs cover the most important part of the input space to the function. Another rule of thumb is to verify that the test fails to ensure no false-positive tests.
One of the greatest benefits of testing modules in isolation is much easier debugging. When a unit test for a module fails, you can be more confident that the bug is found in that module, rather than anywhere in the program.
One of the difficulties in writing unit tests is mocking data for functions. For example, if function A calls a database to do its calculation, the test writer needs to mock the database access. Because we are testing in isolation, we need to mock every external resource the function uses. This process is very time-consuming and requires careful thought from the test writer.
Most programming languages have at least one popular library for writing automated unit tests, including JUnit for Java and unittest for Python, Mocha for JavaScript and TypeScript, etc.
Integration Testing
Integration tests focus on finding bugs in the connection between modules in the program. In certain cases, two modules might do their job correctly, but when combined result in an invalid module. In short, we want to verify the interfaces between modules are compatible. Integration tests usually involve multiple components and often mock data.
Let's look at a car door as an example. The mirror was tested thoroughly and returned a correct result (proper sunlight blocking, wind-blocking, strength, etc.) The door was also tested with positive results (robustness, color, etc.) But how do these two components work together? By composing these modules and adding them together we might find that the window doesn't fit in the door's hinge! In this case, we have two perfectly valid components that fail when connected.
To perform integration tests, we usually compose different units and check for valid behavior. Let's look at two modules, for example, a module that reads and parses data from a database, and a module that receives data and analyzes it for anomalies. An integration test would call module A to read and parse data, and then provide that data as input to module B to analyze it. Having unit tests validate the correctness of both modules individually and integration tests to validate the correctness of the composition of both modules can ensure an error-prone application.
End-to-End Testing
End-to-End testing is a quite simple concept. It tests the whole application in a simulated environment that is close as possible to the real environment. The idea is to see that the application works properly in real scenarios. These tests are often complex and slow.
End to End testing on a car for example includes taking the car to different terrains and simulating real cases. One scenario might be driving downhill for an extended period of time, another would be accelerating on a highway-like road, etc. There are usually endless scenarios we can test, so it is common to test scenarios that occur more often (driving on a highway vs. driving in the middle of a tornado.)
To perform end-to-end tests we usually need to simulate the real-world environment, which might include spinning up databases, micro-services, dependant apps, etc. We then simulate actions that the user might perform on our application and verify their correctness. Using a chat application as an example, we would send a message and check if it was received on the other user's end, we would enter a conversation to see if the notifications disappear, etc. The key idea is that we need to simulate the environment which often requires a lot of work and resources.
Conclusion
Software today is becoming more and more complex. To ensure the quality of our software we use different types of testing methods. Unit tests cover the smallest testable components of our application. Integration tests cover the interaction between different modules. End-to-End tests cover the correctness of our program as a whole, simulating it in an environment that is close to a real-world environment. Utilizing these tools will save you time and money, and bring you happiness!
Snowmate - Early Access
What if you had a solution that generates and maintains your entire backend unit tests 100% automatically? Well, it is now getting built and will soon be released.
If this interests you, sign up for our early access.