What We Test

The discipline of testing has experienced a revival in recent years through agile development and automation. While manual tests, especially smoke tests and usability tests, still exist, the majority of tests, particularly unit tests, are now performed automatically. A large part of what we test are functions or features. These functions and features transform certain input parameters into an observable result.

These input parameters consist of up to three components:

• Direct input values I. These are parameters that are passed directly to a function by value or reference.
• Local state S. This is the set of all values and objects and their states of a unit under test, such as instance variables of a class.
• Global state E. This is the set of all values, objects, and states of the environment, JVM (e.g., system properties), operating system (e.g., environment variables, file system), but also all surrounding systems such as web services or databases and their content.

The observable result can consist of four components:

• Changed input values I', for example, when objects are passed as references, but also calls to callbacks or lambdas.
• Return values O, e.g., the result of a calculation, or a new object.
• Changed local state S' of the unit under test, e.g., when calling a setter.
• Changed global state E', if changes to the environment have taken place, e.g., static variables, system properties, files in the file system, changes in data storage.

A function F is the mapping of the input values G to the observable result T

F: {I, S, E} -> {I', O, S', E'}

and it is exactly this mapping that we want to test. However, we are not interested in how we get from G to T, but whether we get from G to T when a specific event - the call of the function F - occurs.

When writing tests, special attention should therefore be paid to how we create the initial state and which observable result we expect and how we define it.