Integrity Tutorial: Suite Organization
This time you'll learn about organizing tests and calls in suites and using those suites to create a well-structured test execution plan. When you start creating larger test suites, you will absolutely profit from a decent suite structure, as it will enable you to re-use suites in different places and contexts as well as help you keep track of all your functional test cases. Integrity provides tools to help you with that task.
Project Setup
We will require a basic setup as described in Getting Started. You should thus create a new project named "SuiteOrganization", then add the Integrity libraries to it and create a package integrity.tutorial.suiteorganization.
Within this package, we need a simple fixture in order to get something to work with, like this:
package integrity.tutorial.suiteorganization;
import java.math.BigDecimal;
import de.gebit.integrity.fixtures.FixtureMethod;
import de.gebit.integrity.fixtures.FixtureParameter;
public class SuiteOrganization {
@FixtureMethod(description = "Divide $number$ by $divisor$")
public BigDecimal divide(@FixtureParameter(name = "number") BigDecimal aNumber,
@FixtureParameter(name = "divisor") BigDecimal aDivisor) {
return aNumber.divide(aDivisor);
}
@FixtureMethod(description = "Multiply $multiplier1$ with $multiplier2$")
public BigDecimal multiply(@FixtureParameter(name = "multiplier1") BigDecimal aNumber1,
@FixtureParameter(name = "multiplier2") BigDecimal aNumber2) {
return aNumber1.multiply(aNumber2);
}
}
Creating the root suite
If you've read through the Calls and Variables tutorial, you already know the following part. First of all, add a directory named "Integrity" to your project. Then we'll create a package in a separate file named "fixtures.integrity" inside that directory for the fixture definitions:
packagedef suiteorganization.fixtures with calldef multiply uses integrity.tutorial.suiteorganization.SuiteOrganization#multiply testdef division uses integrity.tutorial.suiteorganization.SuiteOrganization#divide packageend
Another Integrity file with the name "suites.integrity" shall contain the root suite. Create this file and insert the following:
import suiteorganization.fixtures.* packagedef suiteorganization with suitedef rootSuite with variable result call multiply multiplier1: 12.5 multiplier2: 8 -> result test division number: result divisor: 8 = 12.5 suiteend packageend
The file is quickly explained:
- The fixture package is imported
- A suite "suiteorganization.rootSuite" is created
- In this suite, two numbers are first multiplied and then divided, with a variable being used to store the temporary result
You can now launch this suite. For the launch configuration, refer to the respective part of the Getting Started tutorial. The parameters for the ConsoleTestExecutor are just a little bit different to account for the package naming differences:
-x result.html suiteorganization.rootSuite Integrity
You probably won't be surprised by the result: the test succeeds.
Introducing suite calls
The next step will be a refactorization of our small test case: The call and test statements will be extracted from the rootSuite into a different suite, which is then called from the rootSuite. Create a new suite definition with name "otherSuite" and move both the variable definition as well as the call/testtandem into this new suite.
But: as the statements have been removed from the root suite, they aren't called anymore in the course of test execution. In order to restore execution of the statement, we need to call the new suite inside the root suite:
suitedef rootSuite with suite otherSuite suiteend
When you execute the test case again, you'll notice a slight difference in the console log:
Loaded test resource 'C:\ew\integrity_test\SuiteOrganization\Integrity\suite.integrity': 0 errors. Loaded test resource 'C:\ew\integrity_test\SuiteOrganization\Integrity\fixture.integrity': 0 errors. Resolving the test model...done! Test execution has begun... Now entering suite 2: suiteorganization.rootSuite Now entering suite 3: suiteorganization.otherSuite Defined variable result with initial value: 8 Now executing call 1: Multiply 12.5 with 8... SUCCESS! Now running test 1: Divide 100.0 by 8... SUCCESS! Now leaving suite 3: suiteorganization.otherSuite Now leaving suite 3: suiteorganization.rootSuite Finished executing 3 suites with 1 tests and 1 calls in 79 msecs! 1 tests finished sucessfully, accompanied by 0 failures and 0 exceptions.
As you can see, the execution path now goes through our new suite, which then contains the test and call statements.
Parameterizing the suite
Now imagine that you want to call the combination of call and test that you've scripted in the "otherSuite" with different parameters, maybe in different places as well. For this, you can easily add an arbitrary number of parameters to any of your suites (except the root suite!). Those parameters can then be used when calling the suite in question using the suite keyword.
In our case, the two numbers which are multiplied make good candidates for parameterization. Add two parameter definitions to the suitedef line, like this:
suitedef otherSuite gets number1 number2 with
Afterwards, you can use the new parameters throughout the suite instead of fixed values:
call multiply multiplier1: number1 multiplier2: number2 -> result test division number: result divisor: number2 = number1
Finally, you need to add the respective values to your suite call:
suite otherSuite number1: 15 number2: 4
Execute the parameterized suite
When you execute the whole test case again, you'll get a result like the following HTML:
As you can see, the values were forwarded into the inner suite as variables, and then used in the call and test statements.
Add a suite dependency
You've seen how you can structure your tests and calls in suites and call those from each other. This is a very explicit thing when using the suite keyword. Now we'll go one step further and introduce implicit suite calls by means of a dependency system.
Any suite can depend on one or more other suites. These dependencies are executed before the actual suite that depends on those is run, but only if the dependent suites haven't already been run in the current "execution stack trace". This sounds complicated, but is actually a very simple system. Let's make an example:
- Suite A, which is the root suite, depends on suite D to be run beforehand
- Another suite B depends on suite D as well, but is called from suite A
If we now start this, we'll first see suite D being executed, since suite A, which is the root suite and therefore the "first" to be run, depends on it. Then suite A is executed. Inside this suite, B gets run, but this time D isn't executed another time, because it has already been run before A, which is still "alive".
The real power of this mechanism becomes visible if you now think about what happens if you decide to use B as your root suite, for example because you only want to execute some of your tests. This time, D gets executed first as well - because B depends on D, and D hasn't been executed yet!
The dependency mechanism is primarily designed for startup and teardown kind of actions, but you're of course free to use it for whatever it's suitable for. In the practical example that we've set up, we may introduce a dependent suite like this:
suitedef rootSuite requires dependency with
{...}
suiteend
suitedef dependency concludedby cleanup with
call multiply multiplier1: 1 multiplier2: 1
suiteend
suitedef cleanup with
call multiply multiplier1: 2 multiplier2: 2
suiteendOkay, of course this example is a little bit made up, as the dependent suite doesn't do anything useful ;-) but nevertheless it showcases the syntax quite well: The requires keyword, followed by one or more suite names, defines a "startup" kind of dependency, which is run before the suite is run, while the keyword concludedby defines a "teardown" dependency that is executed after - and now it gets a bit complicated - the suite that defined the dependency as its requirement has finished executing. So if you've got one suite which, for example, prepares a database, you might have a second one cleaning the database up, and of course you'd like to have that cleanup be executed after the suite has finished for which the database preparation was executed. This is exactly how "requires" and "concludedby" work.
Now execute the tests and see for yourself!
Conclusion
You have now seen basic techniques to structure suites. In the end, suites and all structuring mechanisms that are attached to them are of course just a tool: it must be used wisely in order to get good results, and what "wisely" is depends a lot on your specific projects' requirements.
The next tutorial will be a bit more "low-level" again, dealing with a specific type of test: table tests.