Integrity in a nutshell

Want to know what makes Integrity special? Take a look!

The long story...

There's a lot of frameworks for software testing, even a lot of very different choices specifically for user acceptance testing. But they are usually characterized by taking one of two very distinct approaches: either they avoid anything that looks like "programming" entirely in an attempt to appeal to non-programmers, or they embrace traditional (functional or imperative) programming up to the extreme of just using a full-blown programming language for test authoring, in an attempt to profit from the limitless flexibility gained by this approach, and the already existing, powerful IDEs for that language.

The Integrity Test Framework aims to find a compromise between these two opposites. There's a lot to learn from the evolution of programming that's taken place in the last decades, especially when it comes to the act of authoring programs - or tests, for that matter! Programmers have invested huge amounts of effort in order to perfect their development environments - I'm talking about syntax highlighting, immediate syntax checking as you type, autoformatting, code completion, IDEs that are able to follow references in source files or perform refactorings on them, all this convenience stuff that most Java developers take for granted, thanks to modern IDEs like Eclipse. You definitely want that sort of things for test authoring as well!

But why not just use an existing language with IDEs that offer all this convenience for test authoring? Because constraining yourself to the tools you absolutely need for a given task opens up entirely new possibilities in other places. A programming language is extremely powerful, which is needed for efficient programming, but such limitless power is not at all required for efficient creation of user acceptance tests. Giving up some of these flexibilities does not only make writing tests potentially easier (and thus feasible for people with less programming skills), but it most importantly allows the test framework to apply certain assumptions to the test "scripts" you create, and these assumptions can be the base for tools that actually benefit your task of writing and executing user acceptance tests much more than the limitless freedom of a full-blown programming language. Getting nicely formatted and easily readable test result files - like those produced by Integrity - basically "for free" and without any additional effort is just one example of such a feature which is made possible due to limits imposed on the test creation process.

The compromise that's taken by Integrity involves a custom "scripting language" (an approach which is generally known under the term "Domain-Specific Language", or short "DSL") designed entirely for the purpose of creating user acceptance tests with ease, combined with an extensible interface to so-called "fixtures", which are basically pieces of Java code that link your "System Under Test" (or short "SUT") to the test scripts created in the aforementioned DSL. But having a separate language does not mean "no sophisticated editor tools" in this case! One very important focus of Integrity is to provide all the authoring support that you're used to as a programmer, just specialized on the Integrity DSL and possibly even customized for the very specific fixtures you write for your specific SUT. This includes syntax highlighting, instant syntax validation, autoformatting, reference search, basic refactoring and even a code completion system that can be extended to provide completion proposals specific to your own fixtures!

All of this is done by leveraging two very powerful and sophisticated frameworks, Eclipse and Xtext. Eclipse is well-known in the Java world and beyond as an advanced, open-source IDE with extensive support for creating plugins that seamlessly integrate new features. Xtext is much less known, but definitely one of the most advanced toolkits for the creation of DSLs, especially if you place a huge focus on providing top-notch editing support for a specific DSL inside Eclipse. Integrity uses these tools, which are of a very generic nature, and adds the specialization on user acceptance test authoring and execution, resulting in an entire framework of its own, ready to be specialized a bit further and then used to implement great automated user acceptance tests by YOU!

...and the hard facts!

The Integrity Test Framework is a framework for user acceptance tests, aiming primarily at testing Java-based programs. It sports numerous features, such as:

  • A language for writing tests, which is designed specifically for that purpose. Some language features are...
    • Support for "packages" as a means for structuring your whole test code base (very similar to Java packages)
    • Suites (optionally with parameters) allow for structuring of test statements
    • Tables facilitate repeated invocations of the same tests
    • Local and global variables
    • Constants, which can be initialized with different values depending on the "variant" of a test being executed (think about language variations, for example)
    • Support for "forks", which is a mechanism to run a test script that contains parts to be executed on different processes in a serialized fashion - with all processes being synchronized with each other automatically!
    • Support for various types of static data: Integers, fixed-point decimals (aka BigDecimal), Strings, Date and Time (in 3 different, common notations), Arrays of any type, Complex Objects (essentially nestable collections of Key-Value-Pairs)
    • Autoconversion between all supported data types, as well as Integrity data type to all standard Java data type conversions (including conversion from complex objects to Java Bean instances)
  • A powerful, full-featured editor plugin for Eclipse which offers...
    • Syntax Highlighting
    • Autoformatting (even for tabular data)
    • Continuous Syntax Validation
    • Reference search and "jump to declaration"
    • Basic refactoring capabilities (renaming)
    • Code Completion for the language itself as well as an interface for writing code completion providers which are specific to the application you want to test
  • A highly configurable and customizable test runner for the execution of test scripts
  • An Eclipse plugin for test runner control: launch, display test progress, set breakpoints at statements, single-step test script statements and more!
  • Automatic generation of test result pages
    • Results are both human-readable (HTML) as well as machine-readable (contain raw XML result data)
    • "Single File" approach facilitates handling of results as well as archiving - there's just one file containing everything! No more "I am missing that x.jpg|css|whatever, can you mail it to me?".
    • Fixtures can provide readable, descriptive text for result pages. Fixtures can also influence the way result data is converted to strings for inclusion in result pages, and they can easily inject free-form HTML or images (for example screenshots) into the results, which are incorporated seamlessly - in the same single result file.
    • Standard output and error streams are automatically captured on a per-statement basis and available in the results. This even works when parts of the test script run on a different process (see the "fork" feature) - the outputs of all processes are automatically combined in a single result file for you.
  • A Jenkins Plugin is available to evaluate results of test runs integrated in CI builds as well as archive those results
  • Last but not least: everything is open-source, under the commercial-friendly Eclipse Public License!