Testing the applications you create is a necessary part of writing code. Finding the right tools to perform different testing functions can be difficult. One test may be needed for email functions, while another for Android. Multiple languages may be employed, requiring more intricate and advanced testing.
Integration testing is a valuable component of the DevOps lifecycle, and there are a variety of tools to help you carry out and automate those tests. Ensuring that modules integrate properly is necessary for releasing fully functional, seamless products to the market efficiently. Coupled with a comprehensive security solution like Threat Stack’s Cloud Security Platform®, integration testing tools help you deliver stable, reliable apps. They’re essential tools for your DevOps arsenal. (Looking for SecOps tools? Learn about the three homegrown SecOps tools used by the Threat Stack team.)
In order to aid developers in finding the right solution, we spoke with some experts in the field and compiled a list of 50 integration testing tools. These tests are not ranked, but are separated into a few categories:
- Rare languages or languages with fewer tests: There are several tests on our list that are from more rare languages (e.g. CoffeeScript). These tools are grouped together.
- Tests that offer multiple languages: There are several tests on our list that have been recreated in multiple programming languages and are in a separate category.
Java Integration Testing Tools
DBUnit is an extension of JUnit essentially made for database heavy project testing. One of the more popular capabilities is to put a database into a known state in-between tests.
Some of the highlights of this integration tester are:
- The ability to work with large sets of data
- Importing and exporting full databases
- Tons of supporting documentation on SourceForge
Greenmail is a group of email servers created for programmers to run tests on. The servers support SMTP, POP3 and IMAP.
Greenmail can be used in a number of different scenarios, including:
- Testing email sending code
- Testing retrieving code
- Providing a mock server for development
3. H2 Database
H2 Database Engine is one of many open source testing tools. It’s known as one of the faster database engines available; as well as having some hefty security.
Some of the differentiators of H2 are:
- Incredibly low memory. The tool takes about 2MB
- 3 supported connection modes (embedded, server, mixed)
- Quickstart guides and detailed documentation on a separate website
With more than 1700 comments and 300 stars, JMockit1 is a popular Java testing tool on GitHub. The regular updates continue with multiple versions being released every year.
Some of the features include:
- “Out-of-container integration testing for Java EE and Spring-based apps”
- Code coverage tool
- Ability to create fake implementations
5. JUnit 5
JUnit 5 is one of the newest JUnit testing tools. This version was successfully crowdfunded on Indiegogo. There are modules that run on Java 8 or higher (although you can still test code from previous versions).
A few of the features include:
- The ability to migrate from JUnit 4 and Jupiter
- Can function as an extension model for JUnit Jupiter
- Works with other testing tools by allowing them to plugin to the infrastructure
Creator: Krzysztof Styrc
Most of the integration testing tools on our list are created by companies, or developers within those organizations. Embedded-redis is one of the few published specifically by developers for developers.
Here are a few points as to what it does:
- Provides a redis embedded server to test integrations
- Multiple use cases as easy as a copy/paste run code
- Easily retrieve a full list of ports in the cluster
Mockito is a fun-themed testing tool recently voted the “best mocking framework for java” by the StackOverflow community. It’s clearly one of the more popular tools, but not just it’s fun name and fancy site.
Some of the highlights include:
- In the top 10 Java libraries (not just integration testing tools)
- Mockito gives the ability to “mock creation, verification and stubbing“
- Extensions with other tools (JUnit Jupiter, JUnit 5)
Creator: Rest Assured
Representational State Transfer or REST is an architectural style commonly used in developing web services. It’s particularly hard to test REST services in Java, which is why this tool was created.
Here are some features of this tool:
- Scala support is available
- Troubleshooting docs for dealing with issues (like working with different Java variations)
- Directions for non Maven users
Creator: Roberto Tyley
If you’re a developer and still have trouble understanding the functions and usage of some of these tests, android-screenshot-lib tries to make things clear via illustration. The system is made up of a paparazzo & celebrity. The former takes images when the latter asks for them.
Here’s what this testing tool offers:
- Capture screenshots from an Android device
- Slow image capture
- The app is the “celebrity” and tells the testing tool to take screenshots
Spock is a framework for testing Java (and Groovy) applications. According to the site, this testing tool “is compatible with most IDEs, build tools, and continuous integration servers.” Obviously a play on sci-fi culture, it also has some fun theming thrown in to make things interesting.
Some of what Spock can do includes:
- Detailed breakdowns of definitions to help you understand the lingo
- Not known for speed, but more information is given from tests than some other frameworks
- Super easy mocking and stubbing
11. Spring Test
Spring Test offers various annotations and utilities to assist in testing a web application. This tool offers test support using two modules; the spring boot test has core items, and the spring boot test auto figure supports test auto-configuration. A lot of developers use the spring boot starter tests that imports all the spring tests modules and JUnit, Hamcrest, AssertJ, and various other important libraries
A few of the key features of Spring Test include:
- Uses the dependency injection which enables the code easier for unit testing
- Allows you to mock objects rather than actual dependencies
- Has the test configuration for using an inner test class for customizing the primary configuration.
NUnit and JUnit inspired this testing framework. However, it has new functions which enable it to be stronger and simpler to use. TestNG is made for covering all test categories such as functional, unit, integration, end-to-end and more.
Here’s a breakdown of some TestNG features:
- Default JDK uses for logging and runtime
- Offers support for the parameters
- Strong execution model
Arquillan is highly extensible and innovative testing software for JVM which allows the developers to create easily functional and acceptance tests that have an automatic configuration for the Java middleware. This tool brings tests to runtime and saves you the headache of it.
Available features of Arquillian:
- Integrates with similar testing tools
- Allows launching of tests using present IDE, Maven, and Ant test plugins
- Handles framework initialization
Creator: Akash Manohar
Hound is used in the writing of integration tools and browser automation. This software tool has a number of features that make it ideal for the task at hand.
Some of the features that Hound includes:
- Implementation of WebDriver Wire Protocol
- Supports Chrome Driver, Selenium (Chrome, Firefox) and PhantomJs
- Can run several browser sessions in quick succession
Creator: Daniel Sudol
This is one of the most amazing integration tools on our list. It has made the testing process simpler. Ember Data factory guy makes your process efficient and tests readable.
Some key points of this tool include:
- Uses ember-data
- No need to add files to come up with model relationships
- You can easily push models to their respective stores
This is a great software tool for developing applications. Users are expected to have a great experience using it. App developers can enjoy coming up with beautiful and robust applications. It boasts of several important features that application developers can take advantage of.
Available features of The-book:
- Easier creation of web apps
TESSY does computerized dynamic unit/module and incorporation testing of embedded software. It also determines accurate code coverage. This integration testing is very vital in certifications.
Tessy has some awesome features, including:
- Variant testing
- Fault injection
- Generates tests through value combination
- Imports and exports data to and from Excel
- Develops test documentation
IBM Rational Integration Tester is used in the creation, modification and running service and performance tests. The components of IBM Rational Integration Tester are installed by the help of a Rational Test Workbench. In addition, this tool will help you create tests for business integration processes and SOA messaging.
Features of the Rational Integration Tester include:
- Creates virtual services and removes test dependencies
- Enables early testing
- Provides virtualized test surrounding
- Supports various steps of test development
Creator: Pivotal Labs
Available features from Jasmine:
- Does not have any external dependencies
- It runs all browser and Node tests using a similar framework
20. Core Components
Creator: Nitor Creations
These integration tools serve a wide variety of functions. For instance, the WrappingParameterizedRunner runs parameterized tests while the NestedRunner runs plain and old Java Classes found in the best configurations.
Available features of Core Components include:
- Supports the packaging of integration tools
- Controls how tests are distributed
- Generates standardized XML reports
Nightwatch is one of the most reliable integration tools considering the amount of work it does. It allows the user to carry out a complete website and web application testing. This software also comes in handy during integration testing.
Available features for this tool:
- Works with WebDriver friendly servers
- Can support GeckoDriver, ChromeDriver and Selenium Standalone server
- Is supported with Node.js
Creator: Pixie Labs
Cavy works in your live app while operating on a device such as iOS or Android simulator. It comes up with functions which have the capability of not only referring to but also simulating actions on nested components in your app.
Benefits to expect from Cavy:
- Runs inside an application
- A reliable cross-platform software tool
- Provides results to your console
The protractor is the ideal integration tool for not only Angular but also AngularJS Apps. It conducts several tests against applications that operate on real browsers to interact with it just like a normal user would.
Features of Protractor include:
- Built on WebDriverJS, then utilizes browser unique drivers and native events to work with applications just like a normal user would
- Supports unique angular strategies, that enables you to carry out tests on Angular elements effortlessly
- No need to add waits or sleep on your test
PHP Integration Testing Tools
Some of TestSwarm’s benefits include:
- You can open multiple clients on one browser or on different browsers
- The tool consistently delivers reliable results even when there are browser issues
Integrated is a simple testing tool with PHPUnit. Most of it is now run under the Laravel Core. Integrated has been included under the Laravel extension since Laravel released Laravel 5.
Integrated has a few benefits, including:
- It supports many testing forms
- It is easy to learn and use
- Integration will assist you in testing your APIs
TestDummy is an intuitive tool for generating test data for integration testing. The new version of TestDummy is easy to use.
TestDummy has a few unique features, including:
- It allows for quick setup of test data
- It makes it easier for you to define relationships with pivot tables
- It has its own integrated Faker factory that provides random data for your various factories
- You can use TestDummy to automatically generate your relationship models
Creator: Marcel Pociot
Laravel TestTools is basically a chrome extension that allows you to use Laravel TestTools to run tests. It was based on the assertion that most people don’t actually use tests to run projects. Adding Laravel TestTools as a chrome extension makes it easier for people to run tests.
Available features from Laravel:
- It contains the “Integrated” package which makes testing faster and easier
- The Laravel menu contains some incredible faker options
- The software is free
Python Integration Testing Tools
Buildbot is an integration framework built by Brian Warner and is now maintained by the Bortherders. The general feeling among most users is that Buildbot is somewhat difficult to understand. However, it becomes really easy to understand and use once you get the hang of it.
BuildBot benefits include:
- Different builders can run on different workers depending on your configuration
- You can configure builders to perform different tasks depending on the ultimate goal
- You can use one or more schedulers depending on the setup and the purpose of your build
29. Allure Python
Creator: Allure Framework
This program contains python-based test frameworks that run on the Allure Framework. The Allure framework provides a basis for the test frameworks to give clear and concise test results.
Here are a few of Allure Python’s testing features:
- The program gives a clear list of defects that were revealed during the test
- The clear graphs provide a vivid picture of the results after a test
- The results contain supplementary data that gives you a full picture of the test results
Ducktape contains systems for running performance and integration tests. Testing distributed systems can prove to be a difficult task. It can sometimes require multiple tools to perform a successful test. This library puts all the tools together and provides one software for distribution systems testing.
Ducktape’s benefits include:
- Default isolation of systems which provides for thorough testing
- It provides an all-in-one tool for testing distributed systems
- It presents results in an easy to understand manner
Creator: Mikhail Podgurskiy
There are multiple ways for you to ensure continuous integration with a django project. However, most of these ways require a lot of configuration time and setup time. Some of them might even require you to make changes to your project. This program provides continuous integration for your project without the downsides.
Key features of Django-Jenkins:
- It eliminates the problems associated with previously used nose testing frameworks
- It clearly shows the errors and warnings after each build
- The program has plenty of post-build actions
32. Robot Framework
Creator: Robot Framework
Robot Framework is a testing framework that uses the keyword-testing approach. Its main advantage is that you can add extensions to the framework to satisfy your specific testing demands. For instance, you can add tools and libraries depending on the demands of your test.
Robot Framework’s key benefits:
- You can extend its testing capabilities using library extensions such as Java
- Test cases are clear and easy to understand
- The program has many built-in tools such as Libdoc and Robot that facilitate testing
Creator: Russ Baz
Enforce.py is an integration testing program that runs on Python 3.5+. The program doesn’t run on the previous versions of Python. This software is still in development. This means that it doesn’t work very well while trying to test large structures.
Features available from Enforce include:
- This is a highly configurable system that can be modified for different tests
- It is very easy to use
- One application of the class decorator automatically applies the decorator to every method in that class
Ruby Integration Testing Tools
Test-kitchen has perhaps the most “play on words” approach out of any other testing tool on our list. Using terms like kitchen (obviously), chef and cookbook; users are taken through a visual journey toward using this tool for testing infrastructure.
Test-kitchen allows developers to:
- Learn while creating in a very “hands on” tutorial
- Use the “red, green, refractor” method of coding to write a test first, then use the code that works
- Automate git installation
Creator: Alexander Lang
This clever integration tool actually integrates two other popular tools (one of which is on our list), Cucumber and Celerity. Combine the two and you come up with Culerity
Culerity is useful in a few ways:
- The ability to test the full stack of Rails
- Running an application in Ruby as Celerity runs in JRuby
- Run tests in a developer’s Ruby environment, unlike Celerity which requires JRuby
Creator: Abraham Petrillo
Evergreen’s benefits and features include:
- Integrations with Ruby 2 and Ruby 3
- Uses Selenium to run its tests
Other Programing Languages
Creator: Jimmy Bogard
This integration testing tool helps in rebooting test databases into a clear condition. Rather than erasing information at the test’s end or taking back transactions, this tool will reboot the databases to a clean state through deleting data from the tables.
- Guarantees a known point of starting before every test
- It intelligently checks your SQL Metadata to create a table order for deleting according to foreign key relations among tables
- The object of Checkpoint maintains this list of tables privately to allow only one calculation
Testium is an integration testing tool to offer straightforward, but efficient, software to test web applications in various browsers through selenium and PhantomJS.
This tool allows you to write tests of integration which:
- Use the similar BDD syntax
- Have the synchronous API
- Enable selenium leverage
This tool offers a layer of obstruction between the integration tests and DOM markup. It is also crucial in drying up the definitions of steps and consolidates the way people interact using the DOM in the tests.
- Allows text expansion
- It makes your tests more efficient
- Easy definition of steps
This tool is important for gophers and utilizes the Go test. This Go testing tool can be used in the browser or terminal in regards to your viewing pleasure. With this tool, you will get notifications on your desktop. The console output is readable by anyone, with or without IT. You will also be able to see the coverage of the tests.
Some of Smartystreets’ benefits include:
- Generates Test code
- Large regression tests suite
- Integrates directly with the Go test
In the development of web applications, it is important to utilize services which communicate to the database system. To unit test all these services could prove to be strenuous since mocking dataset is difficult. The Docker system is perfect for doing unit tests since you can rollback containers quickly and destroy them after completion of tests.
Available features from Dockertest:
- Using this tool is simple
- If clean up of container fails, you can set the absolute lifetime on the containers
- Dockertest library offers commands that are simple to use for rolling Docker containers
Creator: Jorge Castillo
Kotlin is a programming language primarily used for Android and it’s growing in popularity to give Java a proverbial run for its money. It gives developers the ability to write API integration tests in the Kotlin language.
Some facts about Hiroaki, include:
- Fairly straight-forward documentation (that may not be enough for some)
- Uses a mocker server to see how your program handles certain server and API issues
According to its page on GitHub, “Subliminal is a framework for writing iOS integration tests.” One of the coolest features here is the ability (via a mechanism) to actually manipulate your application.
Other features of Subliminal include:
- The ability to work with common tools altogether to fully test with ease
- Test virtually every interaction without the need (in many cases) to use private APIs
- Decent documentation and instructional materials
Rust-bio is literally a library of continuously tested implementations for Rust. It is licensed under the MIT open-source and is constantly being updated by a large group of regular contributors.
Rust-bio gives Rust developers:
- In-depth overview of algorithms and structured data supported
- An easily organized and accessible library
- Simple setup instructions for beginners in two steps
Creator: Whisk Labs
Our only tool listed for Docker is Docker-it-scala. Not a ton of documentation here, but enough if you’re familiar with Docker and Scala.
Here are a few points about Docker-it-scala:
- Works with two libraries for docker engine communication
- A pair of “testkits” are available
- The ability to create custom bindings in an environment
Multiple Language Integration Testing Tools
Languages: C, C++
C++ or VectorCAST is a crucial tool in integration testing and is mostly useful for embedded developers for important business systems and security validation. This top-notch automated unit is broadly utilized in medical gadgets, automotive, avionics, railway, financial companies as well as industrial controls.
- It enables developer testing to be a transparent and simple technology
- Takes manual testing to engineering rigor from manual process
- Automates framework build up for isolating newly created code thus reducing costs
Languages: Java primarily, but every language
This wiki web server has a low learning curve and entry. That makes it ideal for integrating with other tools. The specifications of this tool could itself be tested against the framework. FitNesse is a completely integrated independent wiki and framework for acceptance setting.
- The base of the code is not under any ownership
- Highly adaptable and is useful in places like Web/GUI tests and electronic components
- Supports the major languages
Languages: Java, Dockerfile
Selenium is a multi-part integration testing tool. You can use a certain part to quickly create scripts or a more robust part to create larger tests and scripts across multiple environments. This tool is also one of the more popular solutions online.
Some of the core benefits of Selenium include:
- It’s an entire suite of tools including Selenium 1, Selenium 2, Selenium Grid and Selenium IDE
- Detailed documentation on all of the tools available
- Large number of supported browsers for testing
Cucumber tops so many lists when it comes to popular and useful integration testing tools, and it’s for good reason. The site explaining the tool is possibly the best-designed, loaded with documentation and laid out well.
A few of the benefits of Cucumber include:
- Multiple languages for a wide range of development projects
- Large library of video tutorials and other documentation
- List of support options available to help with a number of issues
Languages: Java, PHP, Python
Hilariously named integration testing in seven different programming languages. In addition to the most common (Java, PHP, Python) there is also testing for Ruby, Objective-C, Erlang and Swift.
Some fun and useful notes on Hamcrest:
- Tons of languages to choose from for testing
- Individual pages for each language on GitHub (each with its own documentation)
- Originally created in Java, but expanded