×
THANKS FOR SIGNING UP!
We’ll make sure to share the best materials crafted for you!
5 minutes read
Why Should You Run All Your Tests in Docker?
Written by Burak Kantarcı
Product Manager @Thundra
Containers have become the de facto standard for deploying applications. Whether developers are deploying them in the public cloud or a data center, containerization can bring a lot of benefits like a consistent environment, repeatable builds, and easy deployment patterns.
But so far, the focus has been on running applications in containers, with very little attention given to running tests in containers.
In this article, we’ll see how you can leverage containers for tests as well and why this could prove beneficial to your organization. We’ll also discuss how to effectively handle CI test observability and CI test debugging needed for containerized tests.
Pros of Running Tests in Containers
Running test cases in containers offers a brilliant value proposition with some great advantages:
- Consistent environment: With identical containers in every environment, there is no drift in software versions across dev and test environments. This significantly increases developer velocity and helps deliver code faster.
- Isolation: Containers provide an ephemeral environment to run your tests, making sure your tests run in isolation and don’t interfere with one another.
- Repeatable builds: Containers are built against a consistent definition file, which can be used to recreate containers anytime.
- Quality CI pipelines: Having containerized test cases makes it easy to set up CI pipelines and maintain high-quality builds.
Demo Setup
Before we jump into the demo, you’ll need to make sure you have some prerequisites set up:
- Java 8+ for a sample Java application and unit tests
- Maven 3.6.0+ to download your dependencies and run your tests
- Docker to create a Docker image to run your tests
Code Setup
This demo will entail a simple Java application with unit tests written in jUnit5.
First, create the project using Maven:
mvn archetype:generate -DgroupId=com.example
<-DartifactId=test-in-containers>
<-DarchetypeArtifactId=maven-archetype-quickstart>
<-DinteractiveMode=false>
This will create the folder structure with a sample Java project.
Delete the already existing code files at src/main/java/App.java and src/test/java/AppTest.java.
Next, replace the pom.xml with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>test-in-containers</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>RELEASE</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
</plugin>
</plugin>
</plugins>
</build>
</project>
This configures Maven to use Java 1.8 and add jUnit5 as a dependency and also adds the Maven Surefire plugin for running tests.
Now, create the source code file at src/main/java/Calculator.java:
public class Calculator {
public int add(int a , int b){
return a + b;
}
}
And create the test file at src/test/java/CalculatorTest.java:
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
class CalculatorTest {
@Test
void AddTwoNumbers() {
Calculator calc = new Calculator();
int result = calc.add(1,1);
assertEquals(2, result);
}
@Test
void AddNegativeNumbers() {
Calculator calc = new Calculator();
int result = calc.add(-1,-1);
assertEquals(-2, result);
}
@Test
void AddZeroToANumber(){
Calculator calc = new Calculator();
int result = calc.add(1,0);
assertEquals(1, result);
}
}
And that’s it. Your system and code are ready to proceed to containerization.
Containerize the Tests
In this section, you’ll see how to containerize your tests, after which DevOps can run them in your CI pipeline.
First, create a Dockerfile, which you will use to create a Docker image to be run in your CI pipelines:
FROM maven:3.8.1-openjdk-11-slim
WORKDIR /src
ENTRYPOINT ["mvn", "clean"]
Notice, you’re using Java 11 and Maven 3.8.1 to build the Docker image. You can create many such images with different versions of Java and Maven, which can then run in a single test environment, reducing the maintenance cost of different environments and eliminating interference due to different versions.
Build the docker image:
docker build -t test-in-containers:1.0-3.8.1-openjdk-11-slim .
Here, you are adding Maven and Java versions in the Docker image file tag for easy referencing.
Once the image is built, you can run the tests via the following command:
docker run -v `pwd`/test-in-containers:/src test-in-containers: 1.0-3.8.1-openjdk-11-slim test
Notice how you are mounting the source code inside the container using Docker volumes. You can then hand over the image to DevOps to set up these tests to run on every commit.
You can also mount another volume and generate test reports, but for the purpose of this demo, you will stick to just running tests.
In the prerequisites section, it was noted that you need a certain version of Java and Maven to make sure your tests run properly. Over time, you might need different versions for different projects, and it can get challenging for DevOps to maintain consistent test environments.
For example, one of your applications might need Java 8 and the other might need Java 11. Having them both installed on the same machine might cause interference, or if you run tests on a single Java version, you run the risk of shipping buggy code.
Another scenario might be that you want to test your code for backward or forward compatibility with separate Java versions; for that, DevOps will also need to carefully maintain different environments.
In all of these scenarios, containerizing your unit tests can help alleviate the problem of maintaining different test environments. DevOps just needs to make sure that they have an environment where Docker containers can run and the rest is managed by the developers themselves. Tests running in containers can solve this age-old problem of dev-test environment drift.
Note: We’ve covered how to containerize unit tests, but the same principles can be applied to other tests, namely integration and end-to-end.
CI Test Observability
Running tests in containers is not all rosy and come with its own set of challenges. One of the issues that developers and DevOps face while running their tests in a CI pipeline is test observability, including CI test troubleshooting and debugging. You might have test failures you want to be notified about and resolve, or you may experience a slower build that’s hampering deployment frequency. But there is no straightforward way to get a peek into what is happening inside your CI pipeline.
That’s where Thundra Foresight comes into play. Thundra Foresight is a shift-left tool that can help you gain CI test observability. It provides CI test debugging and CI test troubleshooting capabilities, which can help you lower your build times and increase deployment frequency.
Foresight keeps track of all your test runs and KPIs associated with those runs—things like how much time each test has taken and the reason for slow tests or test failures. Foresight has automated distributed tracing that helps you debug the root cause of slow/failing CI pipelines.
Conclusion
Running your tests (unit, integration, or E2E) in containers alleviates the pain of maintaining cumbersome test environments and helps you streamline your CI pipelines. Containerizing tests can save your organization hours of having to debug issues caused by mismatched software versions. And although CI test observability and CI test debugging of tests can be an issue, Thundra Foresight helps you gain rapid and detailed insights into key metrics for your CI pipelines to keep them on track.
We have integrated Foresight with some popular open-source projects. To discover more about Foresight’s capabilities, explore the build process and the test runs for these open source projects. Also, if you would like to see other open-source projects feel free to contact us!
Containers have become the de facto standard for deploying applications. Whether developers are deploying them in the public cloud or a data center, containerization can bring a lot of benefits like a consistent environment, repeatable builds, and easy deployment patterns.
But so far, the focus has been on running applications in containers, with very little attention given to running tests in containers.
In this article, we’ll see how you can leverage containers for tests as well and why this could prove beneficial to your organization. We’ll also discuss how to effectively handle CI test observability and CI test debugging needed for containerized tests.
Pros of Running Tests in Containers
Running test cases in containers offers a brilliant value proposition with some great advantages:
- Consistent environment: With identical containers in every environment, there is no drift in software versions across dev and test environments. This significantly increases developer velocity and helps deliver code faster.
- Isolation: Containers provide an ephemeral environment to run your tests, making sure your tests run in isolation and don’t interfere with one another.
- Repeatable builds: Containers are built against a consistent definition file, which can be used to recreate containers anytime.
- Quality CI pipelines: Having containerized test cases makes it easy to set up CI pipelines and maintain high-quality builds.
Demo Setup
Before we jump into the demo, you’ll need to make sure you have some prerequisites set up:
- Java 8+ for a sample Java application and unit tests
- Maven 3.6.0+ to download your dependencies and run your tests
- Docker to create a Docker image to run your tests
Code Setup
This demo will entail a simple Java application with unit tests written in jUnit5.
First, create the project using Maven:
mvn archetype:generate -DgroupId=com.example
<-DartifactId=test-in-containers>
<-DarchetypeArtifactId=maven-archetype-quickstart>
<-DinteractiveMode=false>
This will create the folder structure with a sample Java project.
Delete the already existing code files at src/main/java/App.java and src/test/java/AppTest.java.
Next, replace the pom.xml with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>test-in-containers</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>RELEASE</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
</plugin>
</plugin>
</plugins>
</build>
</project>
This configures Maven to use Java 1.8 and add jUnit5 as a dependency and also adds the Maven Surefire plugin for running tests.
Now, create the source code file at src/main/java/Calculator.java:
public class Calculator {
public int add(int a , int b){
return a + b;
}
}
And create the test file at src/test/java/CalculatorTest.java:
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
class CalculatorTest {
@Test
void AddTwoNumbers() {
Calculator calc = new Calculator();
int result = calc.add(1,1);
assertEquals(2, result);
}
@Test
void AddNegativeNumbers() {
Calculator calc = new Calculator();
int result = calc.add(-1,-1);
assertEquals(-2, result);
}
@Test
void AddZeroToANumber(){
Calculator calc = new Calculator();
int result = calc.add(1,0);
assertEquals(1, result);
}
}
And that’s it. Your system and code are ready to proceed to containerization.
Containerize the Tests
In this section, you’ll see how to containerize your tests, after which DevOps can run them in your CI pipeline.
First, create a Dockerfile, which you will use to create a Docker image to be run in your CI pipelines:
FROM maven:3.8.1-openjdk-11-slim
WORKDIR /src
ENTRYPOINT ["mvn", "clean"]
Notice, you’re using Java 11 and Maven 3.8.1 to build the Docker image. You can create many such images with different versions of Java and Maven, which can then run in a single test environment, reducing the maintenance cost of different environments and eliminating interference due to different versions.
Build the docker image:
docker build -t test-in-containers:1.0-3.8.1-openjdk-11-slim .
Here, you are adding Maven and Java versions in the Docker image file tag for easy referencing.
Once the image is built, you can run the tests via the following command:
docker run -v `pwd`/test-in-containers:/src test-in-containers: 1.0-3.8.1-openjdk-11-slim test
Notice how you are mounting the source code inside the container using Docker volumes. You can then hand over the image to DevOps to set up these tests to run on every commit.
You can also mount another volume and generate test reports, but for the purpose of this demo, you will stick to just running tests.
In the prerequisites section, it was noted that you need a certain version of Java and Maven to make sure your tests run properly. Over time, you might need different versions for different projects, and it can get challenging for DevOps to maintain consistent test environments.
For example, one of your applications might need Java 8 and the other might need Java 11. Having them both installed on the same machine might cause interference, or if you run tests on a single Java version, you run the risk of shipping buggy code.
Another scenario might be that you want to test your code for backward or forward compatibility with separate Java versions; for that, DevOps will also need to carefully maintain different environments.
In all of these scenarios, containerizing your unit tests can help alleviate the problem of maintaining different test environments. DevOps just needs to make sure that they have an environment where Docker containers can run and the rest is managed by the developers themselves. Tests running in containers can solve this age-old problem of dev-test environment drift.
Note: We’ve covered how to containerize unit tests, but the same principles can be applied to other tests, namely integration and end-to-end.
CI Test Observability
Running tests in containers is not all rosy and come with its own set of challenges. One of the issues that developers and DevOps face while running their tests in a CI pipeline is test observability, including CI test troubleshooting and debugging. You might have test failures you want to be notified about and resolve, or you may experience a slower build that’s hampering deployment frequency. But there is no straightforward way to get a peek into what is happening inside your CI pipeline.
That’s where Thundra Foresight comes into play. Thundra Foresight is a shift-left tool that can help you gain CI test observability. It provides CI test debugging and CI test troubleshooting capabilities, which can help you lower your build times and increase deployment frequency.
Foresight keeps track of all your test runs and KPIs associated with those runs—things like how much time each test has taken and the reason for slow tests or test failures. Foresight has automated distributed tracing that helps you debug the root cause of slow/failing CI pipelines.
Conclusion
Running your tests (unit, integration, or E2E) in containers alleviates the pain of maintaining cumbersome test environments and helps you streamline your CI pipelines. Containerizing tests can save your organization hours of having to debug issues caused by mismatched software versions. And although CI test observability and CI test debugging of tests can be an issue, Thundra Foresight helps you gain rapid and detailed insights into key metrics for your CI pipelines to keep them on track.
We have integrated Foresight with some popular open-source projects. To discover more about Foresight’s capabilities, explore the build process and the test runs for these open source projects. Also, if you would like to see other open-source projects feel free to contact us!
Containers have become the de facto standard for deploying applications. Whether developers are deploying them in the public cloud or a data center, containerization can bring a lot of benefits like a consistent environment, repeatable builds, and easy deployment patterns.
But so far, the focus has been on running applications in containers, with very little attention given to running tests in containers.
In this article, we’ll see how you can leverage containers for tests as well and why this could prove beneficial to your organization. We’ll also discuss how to effectively handle CI test observability and CI test debugging needed for containerized tests.
Pros of Running Tests in Containers
Running test cases in containers offers a brilliant value proposition with some great advantages:
- Consistent environment: With identical containers in every environment, there is no drift in software versions across dev and test environments. This significantly increases developer velocity and helps deliver code faster.
- Isolation: Containers provide an ephemeral environment to run your tests, making sure your tests run in isolation and don’t interfere with one another.
- Repeatable builds: Containers are built against a consistent definition file, which can be used to recreate containers anytime.
- Quality CI pipelines: Having containerized test cases makes it easy to set up CI pipelines and maintain high-quality builds.
Demo Setup
Before we jump into the demo, you’ll need to make sure you have some prerequisites set up:
- Java 8+ for a sample Java application and unit tests
- Maven 3.6.0+ to download your dependencies and run your tests
- Docker to create a Docker image to run your tests
Code Setup
This demo will entail a simple Java application with unit tests written in jUnit5.
First, create the project using Maven:
mvn archetype:generate -DgroupId=com.example
<-DartifactId=test-in-containers>
<-DarchetypeArtifactId=maven-archetype-quickstart>
<-DinteractiveMode=false>
This will create the folder structure with a sample Java project.
Delete the already existing code files at src/main/java/App.java and src/test/java/AppTest.java.
Next, replace the pom.xml with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>test-in-containers</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>RELEASE</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.0.0-M5</version>
</plugin>
</plugin>
</plugins>
</build>
</project>
This configures Maven to use Java 1.8 and add jUnit5 as a dependency and also adds the Maven Surefire plugin for running tests.
Now, create the source code file at src/main/java/Calculator.java:
public class Calculator {
public int add(int a , int b){
return a + b;
}
}
And create the test file at src/test/java/CalculatorTest.java:
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
class CalculatorTest {
@Test
void AddTwoNumbers() {
Calculator calc = new Calculator();
int result = calc.add(1,1);
assertEquals(2, result);
}
@Test
void AddNegativeNumbers() {
Calculator calc = new Calculator();
int result = calc.add(-1,-1);
assertEquals(-2, result);
}
@Test
void AddZeroToANumber(){
Calculator calc = new Calculator();
int result = calc.add(1,0);
assertEquals(1, result);
}
}
And that’s it. Your system and code are ready to proceed to containerization.
Containerize the Tests
In this section, you’ll see how to containerize your tests, after which DevOps can run them in your CI pipeline.
First, create a Dockerfile, which you will use to create a Docker image to be run in your CI pipelines:
FROM maven:3.8.1-openjdk-11-slim
WORKDIR /src
ENTRYPOINT ["mvn", "clean"]
Notice, you’re using Java 11 and Maven 3.8.1 to build the Docker image. You can create many such images with different versions of Java and Maven, which can then run in a single test environment, reducing the maintenance cost of different environments and eliminating interference due to different versions.
Build the docker image:
docker build -t test-in-containers:1.0-3.8.1-openjdk-11-slim .
Here, you are adding Maven and Java versions in the Docker image file tag for easy referencing.
Once the image is built, you can run the tests via the following command:
docker run -v `pwd`/test-in-containers:/src test-in-containers: 1.0-3.8.1-openjdk-11-slim test
Notice how you are mounting the source code inside the container using Docker volumes. You can then hand over the image to DevOps to set up these tests to run on every commit.
You can also mount another volume and generate test reports, but for the purpose of this demo, you will stick to just running tests.
In the prerequisites section, it was noted that you need a certain version of Java and Maven to make sure your tests run properly. Over time, you might need different versions for different projects, and it can get challenging for DevOps to maintain consistent test environments.
For example, one of your applications might need Java 8 and the other might need Java 11. Having them both installed on the same machine might cause interference, or if you run tests on a single Java version, you run the risk of shipping buggy code.
Another scenario might be that you want to test your code for backward or forward compatibility with separate Java versions; for that, DevOps will also need to carefully maintain different environments.
In all of these scenarios, containerizing your unit tests can help alleviate the problem of maintaining different test environments. DevOps just needs to make sure that they have an environment where Docker containers can run and the rest is managed by the developers themselves. Tests running in containers can solve this age-old problem of dev-test environment drift.
Note: We’ve covered how to containerize unit tests, but the same principles can be applied to other tests, namely integration and end-to-end.
CI Test Observability
Running tests in containers is not all rosy and come with its own set of challenges. One of the issues that developers and DevOps face while running their tests in a CI pipeline is test observability, including CI test troubleshooting and debugging. You might have test failures you want to be notified about and resolve, or you may experience a slower build that’s hampering deployment frequency. But there is no straightforward way to get a peek into what is happening inside your CI pipeline.
That’s where Thundra Foresight comes into play. Thundra Foresight is a shift-left tool that can help you gain CI test observability. It provides CI test debugging and CI test troubleshooting capabilities, which can help you lower your build times and increase deployment frequency.
Foresight keeps track of all your test runs and KPIs associated with those runs—things like how much time each test has taken and the reason for slow tests or test failures. Foresight has automated distributed tracing that helps you debug the root cause of slow/failing CI pipelines.
Conclusion
Running your tests (unit, integration, or E2E) in containers alleviates the pain of maintaining cumbersome test environments and helps you streamline your CI pipelines. Containerizing tests can save your organization hours of having to debug issues caused by mismatched software versions. And although CI test observability and CI test debugging of tests can be an issue, Thundra Foresight helps you gain rapid and detailed insights into key metrics for your CI pipelines to keep them on track.
We have integrated Foresight with some popular open-source projects. To discover more about Foresight’s capabilities, explore the build process and the test runs for these open source projects. Also, if you would like to see other open-source projects feel free to contact us!
