SBN

Thundra Foresight: Now Supporting Azure DevOps CI Platform

5 minutes read


POSTED Jan, 2022

dot

IN
Debugging

Thundra Foresight: Now Supporting Azure DevOps CI Platform

Yusuf Erdem

Written by Yusuf Erdem

Senior Software Engineer at Thundra

linkedin-share

 X

Are you in DevOps and sick of looking at logs while waiting for your build to finish? Or are you a quality engineer running automated tests in the CI pipeline and wondering why your tests failed? Perhaps you’re a developer trying to debug a failed application in the test environment. If any of these sound familiar, you need a full-stack tool that can help you with anything from monitoring a running application in production to provisioning your integration test or capturing screenshots of your end-to-end tests. Thundra Foresight offers all of that, and its central dashboard gives you a bird’s eye view of your application from one place. To make things even better, Foresight now supports the Azure DevOps CI platform, along with TeamCity, Bitbucket, CircleCI, and GitHub integration. In this article, we’ll show you just how easy it is to integrate your project with Thundra Foresight, and how Foresight can improve your CI experience.

Getting Started with Foresight

To get started, we’ll log in to Foresight and create a new project.

Step 1: Create an Account

Your first step will be to sign up for a Thundra Foresight account.

Figure 1: Sign up for a Thundra account

From the sign-up page, you can create an account with an existing GitHub or Google account, or with your email address.

Step 2: Create a New Project in Foresight

Once you sign up with Foresight and sign in to the platform, you’ll see three product options to choose from APM, Foresight, and Sidekick. Choose the Foresight product.

Figure 2: Choose Foresight from among Thundra’s products

Next, click on “connect repositories” to start integrating Foresight with your repository.

Figure 3: Connect a repository to get started with Foresight

After you’ve connected a repository, create a new project in Foresight for monitoring the application. Name your project and hit “Create.”

Figure 4: Create new project with Foresight

Figure 5: There are multiple platforms available with Foresight

Now that you’ve created a project, you have multiple options for getting started. In this article, we’re focusing on how Thundra Foresight can be integrated, so we’ll select the “Test Monitoring” option.

Figure 6: Test monitoring platforms supported

Since we’re integrating Foresight with Azure, we’ll choose the Azure DevOps option.

Step 3: Collect Project and API Keys

When you select Azure DevOps, Thundra automatically provides the project key ID and the API key. You’ll need this information for a future step, so make sure you record it.

Figure 7: Auto-generated configuration for getting started with Azure DevOps

Integrating the Azure DevOps CI Platform with Thundra Foresight

In order to integrate the Azure DevOps CI with Foresight, we’ll need to install the Visual Studio Extension.

Figure 8: Thundra Foresight Initializer installation

From the Thundra Foresight Initializer page, click the “Get it free” button, then select the organization you wish to install Foresight into. The Azure DevOps CI platform has functionalities that support users creating many organizations, and each organization can have its own project. So, when you integrate Thundra Foresight with Azure DevOps CI, it’s important to specify the organization you want to integrate Foresight with.

Once that’s complete, we’ll begin configuring for the pipeline in the Azure DevOps CI. You can choose between “Use the classic editor” for simplicity or creating a YAML config file. For our purposes, we’ll choose the YAML file option because it provides more flexibility.

We’ll now use the keys Thundra auto-generated for us earlier and copy them into the Azure DevOps YAML file.

In our sample end-to-end test job, we’ll assume we’re working with Java and the Maven build tool. The YAML file will look like this:


# Maven
# Build your Java project and run tests with Apache Maven.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/java
trigger:
- main
pool:
  vmImage: ubuntu-latest
steps:
- task: ThundraForesightInitializer@0
  inputs:
    api_key: $(THUNDRA_APIKEY)
    project_id: $(THUNDRA_AGENT_TEST_PROJECT_ID)
    agent_version: '2.7.56'
    instrumenter_version: '0.0.6'
    build_run_type: 'Maven'
- task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    mavenOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/surefire-reports/TEST-*.xml'
    goals: 'test'

For security reasons, we don’t usually expose the credentials to the pipeline. To get around this, we’ll also need to add environment variables into the pipeline in Azure DevOps.

Figure 9: Pipeline with environment variables setup for Thundra Foresight

Now we’re ready to trigger the pipeline.

Note that the configuration pipeline for Gradle and other build tools is similar to Maven. Foresight provides the template for each build tool, so you won’t have to figure out how to write them yourself. Instead, just copy the auto-generated template and paste it into the config YAML file for Azure DevOps.

Running the CI Pipeline from Azure’s DevOps CI Platform

With Foresight, we’re now able to monitor multiple aspects of our application, including:

  • An overview of all tests run
  • Slowest tests
  • Useful logs and metrics

Figure 10: Overview of a test run in Thundra Foresight

The test run overview is important because we often have hundreds of tests. Going through all of them for details each time just isn’t feasible. This summary board illustrates which tests might have a problem, making them easy to target.

If we go into each test’s details, we’ll also see traces for the requests that have been made in our test. The below image is an example integration test among different services in the AWS platform. We can see that the request to the API gateway/products was successful and that its call to the Lambda function for creating a new product was also successful. However, we cannot create a product successfully because we lost the connection to the Amazon DynamoDB table.

Figure 11: Distributed traces collected from the test

Foresight also offers Time Travel Debugging, which gives us a log for each request that has been made in the test. Sometimes a test fails only once, and we can’t reproduce it. The Time Travel Debugging feature allows us to look back in time at test details so we can easily debug it. This saves testers the time they’d otherwise spend setting up an environment for reproducing the bug, and trying different ways to reproduce it.

Figure 12: Time Travel Debugging

We already have the capability to monitor the application and debug the integration tests. How about the end-to-end test that failed?

So often in software testing, end-to-end tests take the most effort to debug. That’s because in end-to-end tests, there are a lot of dependencies, or third-party apps, involved. Luckily, with Foresight, we can save screenshots when we run our end-to-end tests and easily access them from the dashboard. These screenshots provide a lot of information we don’t get through logs.

Figure 13: Screenshot for the end-to-end testing

Foresight offers a complete picture of what your application is doing, what the hidden issues are, why your integration tests are failing, and which of your end-to-end tests are flaky. Open-source software offers some of these capabilities, but it requires you to integrate a lot of tools and dependencies. You’d have to create a new service to handle the logs or to save screenshots for the end-to-end test.

These steps pull your entire software team away from building and improving your application. By integrating Foresight with Azure’s DevOps CI platform, you have all of it instantly.

Conclusion

Building a running CI pipeline might not be a difficult task, but maintaining it over time and getting fast feedback from your tests certainly can be. If your tests are flaky enough, your team may want to remove automated tests altogether in order to gain speed. But this is dangerous, as it can impact quality and lead customers to look elsewhere for better products.

With the help of Thundra Foresight, your team can always backtrack to easily find the root cause of issues. You can finally focus on developing new features and building applications that make your customers happy, rather than struggling to debug failed tests.

Take the headache out of debugging tests and start your Thundra Foresight journey today.

 

×

SUBSCRIBE TO OUR BLOG

Get our new blogs delivered straight to your inbox.

 

THANKS FOR SIGNING UP!

We’ll make sure to share the best materials crafted for you!

5 minutes read


POSTED Jan, 2022

dot

IN
Debugging

Thundra Foresight: Now Supporting Azure DevOps CI Platform

Yusuf Erdem

Written by Yusuf Erdem

Senior Software Engineer at Thundra

linkedin-share

 X

Are you in DevOps and sick of looking at logs while waiting for your build to finish? Or are you a quality engineer running automated tests in the CI pipeline and wondering why your tests failed? Perhaps you’re a developer trying to debug a failed application in the test environment. If any of these sound familiar, you need a full-stack tool that can help you with anything from monitoring a running application in production to provisioning your integration test or capturing screenshots of your end-to-end tests. Thundra Foresight offers all of that, and its central dashboard gives you a bird’s eye view of your application from one place. To make things even better, Foresight now supports the Azure DevOps CI platform, along with TeamCity, Bitbucket, CircleCI, and GitHub integration. In this article, we’ll show you just how easy it is to integrate your project with Thundra Foresight, and how Foresight can improve your CI experience.

Getting Started with Foresight

To get started, we’ll log in to Foresight and create a new project.

Step 1: Create an Account

Your first step will be to sign up for a Thundra Foresight account.

Figure 1: Sign up for a Thundra account

From the sign-up page, you can create an account with an existing GitHub or Google account, or with your email address.

Step 2: Create a New Project in Foresight

Once you sign up with Foresight and sign in to the platform, you’ll see three product options to choose from APM, Foresight, and Sidekick. Choose the Foresight product.

Figure 2: Choose Foresight from among Thundra’s products

Next, click on “connect repositories” to start integrating Foresight with your repository.

Figure 3: Connect a repository to get started with Foresight

After you’ve connected a repository, create a new project in Foresight for monitoring the application. Name your project and hit “Create.”

Figure 4: Create new project with Foresight

Figure 5: There are multiple platforms available with Foresight

Now that you’ve created a project, you have multiple options for getting started. In this article, we’re focusing on how Thundra Foresight can be integrated, so we’ll select the “Test Monitoring” option.

Figure 6: Test monitoring platforms supported

Since we’re integrating Foresight with Azure, we’ll choose the Azure DevOps option.

Step 3: Collect Project and API Keys

When you select Azure DevOps, Thundra automatically provides the project key ID and the API key. You’ll need this information for a future step, so make sure you record it.

Figure 7: Auto-generated configuration for getting started with Azure DevOps

Integrating the Azure DevOps CI Platform with Thundra Foresight

In order to integrate the Azure DevOps CI with Foresight, we’ll need to install the Visual Studio Extension.

Figure 8: Thundra Foresight Initializer installation

From the Thundra Foresight Initializer page, click the “Get it free” button, then select the organization you wish to install Foresight into. The Azure DevOps CI platform has functionalities that support users creating many organizations, and each organization can have its own project. So, when you integrate Thundra Foresight with Azure DevOps CI, it’s important to specify the organization you want to integrate Foresight with.

Once that’s complete, we’ll begin configuring for the pipeline in the Azure DevOps CI. You can choose between “Use the classic editor” for simplicity or creating a YAML config file. For our purposes, we’ll choose the YAML file option because it provides more flexibility.

We’ll now use the keys Thundra auto-generated for us earlier and copy them into the Azure DevOps YAML file.

In our sample end-to-end test job, we’ll assume we’re working with Java and the Maven build tool. The YAML file will look like this:


# Maven
# Build your Java project and run tests with Apache Maven.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/java
trigger:
- main
pool:
  vmImage: ubuntu-latest
steps:
- task: ThundraForesightInitializer@0
  inputs:
    api_key: $(THUNDRA_APIKEY)
    project_id: $(THUNDRA_AGENT_TEST_PROJECT_ID)
    agent_version: '2.7.56'
    instrumenter_version: '0.0.6'
    build_run_type: 'Maven'
- task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    mavenOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/surefire-reports/TEST-*.xml'
    goals: 'test'

For security reasons, we don’t usually expose the credentials to the pipeline. To get around this, we’ll also need to add environment variables into the pipeline in Azure DevOps.

Figure 9: Pipeline with environment variables setup for Thundra Foresight

Now we’re ready to trigger the pipeline.

Note that the configuration pipeline for Gradle and other build tools is similar to Maven. Foresight provides the template for each build tool, so you won’t have to figure out how to write them yourself. Instead, just copy the auto-generated template and paste it into the config YAML file for Azure DevOps.

Running the CI Pipeline from Azure’s DevOps CI Platform

With Foresight, we’re now able to monitor multiple aspects of our application, including:

  • An overview of all tests run
  • Slowest tests
  • Useful logs and metrics

Figure 10: Overview of a test run in Thundra Foresight

The test run overview is important because we often have hundreds of tests. Going through all of them for details each time just isn’t feasible. This summary board illustrates which tests might have a problem, making them easy to target.

If we go into each test’s details, we’ll also see traces for the requests that have been made in our test. The below image is an example integration test among different services in the AWS platform. We can see that the request to the API gateway/products was successful and that its call to the Lambda function for creating a new product was also successful. However, we cannot create a product successfully because we lost the connection to the Amazon DynamoDB table.

Figure 11: Distributed traces collected from the test

Foresight also offers Time Travel Debugging, which gives us a log for each request that has been made in the test. Sometimes a test fails only once, and we can’t reproduce it. The Time Travel Debugging feature allows us to look back in time at test details so we can easily debug it. This saves testers the time they’d otherwise spend setting up an environment for reproducing the bug, and trying different ways to reproduce it.

Figure 12: Time Travel Debugging

We already have the capability to monitor the application and debug the integration tests. How about the end-to-end test that failed?

So often in software testing, end-to-end tests take the most effort to debug. That’s because in end-to-end tests, there are a lot of dependencies, or third-party apps, involved. Luckily, with Foresight, we can save screenshots when we run our end-to-end tests and easily access them from the dashboard. These screenshots provide a lot of information we don’t get through logs.

Figure 13: Screenshot for the end-to-end testing

Foresight offers a complete picture of what your application is doing, what the hidden issues are, why your integration tests are failing, and which of your end-to-end tests are flaky. Open-source software offers some of these capabilities, but it requires you to integrate a lot of tools and dependencies. You’d have to create a new service to handle the logs or to save screenshots for the end-to-end test.

These steps pull your entire software team away from building and improving your application. By integrating Foresight with Azure’s DevOps CI platform, you have all of it instantly.

Conclusion

Building a running CI pipeline might not be a difficult task, but maintaining it over time and getting fast feedback from your tests certainly can be. If your tests are flaky enough, your team may want to remove automated tests altogether in order to gain speed. But this is dangerous, as it can impact quality and lead customers to look elsewhere for better products.

With the help of Thundra Foresight, your team can always backtrack to easily find the root cause of issues. You can finally focus on developing new features and building applications that make your customers happy, rather than struggling to debug failed tests.

Take the headache out of debugging tests and start your Thundra Foresight journey today.

 

Are you in DevOps and sick of looking at logs while waiting for your build to finish? Or are you a quality engineer running automated tests in the CI pipeline and wondering why your tests failed? Perhaps you’re a developer trying to debug a failed application in the test environment. If any of these sound familiar, you need a full-stack tool that can help you with anything from monitoring a running application in production to provisioning your integration test or capturing screenshots of your end-to-end tests. Thundra Foresight offers all of that, and its central dashboard gives you a bird’s eye view of your application from one place. To make things even better, Foresight now supports the Azure DevOps CI platform, along with TeamCity, Bitbucket, CircleCI, and GitHub integration. In this article, we’ll show you just how easy it is to integrate your project with Thundra Foresight, and how Foresight can improve your CI experience.

Getting Started with Foresight

To get started, we’ll log in to Foresight and create a new project.

Step 1: Create an Account

Your first step will be to sign up for a Thundra Foresight account.

Figure 1: Sign up for a Thundra account

From the sign-up page, you can create an account with an existing GitHub or Google account, or with your email address.

Step 2: Create a New Project in Foresight

Once you sign up with Foresight and sign in to the platform, you’ll see three product options to choose from APM, Foresight, and Sidekick. Choose the Foresight product.

Figure 2: Choose Foresight from among Thundra’s products

Next, click on “connect repositories” to start integrating Foresight with your repository.

Figure 3: Connect a repository to get started with Foresight

After you’ve connected a repository, create a new project in Foresight for monitoring the application. Name your project and hit “Create.”

Figure 4: Create new project with Foresight

Figure 5: There are multiple platforms available with Foresight

Now that you’ve created a project, you have multiple options for getting started. In this article, we’re focusing on how Thundra Foresight can be integrated, so we’ll select the “Test Monitoring” option.

Figure 6: Test monitoring platforms supported

Since we’re integrating Foresight with Azure, we’ll choose the Azure DevOps option.

Step 3: Collect Project and API Keys

When you select Azure DevOps, Thundra automatically provides the project key ID and the API key. You’ll need this information for a future step, so make sure you record it.

Figure 7: Auto-generated configuration for getting started with Azure DevOps

Integrating the Azure DevOps CI Platform with Thundra Foresight

In order to integrate the Azure DevOps CI with Foresight, we’ll need to install the Visual Studio Extension.

Figure 8: Thundra Foresight Initializer installation

From the Thundra Foresight Initializer page, click the “Get it free” button, then select the organization you wish to install Foresight into. The Azure DevOps CI platform has functionalities that support users creating many organizations, and each organization can have its own project. So, when you integrate Thundra Foresight with Azure DevOps CI, it’s important to specify the organization you want to integrate Foresight with.

Once that’s complete, we’ll begin configuring for the pipeline in the Azure DevOps CI. You can choose between “Use the classic editor” for simplicity or creating a YAML config file. For our purposes, we’ll choose the YAML file option because it provides more flexibility.

We’ll now use the keys Thundra auto-generated for us earlier and copy them into the Azure DevOps YAML file.

In our sample end-to-end test job, we’ll assume we’re working with Java and the Maven build tool. The YAML file will look like this:


# Maven
# Build your Java project and run tests with Apache Maven.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/java
trigger:
- main
pool:
  vmImage: ubuntu-latest
steps:
- task: ThundraForesightInitializer@0
  inputs:
    api_key: $(THUNDRA_APIKEY)
    project_id: $(THUNDRA_AGENT_TEST_PROJECT_ID)
    agent_version: '2.7.56'
    instrumenter_version: '0.0.6'
    build_run_type: 'Maven'
- task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    mavenOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/surefire-reports/TEST-*.xml'
    goals: 'test'

For security reasons, we don’t usually expose the credentials to the pipeline. To get around this, we’ll also need to add environment variables into the pipeline in Azure DevOps.

Figure 9: Pipeline with environment variables setup for Thundra Foresight

Now we’re ready to trigger the pipeline.

Note that the configuration pipeline for Gradle and other build tools is similar to Maven. Foresight provides the template for each build tool, so you won’t have to figure out how to write them yourself. Instead, just copy the auto-generated template and paste it into the config YAML file for Azure DevOps.

Running the CI Pipeline from Azure’s DevOps CI Platform

With Foresight, we’re now able to monitor multiple aspects of our application, including:

  • An overview of all tests run
  • Slowest tests
  • Useful logs and metrics

Figure 10: Overview of a test run in Thundra Foresight

The test run overview is important because we often have hundreds of tests. Going through all of them for details each time just isn’t feasible. This summary board illustrates which tests might have a problem, making them easy to target.

If we go into each test’s details, we’ll also see traces for the requests that have been made in our test. The below image is an example integration test among different services in the AWS platform. We can see that the request to the API gateway/products was successful and that its call to the Lambda function for creating a new product was also successful. However, we cannot create a product successfully because we lost the connection to the Amazon DynamoDB table.

Figure 11: Distributed traces collected from the test

Foresight also offers Time Travel Debugging, which gives us a log for each request that has been made in the test. Sometimes a test fails only once, and we can’t reproduce it. The Time Travel Debugging feature allows us to look back in time at test details so we can easily debug it. This saves testers the time they’d otherwise spend setting up an environment for reproducing the bug, and trying different ways to reproduce it.

Figure 12: Time Travel Debugging

We already have the capability to monitor the application and debug the integration tests. How about the end-to-end test that failed?

So often in software testing, end-to-end tests take the most effort to debug. That’s because in end-to-end tests, there are a lot of dependencies, or third-party apps, involved. Luckily, with Foresight, we can save screenshots when we run our end-to-end tests and easily access them from the dashboard. These screenshots provide a lot of information we don’t get through logs.

Figure 13: Screenshot for the end-to-end testing

Foresight offers a complete picture of what your application is doing, what the hidden issues are, why your integration tests are failing, and which of your end-to-end tests are flaky. Open-source software offers some of these capabilities, but it requires you to integrate a lot of tools and dependencies. You’d have to create a new service to handle the logs or to save screenshots for the end-to-end test.

These steps pull your entire software team away from building and improving your application. By integrating Foresight with Azure’s DevOps CI platform, you have all of it instantly.

Conclusion

Building a running CI pipeline might not be a difficult task, but maintaining it over time and getting fast feedback from your tests certainly can be. If your tests are flaky enough, your team may want to remove automated tests altogether in order to gain speed. But this is dangerous, as it can impact quality and lead customers to look elsewhere for better products.

With the help of Thundra Foresight, your team can always backtrack to easily find the root cause of issues. You can finally focus on developing new features and building applications that make your customers happy, rather than struggling to debug failed tests.

Take the headache out of debugging tests and start your Thundra Foresight journey today.

 

Are you in DevOps and sick of looking at logs while waiting for your build to finish? Or are you a quality engineer running automated tests in the CI pipeline and wondering why your tests failed? Perhaps you’re a developer trying to debug a failed application in the test environment. If any of these sound familiar, you need a full-stack tool that can help you with anything from monitoring a running application in production to provisioning your integration test or capturing screenshots of your end-to-end tests. Thundra Foresight offers all of that, and its central dashboard gives you a bird’s eye view of your application from one place. To make things even better, Foresight now supports the Azure DevOps CI platform, along with TeamCity, Bitbucket, CircleCI, and GitHub integration. In this article, we’ll show you just how easy it is to integrate your project with Thundra Foresight, and how Foresight can improve your CI experience.

Getting Started with Foresight

To get started, we’ll log in to Foresight and create a new project.

Step 1: Create an Account

Your first step will be to sign up for a Thundra Foresight account.

Figure 1: Sign up for a Thundra account

From the sign-up page, you can create an account with an existing GitHub or Google account, or with your email address.

Step 2: Create a New Project in Foresight

Once you sign up with Foresight and sign in to the platform, you’ll see three product options to choose from APM, Foresight, and Sidekick. Choose the Foresight product.

Figure 2: Choose Foresight from among Thundra’s products

Next, click on “connect repositories” to start integrating Foresight with your repository.

Figure 3: Connect a repository to get started with Foresight

After you’ve connected a repository, create a new project in Foresight for monitoring the application. Name your project and hit “Create.”

Figure 4: Create new project with Foresight

Figure 5: There are multiple platforms available with Foresight

Now that you’ve created a project, you have multiple options for getting started. In this article, we’re focusing on how Thundra Foresight can be integrated, so we’ll select the “Test Monitoring” option.

Figure 6: Test monitoring platforms supported

Since we’re integrating Foresight with Azure, we’ll choose the Azure DevOps option.

Step 3: Collect Project and API Keys

When you select Azure DevOps, Thundra automatically provides the project key ID and the API key. You’ll need this information for a future step, so make sure you record it.

Figure 7: Auto-generated configuration for getting started with Azure DevOps

Integrating the Azure DevOps CI Platform with Thundra Foresight

In order to integrate the Azure DevOps CI with Foresight, we’ll need to install the Visual Studio Extension.

Figure 8: Thundra Foresight Initializer installation

From the Thundra Foresight Initializer page, click the “Get it free” button, then select the organization you wish to install Foresight into. The Azure DevOps CI platform has functionalities that support users creating many organizations, and each organization can have its own project. So, when you integrate Thundra Foresight with Azure DevOps CI, it’s important to specify the organization you want to integrate Foresight with.

Once that’s complete, we’ll begin configuring for the pipeline in the Azure DevOps CI. You can choose between “Use the classic editor” for simplicity or creating a YAML config file. For our purposes, we’ll choose the YAML file option because it provides more flexibility.

We’ll now use the keys Thundra auto-generated for us earlier and copy them into the Azure DevOps YAML file.

In our sample end-to-end test job, we’ll assume we’re working with Java and the Maven build tool. The YAML file will look like this:


# Maven
# Build your Java project and run tests with Apache Maven.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/java
trigger:
- main
pool:
  vmImage: ubuntu-latest
steps:
- task: ThundraForesightInitializer@0
  inputs:
    api_key: $(THUNDRA_APIKEY)
    project_id: $(THUNDRA_AGENT_TEST_PROJECT_ID)
    agent_version: '2.7.56'
    instrumenter_version: '0.0.6'
    build_run_type: 'Maven'
- task: Maven@3
  inputs:
    mavenPomFile: 'pom.xml'
    mavenOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/surefire-reports/TEST-*.xml'
    goals: 'test'

For security reasons, we don’t usually expose the credentials to the pipeline. To get around this, we’ll also need to add environment variables into the pipeline in Azure DevOps.

Figure 9: Pipeline with environment variables setup for Thundra Foresight

Now we’re ready to trigger the pipeline.

Note that the configuration pipeline for Gradle and other build tools is similar to Maven. Foresight provides the template for each build tool, so you won’t have to figure out how to write them yourself. Instead, just copy the auto-generated template and paste it into the config YAML file for Azure DevOps.

Running the CI Pipeline from Azure’s DevOps CI Platform

With Foresight, we’re now able to monitor multiple aspects of our application, including:

  • An overview of all tests run
  • Slowest tests
  • Useful logs and metrics

Figure 10: Overview of a test run in Thundra Foresight

The test run overview is important because we often have hundreds of tests. Going through all of them for details each time just isn’t feasible. This summary board illustrates which tests might have a problem, making them easy to target.

If we go into each test’s details, we’ll also see traces for the requests that have been made in our test. The below image is an example integration test among different services in the AWS platform. We can see that the request to the API gateway/products was successful and that its call to the Lambda function for creating a new product was also successful. However, we cannot create a product successfully because we lost the connection to the Amazon DynamoDB table.

Figure 11: Distributed traces collected from the test

Foresight also offers Time Travel Debugging, which gives us a log for each request that has been made in the test. Sometimes a test fails only once, and we can’t reproduce it. The Time Travel Debugging feature allows us to look back in time at test details so we can easily debug it. This saves testers the time they’d otherwise spend setting up an environment for reproducing the bug, and trying different ways to reproduce it.

Figure 12: Time Travel Debugging

We already have the capability to monitor the application and debug the integration tests. How about the end-to-end test that failed?

So often in software testing, end-to-end tests take the most effort to debug. That’s because in end-to-end tests, there are a lot of dependencies, or third-party apps, involved. Luckily, with Foresight, we can save screenshots when we run our end-to-end tests and easily access them from the dashboard. These screenshots provide a lot of information we don’t get through logs.

Figure 13: Screenshot for the end-to-end testing

Foresight offers a complete picture of what your application is doing, what the hidden issues are, why your integration tests are failing, and which of your end-to-end tests are flaky. Open-source software offers some of these capabilities, but it requires you to integrate a lot of tools and dependencies. You’d have to create a new service to handle the logs or to save screenshots for the end-to-end test.

These steps pull your entire software team away from building and improving your application. By integrating Foresight with Azure’s DevOps CI platform, you have all of it instantly.

Conclusion

Building a running CI pipeline might not be a difficult task, but maintaining it over time and getting fast feedback from your tests certainly can be. If your tests are flaky enough, your team may want to remove automated tests altogether in order to gain speed. But this is dangerous, as it can impact quality and lead customers to look elsewhere for better products.

With the help of Thundra Foresight, your team can always backtrack to easily find the root cause of issues. You can finally focus on developing new features and building applications that make your customers happy, rather than struggling to debug failed tests.

Take the headache out of debugging tests and start your Thundra Foresight journey today.

 

*** This is a Security Bloggers Network syndicated blog from Thundra blog authored by Yusuf Erdem. Read the original post at: https://blog.thundra.io/thundra-foresight-now-supporting-azure-devops-ci-platform