Jenkins integration with tools for SCM, Build Management and Test Management (Covers Azure DevOps)

Jenkins is an Open Source software written in Java. It was originally developed as a Hudson project (in year 2004) which soon became a better option for build servers.

Jenkins is a DevOps tool used for Continuous Integration, build management, testing and continuous delivery. It can work with various plugins to integrate with tools like Ant, Maven, Git, Azure DevOps etc. It provides Pipelines with which implementation of continuous delivery pipelines is possible. Jenkins Pipeline is a suite of features and plugins which helps implement automation of triggering the build at the time of code push to tests till deployment. This can be achieved either by using the Pipeline plugin or using Blue Ocean.

Server-side vs Desktop Build

Jenkins or Azure DevOps provides us a way of server side build.

There is a difference between server side build and desktop build. When a developer triggers a build on his/her machine with local code, it is termed as desktop build. But when the development is being done in a group, for the build, we need to take into consideration all the latest version available in a shared or remote repository, and for that we require Jenkins. It will fetch the latest code from the source control and start the build with a tool like Ant, Maven, MS Build etc.

The following diagram shows some of the tools which can be integrated with Jenkins to create pipelines with various tasks in build.

In a previous article (www.dotnetcurry.com/devops/1477/jenkins-for-devops) I discussed integration of Jenkins with tools like

• Code writing in Java
• Source Control with GitHub
• Build Management with And and Maven
• JUnit and Selenium tests for functional and UI testing
• Continuous Integration (CI) with GitHub push event

In this article, I will discuss:

• Code Writing in Visual Studio 2017 and Eclipse IDE
• Source Control in Azure DevOps as a source control (TFVS or Git)
• Build Management with MS Build
• MS Test and Selenium for testing
• CI with Azure DevOps
• CD with Azure Web App Service

Installation and Configuration of Jenkins is already discussed in the previous article.

Write Code in Java with Eclipse IDE

Create a project in Java using Eclipse IDE. We will add Ant build or Maven build to it later. We will use Source Control for Azure DevOps (formerly called as VSTS – Visual Studio Team Services).

I am assuming that JDK and Eclipse IDE along with Visual Studio 2017 are already installed on the machine

Add support to Azure DevOps from Eclipse

Team Explorer Everywhere (TEE) is the plugin which will connect your Java project to Azure DevOps. Browse to the Eclipse Market place and search for Team explorer and install the plugin. Make sure that Eclipse IDE can connect to internet for downloading.

Once the plugin is installed, you can select and open the perspective of Team Foundation Server as follows:

Provide the credentials for Azure DevOps for the plugin, and select the project you want to use as the Source Control.

The Team Project you are using as Source Control can be either TFVC (Team Foundation Version Control) or Git as per your choice.

Create Free Style project in Jenkins

Use Free Style project to create a new job in Jenkins. Instead of GitHub as Source Control, we need to provide Azure DevOps as the Source Control. We will have to add the plugin for the same to Jenkins. Go to plugins manager and add plugin for Team Foundation Server. Select Jenkins – Manage Jenkins – Configure System and add the organization name to the list of Team Project Collections.

Provide the credentials to connect to Azure DevOps.

Once the Source Control is selected, we can select ‘With Ant’ check box for Ant build with Jenkins.

The code will be fetched from Azure DevOps and the build will be done using build.xml (added to Eclipse project while converting to ant) and binaries created at the workspace. We can add unit and UI tests if required.

Let us now use Visual Studio 2017 to write code. We can use either GitHub or Azure DevOps as the source control.

If you want to use GitHub as Source Control, don’t forget to add the plugin to connect to GitHub from Visual Studio 2017 (use Tools – Extensions and Update – Online tab). I am using Azure DevOps so I have added the connection to the organization using Team Explorer.

Azure DevOps as Source Control, MS Build as Build and MS Test for Test Management

We will write code in C#, using Azure DevOps as repository, MS Build to build and MS test to test the unit tests. We will have to add plugins for MS Build and MS Test to Jenkins by using the plugin manager.

Write code with Visual Studio 2017 C#

Let us create a project in Visual Studio 2017 and use Azure DevOps as the Source Control. You can select project of any type. Use Manage Connection from Visual Studio 2017 to connect to Azure DevOps as well as the Team Project you have created.

Create Free Style project in Jenkins

Create a new job of free style project and select TFS as the Source Control. Provide the organization and project name along with the credentials to connect. If your project uses any NuGet packages, we will have to restore all those packages in the workspace on Jenkins before the build gets created. This can be achieved by running a batch command and using restore option for the solution. Provide the name of the solution so that all the packages will get restored. Download NuGet.exe to do this.

Configure the MS Build plugin by providing the path for MSBuild.exe as follows. Select Manage Jenkins – Global Tool Configuration and select MS Build Installable

Note: Remember the test framework you are using for MS Test and accordingly you will need to provide the path for MS Test either to mstest.exe or vstest.console.exe (for MSTest.TestFramework V2).

Add a build step for MS Build and also for MS Test. If we need to publish report in the form of a trx file we need to provide parameters accordingly. Finally, we can add post build action to publish report from .trx

After the build, the test result will be displayed on the dashboard for that project.

Continuous Integration (CI) with Azure DevOps and Jenkins

If we want to provide the CI for Jenkins build, we need to provide integration between Azure DevOps and Jenkins. I have used ngrok utility to create a public url for Jenkins and used the same to create service hook for the Azure DevOps Team Project.

Create Service hook for Team Project

Service hook provides the capability to perform tasks on other services (Jenkins build in this case). The concept is similar to Webhooks in GitHub. Create a subscription and select the trigger of Code Push, select the repository branch and click next. Provide the url published by ngrok and other details.

Note: ngrok utility will keep the url for a 8 hours duration at a given point of time. You will have to change it accordingly (or get a paid version).

Select the check box for ‘Build when a change is pushed to TFS/Team Services’ from Build Triggers. The build will be automatically triggered the moment you change the code in repository.

Instead of using the ngrok utility, you can also provide the url of the Virtual Machine if Jenkins is installed on it.

Continuous Deployment (CD) with Jenkins to Azure Web App

Finally, I will discuss how to do Continuous Deployment (CD) with Jenkins to Azure Web App. There are quite a few pre-requisites for the same.

I tried using the plugin for “Azure App Service Plugin”. You need to create credentials for this plugin to access Azure Web App Service. As I wanted to deploy a .war created using Java, I created Azure Web App with Java functionality. You can do this by selecting Application Settings and selecting the Java version, minor version along with the container (Apache Tomcat in my case). We also need app registration which will point to the URL for Azure Web Service. Create a client secret for this app registration and store the generated key at a safe place.

As shown in the above figure, we need to configure credentials to connect and deploy to Azure Web App Service.

Getting subscription id for the azure subscription is easy. Client id is the same as application id for the app registration. The client secret is the key you have stored and finally Tenant id is Azure Active Directory id. You also need to create service principal. To do so, please follow this link.

After all the values are supplied, you can verify the principal by clicking on the button for Verify Service Principal. The ID gets automatically generated if everything is ok.

Now comes the part where you add the post build action step for Publish Azure Web App.

Once you select the Azure credentials id, all resource group names will be available and once you select the resource group, you can select the app name.

You need to specify what kind of files needs to be published. I have selected the .war file which will be generated with Ant build.xml.

Disclaimer: I have tried this many number of times and as of this writing, I was not successful. It kept giving the following error:

I will be very happy if anyone knows how to resolve this error. I also tried creating a .zip file for the web application using Visual Studio 2017 and for that, I got the following error:

Finally, I found a not-so-elegant solution with which I used ftp upload to Azure Web App Service. I used WinSCP utility to upload the .war file using ftp credentials. You can get these credentials by using Deployment Centre – FTP. You can also get the FTP endpoint from there.

I have used the same username and password in a batch file as follows. The batch file simply copies the war file to the required folder. Make sure that you provide the wepapps path under site’s root folder as required for Apache Tomcat server.

(
echo option batch abort 
echo option confirm off
echo open ftp://UserName:Password@<ftp server name> -passive=on
echo put -transfer=ascii " <war file>
echo exit
)>temp.txt
<WinSCP path>winscp.com /script=temp.txt /log=<folder>\winscp.log"
del temp.txt /q >nul

I executed this batch file as a step in the project and triggered the build. Thankfully this approach worked and I finally deployed the web site to Azure Web App Service.

And with this, I created a complete CI CD cycle.

Conclusion

In this tutorial, I discussed how Azure DevOps can be used for Source Control for Java apps.

We also discussed how .NET code written using Visual Studio 2017 can be built and tested by using MS Build and MS Test plugins in Jenkins. I also showed how Continuous Integration can be achieved by using service hooks between Jenkins and Azure DevOps. Lastly, I discussed how Continuous Deployment can be achieved with Jenkins.

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

Gouri is a Trainer and Consultant on Azure DevOps and Azure Development. She has an experience of three decades in software training and consulting. She is a graduate from Pune University and PGDCA from Pune University. Gouri is a Microsoft Most Valuable Professional (MVP)  - Developer Technologies (Azure DevOps), Microsoft Certified Trainer (MCT) and a Microsoft Certified Azure DevOps Engineer Expert. She has conducted over 150 corporate trainings on various Microsoft technologies. She is a speaker with Pune User Group and has conducted sessions on Azure DevOps, SQL Server Business Intelligence and Mobile Application Development. Gouri has written more than 75 articles on Azure DevOps, TFS, SQL Server Business Intelligence and SQL Azure which are published on www.sqlservercurry.com and www.dotnetcurry.com. You can connect with her on LinkedIn.