Microsoft Visual Studio Team Services (VSTS) has been a premier integrated set of tools for DevOps for applications based on Microsoft technologies like ASP.NET.
VSTS is the extension that Microsoft created to offer services of Team Foundation Services (TFS) from Azure as Software-as-a-service (SaaS) model. These services include Agile Planning using Work Item Tracking Service, Version Control, Build Automation, Release Management, Packaging etc.
VSTS can support applications that are built on non-Microsoft technologies, just as it supports applications based on Microsoft technologies.
Are you keeping up with new developer technologies? Advance your IT career with our Free Developer magazines covering C#, Patterns, .NET Core, MVC, Azure, Angular, React, and more. Subscribe to the DotNetCurry (DNC) Magazine for FREE and download all previous, current and upcoming editions.
VSTS services for CICD of Java Web App
In this article, I am going to show how these services of VSTS can be used for Continuous Integration – Continuous Deployment (CICD) of a Java Web application. This is a log of my experiments that I am presenting before you.
I am making an assumption here that I am going to use Eclipse as an IDE for doing Java web application development and use Maven for executing a build of that application.
I will be using various versions of Tomcat to host that application in various environments.
The Dev environment here is a Java web application in Azure App Services that supports Tomcat 8.5. The Test environment is a Windows 10 VM that has an installation of Tomcat 9. Finally, the Production environment is an Ubuntu 16.04 VM in Azure that has installation of Tomcat 8.
I will use multiple ways to deploy the web application to these environments. I will use FTP to upload the app to Azure App Service, Azure File Copy task of Release Management to deploy to Windows 10 VM and SSH to deploy to Ubuntu VM.
Figure 1: Flow of Build, Release and Deployments with Environment Details
Build and Release management in VSTS
Let us first understand the infrastructure of build and release management services of VSTS. The building block for that infrastructure is an Agent.
Figure 2: Build and Release Infrastructure
Microsoft Agent is a part of VSTS that can run the workflow of the build, release and also do testing. It can be installed on any computer that is running either Windows or Linux or OS X.
It can also use other software to execute steps of the workflow like Maven or Ant to do a Java build, SSH to deploy built components etc. All these necessary software is required to be installed on same machine where the Agent is installed.
Microsoft offers the ‘Hosted’ agents for our use. These are created and maintained by Microsoft but we have no control over them. We can install our own agents on a machine in our datacenter or a VM on the cloud, so that we can have all the required capabilities for our Build on them.
A VSTS account can have many agents. Agents with similar capabilities are grouped together to form an Agent Pool. When a build is triggered, it is put into the Agent Queue for that team project.
Agent Queue communicates to an Agent Pool to get an agent that is available. If that agent is idle, then the build definition is passed to it to run workflow of that build.
We are going to work in the following scenarios:
Figure 3: Scenario Details
The Agent has installations of JDK 8, Maven, Junit, SonarQube to run code analysis as part of build, Cobertura to compute code coverage of JUnit tests. The developer machine has Eclipse (Oxygen) and Tomcat 9 installation.
Microsoft VSTS is used for Version Control, Build and Release Management. All the environments (Dev, Test, Production) are hosted in Azure.
To run this scenario, I created a new Team Project and named it as “VSTS Java Maven Demo”.
Figure 4: Team Project Created
Now browse to the Settings (that gear like icon at the top center) and from there to Agent Queue and click the button “Download Agent”.
After unzipping the downloaded agent, configure it by running the command “Configure” or “ConfigureAgent” . You may have to give your VSTS URL and provide PAT for authentication. This is how you get the PAT.
For Eclipse to communicate with VSTS, we need to install a plugin developed by Microsoft – Team Explorer Everywhere. You can get it from Eclipse Marketplace.
Figure 5: Install TEE from Eclipse Marketplace
Once it is installed, you can connect to your team project from the perspective of Team Foundation Server.
Figure 6: Connect to Team Project
We now will create a Dynamic Web Project in Eclipse and add an index.jsp page to it.
Figure 7: Create Dynamic Web Project and a JSP
I created another class called Validator that has following code:
Figure 8: Validator Method
This code could have been more elaborate with validation from some data etc. but I want to focus on the process of CICD rather than complexity of code.
Validator class is used in the JSP code as follows:
Figure 9: Index Page
Now I converted this project to a Maven project in Eclipse which created an appropriate Maven POM file with necessary code.
Figure 10: Convert to Maven Project
After the code is ready, we would like to add the code to version control. To do so, I shared the project from Team menu and then selected Team Foundation Server from the options.
Figure 11: Share Project
Now I entered the URL of my VSTS account and then selected the project to add the source to.
Figure 12: Share Java Project to Selected Team Project
Although I have used centralized version control (TFVC), this works equally well with git implementation under VSTS. Now it’s time to Check-in the pending changes. We can do that too from the Team menu item:
Figure 13: Check-in Pending Changes
You can see that we have checked-in even the POM file which has got created. This is the POM that will be used on the build agent by Maven installation.
Now we are ready to create a build definition that will create war file of the web application. To do so, open the team project in the browser and select the Build menu item or click on the “Setup Build” button on the home page. This will start a wizard to create build definition.
Figure 14: Team Project Start Page
Figure 15: Create New Build Definition with Maven Template
In the next step, I selected the POM.xml that was created in Eclipse and checked in.
Figure 16: Agent Queue and Maven POM Selection
On the subsequent page for Maven task, I selected package goal which will ensure that the Java files are compiled and then packaged under the war file.
Figure 17: Selected Maven POM Goal and Other Options
In the same (Maven) task, I also provided path of JAVA_HOME and Maven as they are on the agent machine. It is also possible to select default values of those, instead of giving them explicitly.
Figure 18: JAVA_HOME and Maven Path
As the Maven template demands a capability named ‘Maven’, we will need to set it to the installed agent.
When this build is executed, it creates an output in the form of a war file. We need to hand over this war file to the release management which then will deploy it to various environments.
This handover can be done using Artifact in VSTS. Artifact is a named location on agent which is available to both, the build service and release management service.
Figure 20: Details of Artifact
In the next step of the build, I copied the output of compilation, that is the war file, to the Artifact Staging Directory. VSTS provides a well-known variable for that directory named as Build.ArtifactStagingDirectory.
Figure 21: Copy Output Files to Artifact Staging Directory
In the final step of the build, I published the artifact named ‘drop’.
Now we can save this build definition and trigger the build. We can explore the artifact to check that the war is created below that artifact.
Figure 23: WAR in Build Artifact
To enable Continuous Integration for this build, I set the trigger to Continuous Integration with the scope of team project. With any check-in into this project, this build will get triggered.
This path can be made constrictive if we want it to be triggered for check-in in some particular branch or a folder within the team project. If we check the checkbox of ‘Batch changes while the build is in progress’, additional builds will not be triggered if a long running build is in progress.
Doing so will ensure that at any time at the most only one build of that definition is running.
Figure 24: CI Trigger
Now that the build is ready, we will create a release definition that will deploy the built war file to various environments.
All of our environments are hosted in Azure. To facilitate communication between VSTS and Azure and to ensure the seamless authentication, I created a service endpoint in VSTS for Azure. We can do that from Settings => Services.
Figure 25: Azure Service EndPoint
Preparing Development Environment
To prepare the Dev environment, I created a Web Application using Azure Application Services.
Figure 26: Create Web App on Azure
For this web application, I have set the properties in such a way that it will be using the Java support provided by Azure App Services.
Figure 27: Java Settings for Web App in Azure
I also noted the FTP host address and created credentials to access it.
Figure 28: FTP Details for Java App on Azure
Now to create a release definition, I have used an Empty template and then added a FTP upload task.
Figure 29: FTP Task Selection
As the properties of that FTP Upload task, I have set the Server URL, User Name and Password that was copied from Azure Application.
As the source folder, I selected the folder that contained the war file. In the file pattern, I entered **\*.war so that any war file under the source folder will be deployed. Finally for the remote folder I entered site/wwwroot/webapps which is the Tomcat folder for deployment of war on the Azure App Service.
Figure 30: FTP Task Details
After creating the release and deploying the war, we can view the application from the URL that is mentioned in the Azure App Service.
Defining the Test Environment
Now we can define our Test environment. I created a VM in Azure that has Windows 10 operating system and has the installation of JDK and Tomcat 9. In the release definition, I created another environment and named it as Test. For this environment also, I used Empty template.
Now I added the task of Azure File Copy and set the properties of that as follows:
Figure 31: Azure File Copy Task in Test Environment
Source is the war file from the drop artifact. Machine name for the filter is the name of the machine in Azure (not the FQDN or NetBIOS name). Admin user name and password from the VM and Destination folder is the path of the WebApps folder of Tomcat on the VM.
In addition to these settings, I also want that deployment to Test environment should happen only when a team member authorizes that deployment. This authorization will be given only when testing of the earlier build is over.
I setup the Pre-Deployment authorization for the Test environment. This can be done from the Pre-Deployment condition for the environment. We can also set a Post-Deployment authorizer for any environment to certify that the manual activities of that environment are over and we can do the deployment to next environment.
Figure 32: Set Authorizer
When we run the release workflow, after the deployment to Dev environment is successful, the authorizer will get an email to approve the authorization to Test environment.
Authorizer can also open the release page and provide or reject the authorization.
Figure 33: Authorize
Setting up Production Environment
Now we come to the last environment, Prod.
This environment has an Ubuntu VM in Azure. To copy files on that machine, we will be using SSH. This requires us to create a pair of keys (public key and paired private key). There are many ways to create such a pair but I prefer to use a tool named PuTTYGen which is a part of PuTTY.
Figure 34: Create SSH Key Pair
The generated public key is entered in the property provided for VM in Azure portal.
Figure 35: SSH Public Key Ubuntu
We will then create a Service Endpoint for SSH on our VSTS account. During this endpoint creation, besides the FQDN Host Name of the Ubuntu VM, I provided the private key which was generated with the help of PuTTYGen.
Figure 35: SSH Endpoint Creation
I added a new environment to the release definition using Empty template. I now added the task of Copy Files Over SSH. Properties set for it are Source Folder under drop artifact, Contents – war and Target Folder as it exists on the Ubuntu VM.
I had not set an authorization for this environment for this demo but it will be natural to set authorization for post-deployment on Test environment and pre-deployment authorization on the Prod environment.
Continuous Deployment Triggers
Now to set the triggers for Continuous Deployment, first I set the trigger for release to happen as soon as the build is ready.
Figure 38: Continuous Deployment Trigger
Both of these triggers put together will ensure that as soon as the build is ready, the release will be created and that will be followed automatically by deployment of the application to Dev environment.
In the second iteration of the build definition modification I added a Junit test to the project and executed it as part of the build. In addition to including a Junit test in the code, I had to change the POM.xml for that as shown.
Figure 39: Junit and POM
I then started a SONARQube server (port 9000). I also added a service endpoint for this so that I can run the code analysis on the code that is being built.
Figure 40: Create Endpoint for SonarQube
In the Maven task of the build definition, I enabled the code analysis using SonarQube, enabled publishing the Junit tests results in VSTS and enabled Code Coverage using Cobertura which I have installed on the agent machine.
Figure 41: Code Analysis, Junit, Code Coverage
We can also run a few more tools like Checkstyle, by enabling them in the Maven task.
Figure 42: CheckStyle and Other Tools
In this article I have shown through this walkthrough the use of Microsoft VSTS for doing CICD of a Java Web Application.
CICD is a part of DevOps and VSTS supports many facets of DevOps for development in .NET and many other non-Microsoft technologies. VSTS provides version control support and orchestrates the CICD and gets the actual tasks run by tools like Ant, Maven, Jenkins, SonarQube etc.