Using Test Hub in VSTS to run Automated Tests on Web applications

A couple of years ago, Microsoft introduced Microsoft Test Manager (MTM), a separate set of tools for testers along with Visual Studio 2010. Since then there have been a lot of enhancements for MTM.

MTM is a tool which helps in creating test cases, executing them, creating rich bug (bug that is rich in information) and storing all results automatically in Team Foundation Server (TFS).

Microsoft has also introduced a cool feature of testing with Test Hub if you do not have MTM. I have introduced it earlier and today I am going into details for one of the features of Test Hub.

Following is a list of articles on MTM and Test Hub I have already written

Exploratory Testing XT with Test Hub in Visual Studio Team System

Test Hub Enhancements in Visual Studio Team Services

Testing with Team Web Access using Visual Studio 2013

Exploratory Testing using Microsoft Test Manager 11 (Visual Studio 11)

Automated Tests using TestHub in VSTS

MTM is set of tools mainly used for Manual Testing. However, it can also be used to conduct Automated Testing (Unit Test, Coded UI Test or Web Test) if we follow certain pre-requisites.

You can read about creating of environments using MTM in this article.

I am going to use this feature for execution of Automated Tests with Test Hub.

In this article, I will:

• create a Web Application
• create a Web App Service (using Azure Portal) and deploy Web Application to it
• use Environment (created with Release Management) and
• finally execute our Coded UI Test (CUIT) on the environment.

I will use Visual Studio Team Services (VSTS) for Source Control, Build and Release Management. I will use Visual Studio 2017 for writing web application and later adding Coded UI Test to it.

If you are new to VSTS, read www.dotnetcurry.com/visualstudio/1322/what-is-visual-studio-team-system-vsts.

Pre-requisites

1. Visual Studio Team Services (VSTS) Account (for Source Control, Build & Release Management) – You can create one by using the url https://www.visualstudio.com

2. Azure Account to create Web App Service – You can create a new account here https://azure.microsoft.com

3. Build Agent to execute build in interactive mode

4. Test Agent running in interactive mode

Let us create a Team Project in Visual Studio Team Services (VSTS) and add code to it.

I am using TFVC (Team Foundation Version Control) and Agile as the process. You can choose any version control (TFVS, Git etc) and process template of your choice.

Create a solution and add web application using Visual Studio 2017.

Add Solution to Source Control

Add the solution to the source control. Add it to the Team Project we created earlier.

Let us change some code and create a web application of our choice. Next thing is to check in the code (Commit and Push if you have selected git).

We need to create a build definition for this checked-in code. Browse to the files which have been checked-in. The entire solution and all its files can be seen.

Build Definition

In order to create a build definition, we can use ASP.NET template. Observe that this is still in preview and may change in the future.

This template automatically adds the build parameters and creates zip file for the web application (at the time of build).

Observe the parameter for package as a single zip file. This zip file is used by the WebDeploy technology to deploy web application on a remote machine or on Azure We App.

You can set the trigger for build as Continuous Integration so that every time there is a change in web site and check-in, a build will be automatically triggered.

Build Agent is responsible for selecting the source code, creating build artefacts if any and actually running the build process.

Hosted Build Agent is provided to automatically start building your projects with VSTS. For the time being you can use this agent and trigger the build. Later we will need a build agent on the machine which will be participating in the environment. This machine will also run Test Agent as we will be running automated tests.

Trigger the build and verify that the zip file is created in drop folder in artifact.

Deploy the Web Application to Azure

I am going to create a web app service by using azure portal. You can deploy it on azure machine if required.

Let us create a Web App service now. Go to the Azure portal and log in to your account. Create a new Web app.

Provide a unique name, select existing resource group or create a new one and create the service. Note down the url created which we will use to deploy.

In order to deploy the web application to the web app service, we will create a release definition with a single task to deploy.

Release Management is a process which helps in managing, planning of various environments for build. It can also be used for scheduling for different environment stages.

Select the release tab and clink on  new definition. We need to provide the build which will be taken for creating the release. Add a task of azure app service deployment to the definition.

Configure the task. Provide the name for azure subscription, app service name and save the definition with an appropriate name. Create a new release and after successful release, ensure that the web application is deployed.

Now we have successfully deployed the web application and we can add some automated tests to the solution and use environment to execute those tests.

Adding CUIT

Add a Coded UI Test project to the solution and write the code to launch the web application.

You can record the test by using CUIT Builder or write from the scratch. The following code launches the application by starting the browser.

BrowserWindow browser;
browser = BrowserWindow.Launch(<url for web app service>");
browser.Maximized = true;

You can add more code and test the CUIT by Visual Studio itself by using Test Explorer.

Ensure that the test successfully executes.

Configure Build and Test Agent

We will be using Test hub to assign the release to the test plan.

We need to execute automated tests, hence we need to provide the location where the tests assemblies are located.

We will need the default agent which will be running in interactive mode. We also need to run the test agent on the machine which is participating in environment so as to run CUIT. This test agent should also run in interactive mode as the test requires user interface.

If we use V2.0 of testing task, then test agent is not required. It requires updated agent which works as agent for Build, Release and Test also. This works as unified agent.

Create Personal Access Token (PAT)

Create a Personal Access Token (PAT) which will be used to configure build agent. Select security tab for the VSTS account, create a PAT and store the token safely.

Download the build agent and unzip it in a location.

To download, Select the Setting (wheel / gear icon) shown for the Team Project account and select Agent queues and you will find the link for Download Agent.

Unzip the agent at a location of your choice.

In order to configure the build agent, start command prompt with administrative privileges. Browse to the folder where the agent is unzipped and run config.cmd.

We need to provide the url for VSTS account, PAT and other details for configuring the agent. We do not want the agent to run as a service so later we need to run the ‘run’ command

Keep the command window running so that the agent keeps listening.

Download and Configure Test Agent

For running automated tests, we need the test agent running in an interactive mode. You can download it from this link.

After downloading you can install it and configure. For configuring, it is not necessary to register it with Test Controller. Ensure that the agent is running in an interactive process mode. The executable for vstest.console.exe (required for execution of automated test) gets installed after configuration. It will be available if you have Visual Studio installed.

Build the Automated Test and Create Artefacts

Change the build definition so as to drop the test assembly.

We need to ‘check in’ the Coded UI test and create a build which drops the test assembly as an artefact. The build will be automatically triggered (because of Continuous Integration trigger) and you will see the test gets executed as a part of Build Verification Test (BVT).

Associate Test Case

We will need to create a Test Plan with the Test Hub to execute test case(s). Let us create a user story and link test case to it.

Visual Studio 2017 provide the option to associate the test case with the test written in the assembly. This association can be done using Test Explorer.

When we select the Associate Test Case option, we need to provide the test case by giving the test case id.

Once the code is associated with the test case, we can find the test name and assembly with the help of Associated Automation tab in test case IDE as follows.

Change the Release Definition

We need to add the task of executing the test to the release definition.

Configure the task with Test run (on-demand runs) for Select tests using and change the version to 2.

Click on Run on agent and specify the Default build agent for release definition.

Test Plan Settings

Select the Test tab from the VSTS account and add a new test plan to it. Add a requirement based test suite which will add the test case to the test suit.

The Test Plan provides the new option for Test Plan Settings.

We can configure the Test plan settings now.

Provide the build definition name and select latest build to select. Provide the release definition name and also the environment name.

Execute with Test Hub

Select the test case, right click and select run with options.

The execution of test starts and we get the verification dialog.

Click on View Test run to observe that the test currently is in a not started state and later we see the test getting executed. Once the test is successfully executed, we can find the test run.

We can select the trigger for Continuous Deployment so as to make it complete CI CD.

Now we can make some changes in the code and a check-in will trigger the build and later the release, automatically.

Conclusion

In this article, we discussed how to execute an automated test with the help of Test Hub in Visual Studio Team System (VSTS). We used the environment created in Release Management for this.

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

Gouri is a Trainer and Consultant on Azure DevOps and Azure Development. She has an experience of three decades in software training and consulting. She is a graduate from Pune University and PGDCA from Pune University. Gouri is a Microsoft Most Valuable Professional (MVP)  - Developer Technologies (Azure DevOps), Microsoft Certified Trainer (MCT) and a Microsoft Certified Azure DevOps Engineer Expert. She has conducted over 150 corporate trainings on various Microsoft technologies. She is a speaker with Pune User Group and has conducted sessions on Azure DevOps, SQL Server Business Intelligence and Mobile Application Development. Gouri has written more than 75 articles on Azure DevOps, TFS, SQL Server Business Intelligence and SQL Azure which are published on www.sqlservercurry.com and www.dotnetcurry.com. You can connect with her on LinkedIn.