Using Secrets in Azure Pipelines

In the DevOps team, managing secrets is everyone’s responsibility. These secrets may be passwords, other credentials, certificates etc. used for accessing other resources. Secrets used to access resources should not be stored in a way that will make it possible for an unauthorized person to view that secret. Azure DevOps makes use of Azure Key Vault to store the secrets in a safe way and use them as and when needed without making it visible to anyone.

Azure Key Vault is a service from Azure that allows storage of keys, secrets and certificate in a safe and encrypted repository. It implements a Role Based Access Control (RBAC) to ensure that the stored artifacts are not accessible to unauthorized persons and accounts used by services (Service Principles). This is an excellent option to store secrets like passwords, connection strings etc. which we require while doing the deployment through Azure DevOps. In this article, I am going to show you different ways that Azure Key Vault can be used by Azure DevOps Pipelines.

Using Secrets in Azure - Scenario

I am putting a scenario in front of you.

I have a web application that is to be deployed to a VM hosted in Azure. Code of that application is under the version control in Azure DevOps – git. I have created a pipeline to build that application. The output of that build is a WebDeploy package that I want to copy and run on a VM. To copy the package to that VM, I need to provide user name and password. I want to store that password securely so that only I can view it and change it if necessary but others in my project can use that without able to view it while doing the deployment.

Walkthrough

Using secret variables in Azure Pipelines

Let us start the walkthrough by creating a new release pipeline which has a stage to do the deployment to the VM. In this release pipeline, we will create variables to store user name and password. The variable for user name (UserName) is a simple variable where we can store the user name that we will use to access the VM. Another variable is for password. We want to keep password a secret so that no one else can view it even if they can view the release pipeline definition. We can see a lock icon on the right of the variable named Password that we create. This lock icon can make the value of that variable as a secret.

Pipeline variables

After clicking that lock icon, we can now enter the value of Password variable. It will appear with password characters. When we save the pipeline definition, this value is encrypted and stored in Azure Key Vault. Now only the authorized accounts that are part of our team, can access that value.

Secret variables in the Azure Pipeline

Let us now add a task to copy the package to the VM. Since the VM is in Azure, I suggest using the task of Azure File Copy. On this task we provide the necessary information like the Source to be copied, Azure Subscription to be used (you will need to Authorize Azure DevOps to access resources under subscription), Destination VM name etc. For the Admin Login we will use the variable UserName. The value of that can be accessed using $(UserName). For Password parameter of the task we will use the value of variable Password we had created using $(Password).

Using variables and secret variables in a task

At the run time, the value of Password will be retrieved from the Azure Key Vault and assigned to the variable and through that to the parameter of the task to login to VM. In this case access to the Azure Key Vault, to create and to retrieve the secret, is done using the service account and we have no control over the created secret in Azure Key Vault. This secret variable value is accessible only in the pipeline where we have created the pipeline.

Create and use secrets from Azure Key Vault instead of using the implicit way shown above

There are a few reasons to create our own secret in the Azure Key Vault.

1. We want to keep secret in your own account instead of allowing Azure DevOps to create in a default account.

2. We want to share the secret in multiple pipelines in the same project and also across multiple projects.

3. We want to set an expiration policy for the secret.

Create a new Azure Key Vault and secret in that using Azure Portal

Let’s first create a secret in the Azure Key Vault and then use it in our pipelines. To create the secret, we will use Azure Portal (https://portal.azure.com) and login to it using our credentials. If Azure Key Vault does not exist, we will create one. Give it an appropriate name.

Create a new Azure Key Vault

We can now create a secret in this key vault.

Create new secret under Azure Key Vault

Let’s give the secret a name, ssgsemspwd by which we can access it. Store the password of the VM in the Value box.

Set value to a secret

Using secret stored in Azure Key Vault in Azure Pipelines

We have two options to access this Key Vault and the secret from it in the Azure DevOps.

1. Using Variable Group

2. Using Azure Key Vault Task

Using variable groups to access secrets stored in Azure Key Vault

To use the first method, open the Libraries hub under Pipelines hub group. If not already selected, select the tab of Variable Groups. We will create a new variable group which will give us access to our Azure Key Vault.

Create variable group

Click the button to create new Variable Group. Give it an appropriate name such as “SSGS EMS Creds”. Click the toggle to “Link secrets from an Azure Key Vault as variables” to switch it on.

Select Azure Key Vault to link in the variable group

We will need to select our Azure Subscription. If the Azure and Azure DevOps subscription are created using same Microsoft account (same email account) then the authorization process is seamless and simple. If it is not using the same account then we will need to create a Service Connection to Azure. For guidance to create such a service connection, go through the instructions given in this article on DevCurry.com (https://www.devcurry.com/2019/08/service-connection-from-azure-devops-to.html).

Authorize Azure DevOps to access selected Azure Key Vault

During the authorization process, in the background a Service Principle is created to represent the Azure DevOps to access Azure Key Vault and is added in the role to access the secrets stored in the Key Vault.

Once the authorization process is successful, we can select the key vault, the one that we had created in earlier step. Finally we can select the secret that we had created earlier. We can save the Variable Group now.

Select secret to use and save variable group

In the release pipeline that we create, we can open the Variables section and then click the tab of Variable Groups and then click the button to “Link variable group”.

Link variable group to a release pipeline

We will select the variable group that we have created in the last step, to link. We can link this variable group to either the entire release or to one or many steps.

After linking variable group to release pipeline

After linking this variable group, we can make use of the variable under that, which provides us the value of secret stored in Azure Key Vault, in the task as any other variable.

Using the secret through the variable in a variable group

Using a Azure Key Vault task to access the secret

Another option for accessing the secret in Azure Key Vault is to use a task provided for that purpose. We can add that task by searching for Azure Key Vault in the tasks catalog. Once it is added, we can select the Azure Subscription, Key Vault and optionally a filter for the secret.

Using Azure Key Vault task to access secret

After this task, we can access the secret with its name as if it is a variable, in any subsequent tasks.

Summary:

In this article, I have shown you how to store a secret in Azure Key Vault using Azure Portal and then access it in the Azure DevOps Pipelines as and when required. We used variable with option to make it secret. This action stores the secret variable in the Azure Key Vault. That is an implicit use of Azure Key Vault. We can also explicitly create a secret in Azure Key Vault and access it using either the Variable Group feature of Azure Pipelines or using the task that is provided for accessing the secrets in Azure Key Vault.

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

Subodh is a Trainer and consultant on Azure DevOps and Scrum. He has an experience of over 33 years in team management, training, consulting, sales, production, software development and deployment. He is an engineer from Pune University and has done his post-graduation from IIT, Madras. He is a Microsoft Most Valuable Professional (MVP) - Developer Technologies (Azure DevOps), Microsoft Certified Trainer (MCT), Microsoft Certified Azure DevOps Engineer Expert, Professional Scrum Developer and Professional Scrum Master (II). He has conducted more than 300 corporate trainings on Microsoft technologies in India, USA, Malaysia, Australia, New Zealand, Singapore, UAE, Philippines and Sri Lanka. He has also completed over 50 consulting assignments - some of which included entire Azure DevOps implementation for the organizations.

He has authored more than 85 tutorials on Azure DevOps, Scrum, TFS and VS ALM which are published on www.dotnetcurry.com.Subodh is a regular speaker at Microsoft events including Partner Leadership Conclave.You can connect with him on LinkedIn .