Using Secrets in Azure Pipelines

Posted by: Subodh Sohoni , on 9/2/2019, in Category DevOps
Views: 1679
Abstract: Azure Key Vault is a service from Azure that allows storage of keys, secrets and certificate in a safe and encrypted repository. In this tutorial, I am going to show you different ways that Azure Key Vault can be used by Azure DevOps Pipelines.

In the DevOps team, managing secrets is everyone’s responsibility. These secrets may be passwords, other credentials, certificates etc. used for accessing other resources. Secrets used to access resources should not be stored in a way that will make it possible for an unauthorized person to view that secret. Azure DevOps makes use of Azure Key Vault to store the secrets in a safe way and use them as and when needed without making it visible to anyone.

Azure Key Vault is a service from Azure that allows storage of keys, secrets and certificate in a safe and encrypted repository. It implements a Role Based Access Control (RBAC) to ensure that the stored artifacts are not accessible to unauthorized persons and accounts used by services (Service Principles). This is an excellent option to store secrets like passwords, connection strings etc. which we require while doing the deployment through Azure DevOps. In this article, I am going to show you different ways that Azure Key Vault can be used by Azure DevOps Pipelines.

Using Secrets in Azure - Scenario

I am putting a scenario in front of you.

I have a web application that is to be deployed to a VM hosted in Azure. Code of that application is under the version control in Azure DevOps – git. I have created a pipeline to build that application. The output of that build is a WebDeploy package that I want to copy and run on a VM. To copy the package to that VM, I need to provide user name and password. I want to store that password securely so that only I can view it and change it if necessary but others in my project can use that without able to view it while doing the deployment.

Walkthrough

Using secret variables in Azure Pipelines

Let us start the walkthrough by creating a new release pipeline which has a stage to do the deployment to the VM. In this release pipeline, we will create variables to store user name and password. The variable for user name (UserName) is a simple variable where we can store the user name that we will use to access the VM. Another variable is for password. We want to keep password a secret so that no one else can view it even if they can view the release pipeline definition. We can see a lock icon on the right of the variable named Password that we create. This lock icon can make the value of that variable as a secret.

pipeline-variables

Pipeline variables

After clicking that lock icon, we can now enter the value of Password variable. It will appear with password characters. When we save the pipeline definition, this value is encrypted and stored in Azure Key Vault. Now only the authorized accounts that are part of our team, can access that value.

secret-variables-in-the-azure-pipeline

Secret variables in the Azure Pipeline

Let us now add a task to copy the package to the VM. Since the VM is in Azure, I suggest using the task of Azure File Copy. On this task we provide the necessary information like the Source to be copied, Azure Subscription to be used (you will need to Authorize Azure DevOps to access resources under subscription), Destination VM name etc. For the Admin Login we will use the variable UserName. The value of that can be accessed using $(UserName). For Password parameter of the task we will use the value of variable Password we had created using $(Password).

using-variables-secret-variables

Using variables and secret variables in a task

At the run time, the value of Password will be retrieved from the Azure Key Vault and assigned to the variable and through that to the parameter of the task to login to VM. In this case access to the Azure Key Vault, to create and to retrieve the secret, is done using the service account and we have no control over the created secret in Azure Key Vault. This secret variable value is accessible only in the pipeline where we have created the pipeline.

Datadog, a monitoring and analytics platform that integrates with more than 250 technologies, unites metrics, traces, and logs in one platform so you can get full visibility into your infrastructure and applications. With powerful dashboards, anomaly and outlier detection, and distributed tracing, Datadog helps you get a handle on the performance of your .NET applications, as well as their underlying infrastructure.

Click here to try Datadog’s full-stack monitoring for free! (No Credit Card or Phone required)

Create and use secrets from Azure Key Vault instead of using the implicit way shown above

There are a few reasons to create our own secret in the Azure Key Vault.

1. We want to keep secret in your own account instead of allowing Azure DevOps to create in a default account.

2. We want to share the secret in multiple pipelines in the same project and also across multiple projects.

3. We want to set an expiration policy for the secret.

Create a new Azure Key Vault and secret in that using Azure Portal

Let’s first create a secret in the Azure Key Vault and then use it in our pipelines. To create the secret, we will use Azure Portal (https://portal.azure.com) and login to it using our credentials. If Azure Key Vault does not exist, we will create one. Give it an appropriate name.

create-new-azure-key-vault

Create a new Azure Key Vault

We can now create a secret in this key vault.

create-new-secret-under-azure-key-vault

Create new secret under Azure Key Vault

Let’s give the secret a name, ssgsemspwd by which we can access it. Store the password of the VM in the Value box.

set-value-to-a-secret

Set value to a secret

Using secret stored in Azure Key Vault in Azure Pipelines

We have two options to access this Key Vault and the secret from it in the Azure DevOps.

1. Using Variable Group

2. Using Azure Key Vault Task

Using variable groups to access secrets stored in Azure Key Vault

To use the first method, open the Libraries hub under Pipelines hub group. If not already selected, select the tab of Variable Groups. We will create a new variable group which will give us access to our Azure Key Vault.

create-variable-group

Create variable group

Click the button to create new Variable Group. Give it an appropriate name such as “SSGS EMS Creds”. Click the toggle to “Link secrets from an Azure Key Vault as variables” to switch it on.

azure-key-vault-link-variable-group

Select Azure Key Vault to link in the variable group

We will need to select our Azure Subscription. If the Azure and Azure DevOps subscription are created using same Microsoft account (same email account) then the authorization process is seamless and simple. If it is not using the same account then we will need to create a Service Connection to Azure. For guidance to create such a service connection, go through the instructions given in this article on DevCurry.com (https://www.devcurry.com/2019/08/service-connection-from-azure-devops-to.html).

authorize-azure-devops-azure-key-vault

Authorize Azure DevOps to access selected Azure Key Vault

During the authorization process, in the background a Service Principle is created to represent the Azure DevOps to access Azure Key Vault and is added in the role to access the secrets stored in the Key Vault.

Once the authorization process is successful, we can select the key vault, the one that we had created in earlier step. Finally we can select the secret that we had created earlier. We can save the Variable Group now.

azure-secret-use-save-variable

Select secret to use and save variable group

In the release pipeline that we create, we can open the Variables section and then click the tab of Variable Groups and then click the button to “Link variable group”.

link-variable-group-release-pipeline

Link variable group to a release pipeline

We will select the variable group that we have created in the last step, to link. We can link this variable group to either the entire release or to one or many steps.

linking-variable-group-release-pipeline

After linking variable group to release pipeline

After linking this variable group, we can make use of the variable under that, which provides us the value of secret stored in Azure Key Vault, in the task as any other variable.

using-azure-vault-secret

Using the secret through the variable in a variable group

Using a Azure Key Vault task to access the secret

Another option for accessing the secret in Azure Key Vault is to use a task provided for that purpose. We can add that task by searching for Azure Key Vault in the tasks catalog. Once it is added, we can select the Azure Subscription, Key Vault and optionally a filter for the secret.

azure-key-vault-task-access-secret

Using Azure Key Vault task to access secret

After this task, we can access the secret with its name as if it is a variable, in any subsequent tasks.

Summary:

In this article, I have shown you how to store a secret in Azure Key Vault using Azure Portal and then access it in the Azure DevOps Pipelines as and when required. We used variable with option to make it secret. This action stores the secret variable in the Azure Key Vault. That is an implicit use of Azure Key Vault. We can also explicitly create a secret in Azure Key Vault and access it using either the Variable Group feature of Azure Pipelines or using the task that is provided for accessing the secrets in Azure Key Vault.

This article has been editorially reviewed by Suprotim Agarwal.

Absolutely Awesome Book on C# and .NET

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce the The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this eBook aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on .NET Standard and the upcoming C# 8.0 too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

What Others Are Reading!
Was this article worth reading? Share it with fellow developers too. Thanks!
Share on LinkedIn
Share on Google+

Author
Subodh is a consultant and corporate trainer. He has overall 28+ years of experience. His specialization is Application Lifecycle Management and Team Foundation Server. He is Microsoft MVP – VS ALM, MCSD – ALM and MCT. He has conducted more than 300 corporate trainings and consulting assignments. He is also a Professional SCRUM Master. He guides teams to become Agile and implement SCRUM. Subodh is authorized by Microsoft to do ALM Assessments on behalf of Microsoft. Follow him on twitter @subodhsohoni


Page copy protected against web site content infringement 	by Copyscape




Feedback - Leave us some adulation, criticism and everything in between!

Categories

JOIN OUR COMMUNITY

POPULAR ARTICLES

C# .NET BOOK

C# Book for Building Concepts and Interviews

Tags

JQUERY COOKBOOK

jQuery CookBook