DevOps can be defined as extending agile practices to operations. An important part of ‘Operations’ is to monitor the health and performance of the software after it is deployed to production, and provide a feedback about it to the development team. Microsoft has built an Azure based infrastructure with tools to monitor, collect data and analyze health and performance of applications. This service offered by Azure is called Application Insights.
In this article, I am going to document some of the important details related to Application Insights. This article will cover:
- How to install and enable Application Insights for a web application?
- What monitoring data is collected by Application Insights?
- How the collected data is reported?
- How does Application Insights allow us to analyze that data?
Application Insights and DevOps
Software Operations was done by a separate team (other than the development team).
For the understanding of developers, let me first enumerate activities that are done by the operations team:
- Provisioning – Operations team builds the environment to which the deployment is to take place. Tasks under that activity are
- Create the virtual machines or take the existing virtual machines back to pristine stage
- Install required prerequisites and configure them
- Configure supporting software like WinRM, if required
- Create necessary user accounts and groups. Set permissions for them.
- Deployment – Copy files on target machines and install any of the binaries if needed.
- Feedback and Testing – Run the application and do smoke tests. Do exploration of the application after deployment on the target machine to ensure the sanity of the software. Run automated tests if configured and send the results back to development team.
- Health monitoring – Continuously monitor the availability, performance and other parameters to optimize the product.
The entire development and operations cycle could be depicted like this:
Figure 1 - DevOps Cycle
Today we are going to set focus on the health monitoring of the deployed application. As part of the health monitoring, I am expecting the following features:
- Continuous check for availability of the application
- Time-to-time monitoring to know the performance of the application
- Status of the components of the product and the servers on which they are deployed
- Faults and errors including those of dependencies of product
As a team member of development team, I also would like to get some statistics and analysis of the usage of the product or the application. Statistics that I expect is for requests that are received by the application, page views, clients, exceptions etc.
Microsoft has created a service in Azure that is called Application Insights that provides health and performance monitoring, with analytics. Any application that can access Internet can publish the data for Application Insights. It can be ASP.NET web application hosted in cloud, or on a server in premises, or any other type of windows application including windows services or a Java web application and so on.
Today, we are going to take the case of Application Insights for an ASP.NET Web Application hosted in the App Services of the Microsoft Azure Cloud.
Enable Application Insights for an ASP.NET application
For an application to publish data to Application Insights, we need to add Application Insights SDK to it. This can be done while the application is created in Visual Studio, or afterwards, when it is ready to be published.
Figure 2 – Add Application Insights SDK to the application in Visual Studio
Once the application is ready to publish the Application Insights data, it needs to be registered to the Application Insights service of Azure.
I am assuming here that an Azure account is already created and available.
Figure 3 – Register application with Application Insights
As soon as the application is registered with Application Insights and starts running (even in debug mode in Visual Studio), it starts publishing data. We can observe this data on Application Insights of the Azure portal. The default blade of Application Insights shows overview of data related to health of the application, and requests received for each page of the application.
Figure 4 – Health and performance overview of the application
Details of the data collected and reported by Application Insights
Each of the blades of Application Insights provides us with a different perspective to data that is collected. It is an analysed view of the data in a graphical format. We can set the begin and end timestamp to set the scope of analysis on the timeline.
Application Map – This blade shows us graphics of various components of the application, and related statistics. It also shows summary of the status of each component. That summary includes the requests or calls received, performance per request, errors or failures and link to drilldown for details of these parameters.
Figure 5 – Application Map
Live Metrics Stream – It is a dashboard of live data being collected from the application. It shows the data related to:
- Incoming Requests
- Requests Per Second
- Request Duration
- Requests Failed per second
- Outgoing Requests
- Dependency calls per second (e.g. SQL Server Database Calls)
- Dependency call duration
- Dependency call failures
- Overall Health
- Memory Usage
- CPU Usage
- Unhandled Exceptions Per Second
- For each Server, all the above 9 parameters are used
- Recent Failures with drilldown to the exception details if available or request details
Figure 6 – Live Metrics Streams Dashboard
Availability – To check availability, Application Insights uses web tests that run at a predetermined periodicity. Initially there are no web tests available for us and we need to create at the least one web test for collecting the availability data. In Figure 7, you can see that we can set the frequency at which the test will execute, success criteria, alert criteria etc. This test will be run at a time from 5 locations. Failure of test from 3 locations will send an alert for failure of test.
Figure 7 – Add a web test to gather availability data
Once the test is configured, we can view the availability metrics.
Figure 8 – Availability Metrics
Failures – This metrics shows following details of failures
- Request failures (Any http response with response code more than 400)
- Application exception that are not handled
- Dependency failures
- Server exceptions
- Page faults
- Details of all exceptions and requests in the specified time period
Figure 9 – Failures metrics
Performance – This blade shows the following performance details:
- Server response time
- Dependencies response time
- Per operation loading time
- Dependency response average time
Figure 10 – Performance Metrics
Browser – To enable browser side metrics, certain code has to be added to the master page of the site. That code is as follows:
Figure 11 – Code to enable browser metrics
Browser metrics show the data related to the client application. Those metrics are:
- Page view load time
- Page views
- Client processing time
- Send request time
- Receive response time
- Page load networking time (time required for DNS Lookup and transport connection)
- Operation time on each page
Figure 12 – Browser related metrics
Usage – This blade shows the data of:
- Page views
Figure 13 – Usage statistics
Application insights Analytics
Application Insights provides a powerful query mechanism to do custom analysis and view the results as a list or a graph. Analytics schema allows us to create query based upon the following data:
- Custom Events
- Page Views
- Availability Results
- Custom Metrics
- Performance Counters
- Browser Metrics
Queries use a special language that is similar to SQL. We can create a query graphically by first selecting the metrics to be queried, say pageViews. Then we can apply filters on that to get list or a graph of the results. For example, pageViews | where operation_name == “/” will show the results in a list / tabular form as in Figure 14.
Figure 14 – A simple query and its tabular results
We can also view the results in a graphical form. A query on availabilityResults summarized by location of test, shows a pie chart as in Figure 15.
Figure 15: Simple query results in graphical form
For knowing more about Application Insights Analytics visit https://docs.microsoft.com/en-us/azure/application-insights/app-insights-analytics
Application Insights Alerts
Application Insights also facilitates setting email alerts for various conditions. For example, we can set alert for availability based upon the web test that we had created. We can set a threshold limit and if availability is below the limit, we can have an email delivered to us by Application Insights.
Figure 16 – Setting alert for availability going below threshold
This alert will look like the one shown in Figure 17.
Figure 17 – Availability below threshold limit alert
DevOps strategy cannot be complete without monitoring and analyzing the health and performance of the application that is deployed.
Microsoft has provided Application Insights, an Azure based service which makes it possible to monitor any application to constantly know availability of the application, failures and performance. It also provides an easy interface to query and analyse data for creating tabular and graphical reports.
For critical conditions, Application Insights, can also be configured to send alerts to our email addresses.