ASP.NET WebHooks - Consuming WebHooks from Github

The ASP.NET web stack provides solutions to most of the problems we face while building server-based web applications. It has support for building Web applications with views using Web Forms and MVC, building REST APIs using Web API, and support for real-time communication using SignalR. The platform continues to grow with each of these supported technologies receiving frequent updates to make each of them even better. A recent addition to the ASP.NET stack is WebHooks. In this article, we will see what WebHooks are, examine their usefulness, and we will implement an application consuming WebHooks from GitHub.

A special note of thanks to Rion Williams for reviewing the article and adding important bits to it.

This article is published from the DNC Magazine for .NET Developers and Architects. Download this magazine from here [Zip PDF] or Subscribe to this magazine for FREE and download all previous and current editions

What are WebHooks?

WebHooks are user-defined HTTP-based callbacks. They are configured with an event so that the callback will be invoked when the triggering event occurs. These callbacks can be configured and managed by both users of the application (including third-parties) and developers. The source of these events may be in the same application or, in a different application.

The REST API is called when the triggering event occurs and then data related to the event is sent to the POST API. Basically, a user or developer can create a notification and when a specific trigger occurs, they will be notified via an HTTP POST. Web Hooks are most commonly used in systems such as continuous integration systems or messaging queues; but because of their flexibility (events and callbacks can be just about anything), their potential could really be endless.

ASP.NET WebHooks

ASP.NET WebHooks is the implementation of WebHooks in the context of ASP.NET. This feature is currently implemented in ASP.NET 4.5 and it is in preview at the time of writing this article. ASP.NET WebHooks supports both sending and receiving WebHooks. It uses ASP.NET Web API for sending and receiving WebHooks under the covers.

Online services like GitHub, Trello, Dropbox, Slack and a few others expose WebHooks to notify actions. ASP.NET team has implemented a set of NuGet packages to make it easier to talk to these services. It is also possible to subscribe to other services using the custom receiver NuGet package.

In this article, we will see how this feature can be used to receive WebHooks exposed by Github in ASP.NET. Before we start writing code, we need to understand how this communication is designed to work. Figure-1 explains the flow of data when a WebHook is raised.

Figure 1: Flow of Data when a WebHook is raised

When a WebHook is raised by a source,

• The source calls the receiver’s HTTP POST endpoint
• The Web API controller handling the HTTP POST callback forwards the notification to the receiver specific to the sender or, to the generic receiver
• The receiver validates the request and then sends it to the WebHook handler

WebHook handler is the component that gives us access to the data received from the source. Then it is responsibility of the application to use this data. The data may be stored in a database, may be sent to the client and displayed on the UI or, may be logged on the server based upon the need of the application.

Implementing a GitHub WebHook Client using ASP.NET WebHooks

Now that we have some understanding on what WebHooks are and how they work in ASP.NET, let’s build an application to understand how WebHooks from GitHub can be received in an ASP.NET application.

Setting up the Application

Open Visual Studio 2015 or, 2013 and create a new empty ASP.NET 4.5 application. To make it a bit easier to setup, choose the Web API checkbox in the New ASP.NET Project dialog.

Figure 2: Empty ASP.NET Project

As most of the WebHook providers are hosted on HTTPS and they expect the receivers also to use HTTPS protocol, host this application on Azure to get SSL for free. In addition to this, since only a public domain can ping back another public domain, hosting on Azure makes our app public. In the Azure Web App hosting dialog box, enter URL of the application and hit the OK button.

Now we need to add WebHooks to this application. This can be done in one of the two ways:

a. installing the Nuget packages (Microsoft.AspNet.WebHooks.Common, Microsoft.AspNet.WebHooks.Receivers and other specific receiver packages), configuring the WebHook receiver and manually writing the WebHook handler OR

b. taking advantage of the WebHooks extension. Brady Gaster has a great video in the announcement blog post of the extension demonstrating usage of the extension. For the purposes of this article, we will be demonstrating the manual, NuGet approach.

Before we configure the project to use GitHub WebHooks, we need to configure WebHooks on a GitHub repo. As we need to make the site public, let’s deploy the application on Azure. Right-click on the project and choose the Publish option. The dialog box will be prepopulated with the data using the Azure account associated with your instance of Visual Studio. After reviewing the information, click the Publish button to publish the site.

Figure 3: Deploy application on Azure

Now the site should be deployed on Azure and will be loaded into an instance of your default browser.

Setting up WebHooks on a GitHub Repo

Login to GitHub using your GitHub account credentials. If you don’t already have an account on GitHub, then create one and login. Create a new repo under your account and give it a name of your choice. We will use this repo to play with GitHub’s WebHooks.

Once the repo is created, open the Settings page of the repo and from the menu on the left side, choose WebHooks and Services. In this page, click the “Add WebHook” button. This will ask for password. Enter your password and now you will see a page asking for details of the WebHook. Fill in the following details into the WebHook dialog:

• Payload URL: Payload URL for GitHub would follow this pattern: https://your-site-name.azurewebsites.net/api/webhooks/incoming/github
• Content type: application/json
• Secret: You can generate a secret using the Free Online HMAC Generator
• Which events would you like to trigger this webhook? : Send me everything

Click the “Add Webhook” button after filling these details. This will create a WebHook on GitHub. It will send a WebHook to the site as soon as it is created, but it will fail as our server is not yet ready to listen to this WebHook. Let’s make our server to listen for this event and capture the WebHooks.

Creating GitHub WebHooks Receiver

To enable listening to GitHub WebHooks, we need to install the following NuGet package in the project by typing the following into the Package Manager Console:

> Install-Package Microsoft.AspNet.WebHooks.Receivers.GitHub –pre

Note: The package can also be installed from the NuGet Package Manager GUI by searching for the appropriate package.

As the package is still in prerelease state (as of this writing), we need to include the –pre option in the command. The above package brings the packages Microsoft.AspNet.WebHooks.Receivers and Microsoft.AspNet.WebHooks.Common, as it depends on them.

Open the Web.config file and add the following entry under the appSettings section:

<add key="MS_WebHookReceiverSecret_GitHub" value="your-github-secret" />

The “your-github-secret” value in the above entry should be same as the secret entered previously on the GitHub site. Now we need to enable WebHooks in the current project. For this, add a new C# class file to App_Start folder and name it WebHookConfig. Open this file and add the following code to it:

public static class WebHookConfig
{
    public static void Register(HttpConfiguration config)
    {
        config.InitializeReceiveGitHubWebHooks();
    }
}

We need to invoke this method when application starts to activate WebHooks. Open Global.asax.cs and add the following statement to Application_Start event:

GlobalConfiguration.Configure(WebHookConfig.Register);

Publish the project again after making the above change. Now we have WebHooks active and running on the site. Go to GitHub, open the project for which WebHook is configured and change the readme file using the edit option on GitHub and commit it. This should generate a WebHook. You can check status of the WebHook under the same page where the WebHook was configured. It should show a succeeded WebHook delivery similar to the one below:

Figure 4: A WebHook generated and received

Creating a WebHook Handler

The screenshot in Figure 4 indicates that the WebHook is received by the Web API in our application. The Web API then looks for presence of WebHook handlers in the application. If it finds them, the information received is sent to the handler for further processing. If the application has to use the data received, then WebHook handler is the way to go.

Add a new folder to the application and name it WebHookHandlers. Add a new C# class file to this folder and name it GitHubWebHookHandler. Add the following code to this file:

public class GitHubWebHookHandler : WebHookHandler
{
    public override Task ExecuteAsync(string receiver, WebHookHandlerContext context)
    {
        if ("GitHub".Equals(receiver,StringComparison.CurrentCultureIgnoreCase))
        {
            string action = context.Actions.First();
            JObject data = context.GetDataOrDefault<JObject>();
            var dataAsString = Newtonsoft.Json.JsonConvert.SerializeObject(data);
        }

        return Task.FromResult(true);
    }
}

In the above snippet:

· The class GitHubWebHookHandler is extended from the abstract WebHookHandler class. WebHookHandler is the contract defined by WebHooks and any class extending this abstract class is instantiated as soon as a WebHook is received

· It overrides the abstract ExecuteAsync method. This method gets access to name of the receiver and the WebHookHandlerContext object. The context object contains the actions that caused the WebHook, ID of configuration of the WebHook and the data received by the WebHook

The data in the context object has the type object. We need to convert it to a type of our choice before using it. Out of all possible activities on a GitHub repo, I chose to handle commits, opening issues, closing issues and commenting on an issue. After these changes, Build and publish the application to Azure.

Connect to your Azure account within the Server Explorer of Visual Studio and start debugging the application using the Attach Debugger option.

Figure 5: Preparing to debug the application

Place a breakpoint inside the if block of the code snippet below and perform a commit operation on the GitHub repo. A WebHook will be sent by GitHub to the application about this operation and we will be able to see details of the action if you inspect data object in the above snippet:

Figure 6: Inspecting data object

Similar objects would be sent for other operations as well, their structure would differ as they hold different data. In the next section, we will store selected values from this payload in the database.

Saving the Data Received in a Database

To save the data received, create a database on SQL Azure. Connect to this database using either SQL Server Management Studio or, using SQL Server tools on Visual Studio. Run the following query on this database to create a table where we will save the data:

CREATE TABLE GitHubActivities
(
    ActivityId INT IDENTITY PRIMARY KEY,
    ActivityType VARCHAR(20),
    [Description] VARCHAR(MAX),
    Link VARCHAR(MAX)
);

The table would contain type of the event, the description (commit message in case of commits, title of the issue when an issue is opened or closed and text of the comment when a comment is posted) and a URL of the page where the event can be seen. We need to transform the data received from GitHub to this form before saving. We will do it in the next section.

Add a new ADO.NET Entity Framework Data Model to the Models folder and name it GitHubActivityEntities. Configure this model to point to the SQL Azure database we created for this application and choose the GitHubActivities table in the wizard.

We need to perform two types of GET operations, one to fetch all activities and the other to fetch activities that were posted after a given activity id. In addition to these, we need to perform an insert operation to insert details of the new activities. The following is the repository class that performs these operations:

public class GitHubActivitiesRepository: IDisposable
{
    RaviDevDBEntities context;

    public GitHubActivitiesRepository()
    {
        context = new RaviDevDBEntities();
    }

    public List<GitHubActivity> GetAllActivities()
    {
        return context.GitHubActivities.ToList();
    }

    public List<GitHubActivity> GetActivitiesAfter(int activityId)
    {
        return context.GitHubActivities.Where(gha => gha.ActivityId > activityId).ToList();
    }

    public GitHubActivity AddNewActivity(GitHubActivity activity)
    {
        var activityAdded = context.GitHubActivities.Add(activity);
        context.SaveChanges();

        return activityAdded;
    }

    public void Dispose()
    {
        context.Dispose();
    }
}

We will invoke the GET operations from a Web API controller to make the data retrieval possible from JavaScript and the add operation will be performed from the GitHubWebHookHandler class. Before calling the add method, we need to convert data to GitHubActivity type. The data received from GitHub differs for every event and we need to understand the pattern before converting. After a few observations, I understood that we need an intermediate object to help us in this scenario. After some analysis, I created the following class to hold the required information from the payload we receive and the enum to represent type of the event:

public class GitHubMessage
{
    public JObject[] Commits { get; set; }
    public JObject Issue { get; set; }
    public JObject Comment { get; set; }
    public string Action { get; set; }
    public GitHubActivityType ActivityType { get; private set; }
}
public enum GitHubActivityType
{
    Commit,
    IssueOpened,
    IssueClosed,
    Comment
}

The payload would contain some of the properties of the class and we will use JSON.NET to convert the data. The value of the property ActivityType has to be calculated based on values in the other properties. Add the following method to the GitHubMessage class to find the ActivityType:

public void SetActivityType()
{
    //If the payload has commits, it is a commit
    if (Commits != null && Commits.Length > 0)
    {
        ActivityType = GitHubActivityType.Commit;
    }
    //if the payload has Issues property set and Action is set to opened, an issue is opened
    else if (Issue != null && string.Equals("OPENED", Action, StringComparison.CurrentCultureIgnoreCase))
    {
        ActivityType = GitHubActivityType.IssueOpened;
    }
    //if the payload has Issues property set and Action is set to closed, an issue is closed
    else if (Issue != null && string.Equals("CLOSED", Action, StringComparison.CurrentCultureIgnoreCase))
    {
        ActivityType = GitHubActivityType.IssueClosed;
    }
    //if the payload has comment set, it is a comment
    else if(Comment != null && string.Equals("CREATED", Action, StringComparison.CurrentCultureIgnoreCase))
    {
        ActivityType = GitHubActivityType.Comment;
    }
}

Comments in the snippet explain how the ActivityType is assigned. Now that we have the type, we need to find Description and Link properties of the GitHubActivity class and assign them to an object. The following method does this:

public GitHubActivity ConvertToActivity()
{
    var activity = new GitHubActivity();

    activity.ActivityType = ActivityType.ToString();
    if (ActivityType == GitHubActivityType.IssueClosed || ActivityType == GitHubActivityType.IssueOpened)
    {
        activity.Link = Issue.GetValue("html_url").Value<string>();
        activity.Description = Issue.GetValue("title").Value<string>();
    }
    else if (ActivityType == GitHubActivityType.Commit)
    {
        activity.Link = Commits[0].GetValue("url").Value<string>();
        activity.Description = Commits[0].GetValue("message").Value<string>();
    }
    else
    {
        activity.Link = Comment.GetValue("html_url").Value<string>();
        activity.Description = Comment.GetValue("body").Value<string>();
    }

    return activity;
}

These methods have to be called from the GitHubWebHookHandler class and the final result has to be used to call add method on the repository class. Modify the class GitHubWebHookHandler as follows:

public class GitHubWebHookHandler : WebHookHandler
{
    GitHubActivitiesRepository repository;

    public GitHubWebHookHandler()
    {
        repository = new GitHubActivitiesRepository();
    }

    public override Task ExecuteAsync(string receiver, WebHookHandlerContext context)
    {
        if ("GitHub".Equals(receiver,StringComparison.CurrentCultureIgnoreCase))
        {
            string action = context.Actions.First();
            var message = context.GetDataOrDefault<GitHubMessage>();
            message.SetActivityType();

            var activityToSave = message.ConvertToActivity();
            repository.AddNewActivity(activityToSave);                
        }

        return Task.FromResult(true);
    }
}

Save all files, build and publish the application and then perform some operation on the repo. You will immediately see the data getting added to the table on SQL Azure.

Building a Web API Controller and a Page to View Activities

Now that we have data flowing into our database whenever an action takes place, let’s view this data on a page. Let’s build an ASP.NET Web API controller to serve this data. As already discussed, the controller will have APIs to serve all records from the table and a set of records after a certain activity. Add a new Web API controller to the Controllers folder and name it ActivitiesController and add the following code to it:

public class ActivitiesController : ApiController
{
    GitHubActivitiesRepository repository;

    public ActivitiesController()
    {
        repository = new GitHubActivitiesRepository();
    }

    public IEnumerable<GitHubActivity> GetAllActivities()
    {
        return repository.GetAllActivities();
    }

    public IEnumerable<GitHubActivity> GetActivitiesAfter(int id)
    {
        return repository.GetActivitiesAfter(id);
    }
}

Add a new HTML file to the project and name it Index.html. Add references of jQuery library and Bootstrap’s CSS file to this page.

It is up to you if you want to add these references from CDN or, using bower. We will have a table displaying the list of activities on this page and the table will be updated after every two minutes to show the latest activities. Copy following markup into body section of the page:

Finally, we need some JavaScript to call the API and fill the table with rows. It also has to keep checking for new activities at an interval of two minutes and add the new entries to the table.

Add a new JavaScript file app.js to the application and paste following code in it:

(function () {
    $(function () {
        var table = $("#tblGithubActivities");
        var lastActivityId;

        loadActivities().then(appendRows);

        setInterval(function () {
            loadActivitiesAfter(lastActivityId).then(function (data) {
                if (data.length > 0) {
                    appendRows(data);
                }
            });
        }, 2*60*1000);

        function appendRows(data) {
            lastActivityId = data[data.length-1].ActivityId;

            var rows = "";            
            data.forEach(function (entry) {
                rows = rows + "<tr><td>" + entry.ActivityType + "</td><td>" + entry.Description + "</td><td><a href='" + entry.Link + "'>Check on GitHub</a></td></tr>";
            });
            table.append(rows);
        }
    });

    function loadActivities() {
        return $.get('/api/activities');
    }

    function loadActivitiesAfter(activityId) {
        return $.get('/api/activities/' + activityId);
    }
})();

Save all the files, build and publish the application. Now you will be able to see a page similar to Figure – 7.

Figure 7: Github Activities using WebHooks

Conclusion

WebHooks provide a nice and simple way to keep monitoring activities on an online service. The data collected from WebHooks from certain services can even be used to take some crucial business decisions or, build a notification system on top of it. ASP.NET WebHooks provides abstractions on top of most widely used services and it has extensibility points to define our own senders and receivers. We will look at more capabilities of this feature in a future post.

Download the entire source code from GitHub at bit.ly/dncm22-aspnet-webhook

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

Rabi Kiran (a.k.a. Ravi Kiran) is a developer working on Microsoft Technologies at Hyderabad. These days, he is spending his time on JavaScript frameworks like AngularJS, latest updates to JavaScript in ES6 and ES7, Web Components, Node.js and also on several Microsoft technologies including ASP.NET 5, SignalR and C#. He is an active blogger, an author at SitePoint and at DotNetCurry. He is rewarded with Microsoft MVP (Visual Studio and Dev Tools) and DZone MVB awards for his contribution to the community