Language Understanding in Azure With LUIS

As artificial intelligence (AI) and machine learning (ML) proliferate within modern software applications, the need for these advanced skillsets become more difficult to retain. However, there is a growing shift to democratize AI/ML with cloud providers offering an ever-increasing menu of managed services to satisfy access to AI capabilities such as speech recognition, image classification, and computer vision including face and object detection.

Included with Microsoft Azure’s AI/ML offerings is a service called Language Understanding or LUIS.

In this article, I will give an overview of LUIS and demonstrate some of the features using a real-world example of natural language understanding in action.

Are you keeping up with new developer technologies? Advance your IT career with our Free Developer magazines covering C#, Patterns, .NET Core, MVC, Azure, Angular, React, and more. Subscribe to the DotNetCurry (DNC) Magazine and download all previous, current and upcoming editions. Yes! I'm Ready to Boost My IT Career

What is LUIS?

Under the umbrella of Azure Cognitive Services, the Language Understanding Intelligent Service (LUIS) offers a simplified method of including natural language understanding into your applications.

LUIS can be used directly by your applications or in combination with other Azure Cognitive Services that are directly related to Language and Speech processing and analysis including Azure Bot Service, QnA Maker, Text Analytics, Bing Spell Check, and Azure Search – to name just a few.

In addition, there are many cases for using a combination of LUIS with the Microsoft Bot Framework for strengthening the language understanding and intent for custom Bot applications.

Role within Azure

Azure offers a grouping of services within the cognitive services space that includes Decision, Language, Speech, and Vision. LUIS is included in the cognitive services offering under the Language category along with QnA Maker, Text Analytics, and Translator.

Benefits

The largest benefit you will receive by successfully implementing LUIS is that the alternative would require hiring a team of highly educated and experienced artificial intelligence and machine learning experts within the domain of natural language processing and understanding. These skills are in high-demand and smaller companies simply cannot afford to retain the necessary talent.

Before we dive into LUIS and the associated concepts, let’s quickly consider how one might approach a language processing application without the services of experienced AI and ML resources for processing text-based content and determining a next course of action.

For example, imagine you had to create a software application that could act on the simple concept of someone saying “Goodbye”. An easy solution would be to process the incoming text-based content and parse for the word “Goodbye” – if found, then do the next step.

Now consider someone saying “Goodbye”, “See you later”, or “I’m out of here”. As a human, you would immediately be able to gather that the person is leaving (their intended action). However, processing these sentence structures to allow for a computer to immediately understand the intent and therefore, the necessary action, presents more of a challenge.

Would you really want to program the control flow of your application to look for a static string such as “See you later” to determine a leaving action and then decide based on that direct match?

You could, but then you would have to consider all the various iterations of what a leaving intent could include. Doing so would require many potential control-flow statements in your application based on static strings and would quickly become unmanageable. For this example, let LUIS intervene, as it can help by processing many similar forms of a word or grouping of words and distil them down to a singular intent or actionable decision.

LUIS – Getting Started

In this section, we will briefly describe what natural language is, give a brief description of the application we will be building with LUIS, and highlight the core concepts that are important to grasp before using LUIS effectively.

Natural Language

The “L” in the LUIS acronym is for “Language”. This might be clear enough immediately, but what “language” is it referring to?

Language can be viewed as naturally occurring or constructed. Constructed languages could be a software language (i.e., C#, JavaScript) – built for purpose and derived for instructing computation. Naturally occurring languages are those that have evolved over time and have changed organically based on generally accepted terms and conventions over the course of the language’s lifespan. These languages that we are familiar with include English, Spanish, German, etc.

The language “type” that we (and LUIS) are focused on in this article is the natural language. LUIS supports several, but we must teach LUIS to understand how to interpret the language’s conventions.

The Application

Before we dive into the core concepts of LUIS, let’s talk about the application we are looking to support.

Let’s imagine that we currently work for a cable TV company, and we want to offer a way for our customers to send in questions or commands regarding our programming using either a direct SMS message from their mobile phone or by interacting with a Bot we will be setting up on our website. We won’t cover the client applications portion here such as the creation of the Bot or ingesting the SMS message, that will be left for another team to look after. However, after those applications are built and the text is extracted from the SMS message or the Bot application, it will be coming to our LUIS application for processing.

We are going to work with LUIS to build out the processing mechanism for the incoming text phrase. Keep in mind that we need to teach LUIS what it can expect from a customer’s language and how it should determine what the customer is requesting. With our help, LUIS will receive the proper training for success.

LUIS Core Concepts

Intents

What does the user want to do? Consider when a customer asks if he or she can find out when a favorite TV show is scheduled. They might task “when is Law & Order on tonight?” or “is Law & Order on tonight”. Intuitively, a human would know what the customer’s intent is based on the phrase used. In this case, look up the times for the TV show for tonight’s broadcast and return the answer to the customer. The human knows that the customer wants specific information based on the phrase and implied intent – the customer is wanting to know scheduled times for when a TV show airs.

Utterances

An utterance refers to the natural language phrase. As with our example above, “when is Law & Order on tonight?”. It is these utterances that LUIS will ingest, and it is up to you, as the LUIS application developer, to ensure that LUIS can understand the intent and deliver the result.
Entities
Continuing with our example, the entity would be the TV show. Again, this is easy for a human to recognize and understand, but it must be taught to LUIS. As we will see, there are four main types of entities, Machine learned, List, Regex, and Pattern.Any. The Machine learned entities can also be made up of composites, where a larger entity can be made up of child entities. This would be analogous to an object with several properties.

Models

“Models” or “Language Models”, used interchangeably, are made up of intents and entities. LUIS is continually trying to determine the intent based on a given utterance and then filter out the entities that the intent includes. Going back to our example, based on the utterance from the customer, you quickly determine he or she wants to know a list of times when a TV show is broadcasting. Asking for the time is the intent and the TV show is the entity.

Patterns

Patterns are meant to balance the number of trained utterances across all intents. The patterns can be used to help strengthen your intents. To use a concrete example, if you have 5 utterances against one intent, and 105 against another, there is an imbalance that a pattern may be able to alleviate.

The “None” Intent

Every application created in LUIS contains a default intent called the “None” intent. This intent is important and there is no option to delete or rename it. The “None” intent is basically just that – there is no intent derived from the text phrase that is processed. It is used as a fallback intent like the “default” option in a case statement – there must be an action that can be taken if no intent is derived. From an application perspective, the user may need to provide additional or clearer details when the intent is unknown.

Active Learning

As you will see, the concept of active learning takes place as LUIS singles out utterances (text-based phrases that require validation). If the confidence or prediction score is low and the intent can’t be determined, these phrases get flagged and sent to the “review” collection where you, as the administrator of the application, can provide additional direction for the next time this phrase (or fragment of the phrase) is ingested for processing.

Set it and Forget it

Remove the notion of “set it and forget it” before diving in. LUIS is not a “set it and forget it” solution. Working with LUIS requires an on-going commitment. We will see this in practice below, but keep in mind that LUIS needs to be taught continually to be effective. The great part is that the effectiveness will increase over time providing it is continually learning based on your commitment to teach it.

How Does LUIS Handle and Process Data?

Typical use cases for utilizing LUIS include conversational and human assistant bots in addition to command-and-control applications (application performs an action based on intent). It is important to note that your application data is sent to LUIS for processing only but that utterances can be logged in LUIS, if enabled. It is recommended that personally identifying or other sensitive data is limited to a flow-through for LUIS (and not logged explicitly).

In any case, this is the responsibility of the application developer but keep in mind that utterances containing sensitive data could be persisted outside of your application’s data store and this scenario should be considered when applying your use case to LUIS.

We won’t go into details in this article, but optional security measures can be taken where your logged data is stored in containers for you to manage directly.

Taking LUIS for a Ride

It’s now time for an example to demonstrate how the LUIS services are created within Azure and how features of LUIS are used to apply the core concepts against a new language model that can be used by your applications.

Creating Resources

Creating a LUIS resource is like creating any other Azure service. A simple search for “luis” in the Azure marketplace will find the Language Understanding service.

Searching for the LUIS Service

Clicking “Create” will bring you to the next screen where you can pick the appropriate options based on your existing Azure setup.

Creating the LUIS Services

This is familiar territory when creating an Azure service, however there are a few minor differences.

In the create options, there is a choice for creating an Authoring service, a Prediction service, or both. We will create both, ensuring that the appropriate resource group is either created or selected, the resource is adequately named, and the region and pricing are selected for each. Selecting the Free pricing tier is acceptable for testing the services and can also provide enough runway for an application will limited scale. We will review pricing options later in the article.

To reason about how the two services differ, consider the authoring service as the place you go to build and manage your LUIS models and the prediction service is where you go to send requests for the model to process.

Clicking “Review + Create” will create the following two resources:

Newly Created LUIS Services

Azure has taken the name given in the creation options screen, “dnc-tv” and used it for both resources while appending the “-Authoring” convention for the authoring service. The prediction service used the original name as the convention.

LUIS Portal

At this point, we will leave the Azure portal and explore the LUIS.ai portal where we can connect back to the created resources and work with them directly using the features of LUIS through the portal interface. The portal address is https://luis.ai/.

LUIS Portal

When signing in, you will use the same domain account to sign into the LUIS portal that you would for the Azure portal. Once signed in, you will be presented with the authoring tools that we will explore next after ensuring that you are setup with the correct subscription and Azure connection.

Welcome Modal

You will notice that LUIS has reached into the Azure account and retrieved the list of subscriptions. We will select the appropriate subscription and continue with the selection of our authoring resource we created previously in the Azure portal.

Authoring Service Selection

You can see here that the authoring resources are listed based on the subscription selected. There is also the option to create a new authoring resource. For now, we will select the “dnc-tv-authoring” service and click “Done”.

Empty Application List

We are now in the LUIS portal and connected to our desired authoring service. The first step for us will be to create a new “Conversation app” and begin with authoring the application.

Creating a New Application

In the “Create new app” modal, we want to make sure that we give the application and appropriate name. In our case, we will name the application “whats-on”.

Culture is the language that you will use for the text utterances (phrases) sent to LUIS. This would typically correlate to your client applications. Currently there are 13 supported languages and 6 in preview:

*The simplified Chinese character set is expected instead of the traditional character set. Naming for intents, entities, features, and regular expressions accept either Chinese or Roman characters.

**LUIS will not understand the difference between Keigo and informal Japanese due to the lack of syntactic analysis in this regard. Therefore, the incorporation of varying levels of formality as training examples are required.

The prediction resource list is the pulling from our Azure subscription and correlates back to the service we created earlier – along with the authoring service.

Application List

After our application is created, it will be listed in the applications table.

Lay of the Land

Now that we have successfully signed in, connected to our Azure instance, and created our first application, let’s quickly breakdown the various menu options before diving into creating some intents and entities.

Top Menu

You can see that the horizontal toolbar shows that we are currently working with the “whats-on” application. You will also notice that the application has a version designation. Applications in LUIS can be cloned and versioned. This is helpful for when you have an application published for production use, but you want to explore additional possibilities with a separate version.

The three main menu options are “DASHBOARD”, “BUILD”, and “MANAGE”.

DASHBOARD provides analytics that include the number of interactions with your LUIS application. BUILD is where we will spend most of our time in this example and, you can probably guess that this is where we will build out the intents, entities, and utterances that are critical components of the application. MANAGE, which we will touch on briefly later, is useful for adjusting settings and also accessing authorization keys for integration. The “TRAIN”, “TEST”, and “PUBLISH” options, we will get to shortly.

Side Menu

The side menu allows you to access the editing and management features for the main components of the LUIS application. Let’s click on “Intents” to get started.

Empty Intents List

Here you will see that the default “None” intent has already been created for us. As mentioned earlier, this intent is required for every application and cannot be renamed or deleted.

Let’s create our first intent. Click the “Create” button to open the create modal.

Create New Intent

We want our first intent to reflect the intent that a user might imply when sending in an utterance (text phrase) to indicate that he or she is looking for scheduling information regarding a TV Show. As such, we will call this intent “TVShowTimes”.

Click “Done” to create the intent.

TVShowTimes Edit Screen

Here we can see our new TVShowTimes intent. We will need to enter some utterances, but before we do, we will first go and create some entities.

This can be accessed by clicking “Entities” in the side menu.

Empty Entity List

Click “Create” to create your first entity.

Create an Entity

Here we can see the four entity types that are available to us. We will create a Machine Learned entity later. For now, select the “List” type and click “Create” to create the list entity.

New Entity Created

With our list entity, “TVShow” now created, we can go ahead and click on it to access the edit screen.

Entity List Values

Here we are in the “TVShow” edit screen. To enter new items, simply enter the name in the “Type in a list item…” text box and hit enter. I have gone ahead and entered several so we can breakdown what these are and how the “Synonyms” relate. These items all identify a TV show that we, as the cable company, know we include in our schedules. When the user sends in a text request, we want LUIS to be able to easily parse these values from the text phrases. We use the synonym values to associate potential additional ways that a user may request a TV show. “America’s Got Talent” is often referred to as “AGT”, so we use synonyms to create this association.

Let’s now go back and enter a different type of entity, the “Prebuilt” entity.

Prebuilt Entities List

LUIS comes with prebuilt entities that we can take advantage of without having to reinvent the wheel and start from scratch to train LUIS on heavily used entities such as a datetime value. Let’s go ahead and select the “datetimeV2” option and click “Done”.

To complete our entity list, create a “Machine learned” entity called “Recording” and we can make some modifications to it to round out our entities before going back to update our intents.

Entity List

From the entities list, click the “Recording” entity and we will make some slight modifications to allow for a composite entity.

Edit Recording Entity

In the edit screen for the “Recording” entity, click the “+” button to add a few child properties.

List of Child Properties

Our goal here is to create a composite entity that represents a TV show recording request. We want the “Recording” entity to have a “TVShow” and “RecordTime” property since both values will be required for scheduling the recording. We will use the “Add feature” option to create the association to the relevant entities.

Select TVShow Feature

You will see that clicking on the “Add feature” section revealed a dropdown with our two existing entities we have created previously. Let’s create the associations to each child property.

Recording Features

At this point we will go back and add one more intent before beginning to train our model. We will name this intent “RecordTime”, and it will be used to determine from any utterance sent by the user that a TV show should be recorded and at which time.

Intents List

Looking at our intent list now, we can see that we have three intents.

It is now time to start training LUIS. Let’s click on the “TVShowTimes” intent and start to enter utterances which LUIS will use to create the necessary learning model when responding to real-world requests.

Empty Utterance List

Enter the phrase “can you tell me when criminal intent is on today?” into the text box and hit “Enter”.

Example TV Show Utterance

You can see that LUIS has picked out from the utterance that the list type of TVShow has been highlighted. In addition, LUIS has also signaled out “today” as referring to a prebuilt entity type that we added to represent a datetime value.

Let’s go ahead and enter more utterances to help LUIS understand what intent is inferred from these types of text-based phrases.

TV Show Utterance List

We will now go and enter some utterances to help train LUIS further on the RecordTVShow intent

Record Utterance

You can see that LUIS has parsed the “TVShow” and “datetimeV2” entities, but we also want to ensure the “Record” object can be parsed as well. Let’s help LUIS out here. We are going to highlight a portion of the text and make some adjustments.

Select the Recording Entity

Click “Recording” and you will see that LUIS now understands that this block of text is a higher-level object with child entities that it has already detected.

Recording Utterance

Now we will continue to add some more utterances in the “Record” intent to ensure LUIS is receiving additional training.

Recording Utterances

Let’s now explicitly tell LUIS to lock in the training based on our guidance. Click the “Train” button to train LUIS on the information provided so far.

Train Button

From here, we can click the “Test” button to see how LUIS is doing.

Test Panel

You can see that LUIS has determined that the intent from our phrase is “TVShowTimes” and that the entities have been singled out for both the “TVShow” and the “datetimeV2” entities!

Something to point out here is that the intent score is low at 0.672. This is a good indication that LUIS needs further training. In this case, additional utterances can be added to ensure that LUIS is getting smarter.

The last thing to do now is to publish so integration with the external applications can also begin.

Go ahead and click the “Publish” button.

Publishing Completed

After publishing is completed, click the “Access your endpoint Urls” to jump over to the “Prediction Resources” screen where we will find the values to use when integrating with the LUIS from your client applications. For enhanced security, the endpoint Urls enforce TLS 1.2

Prediction URL

We want to copy the Example Query and include a custom natural language phrase into the “query” parameter. The query below contains the relevant domain specification, security keys, and query parameter to successfully access your LUIS instance and application.

https://dnc-tv.cognitiveservices.azure.com/luis/prediction/v3.0/apps/1c30df54-764a-4dd5-a037-d9db349503c5/slots/staging/predict?subscription-key=9a1bf57b556a4e789cd0f8a938e0ecfa&verbose=true&show-all-intents=true&log=true&query=what%20time%20is%20seinfeld%20on%20thursday?

Copy the created query and paste it into your browser’s location bar and hit “Enter”. The following output will be produced:

Query Result

The JSON document in the response includes a nicely formatted set of data values that your client application can easily hydrate into an object and use for additional application logic. Your client application can use these values to look up the tv show for the relevant date and return all the associated times. This decisioning mechanism is possible since LUIS will do the heavy lifting of parsing the intent and relevant entities from a natural language structure that your application does not have to parse directly.

Other Ways to Interact With LUIS

In addition to using the portal for authoring, testing, and training the created language models, there are other interaction methods that can be utilized to both author and query the trained and published models. These methods include direct access using either a REST API or SDK client libraries for C#, JavaScript, and Python.

Prebuilt Domains

While you can create application models that are custom to your domain, LUIS also offers a selection of prebuilt domains that you can readily access and include in your applications.

Prebuilt Domains

There is a nice mix of prebuilt domains available to help get you started, or you may choose to take full advantage of one of these domains if your application currently requires the language understanding capabilities provided out of the box with LUIS.

Pricing

Pricing for the LUIS service ranges from free which has a limit on transactions per second (TPS) and total transactions per month, to a paid tier that includes a higher number of TPS and allowable monthly transactions. The authoring service and prediction service are not offered in every region so you may have to pick separate regions for each of the authoring and prediction service.

Region: West US

Conclusion:

In this article, I gave an overview of the capabilities of LUIS.

LUIS is a mature service included with the growing array of managed AI/ML services that Microsoft is committed to developing under the expanding Azure Cognitive Services umbrella. Democratizing access to natural language understanding using managed services will allow your applications to include complex natural language understanding (NLU) capabilities at a fraction of the cost when compared to hiring and retaining specialized skillsets.

In a subsequent article, we will look at integrating directly with LUIS to enhance a chatbot application.

This article was technically reviewed by Vikram Pendse.

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

Darren Gillis is a Toronto-based software developer and technologist with 20+ years of experience primarily with Microsoft technologies.  He is a big fan of data and Microsoft Azure, having architected numerous cloud-based software projects leveraging many of the features that Azure has to offer.  He is currently developing a SaaS based compliance platform using C#, React, and PostgreSQL.  Feel free to connect with him on LinkedIn, or follow him on twitter @darrengillis.