Service Oriented Solutions using MSMQ and WCF

There are many types of messaging systems and architectures available at our disposal. All encompass one main objective, which is to be able to send a message (or “object”) from a source to its destination, wherever the destination may be.

Some systems commonly send a request to a service (such as WCF services or even communicate with the legacy ASMX web services) for it to do some internal processing (optionally) and finally insert data into the database, such as Microsoft SQL Server. Such systems are very common and used almost everywhere in today’s world of technology.

These systems, like with anything, can have a downtime such as server issues or DB issues and therefore the request coming from the client can fail and therefore no data will be submitted. Sure, you can implement a cluster of servers to attain a higher uptime level, but sometimes this is quite expensive in terms of cost and resources; as you would be dealing with different pieces of software to make it work together. Moreover to make it work in such an environment (SQL Server, Web servers etc…), there will be a number of moving parts to be configured and possibly even re-written/reviewed.

This article is published from the DNC Magazine for .NET Developers and Architects. Download this magazine from here [PDF] or Subscribe to this magazine for FREE and download all previous and current editions

At times, you take a step back and help but wonder if there is any way to simply send a message down the wire without having the systems always being ON all the time, and also without it having to lose messages or data. There must be some way of receiving the message successfully from a client and then be able to process it at a later time, or when other systems are up and running.

This is where MSMQ (Microsoft Message Queuing) comes into play. MSMQ has been in Windows for a very long time and is also included in some Windows CE versions. It allows remote computers to connect and send messages to the destination in a transactional and non-transactional manner and allows you to persist these incoming messages to the disk. So in case of a power failure, the messages are still on the disk, waiting to be processed the next time the system is functional.

You do not always have to have the server with MSMQ up and running. The sender will keep trying to connect and send messages in the queue until the destination it is aiming for, is back up and running; so the client will hold onto the message until there is a connection. This means that the message will not be lost but also that it will automatically be sent when the connection is established.

This is one of the fundamental things about MSMQ that has been with us for years and the best part is that it is available for Free, as it is built into the OS (but you do need to manually install it). It has been available since Windows 95 clients and Windows 2000 for servers with different versions of MSMQ being evolved overtime. But from Windows XP (Professional), it has the ability to communicate via SOAP formatted messages and to be able to multicast messages.

When WCF was developed by Microsoft on the .NET Framework, MSMQ was a story that was included where it would make it easy for developers to integrate with the system. It also meant that WCF and MSMQ can be used to provide secure and reliable transportation of messages which was a big bonus for WCF developers. So with this, you can send and also receive messages to and from the MSMQ queue and also have the ability to read the messages from the queue and start processing them.

MSMQ Usage and some examples

So now that we know what MSMQ is and that we can use WCF on the .NET Framework to queue messages and read them from the queue, where do we use MSMQ? Well the fact is that you can use it anywhere. Some systems use it to communicate with one another if there is no way to interact via services for instance.

Let’s see an example of a fictitious goods and a shipping company.

Firstly, the systems are setup to be transactional, so it is guaranteed that if there are multiple operations on different queues for a message, then either all or none of the operations are completed and that nothing is in a "half limbo” state.

Imagine there’s a goods company, “DNC Goods”, selling goods to customers. Also imagine that the shipping company “DNC Acme Shippers” is the shipping company DNC Goods uses to request and ship orders. Both have MSMQ and Distributed Transactions installed in their system.

DNC Acme Shippers operates between 9-5pm and they do not accept any orders beyond this time. They also do not want to poll a database for orders which have been approved and are waiting to be shipped, because it’s expensive to do so and error prone at times. They however need to agree on a format/contract that both systems can understand. This is where WCF is a great candidate to use in such a system as it is based on a contract binding/agreement basis.

DNC Acme Shippers always switch on their systems between these times to receive requests from external companies such as DNC Goods for shipping goods. They don’t want to know anything else outside of these hours.

Now imagine a customer logging onto the DNC Goods website, placing an order and paying for the order. The website takes the order request (checks availability etc…) and when the card payment is approved almost immediately, it will create a message object which will contain the order details and finally send this off to DNC Acme Shippers.

DNC Goods website does not need to know the technical implementation detail such as how the communication channel works or even if it is between the business hours that DNC Acme Shippers are operating on. All the website needs to do is place the MSMQ message on the queue – that is literally it.

MSMQ will then try to connect to the destination (which is provided in the MSMQ message object) and send the message. If it cannot connect, it will try again later. MSMQ takes care of this communication for us, amongst other things that is exposed in the MSMQ message object.

This system provides us the solution for the requirements needed for DNC Acme Shippers where they only want orders for shipping between their business hours and that the message they expect is of a certain agreed standard.

The DNC Goods website processes the order and by executing its own internal logic, processes the card transaction. Finally once the transaction is approved, it creates a message ready to be submitted to DNC Acme Shippers for shipping. The shipping company will take care of the order and email the customer when the goods are shipped.

Setup MSMQ on your System

To setup the system, we will not get into the technical details about setting up a full network architecture, but instead focus more on the software configuration and assume that it is a simple direct end to end connection.

The technologies used for this specific example were as followed:

- Windows 7 Ultimate

- MSMQ (installed from control panel)

- Visual Studio 2012 (non express SKU)

- WCF

- .NET Framework 4.0

- C#

We install MSMQ (Message Queuing) via “Programs and Features/Turn on or off Windows Features” and install with HTTP Support and Multicasting support. We also install Distributed Transactions.

This is all that is needed for DNC Goods.

For DNC Acme Shippers, we need to create a queue, which is transactional. To do so, open the MMC console, and go to File > Add/Remove Snap-ins and select “Computer Management” from the available snap-ins:

Figure 1. Setting up MSMQ

Press “Add” and then on the dialog screen that appears, press “OK” to connect to the local computer.

Expand the “Services and Applications” and then “Message Queuing”. You should see the following:

Figure 2. Message Queueing

In the Private Queues, we will create a transaction queue called “ShippingOrders”. Right click on the Private Queues and select “New > Private Queue”.

Type “ShippingOrders” in the Queue name and check the “Transactional” box and press OK:

Figure 3. Creating a transactional MSMQ

Now that we have MSMQ installed with a transactional queue created, we need to write some client code which will place an order to the queue. We first need to create a common project which has our contract agreement class “Order” which will be created and placed on the queue.

Create a new project in Visual Studio (using .NET 3.5 or higher) named “Acme.Shared.Contracts” and create a class named Order. Add a reference to “System.Runtime.Serialization” which is a .NET assembly and will expose the attributes we need to apply to the class and properties, in order to serialize the object to and from the MSMQ using WCF as the communication protocol.

The following code will be written as a data contract:

using System;
using System.Runtime.Serialization;

namespace Acme.Shared.Contracts
{
    [DataContract]
    public class Order
    {
        public Order()
        {
        }

        [DataMember(IsRequired = true)]
        public int OrderID { get; set; }
        
        [DataMember(IsRequired = true)]
        public DateTime SubmittedOn { get; set; }
        
        [DataMember(IsRequired = true)]
        public string ShipToAddress { get; set; }

        [DataMember(IsRequired = true)]
        public string ShipToCity { get; set; }

        [DataMember(IsRequired = true)]
        public string ShipToCountry { get; set; }

        [DataMember(IsRequired = true)]
        public string ShipToZipCode { get; set; }

    }
}

In order to simulate the messages being sent (and received), we first create a simple application (Console/Winforms/WPF) which will create a message and place it on the MSMQ named “ShippingOrders”. Be aware that this is not production quality code but simply to illustrate how messages can be sent and received.

Create a new project named “Acme.Dispatcher” and add a reference to the “System.Messaging” .NET assembly, which gives us access to the MSMQ classes to interact with, and the “System.Transactions” assembly which gives us access to the TransactionScope class. Also add a project reference to the “Acme.Shared.Contracts” project, as it contains the data contract we will be using to create and send the Order message. The order message will be sent to the transactional private queue we created above “ShippingOrders”.

The code shown here is what is used to create a sample Order and place it on the queue:

namespace DNCDispatcher
{
    class Program
    {
        static void Main(string[] args)
        {
            // create a fake order, for simulation:
            var anOrder = new Order { OrderID = 1, ShipToAddress = "123 Abc avenue", ShipToCity = "DNC", ShipToCountry = "A country", ShipToZipCode = "12345", SubmittedOn = DateTime.UtcNow };
            
            // create a MessageQueue to tell MSMQ where to send the message and how to connect to it
            var queue = new MessageQueue(ConfigurationManager.AppSettings["MessageQueuePath"]);

            // Create a Message and set the body to the order object above
            var msg = new Message { Body = anOrder };

            // Create a transaction
            using (var ts = new TransactionScope(TransactionScopeOption.Required))
            {
                queue.Send(msg, MessageQueueTransactionType.Automatic); // send the message
                ts.Complete(); // complete the transaction
            }

            Console.WriteLine("Message Sent");
            Console.ReadLine();
        }
    }
}

The “MessageQueuePath” is a setting pulled from the app.config file which contains the queue path for MSMQ to know where to send the message to. The app.config setting looks like as follows:

<appSettings>
    <add key="MessageQueuePath" value="FormatName:Direct=TCP:xx.xx.xx.xx\private$\ShippingOrders"/>
</appSettings> 

The “FormatName” is used when sending messages directly to a computer or over the internet, or reading them while operating in a domain or workgroup environment, or even in an offline mode. It is also used to send messages when authentication, routing and encryption is not needed and in this setup, it is not needed. For more information about FormatName, please visit the MSDN resource here: https://msdn.microsoft.com/en-us/library/ms700996(v=vs.85).aspx

We are specifying that we want to send the message to the remote computer. The “xx.xx.xx.xx” should be replaced with the IP Address you are intending on sending the message to.

When you run the simulator, you will see that the message was sent, and when you pull up MSMQ in Microsoft Management Console (MMC), you will see that the number of messages will be set to “1”:

Figure 4. Message waiting

Now we know our order is waiting to be read, we can now create a final project which will be the one to read the message from MSMQ. To do this, for this exercise, let us create another Console/Winforms/WPF project named “Acme.OrderReader” and add the “Acme.Shared.Contracts” project reference. Also add the “System.Transactions”, “System.Messaging”, “System.Runtime.Serialization” and “System.ServiceModel” .NET assembly references for us to be able to process the incoming messages from MSMQ through WCF.

In order for MSMQ to dispatch the message to the reader application through WCF (since WCF is the one to read the messages from the queue), WCF needs to know the type of the object we are expecting so that it can deserialize it and finally dispatch it to our application for processing. To do so, we use the ServiceKnownType attribute.

Let us create an interface called “IOrderInboundMessageHandlerService” and provide a single method that the implementer must implement:

namespace Acme.OrderReader.Interfaces
{
    [ServiceContract]
    [ServiceKnownType(typeof(Order))]
    public interface IOrderInboundMessageHandlerService
    {
        [OperationContract(IsOneWay = true, Action = "*")]
        void ProcessIncomingMessage(MsmqMessage<Order> incomingOrderMessage);
    }
}

Following this interface, let’s create a class called “OrderInboundMessageHandlerService” which implements this interface and set the ConcurrencyMode enum to “Multiple” so it can process multiple incoming messages. You also have the option of creating a single context service or multiple. This is a design choice that you must decide upon depending on the environment. For this purpose, we will make it “Multiple”. As for the transaction behaviour, we will autocomplete the transaction and ensure that a transaction scope is required, since the queue is transactional. With these configuration parameters in mind, the class will look similar to the following:

namespace Acme.OrderReader
{
    [ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Single, InstanceContextMode = InstanceContextMode.Single, ReleaseServiceInstanceOnTransactionComplete = false)]
    public class OrderInboundMessageHandlerService : IOrderInboundMessageHandlerService
    {
        #region IOrderInboundMessageHandlerService Members

        [OperationBehavior(TransactionScopeRequired = true, TransactionAutoComplete = true)]
        public void ProcessIncomingMessage(MsmqMessage<Order> incomingOrderMessage)
        {
            var orderRequest = incomingOrderMessage.Body;
            Console.WriteLine(orderRequest.OrderID);
            Console.WriteLine(orderRequest.ShipToAddress);
            Console.WriteLine(orderRequest.ShipToCity);
            Console.WriteLine(orderRequest.ShipToCountry);
            Console.WriteLine(orderRequest.ShipToZipCode);
            Console.WriteLine(orderRequest.SubmittedOn);
        }

        #endregion
    }
}

As you can see, with the parameters we defined above, we should now be able to receive the message and for the purpose of this exercise, display the details about the order on the console. In reality, you would process the message to the business requirements definition (i.e check the order does not already exist in the system or insert into the database or print out shipping labels etc…).

We now need to host the reader so the reader can read and process incoming messages placed on the queue. To do this, we use the WCF ServiceHost object and then host the OrderInboundMessageHandlerService like so:

using System;
using System.ServiceModel;

namespace Acme.OrderReader
{
    class Program
    {
        static void Main(string[] args)
        {
            ServiceHost host = new ServiceHost(typeof(OrderInboundMessageHandlerService));
            host.Faulted += host_Faulted;
            host.Open();
            Console.WriteLine("The service is ready");
            Console.WriteLine("Press <ENTER> to terminate the service");
            Console.ReadLine();
            if (host != null)
            {
                if (host.State == CommunicationState.Faulted)
                {
                    host.Abort();
                }
                host.Close();
            }
        }

        static void host_Faulted(object sender, EventArgs e)
        {
            Console.WriteLine("Faulted!"); // Change to something more sensible – this is just an example showing what happens when the host has faulted.
        }
    }
}

We are getting very close to completion with our project. The final step is to setup the application so that it will open the WCF service host and start reading the messages from the MSMQ. To do so, we use the ServiceHost class. But first, we must configure WCF settings in the config file (app.config). We simply need to tell WCF the details about the service such as the A B C (Address, Binding, Contract). The following is what we would enter in the config file:

<system.serviceModel>
    <behaviors>
      <endpointBehaviors>
        <behavior name="IncludeExceptionDetails">
          <callbackDebug includeExceptionDetailInFaults="true" />
        </behavior>
      </endpointBehaviors>
    </behaviors>
    <services>
      <service name="Acme.OrderReader.OrderInboundMessageHandlerService">
        <endpoint address="msmq.formatname:DIRECT=OS:.\private$\ShippingOrders" binding="msmqIntegrationBinding" bindingConfiguration="IncomingMessageHandlerBinding" contract="Acme.OrderReader.Interfaces.IOrderInboundMessageHandlerService">
        </endpoint>
      </service>
    </services>
    <bindings>
      <msmqIntegrationBinding>
        <binding name="IncomingMessageHandlerBinding"
                 closeTimeout="00:30:00"
                 receiveTimeout="01:00:00"
                 retryCycleDelay="00:00:10"
                 receiveRetryCount="0"
                 exactlyOnce="true"
                 maxRetryCycles="3"
                 receiveErrorHandling="Move">
          <security mode="None"/>
        </binding>
      </msmqIntegrationBinding>
    </bindings>
  </system.serviceModel>

Notice that the endpoint address is set to look at the local computer MSMQ – this should once again, be changed to the machine where the queue is located on the receiving end. For this exercise, we are sending to a remote computer and the OrderReader is running directly on the machine where the messages are being sent to, thus the reason to read the messages from the local MSMQ endpoint.

We are now finally ready to run the solution. First, fire up the OrderReader app and then secondly fire up the Dispatcher app. The dispatcher will send the message and the OrderReader will almost immediately read the incoming message and display the results in the console:

Figure 5. Order sent and received

As you can see, the message was sent and the order was received.

Whilst this solution works – in the real world, things differ slightly. You can have messages that are invalid and the application not expecting it, therefore it would be known as a poison message and MSMQ will automatically place it in its own queue because the reader is unable to process the message (i.e it cannot deserialize it and does not know its type) and terminate the transaction.

You can allow the receiver (OrderReader) to handle these poison messages if you wish within the code. For more information, please visit the following MSDN resource.

Conclusion

DNC Goods and DNC Acme Shippers now have a solution that both parties are satisfied with, where orders are sent and are only received by DNC Acme Shippers when they open for business during their own business hours. The solution shows us that the technologies are readily available at very little cost, generally speaking. We can create a reasonably straight forward service oriented solution that meets the demands of businesses, using the existing and ever evolving technologies.

Download the entire source code of this article (Github)

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

After leaving Microsoft, Ahmed Ilyas ventured into setting up a consultancy company, Sandler Ltd, offering the best possible solutions for a magnitude of industries and providing real world answers to those problems. He uses the Microsoft stack to build these solutions and has been able to bring in best practices, patterns and software to his client base. This resulted in him being awarded the MVP title thrice in C# by Microsoft for 'providing excellence and independent real world solutions to problems that developers face'. His reputation and background has resulted in him having a large client base in the UK and Sandler Software (USA) now includes clients from different industries from medical to digital media and beyond.