State in Multi-threaded C# Applications

In a previous article, I talked about Global state in C# applications. I talked about why people tend to use global variables and suggested some solutions.

In this part, I will talk about state in multithreaded C# applications.

C# Multithreaded application – The Example

I am going to continue using the same example application I used in Part 1. The source code can be found here: https://github.com/ymassad/StateExamples. I added new projects to the solution for this part.

Modifying state without any synchronization

Take a look at the MultithreadingAndRefStateParameters project. This project is a modified version of the PassingStateViaRefParametersWithIOC project that we ended up with in Part 1.

The difference between the two projects is that in the MultithreadingAndRefStateParameters project, documents are processed in parallel.

Take a look at the TranslateDocumentsInFolderInParallel method in the FolderProcessingModule class. This method uses the Parallel.ForEach method to process the documents in parallel. The Parallel.ForEach method automatically decides on the degree of parallelism based on the number of CPU cores on the machine and other factors.

Another difference is that I added a ref int numberOfTimesCommunicatedWithServersState parameter to both the TranslateFromGerman and TranslateFromSpanish methods to allow these methods to increment such state every time they communicate with a translation server. I also defined a numberOfTimesCommunicatedWithServersState variable in the Main method to hold such state.

Note: In Part 1, I used the Server1State class to represent the state of server 1 (if it is down and since when). All the projects I created for Part 2 don’t have this state class. In these new projects, the choice between server 1 and server 2 is random.

The number of documents the application will try to process is set to 1000 in this application and each document has two paragraphs. Therefore, the application is expected to communicate with the servers (fake servers in this demo application) 2000 times.

If you run this sample application, the application will print the value of numberOfTimesCommunicatedWithServersState after all documents have been processed. In one run on my machine, I got the value 1974. In another run, it was 1946. I expect it to give you a number smaller than 2000 when you run it (unless you have a single CPU core or Parallel.ForEach decided to use a single thread for some reason).

We get a number smaller than 2000 because we have a race condition. The issue is related to the following code:

numberOfTimesCommunicatedWithServersState++;

This code exists in four places in the project: two in TranslateFromGerman() and two in TranslateFromSpanish().

This code increments the value in the numberOfTimesCommunicatedWithServersState parameter. However, such increment operation is not atomic. It consists of three operations:

1. Reading the value of the state parameter

2. Adding one to the value (not the state parameter, think of this as a temporary variable)

3. Writing the result to the state parameter

Now, imagine the following scenario: The current value of the state is 100, and two threads are about to execute the code above. They execute the operations like this:

Because each thread communicated with the service once, we expect the new state value to be 102. Instead, it ended up being 101!

To solve the problem, we need to treat the increment operation as an atomic operation.

To increment an integer in C# as an atomic operation, we can use the Interlocked.Increment method. We can replace the code line that increments the state parameter with the following line:

Interlocked.Increment(ref numberOfTimesCommunicatedWithServersState);

The Increment method increments the value stored in numberOfTimesCommunicatedWithServersState as an atomic operation.

The lock statement

What if we want to multiply the state variable by some number? Or do some other type of complex update to a more complex state object?

For example, let’s say we want to have a complex state object to keep track of the number of times the application communicated with each server and the total time spent communicating with each server.

We can use the lock statement to synchronize any kind of state manipulation.

Take a look at the MultithreadingAndRefStateParametersAndLockStatement project. The ServerCommunicationStatistics class is an immutable class that contains four properties that keep track of the number of times we communicate with each server and the total time spent communicating with each server.

The TranslateFromGerman and TranslateFromSpanish methods take a ServerCommunicationStatistics state parameter (by reference, of course) instead of a simple int state parameter. They also take a parameter of type object (statisticsStateLockingObject) that they use for synchronization. Here is the relevant code:

Stopwatch stopwatch = Stopwatch.StartNew();

var result = TranslateFromSpanishViaServer1(text, location);

var elapsed = stopwatch.Elapsed;

lock (statisticsStateLockingObject)
{
    statisticsState = statisticsState
        .WithTotalTimeSpentCommunicatingWithServer1(
            statisticsState.TotalTimeSpentCommunicatingWithServer1 + elapsed)
        .WithNumberOfTimesCommunicatedWithServer1(
            statisticsState.NumberOfTimesCommunicatedWithServer1 + 1);
}

This code measures the time spent communicating with server 1. It then uses the C# lock statement to synchronize the update to the statisticsState parameter (of type ServerCommunicationStatistics). All access to the statisticsState object in the TranslateFromGerman and TranslateFromSpanish methods is protected by locking on the statisticsStateLockingObject object. This guarantees that only a single thread can be inside the lock statement block at any given time. This means that when a thread reads the value of statisticsState, no other thread will have the chance of reading or updating the value of statisticsState before the first thread is done updating it. This eliminates the race condition.

Inside the lock block, we create a new ServerCommunicationStatistics instance that has the updated property values. I use With methods here to do this. For more information about With methods, see the Designing Data Objects with C# and F# article.

Note that in the Main method, we pass the same object instance (serverCommunicationStatisticsStateLockingObject) to both TranslateFromGerman and TranslateFromSpanish. If we pass two different instances, updates to statisticsState will not be synchronized and we will have the race condition again.

But why have the statisticsStateLockingObject parameter? Why not use the statisticsState parameter?

The statisticsState parameter (being a state ref parameter) does not always refer to the same object instance. Therefore, when two threads read the value of this parameter, they might get references to two different ServerCommunicationStatistics objects if they read it at different times.

We must lock using the same instance.

If we turn ServerCommunicationStatistics into a mutable object, then we can pass it normally (not as a ref parameter), and then we can use it for locking. However, having mutable objects passed as parameters makes methods harder to understand. For more details, see the Designing Data Objects with C# and F# article.

Using Interlocked.CompareExchange

The lock statement is the recommended way of protecting shared resources in .NET. However, there is a way to use the Interlocked class to update a complex state object which does not involve a lock.

Take a look at the MultithreadingAndRefStateParametersAndComplexCAS project. Because we need no locks in this project, I removed the statisticsStateLockingObject parameters. Here is how I update the state (in the TranslateFromGerman method for example):

Utilities.UpdateViaCAS(ref statisticsState, state =>
    state
        .WithTotalTimeSpentCommunicatingWithServer1(
            state.TotalTimeSpentCommunicatingWithServer1 + elapsed)
        .WithNumberOfTimesCommunicatedWithServer1(
            state.NumberOfTimesCommunicatedWithServer1 + 1));

I call a method called UpdateViaCAS passing a reference to the state and a lambda. This lambda will be called by the UpdateViaCAS method to calculate a new state value from the current state value.

Here is how the UpdateViaCAS method looks like (CAS stands for Compare and Swap):

public static void UpdateViaCAS<TState>(
    ref TState state, Func<TState, TState> update) where TState : class
{
    var spinWait = new SpinWait();

    while (true)
    {
        TState beforeUpdate = state;

        TState updatedValue = update(beforeUpdate);

        TState found = Interlocked.CompareExchange(
            location1: ref state,
            value: updatedValue,
            comparand: beforeUpdate);

        if (beforeUpdate == found)
            return;

        spinWait.SpinOnce();
    }
}

The trick in the UpdateViaCAS method is that it uses the Interlocked.CompareExchange method to update the state object only if it has not changed by another thread.

The idea is like this: We first read the value of the state into the beforeUpdate variable. We then invoke the update function to calculate the updated state object. We then invoke Interlocked.CompareExchange.

Interlocked.CompareExchange will only update the state if it finds out that no other thread has already updated it since we read it. It does that by comparing the current state value with beforeUpdate. The compare operation and the swap operation (setting the new state value) happen as a single atomic operation.

The Interlocked.CompareExchange method returns the value of the state as it found it when attempting to swap. If we find out that this returned value is the same as the value we read before attempting to swap, then our mission is complete. Otherwise, we repeat the whole operation. This means that we might end up calling the update function multiple times, each with a different state value.

This is an advanced technique and as I mentioned before, using the lock statement is the recommended way to protect a shared resource in .NET.

Still, using the CAS approach might give you some performance benefits in certain cases.

When it comes to performance, though, always test to see which approach is better in your case.

Note: In the example above, I use SpinWait to wait for a very small time after each failed attempt to set the new state before trying again. This is intended to reduce the chance of failing for the next attempt. The reason I use this and not Thread.Sleep is for performance reasons which are out of the scope of this article.

Extracting state updating to a StateHolder object

Usually, you don’t want your functions to know exactly how state is stored and which synchronization technique is used to protect access to it.

Take a look at the MultithreadingAndRefStateParametersAndStateHolder project. Look at the TranslateFromGerman method for example. It does not take a ref ServerCommunicationStatistics parameter, nor an object, to use for locking. Instead, it takes a parameter of type IStateUpdater<ServerCommunicationStatistics>. The IStateUpdater interface has the following method:

void UpdateState(Func<TState, TState> updateFunction);
The TranslateFromGerman method uses the UpdateState method like this:
statisticsStateUpdater.UpdateState(statisticsState =>
    statisticsState
        .WithTotalTimeSpentCommunicatingWithServer1(
            statisticsState.TotalTimeSpentCommunicatingWithServer1 + elapsed)
        .WithNumberOfTimesCommunicatedWithServer1(
            statisticsState.NumberOfTimesCommunicatedWithServer1 + 1));

The UpdateState method takes an updateFunction parameter. This function parameter allows the UpdateState method to call back so that we can tell it how to produce a new state value given the current state value (represented by the statisticsState lambda parameter above).

The IStateUpdater parameter of the TranslateFromGerman method makes it clear to readers that this method might update state.

In the Main method, create an instance of the ThreadSafeStateHolder class and pass it to the TranslateFromGerman and TranslateFromSpanish methods. This class implements the IStateUpdater interface (and other interfaces). Internally it uses the lock statement to protect access to the state. We can create another state holder class that uses Interlocked.CompareExchange if we wanted to.

Using the return value of a function to communicate the state back to the caller

In multithreaded applications, functions running on different threads might require access to a shared copy of some state.

For example, consider the Server1State class (the state discussed in part 1). If code running on thread #1 discovers that server1 is down and sets the state accordingly, then thread #2 which runs in parallel with thread #1 should be able to read that same state.

In these scenarios, using an IStateUpdater parameter (or a ref parameter) to update state is required because a function must be able to have some outputs (the new state) while it executes, i.e., before it returns.

The IStateUpdater interface (and ref parameters) allows functions to make such outputs before they complete. They can simply invoke the UpdateState method. Other functions will be able to read the new state (via IStateGetter for example) when they wish.

ServerCommunicationStatistics is different. We only really read these statistics at the end of the application, after all the processing is done.

Each thread can have its own instance of ServerCommunicationStatistics, and translate a subset of the documents. We can then combine the ServerCommunicationStatistics objects from different threads to produce a single instance of ServerCommunicationStatistics that we then use to display the results in the console.

See the MultithreadingAndUsingReturnValue project. In this project, there are no ref parameters or State Holders. A method interested in reading or updating state takes the current value of the state as input, and returns a new value of the state as part of the output.

For example, here is the signature of the TranslateFromGerman method:

public static (Text text, ServerCommunicationStatistics newState) TranslateFromGerman(
    Text text,
    Location location,
    ServerCommunicationStatistics statisticsState)

The method takes a normal (not ref) ServerCommunicationStatistics parameter and returns a new ServerCommunicationStatistics object. Notice that I used a tuple here to represent two outputs; the original translated Text object, and the new ServerCommunicationStatistics object.

Take a look also at the TranslateDocumentsInFolderInParallel method:

public static ServerCommunicationStatistics TranslateDocumentsInFolderInParallel(
    string folderPath,
    string destinationFolderPath,
    Location location,
    ServerCommunicationStatistics statisticsState)
{
    IEnumerable<Document> documentsEnumerable = GetDocumentsFromFolder(folderPath);

    object lockingObject = new object();

    var state = statisticsState;

    Parallel.ForEach(
        documentsEnumerable,
        localInit: () => ServerCommunicationStatistics.Zero(),
        body: (document, loopState, localState) =>
        {
            var result = DocumentTranslationModule.TranslateDocument(
                document, location, localState);

            WriteDocumentToDestinationFolder(result.document, destinationFolderPath);

            return result.newState;
        },
        localFinally: (localSum) =>
        {
            lock (lockingObject)
            {
                state = state.Combine(localSum);
            }
        });

    return state;
}

This method uses a different overload of the Parallel.ForEach method. This overload allows us to have a thread-local state (or Task-local state since Parallel.ForEach uses the Task Parallel Library, and does not use threads directly).

When each task runs, it starts with an initial state. In the method above, I use ServerCommunicationStatistics.Zero to construct an empty ServerCommunicationStatistics object. You can find this code above in the localInit function parameter.

Then, when processing each item, in the body function parameter, we get to access the task-local state. In the method above, the localState lambda parameter represents such state. After I call DocumentTranslationModule.TranslateDocument, I return the new state object as the return value of the lambda. We don’t need to synchronize any access to this state because it is guaranteed that only a single task (and thus a single thread) is going to access it.

However, when a task completes its subset of documents, the localFinally function parameter is invoked. The localSum lambda parameter in the code above will be the state object produced by translating the final document run by the task. Here I pass a lambda that updates state; a local variable representing the final result for all tasks.

Notice that here I must synchronize access to the state variable since there might be multiple tasks that finish at the same time and thus attempt to update the state variable at the same time.

Without locking, we can have a race condition here.

Please notice that in this project, there is no Inversion of Control (IOC). Functions call each other directly. This means that the call hierarchy is polluted with the state parameters/return values. Go to the programs functions and see how each of the functions had to deal with the state parameter/return value.

Here are some questions:

1. What are the benefits of having the state returned via function return values in both multithreaded and single-threaded applications?

2. Can we use the return values of functions to return state without polluting the whole call hierarchy?

I will address these questions in an upcoming article.

Note: In the case of the ServerCommunicationStatistics state, we really don’t even need to have a parameter of this type, just a return value. This is true because the functions don’t really read the state, they just update it. We can make methods return objects representing the changes that they made, and then aggregate them higher in the call stack. For example, when translating a document, we translate multiple paragraphs. We can get the changes done by each paragraph translation and aggregate that and return it to the caller.

However, this is just a special case. In general, state can be both read and written to by a function.

Conclusion:

In this article, we discussed handling state in multithreaded applications. I talked about race conditions. When two threads try to update a shared value, a race condition can happen if the updates are not atomic.

We explored the Interlocked class to atomically update state, and using the lock statement to do so.

We also explored encapsulating state changing into a StateHolder object so as to separate concerns.

Finally, I explained a way to deal with updating state by actually returning the new state value as part of the function’s return value, instead of having access to the state via a parameter.

This article was technically reviewed by Damir Arh.

This article has been editorially reviewed by Suprotim Agarwal.

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

Author

Yacoub Massad is a software developer and works mainly on Microsoft technologies. Currently, he works at Zeva International where he uses C#, .NET, and other technologies to create eDiscovery solutions. He is interested in learning and writing about software design principles that aim at creating maintainable software. You can view his blog posts at criticalsoftwareblog.com. He is also the creator of DIVEX(https://divex.dev), a dependency injection tool that allows you to compose objects and functions in C# in a way that makes your code more maintainable. Recently he started a YouTube channel about Roslyn, the .NET compiler.