Microsoft Fakes Framework in Visual Studio 2012

Posted by: Raj Aththanayake , on 1/1/2014, in Category Visual Studio
Views: 50695
Abstract: The article introduces the Microsoft Fakes framework in Visual Studio 2012 Ultimate and gives a quick run-down on how to use Fakes, Shims, Stubs and Observers to build a robust test harness for your .NET applications

In Unit Testing/Test Driven Development (TDD) Isolating dependencies, sometimes referred to as “mocking objects” or “faking objects” is a norm. There are many books/articles that have been written on this topic. There are also great frameworks out there, which allow us to isolate dependencies without having to use any hand written mocks/fakes. Visual Studio 2012 new Fakes framework takes faking/mocking objects to a new level. In this article, I will explain the features of Visual Studio 2012 Fakes framework. Also please remember that, currently the Fakes framework is only available in Visual Studio Ultimate edition.

 

Some history behind the Fakes Framework

If you have used Microsoft Moles before, you will find the Fakes framework has a very similar usage workflow. Moles were originally designed to support the development of Microsoft Pex , which is another great Unit Testing tool by Microsoft Research. Microsoft Research did not use other isolation frameworks because they wanted something powerful that could mock pretty much anything, was simple and had absolutely no overhead of using it. As you would imagine, most isolation frameworks use dynamic proxies to generated mock objects, which have some overhead of using it. Moles provided a simple type safe detour framework with very clear semantics. In a nutshell, Moles replaces any .NET method with a delegate. In VS 2012, with some improvements, Moles framework became the Microsoft Fakes framework, made available in Visual Studio 2012 Ultimate Edition.

Hard to test code

As you probably know, static methods by nature, are harder to test. A static method itself is easy to test, but the code that is calling the static methods become harder to test because it is tied into the type of the class and thus cannot be easily replaced by something else.

For example, let’s say that we want to test a piece of code that throws an exception when the current date reaches 21st of December 2012. (Reminds me of the movie “2012 ” but it is not doomsday . If you are reading this article, this date has passed already and we are all good :) ).

public static void DoomsDay()
{
if (DateTime.Now == new DateTime(2012, 12, 21))
  throw new Exception("Boom!");
}

Let’s say I want to test this method. I want the exception to be thrown when the current date equals to 21st of December 2012. Assuming today’s date is not 21st of December, the test would not throw an exception. So how can we make the test to fail with the “Boom!” exception? We need to get hold of DateTime.Now property. Previously I mentioned that “replacing any .NET method with a delegate”. For example, I should be able to easily replace calls to the DateTime.Now with a delegate as below.

DateTime.Now = ()=> new DateTime(2012, 12, 21);

This code cannot be compiled as in .NET DateTime.Now property cannot be assigned to a delegate. However using the new VS Fakes framework, you can replace this property with a delegate. I will revisit this example a bit later, but now let’s look at how to enable VS Fakes within your project.

Enabling Visual Studio Fakes

Enabling Fakes is about creating a fake assembly that allows you to use (fake) types that replace .NET types with delegates. In this case it is the System.dll, which wraps the mscorlib.dll that contains the DateTime object.

enabling-fakes-framework

Figure-1

Once you “Add Fakes Assembly”, there are some new files/assemblies that get added to the test project as seen in Figure 2.

fakes-test-project

Figure-2

Note that mscorlib.4.0.0.0.Fakes assembly contains a special type called Shims. Shims are strongly typed wrapper that allows you to replace .NET types with delegates.

Using the Shim for DateTime class

We just generated the Fakes library that contain Shims and now we can look at replacing the DateTime.Now with a delegate. In here, instead of using the real DateTime.Now property, you can use the Shim DateTime as below.

[TestMethod]
public void TestMethod1()
{
ShimDateTime.NowGet = ()=> new DateTime(2012, 12, 21);
ClassA.DoomsDay();
}

Notice that there is a naming convention. You need to prepend the word “Shim” to the real .NET type and call the appropriate property that uses the delegate.

Now if I run the test, my test fails! But if you look at the exception, it failed for a different reason. The complete exception is below.

Test method FakesDemo.UnitTests.UnitTest1.TestMethod1 threw exception: Microsoft.QualityTools.Testing.Fakes.Shims.ShimInvalidOperationException: A ShimsContext must be in scope in order to register shims. Use the snippet below to fix your code.-- C#:using Microsoft.QualityTools.Testing.Fakes;using(ShimsContext.Create()){ // your test code using Shims here}--

The Shims require running their own context. This guarantees the level of isolation that is required by Shims, have no impact on other domains whatsoever. This means that whenever you use a Shim, you need to create a context that Shim persists and destroy the Shim when the context finishes execution. You can create the ShimContext as below.

[TestMethod]
public void TestMethod1()
{
using(ShimsContext.Create())
{
  ShimDateTime.NowGet = () => new DateTime(2012, 12, 21);
   ClassA.DoomsDay();   
}
}

 

test-method-exception

Figure-3

As you see in the screenshot, we hit the “Boom” exception. The ShimDateTime.NowGet replaces the real DateTime.Now value with our own DateTime value, which we have provided via the delegate.

Before we see more examples, let’s see what’s really happening under the hood. We set a breakpoint on the delegate new DateTime(2012, 12, 21); and re-run the test. Once the breakpoint hits you should see the stack trace as below.

FakesDemo.UnitTests.dll!FakesDemo.UnitTests.UnitTest1.TestMethod1.AnonymousMethod__0() Line 16

mscorlib.dll!System.DateTime.Now.get() + 0xa9 bytes

FakesDemo.dll!FakesDemo.ClassA.DoomsDay() Line 13 + 0x8 bytes

FakesDemo.UnitTests.dll!FakesDemo.UnitTests.UnitTest1.TestMethod1() Line 17 + 0x6 bytes

[Native to Managed Transition]

….

Once the code executes the DateTime.Now property in mscorlib.dll, it has taken detour to an anonymous method specified by the delegate. Using .NET reflector, if we take a close look at the implementation of ShimDateTime.NowGet, you can see the ShimRunTime replaces the DateTime.Now-> Get property with the delegate, which we have set in the test method.

public static FakesDelegates.Func NowGet
{
  [ShimMethod("get_Now", 24)] set
  {
    ShimRuntime.SetShimPublicStatic((Delegate) value,
     typeof (DateTime), get_Now",
     typeof (DateTime), new Type[0]);
  }
}

So far we have discussed an example of a Shim. Shims allow you to replace methods that are Static, Non-override-able, or even private methods (I called them all unfriendly) with delegates. It re-writes the code at runtime. With Shims, you can even fake the deletion of your C:\ drive without actually deleting it. You can also generate code and inject them into the system. For example, using ShimDirectory.BehaveAsNotImplemented() method you can inject the code to throw NotImplementedException

//Warning - Do not attempt this outside the test context
public static void DeleteC()
{
Directory.Delete(@"C:\TEMP", true);
}

[TestMethod]
public void TestMethod2()
{
using (ShimsContext.Create())
{
  ShimDirectory.BehaveAsNotImplemented();
   ClassB.DeleteC();
}
}

test-method-failed

Figure-4

Stubs

In Unit Testing, stubs provide canned answers to the System Under Test (SUT). As Martin Fowler puts it:

Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what's programmed in for the test. Stubs may also record information about calls, such as an email gateway stub that remembers the messages it 'sent', or maybe only how many messages it 'sent'.

The Fakes framework also has the ability to generate stubs; these have more to do with virtual types. For example, Stubs can be used to fake virtual types such as interfaces, abstract classes, virtual methods etc. Stubs cannot be used in non-virtual context such as private, static or sealed types.

Let’s look at an example of a Stub.

System Under Test (SUT):
public interface ILogger{
bool ShouldLog();
void Log(string message);
}

public class ClassA
{
  public string GetName(ILogger logger)
  {
   if (logger.ShouldLog())
   {
    logger.Log("log something");
    return "some name";
   }
   return string.Empty;
  }
}

Test method:

In order to use stubs in our test, we need to add a Fakes assembly for the library that contains the virtual type/interface. In our previous Shims example, we add the Fakes assembly for the System.dll. In this case, it is the FakesDemo.dll. You only have to do this once, and then every time when you build the application the Fakes assembly get auto generated.

 

add-fakes-assembly

Figure-5

Once we add a Fakes assembly, FakesDemo.Fakes.dll gets generated. This fakes assembly contains the Stub types that we can use to replace the ILogger implementation. Please see below.

[TestMethod]
public void TestMethod1()
{
//Arrange
var stubLogger = new StubILogger
{
  ShouldLog = () => true
};
var sut = new ClassA();
//Act
var result = sut.GetName(stubLogger);
//Assert
Assert.IsTrue(!string.IsNullOrEmpty(result));
}

StubILogger is a type that generated for you by the Fakes Framework. ShouldLog is a property delegate (Func) within the StubILogger.

If you are interested in how the FakesDemo.Fakes.dll and the stub types get generated in the first place, it is done by the Microsoft.QualityTools.Testing.Fakes assembly. This assembly generates a Fakes project (i.e f.csproj) and the associated C# classes (i.e f.cs) and compiles them into a dll (i.e FakesDemo.Fakes.dll). The source code for the auto generated class and the other artefact can be found in the test project’s obj/Debug/Fakes/...

Below is a trimmed down version of the auto generated class StubILogger

namespace FakesDemo.Fakes
{
public partial class StubILogger
  {
  ///

Sets the stub of ILogger.ShouldLog()
  public mqttf::Microsoft.QualityTools.Testing.Fakes.FakesDelegates.Func ShouldLog;
  /// Sets the stub of ILogger.ShouldLog()
  bool fd::FakesDemo.ILogger.ShouldLog()
  {
   mqttf::Microsoft.QualityTools.Testing.Fakes.Stubs.IStubObserver ___observer
    = ((mqttf::Microsoft.QualityTools.Testing.Fakes.Stubs.IStubObservable)this).InstanceObserver;
   if ((object)___observer != (object)null)
   {
    mqttf::Microsoft.QualityTools.Testing.Fakes.FakesDelegates.Func ___currentMethod = ((fd::FakesDemo.ILogger)this).ShouldLog;
    ___observer.Enter(typeof(fd::FakesDemo.ILogger), (global::System.Delegate)___currentMethod);
   }
   mqttf::Microsoft.QualityTools.Testing.Fakes.FakesDelegates.Func ___sh = this.ShouldLog;
   if ((object)___sh != (object)null)
    return ___sh.Invoke();
   else
   {
    mqttf::Microsoft.QualityTools.Testing.Fakes.Stubs.IStubBehavior ___behavior
                 = ((mqttf::Microsoft.QualityTools.Testing.Fakes.Stubs.IStub)this).InstanceBehavior;
    return ___behavior.Result(this, "FakesDemo.ILogger.ShouldLog");

   }
}

The highlighted text shows 2 important parts

· We have a public delegate/Func field ShouldLog, which can be used to replace with our own delegate.

· private FakesDemo.ILogger.ShouldLog() method, which has the implementation of the invocation of the ShouldLog delegate.

mqttf::Microsoft.QualityTools.Testing.Fakes.FakesDelegates.Func ___sh = this.ShouldLog;
if ((object)___sh != (object)null)
return ___sh.Invoke();

As you see above, once the ShouldLog delegate is not null, it invokes the delegate. Previously explained Shims does the same thing, but it all happens dynamically at run time.

Observers

Observers allow you to record the calls on stubs. Fakes framework ships its own observer, which allows you to record every call, every argument made in the stub. This is similar to what you have seen in standard isolation frameworks, where you would verify (i.e mock.Verify(x => x.SomeMethod()) the interaction of a method call.

[TestMethod]
public void TestMethod1()
{
//Arrange
var stubLogger = new StubILogger {ShouldLog = () => true};
var sut = new ClassA();
var observer = new StubObserver();
stubLogger.InstanceObserver = observer;
//Act
sut.GetName(stubLogger);
var calls = observer.GetCalls();
}

If you see the debug information of the “calls” variable, you can see the following.

debug-info

Figure-6

There are two calls have been made – ShouldLog() and Log(message). Each call has arguments (if any), stub method, stub MethodInfo, and stub method type etc.

This is valuable information if you want to verify a certain method has been called with certain parameters. Some mock frameworks allow us to verify the number of occurrence of each method call, but I could not see a similar feature in the built-in StubObserver. Also important to note that you can write your own Observer that allows you to extend additional features.

Using Shims vs. Stubs and recommendations

As I mentioned before, Shims are used mainly to test the untestable code. You find more of these code third party components that contains lots of statics, privates etc. Often this implies there is some code smell. It is recommended to use Stubs, because the virtual types are extendable, replaceable and easily testable. The other aspect is that if you see the code that is hard to test, and you have no confidence of changing code, you can use Shims to write integration type tests and refactor the code incrementally to make the code testable.

Summary

We looked at some of the features of the new Microsoft Fakes framework. The key take away is that you can replace any .NET method with a delegate using the Fakes framework. The statically typed detour framework provides some powerful features, which allow you to stub hard-to-test methods such as statics and privates.

With Shims, the detour/delegation occurs dynamically at run time. ShimContext ensures that there is no side effect by creating a new app domain for the entire test context which detour occurs.

Stubs are lightweight wrappers which allow you to stub virtual types. Fakes Framework tools auto generate necessary code and compiled into fakes dll, which you can use the stub types without having to hand craft them.

Observers are a great way to record stub interactions in the SUT (System Under Tests) and verify them within the test.

Entire Source Code for the Sample Application can be downloaded from http://bit.ly/dncm-04-vsfgh

What Others Are Reading!
Was this article worth reading? Share it with fellow developers too. Thanks!
Share on LinkedIn
Share on Google+

Author
Raj Aththanayake is a Microsoft ASP.NET Web Developer specializing in Agile Development practices such Test Driven Development (TDD) and Unit Testing. He is also passionate in technologies such as ASP.NET MVC. He regularly presents at community user groups and conferences. Raj also writes articles in his blog http://blog.rajsoftware.com. You can follow Raj on twitter @raj_kba


Page copy protected against web site content infringement 	by Copyscape




Feedback - Leave us some adulation, criticism and everything in between!
Comment posted by praveen on Friday, January 3, 2014 12:10 AM
Thank you for given information about .Net Fakes Frameworks.....
Comment posted by Glauco on Saturday, January 4, 2014 5:33 AM
very interesting article!!!
And what about mocks?
Comment posted by Phillipe on Tuesday, January 7, 2014 7:07 PM
Check this for mocking http://www.dotnetcurry.com/showarticle.aspx?ID=767
Comment posted by Amit on Friday, September 26, 2014 4:29 AM
Very Useful Info. I have a question regarding check in the code to  build server (TFS) . When add a new fake assembly , it creates a reference in *cs proj file of test project so at the time of check should we check in both cs proj and fake dll or only cs proj. My overall intention over this question is to know will it give build errors after check in on some other persom machine if/she takes latest and build the test project.

Categories

JOIN OUR COMMUNITY

POPULAR ARTICLES

FREE .NET MAGAZINES

Free DNC .NET Magazine

Tags

JQUERY COOKBOOK

jQuery CookBook