If you had the opportunity to download Visual Studio 12 Beta, you might have seen some of the nice improvements around Unit Testing. We will look at some of the reasons behind these changes and an overview of the new features in VS 2012 Beta.
MS-Test and Unit Testing
Over the years, there has been some criticism over the Unit Testing capabilities in MS Unit Testing suit. This includes the lack of extensibility features, sluggishness, and less TDD (Test Driven Development) friendly features.
Some developers continue to work with MS-Test as it is part of the Visual Studio Development environment. Some developers have moved to third party Unit Testing frameworks such as NUnit, xUnit.NET and MBUnit. Most of these frameworks are extensible, lightweight and provide tooling support for Test Driven Development. However up until now, these tools had lesser integration support within Visual Studio.
In the upcoming VS version, Microsoft has taken few steps towards addressing some of these issues. These are most resolved around the tooling support. As you would imagine, there is still lot of room to improve around the MS-Test framework itself.
New Features in VS 2012 Beta Unit Testing
Below are some of the features that we see in the new VS12 Beta release. Of course some of these features are subject to change, and may even have extra features before the RTM release.
Upgrading from VS2010 to VS12
If you can remember, there were few issues when upgrading Unit Test projects from VS2008 to VS2010. Unlike earlier versions, VS2010 Unit Test projects will continue to work with VS12. They both share the same assembly, which is Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll v10.0.0.0, and has same runtime v2.0.50727.
New Unit Test Explorer
I think the Unit Test Explorer is the biggest change in terms of the MS-Test tools. Here’s a preview of the VS 2010 Unit Test View, Test Result and Results:
Now compare the above with the new Visual Studio 12 Unit Test Explorer. The new Unit Test Explorer is simple, seamless, and easy to use. You feel that your Unit Tests are more connected with your development work than before. Unit Test View and the Unit Test Result windows have been removed and the Unit Test Explorer provides both tests and the results in a single window.
As you see the failing tests are listed before the passing tests, which make them attention first. The failed Unit Test and the exception also clearly displayed, which make it easier to troubleshoot issues. Previously it was an extra step to open the Result window which had the actual exception and the call stack etc.
The new Search feature allows you to search Unit Tests within the Unit Test Explorer. It also allows you to specify filter criteria such as “Result:Failed”, which filter only the failing tests. Test Groupings for an example, group by assembly name, class name etc have been missing and hopefully this will be available in the final release. You may also notice that Unit Test timings are displayed next to the Unit Test result, which instantly gives an indication on the test execution time.
What I personally like to see is that the ability to run Unit Tests from the Solution Explorer. I think third party tools such as ReSharper supports up to some extent, but it is a nice feature to have built-in within VS.
Third Party Unit Testing framework support
Previously I mentioned that the earlier versions of Visual Studio had less tooling support for third party Unit Testing frameworks. The new Unit Test Explorer can also be configured to run Unit Tests from other frameworks such as xUnit.NET, MbUnit and NUnit. All you need to do is to install the appropriate Unit Test adapter. Please see the available Unit Test adapters for Visual Studio 12.
If you have your own testing framework, you can also create your own adapter and load it via the Explorer. As long as the Test Adapters are installed, now you can run Unit Tests from 2 different Unit Testing frameworks within the same Unit Test Explorer.
Code Coverage improvements
You probably have noticed that it is not straight forward to generate code coverage with the earlier versions of MSTest. There is a bit of work to setup the environment, configuration etc. This has been changed now and you can now easily generate Code Coverage from the Unit Test Explorer window.
Here you can also import existing code coverage results, export current results, show coverage code (highlighted) etc.
You also have the option to run the code coverage for the whole solution or selected Unit Tests.
Unit Testing Async methods
.NET 4.5 has a great support for writing asynchronous code. For example, in .NET 4.5 the compiler supports two new keywords: “async” and “await” for asynchronous task based operations.
You can now include “async” keyword part of the test method, and the “await” keyword within the test method body.
If you are interested in Async and Unit Testing, here’s a good place to start.
Note: The new version of xUnit.NET (1.9) also supports asynchronous Unit Testing. You can find more information here.
Visual Studio Fakes
This is something quite interesting and very similar to what Moles Isolations framework does. Microsoft has introduced Fakes Framework, which you can stub and inject dummy implementations to the system under test (SUT). The idea is that you replace real methods with delegates.
Below is a simple example of how you would stub an interface method.
Note: StubILoggerService is automatically generated by the Fakes Framework.
For more information on creating Fakes assemblies and tests, please refer to this link.
Post Build Test Run
Many TDD developers manually run all Unit Tests pretty much after every successful build. This is because the test drives the code and every re-factor require code to be build-run tests as a loop. Basically this is about the constant refactoring on the production code and constant Unit Test execution. With the new MS-TEST suit you can enable/disable running all Unit Tests after every successful build as below.
Also not sure why, this feature is only available in Ultimate & Premium versions.
As you see, VS 2012 has taken some important steps towards improving the Unit Testing capabilities within MS-Test. This means more developers get encouragement to write good Unit Tests and follow best practices such as TDD. Hopefully there will be more improvements around the MSTest framework itself for greater extensibility.