Over the years, we have seen a lot of enhancements in different versions of Visual Studio. Visual Studio provides many tools like Static Code Analysis, Code Metrics, IntelliTrace, Code Profiler, Unit Testing mechanism, Code Map etc. that facilitates writing code the right way. All these tools come integrated in Visual Studio, depending on the version you are using. You can read an article authored by Subodh which talks about Code Optimization Tools in Visual Studio 2013.
Microsoft is now on the verge of releasing a newer version of Visual Studio i.e. Visual Studio 2015 which boasts of features which will make it even more simpler to create high quality code without any compromise on productivity. In this article, I will introduce you to one such feature called Smart Unit Tests in Visual Studio 2015. Visual Studio 2015 is in CTP5 at the time of this writing and some features may change in the final release.
Unit Testing and Visual Studio
Unit Testing helps in ensuring that the code is working as expected. Testing helps in finding out the quality of the software. The earlier you start testing your code, the easier it becomes to evaluate the quality of code. Unit Testing feature was available from Visual Studio 2005 as an integral part of MSTests. Over the years a lot of enhancements have been made to it - like auto generation of Unit Tests in Visual Studio 2010, PEX framework as a separate installable for generating parameters since Visual Studio 2010, support for third party unit tests framework from Visual Studio 2012 and so on.
With Visual Studio 2015, unit testing goes to the next level with the introduction of Smart Unit Tests.
Smart Unit Tests
Developers need to write unit tests for a variety of reasons. Some of these reasons are as follows:
- To maintain high quality code. Teams would like to release the software with fewer bugs. Writing unit tests reduces the bugs in new features as well as existing features
- Teams need to deliver code early. Contrary to the belief, writing unit tests makes development faster
- Writing unit tests for code makes your code more testable for non-functional testing like performance testing
- The software which is not sufficiently tested becomes more bugs prone, which leads to more time in creating error free code, thereby increasing costs. So by writing unit tests, overall cost of the software is reduced.
Eli Lopian has covered some Unit Testing Myths and Practices on DotNetCurry.com which I recommend reading.
As I have already mentioned, from Visual Studio 2012, Microsoft provided support for third party testing frameworks. PEX framework was available as a power tool. With PEX you could generate test suites with high code coverage. Visual Studio 2015 extends it and provides this feature under the name of Smart Unit Tests.
Smart Unit Tests can be helpful in finding out if the code is testable. It can also help in getting code coverage information like which code paths were visited etc. We need to decide when to fail or pass the test. This can be achieved by adding assertion in the method. We have the liberty of adding as many assertions as needed.
Smart unit test instruments the code under test. The tests are run with a simple input value (depending upon the parameters for the test). The test engine finds out how the input value flows through the code and thus computes the code coverage. There can be various logic branches in the code. If the input value does not reach some paths of the code, then another input value is picked up and then decided if the un-reached branches are getting executed now. Smart unit tests also specify if there are any input values that are not considered as a part of code (for example null value not checked).
As an illustration of how smart unit testing works, I am going to create a class library using Visual Studio 2015 in which I will add some code and add smart unit tests for the methods in the code.
I have written a method to validate the email address entered. The method takes one parameter of the type string which is validated.
Visual Studio 2010 provided the feature of creating unit tests on a method to be tested automatically. In later versions of Visual Studio, this was available as a separate feature from Visual Studio Gallery (but is only used to add references to the class library and provide a test method stub). Now we can generate smart unit tests and execute it even before adding them in a test project.
Right click on the method in Visual Studio 2015 to bring up the option for Smart Unit Tests.
A test suite is generated for the current method. The tests are executed and we get the result. It generates various input values depending upon the parameter(s) required for the method to be tested.
Different input values are created in order to find the flow of all the paths in the method. The number of test data depends upon the types and number of parameters for the method.
When you run these smart unit tests, you can find out which tests are passing and which are failing.
The option of generating smart unit tests for all the public methods in the class can be availed by selecting the class, and choosing the option of Smart Unit Tests. It will give us a test suite with all the test methods along with any global events.
Selecting All Events will show a list of events. It will also show us if there are any boundary violation. These issues can be fixed on clicking on the Fix button (see icon in the image).
Observe the Fix button provided to fix this issue. This fix will change the count for max runs to 200 as shown in following diagram. This gives us maximum number of consecutive runs without a new test being emitted.
We can save these test suites in a test project. Select all the test data and the Save button will be enabled.
Observe the test project and the method with parameter. PexMethod attribute is added to the method along with TestClass attribute for the class. In Test Explorer, you can view a list of all the tests added. As usual, build (desktop build) will provide a list of all the test methods added. It can be seen with the help of Test Explorer.
We can execute all the tests. Thus we have created test suite for regression. A file is automatically added to the test project with test data and required assertions in it. If the method code changes, you need to re-run the smart unit test
You can also find out the code coverage information about the assembly which is instrumented. Select the methods to be executed from Test Explorer, right click and choose the option for Analyse Code Coverage for Selected Tests.
Expand the assemblies shown in Code Coverage Results. We can drill down to the code and find out the paths which did not execute. The code paths which executed are shown in blue colour and the others in red colour. With code coverage, we can find out about efficiency of the code.
Generated Parameters in Smart Unit Tests
Let us try and find out how the parameters are generated for Smart Unit Test. If you refer to the original values of the parameters, you can observe that the code is not testing for null. The first value for parameter passed is given as null and an exception is thrown.
I changed the equality to null in the code as seen in the following code and now I get a little change in the input values and the results shown.
The test method generated can be seen in the following image once you select one of the test cases.
The code is run with the simplest value for the input parameter. That is the first value for the parameters generated.
This result shows that it has hit the first if statement. The next value for parameter is going to the next if statement and finally the third value gives the valid email address. This determines that not only are the random values for parameters generated, but the test engine keeps track of which paths in the code were reached and accordingly generates input values. Any exception encountered is termed as a separate branch for the code. Smart!
Conclusion
In this article, we discussed Smart Unit Tests in Visual Studio 2015. Smart Unit Tests are used for automated white box testing along with code coverage. Smart unit tests are an extension of Pex framework with a lot of enhancements. The usual time taken to release a software is reducing day by day and Smart Unit Tests is one such tool that help us in doing so.
This article has been editorially reviewed by Suprotim Agarwal.
C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.
We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).
Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.
Click here to Explore the Table of Contents or Download Sample Chapters!
Was this article worth reading? Share it with fellow developers too. Thanks!
Gouri is a Trainer and Consultant on Azure DevOps and Azure Development. She has an experience of three decades in software training and consulting. She is a graduate from Pune University and PGDCA from Pune University. Gouri is a Microsoft Most Valuable Professional (MVP) - Developer Technologies (Azure DevOps), Microsoft Certified Trainer (MCT) and a Microsoft Certified Azure DevOps Engineer Expert. She has conducted over 150 corporate trainings on various Microsoft technologies. She is a speaker with Pune User Group and has conducted sessions on Azure DevOps, SQL Server Business Intelligence and Mobile Application Development. Gouri has written more than 75 articles on Azure DevOps, TFS, SQL Server Business Intelligence and SQL Azure which are published on
www.sqlservercurry.com and
www.dotnetcurry.com. You can connect with her on
LinkedIn.