Anyone that has a vegetable garden knows the importance of harvesting the veggies at exactly the right time. If you pick a vegetable too soon, it won’t have good flavor. Pick it too late and it will be soft, mushy, and rotten. Just like the gardener, we need to harvest our software applications when they are at the height of ripeness. Too soon and the app will be full of bugs. Too late and we’ll have lots of feature creep or the business needs will have changed and the application will be full of rotting code.
How do you determine the right time to harvest or release your application? Applying Agile, you release often. In fact, every two weeks is very common. This schedule helps you know that the application is not overly ripe. But that doesn’t help you know that application is actually properly ripened and ready for release. What does help you know if the code is ready for release, is by applying Continuous Delivery techniques.
This article is published from the DotNetCurry .NET Magazine – A Free High Quality Digital Magazine for .NET professionals published once every two months. Subscribe to this eMagazine for Free and get access to hundreds of free .NET tutorials from experts
Continuous Delivery uses what is called the “deployment pipeline”. Code goes in one side and fully tested applications come out the other. The idea of the deployment pipeline is that the further to the left you find a bug, the less costly it is to fix. This is commonly called “fail fast”.
Figure 1. The Deployment Pipeline
The other idea of the Deployment Pipeline is that code progresses in an automated fashion from left to right. This means that you automate deployments and automate testing, as much as possible. In fact, you can automate almost all the testing including unit, regression, integration, capacity, acceptance, functional, system, stress, performance, and deployment. Manual testing includes things such as exploratory, usability, showcases, look and feel, and worst-case testing.
You can see that Harvesting is a large topic. I only have space to discuss one aspect, so let’s talk about the first step, Continuous Integration. It’s here that your code is integrated with code from your co-workers and then unit tested to ensure the minimal functionality is still, well, functional. Ideally, the integration happens with each check-in to version control and takes five minutes or less to complete.
Figure 2. The Continuous Integration process
Using Continuous Integration (CI), each developer builds and tests their code locally. When the functionality is complete and their tests pass, updated code that has been written by other team members is brought locally and merged. This is the integration step. Tests are then run again locally to ensure code still passes. This is important. Once you get the unit tests to pass, check in the code. Ideally, this happens several times a day. Hence, “continuous integration”. Once the code passes this step, you check in your code.
On the other side of the version control system is a server that watches for code check-ins. When one occurs, it grabs the code and compiles it. If the compilation is successful, unit tests are run against the code. This unit testing step is critical as it identifies areas where code from one developer has side effects on code from another developer. The code doesn’t integrate. If you don’t have unit tests, you don’t have Continuous Integration. You only have an automated build.
Some systems, such as TFS, use a “gated check-in”. When you check in the code, it’s put in isolation and unit tests are again run. Only if those tests pass, is the code allowed into the version control repository. In this column, I will show you how to setup Team City for your CI server. Because Team City does not have its own source repository, it cannot force code to pass tests before check-in. However, Team City does support a “personal build” that you can run before check in. Note I will not show the personal build, but it is fully documented on the Team City web site.
The outputs of the CI build and test process are typically assemblies and other files. These are called “artifacts” in CI parlance. Hopefully your tests passed so you can get these artifacts.
The final step is reporting back to the team on the outcome of the integration. This is typically done through a web site.
The CI server software is a program designed to make this process easier. There are many CI servers that you can choose from including TFS, Go, Jenkins, TeamCity, and others. I will use Team City for my examples because I believe Team City is best of breed, even surpassing TFS. Why? TFS is very process heavy and you’re pretty much tied to how Microsoft thinks things should be done. Team City allows you to pick best of breed for version control, bug tracking, project management, deployment, etc. Depending on which other tools you choose, Team City or the other tool may have built-in integration points with them. Also, if your team uses languages and tools other than .Net, Team City may work out to be a much less expensive option. The cost is free up to 20 configurations (I’ll explain that in a bit), then its $2000 US. Really a small price to pay for a great tool. You may also be thinking that Jenkins, which is always free, is cheaper. Yes, initially it is, but you have to deal with lots more XML files and do more manual configuration, making it more costly in the end.
Before getting into Team City, we need to talk about MSBuild. Every time you build in Visual Studio, an MSBuild file is used. In fact, .csproj and .vbproj files are really MSBuild files. We’ll use a special MSBuild file for Team City.
A bit of explanation is needed. Targets are the sections that do the work. Think of them as procedures. There is one target, ReleaseBuild, in this project. A PropertyGroup is basically a property bag. You can see our solution name is TeamCity. In my example, you will save this file as Build.proj in the same folder as the solution.
You can open a Visual Studio command prompt, navigate to the solution folder and type MSBuild Build.proj and see the output in the console window. Yes, MSBuild is a command line tool. It originally shipped as part of the .Net Framework but as of Visual Studio 2013 it has been broken out into its own package called Microsoft Build Tools. It gets installed with Visual Studio, but will need to be installed separately on the build server. Never install Visual Studio on your build server.
Setting up Team City
I will use the term “build server” when referring to the actual server that runs the CI software, and “CI Server” as the generic term for the CI server software. You can download Team City from the Jet Brains web site. You can connect Team City to use Active Directory or use its own internal security. You also need a database for Team City to store configuration information. It ships with a small database server, but it is recommended you use it only for evaluation purposes. I typically use SQL Server Express for production build servers.
You will also need to install the version of the .Net Framework you are using, the Microsoft Build Tools, and NuGet. These are all free so you don’t need to worry about licensing costs.
Team City uses a hierarchy of objects to manage your build. At the top is a project. You can think of this as a Visual Studio solution. You can nest one Team City project inside another if you wish, but I haven’t had a need to do this.
Figure 3. The Team City object hierarchy
Next is the Configuration. This is where the work is done. For example, everything needed to do an Integration Build is a Configuration. If you do a nightly build to run additional tests, you have another Configuration. If you have Team City deploy to your QA server, that’s another Configuration. Remember, the free version of Team City supports up to 20 Configurations.
Inside a Configuration you have several types of objects such as Build Steps (we’ll see those in a moment), Triggers that determine when the build runs, Dependencies (one Configuration can depend on another), etc. We’ll look at some of these objects as we setup an Integration build.
Creating the Team City project
Once you have Team City and the other tools installed, navigate to the Team City website on your server. In the upper right-hand corner, click “Administration” then “Create Project”. I usually give the Team City project the name of the application I’m working on and then accept the default value for the Project ID. In my example, I named the Team City project DNC even though my Visual Studio solution is named TeamCity.sln.
Figure 4. Team City project settings
Looking at the left-hand side of the TeamCity configuration, you’ll now see the Project Settings menu. This menu is context sensitive and will change based on where we are in our build configuration.
Connecting to version control
The first step is setup the connection to version control. Click on VCS Roots then Create VCS root. I’ve found it’s generally easier to select the type of VCS. You’ll need to supply the URL to the VCS source, a username, and password. Depending on your version control system, you may need to provide additional configuration data.
Figure 5.: Configuring the VCS Root
Once you have finished configuring the VCS Root, click Create. You will see a screen listing all VCS Roots for this project. The next thing is to configure the build.
Configuring the build
Click General Settings on the Project Settings menu then click Create build configuration. You will need to enter the name of the build. As this is the integration build, I selected “100 – Integration Build”. By numbering the different builds, they will be displayed in the same order that they run. So, if the next build is to deploy to QA, you would name that build “200 – Deploy to QA”. Once you’re named this build, click Create.
You need to tell this build to use the VCS Root you created earlier. Select it from the dropdown and click Attach. Team City now examines the files in version control and suggests different methods to do the build. I prefer to do this manually. Click the link that says “configure build steps manually” then on the next page, select NuGet Installer as the build runner type.
Figure 6: Restoring NuGet files
It’s necessary to restore NuGet packages because they shouldn’t be checked into version control. It will be impossible to build without these files. In Figure 6, you can see that I named this step NuGet Restore, I selected the version of NuGet to use and entered the path to TeamCity.sln, the Visual Studio solution. Note also that the left context menu has changed to Build Configuration Settings. Save this step then Add a build step. This next step will be the actual build so select MSBuild as the build runner type
Figure 7: Settings for the Compile build step
Name the step, point to Build.proj for doing the build, and select the proper MSBuild version. Then click Save. We have one more build step to add. This one will run unit tests. Select NUnit for the build runner type.
Figure 8: Configuring unit tests
As part of the unit test configuration, you need to enter the path to the unit test .dll file. This path is relative to the root directory of the Visual Studio solution file. Click Save. Team City has pre-build runners for NUnit and MSTest, but you can hook up any unit testing framework that has its own console runner.
Setting the version number
Before we go too much further, let’s step back just a bit and add some information about the build itself. Click General Settings. One of the features of Team City is the ability to number the build. Think about this for a moment. If you have five developers, each will have a different build number on their local machine. How will Team City know which version number is correct? Well, because the most recent check-in wins, it will be the more recent version of AssemblyInfo.cs. Let’s setup TeamCity to handle the build number. This is done in two steps.
Figure 9: Setting the build number and artifacts
The first step is the build number format. I’ve used 1.0.0.%build.counter%. Team City will increment the build counter with each build. So, the first build will be 18.104.22.168, the second 22.214.171.124 and so on. When you’re ready to work on version 2 of your application, just set the build number to 2.0.0.%build.number% and Reset the build counter.
Before moving to step two, we need to tell Team City to keep the generated artifacts. In the Artifacts paths box, enter ** => %system.build.number%.zip. This tells Team City to take all the files that it checked out and those it created in the build and put them in a zip file with a file name, the same as the build number.
Now Click Save.
The second step is to tell TeamCity to update AssemblyInfo.cs with the new build number. On the Build Configuration Settings menu, click Build Features then Add build feature. And select Assembly Info Patcher. Team City will automatically select the proper build number format. Click Save.
Tagging version control
There’s one more thing we need to configure on this build. It would be nice if we had Team City tag version control with the build number. This way, we always know what version of source control files went into a specific build number. Again, click Build Features and Add build feature. This time, pick VCS Labeling and then select the VCS Root for this project.
Figure 10: Tagging version control with the build number
Once you configured this feature, click Save.
Scheduling the build
There’s one last thing we need to setup. We need to schedule this build to run whenever Team City detects a change in version control. To do this, click Triggers then Add new trigger and select VCS Trigger from the drop down. Note you can also manually run the build or schedule the build to run on a specific day or time.
If you have a busy build server, Team City will queue up the builds and run them in order or you can purchase additional build agents and install them on another server (The Team City Enterprise license gives you three build runner licenses.) Using this scenario, one server hosts Team City and handles farming out the builds to other servers.
Finally, the only thing left is to run the build. There are a couple of ways to do this. You can modify a file in the Visual Studio project and check it in or you can manually trigger the build. Return to the Team City home page, expand the DNC project and click the Run button for the build configuration we just created.
Team City will go out to version control, get the source files, copy them locally, then run each build step in order. If all went well, you’ll see the build result will be green. If not, it will be red, and you’ll have to drill into the build results to determine what went wrong.
Developers can get results in three different ways. First is a system tray application that simply shows a red/green condition for all builds you are monitoring. The second is a Visual Studio plug-in. Third is by going to the Team City site of your build server and looking at the build status. If a build fails for any reason, this is where you go to get the results. You can drill-down to the details.
Figure 11: Results of the build on the Team City home page
One final note. You can also hook up additional tools like StyleCop or FxCop, but I don’t recommend these for an Integration build. Why? FxCop is a bit outdated and doesn’t support .Net 4.x very well. Additionally, Team City includes its own inspections module that does the same basic job (this is the same inspections that you can run from Resharper). The second reason for not running FxCop is the same reason I wouldn’t run StyleCop, SQLCop, etc as part of the integration build…they take time. Ideally, an integration build should be run in five minutes or less and running these additional pieces adds time to the build.
Instead, create an Inspections Build and run it at night. This is when you run inspections such as StyleCopy, TeamCity Inspections, SQLCop, or any other review tool you wish. In other words, inspections are not part of Continuous Integration and out of scope for this instance of Software Gardening. I’ll come back and address them in the future.
So, how important is Continuous Integration? Well, the Agile Manifesto says you are to deliver working software. If you are trying to deliver every two weeks, there is no way you can deliver working software without using CI. It would be near impossible to determine everything is working correctly without it.
Hopefully you understand what CI can do for you and how easy it is to setup in Team City. After all, it is only by using CI as the first step in your deployment pipeline that you can ensure your software stays lush, green, and vibrant.
About Software Gardening - Comparing software development to constructing a building says that software is solid and difficult to change. Instead, we should compare software development to gardening as a garden changes all the time. Software Gardening embraces practices and tools that help you create the best possible garden for your software, allowing it to grow and change with less effort. Learn more in the Software Gardening article series.