DevOps Timeline

Posted by: Subodh Sohoni , on 3/7/2020, in Category DevOps
Views: 10345
Abstract: DevOps concepts can be understood if we first have a clarity about Agile development and activities in its timeline. In this tutorial, I have shown how the activities and strategies of Agile engineering can be brought on the same timeline so that each team member playing any role in DevOps team knows their responsibilities.

Concept of Agile teams is a passé.

DevOps teams is the new ‘in thing’. It is simple to understand that DevOps is just extending the concepts and scope of Agile to the operations and testing functions in the organization, if they are not already present.

Many organizations do not consider to build a DevOps teams. DevOps cannot be ‘implemented’. DevOps is the way teams are designed and it becomes a part of the mindset of the team. These teams consist of representation from business, architecture, development, operations, testing and any other function that will make possible the agile promise of ‘early and frequent delivery of value to the customer’.

When I try and capture the context of DevOps in one diagram, I usually end up with the one shown in Figure 1. Although it shows all the activities in an order that is easy to understand, it does not depict the correct picture of how practically a team may execute those activities.

devops-scope

Figure 1: Scope of DevOps

DevOps Timeline – Issue at hand

When so many disciplines are going to work together, it becomes very difficult to imagine the coherence of all activities of all of those disciplines. It is even more difficult to put those on the same timeline without conflicting with the others, and at the same time, supporting others to improve the productivity of the entire team.

The Scrum concept of Sprint gives one way of putting many activities in the timebox of sprint and provides a guideline for certain timeline. These activities start from a development team starting a sprint with sprint planning, and ends with the same team doing a retrospection to inspect and adapt to improve.

These activities are fairly linear and not many are on a parallel or diverging track. May be, backlog grooming is the only parallel track that can be thought of. Another observation is that all these activities are related to managing the team, requirements and efforts involved. None of them are for agile practices that focus on development and engineering.

I can easily list these activities for you, although it is not an exhaustive list:

  • Practices related to version control like branching and merging
  • Builds with automated triggers like continuous integration
  • Code review
  • Provisioning of environments to make them ready for deployment
  • Deployment of packages to the provisioned environments
  • Testing the product after it is deployed. These may be functional tests or non-functional tests like performance testing
  • Monitoring the health of the product

How to put all these activities in the timebox of sprint and on the same timeline of earlier identified agile practices, is an issue that many teams fail to address successfully.

What it means is that teams follow agile planning and management, but cannot correlate those to continuous delivery(CD) using agile development practices. I have observed many teams finding a solution of their own for this issue. Those solutions are applicable to their situations and scenarios.

This article is an effort to showcase the best of such practices that I have observed on one timeline, if they are generically applicable.

Let us first define the context of the problem. We have a timebox of sprint, which is fixed for a team anywhere between two to four weeks. In this timebox, there are predefined activities:

  • Sprint planning
  • Daily Scrum
  • Development, which may involve architecture changes, design, coding and code quality maintenance like code analysis, unit testing and code review
  • Sprint Review
  • Retrospective

Start of the timeline with known Scrum activities

I am going to put it on a timeline of the sprint for better visualization.

scrum-events-timeline

Figure 2: Scrum events on timeline

Now, on this timeline, I would like to put the other variables that I have listed earlier. I will begin with practices of version control – branching and merging.

 

Version Control – Align branching and merging in iteration (sprint)

Feature Branches

For a team that is developing new software, I recommend using GitFlow branching strategy which is based upon features.

At the beginning of the sprint, for every feature that is to be developed, a branch is created. These branches are created from a long running branch, called Dev branch as a convention. Some feature branches are long running. They span multiple sprints and are continued from an earlier sprint. All the code that is to be deployed in the sprint will be developed in feature branches, and once developed, they will be merged in the Dev branch. This merge will be done after a code review, say by a pull request on the feature branch. It is also suggested to have synchronization of code by forward and backward merge. Frequency of such synchronization should not exceed two days to reduce the chances of large conflicts getting created.

I am not showing those synchronization merges to avoid clutter on my diagrams.

Release Branch

A short running Release branch is created from Dev branch once all the feature branches of the sprint are merged in the Dev branch. Release branch is where integration testing will be done on the code. There are no direct commits or check-ins in the Dev and Release branch. Development of long running features will continue in their own branch. After the integration testing and any other non-functional testing, the release branch is merged in another long running branch, conventionally called as Production or master branch.

Production (master) branch

The production environment receives code from this branch.

Hotfix branches

You can also take care of urgent fixes for critical bugs in the production code by creating hotfix branches from the Production branch. These get merged back in the Production branch as well as in the Dev branch and in the Release branch, if it is there at the time when the fix was created.

We can view these branches on the timeline as shown in Figure 3.

branching-strategy-with-iterations

Figure 3: Branching strategy in iteration

Version Control – Branching and merging strategy without iterative development

For a team that is maintaining a developed software by doing bug fixes and small enhancements, concept of the timebox of sprint is not applicable.

We are now in the context of continuous delivery of value to customers without the period of development of batch of requirements. Every bug or a change request has to be delivered as soon as its code is ready for delivery. In such conditions, it is necessary to have a different timeline and a branching strategy. There is no start and end of a timebox, like a sprint, and each bug or a change request will have its own branch from the Dev branch. Hotfix branches will remain as in the earlier model.

Note: Although I am considering and showing this model here, in the subsequent topics and diagrams, I have considered the timebox model only, since that is more prevalent.

branching-strategy-without-iterations

Figure 4: Branching strategy without iterations

Build Strategy on the timeline

Now, let us think about the builds. A common build strategy is to have a build pipeline defined for each of the branch.

  • Trigger for build of the feature branches, as well as for hotfix branches, should be Continuous Integration(CI). This ensures that whenever code is pushed in the branch, it will be built to ensure its integration with the code that was pushed by other team members in the same branch.
  • Triggers for build on Dev and Production branch need to be against the code being merged from other branches, which again is the trigger of Continuous Integration, but those branches will get built less often.
  • Trigger for build of Release branch needs to be manual so that it gets built only as often as required.

build-strategy

Figure 5: Build strategy on timeline of iteration

Provisioning and Deployment

Let us now move our attention to the next important agile practice and that is continuous deployment (CD). This practice is divided in two parts, Provisioning and Deployment.

Provisioning

For the CD to work, we need to ensure that the environments on which the deployment is to be done, is in a state where deployment is possible.

For example, if a web application is to be deployed, it is necessary that the server that is going to host the application has the necessary prerequisites like IIS installed and configured. If an earlier build is already installed on it, then that needs to be removed.

This preparatory work is called provisioning. We need to define environments for deployments for the purpose of development (smoke testing), testing, UAT, production etc. These environments may need to be created from scratch or may need to be refreshed every time a new set of software packages are to be deployed.

The Production environment usually will not change. Other environments may be provisioned initially once in the sprint, or can be provisioned every time a new build is to be deployed. Deployment of builds of feature branches and hotfix branches will happen on the Dev environment and that is provisioned every time the build is to be deployed, just before the deployment is triggered.

For this activity to be idempotent (resulting in same outcome every time, if executed multiple times), we need to automate it. This is done using either the ARM templates or any other technology that the platform supports. For more information about it, please go through my article at www.dotnetcurry.com/visualstudio/1344/vsts-release-management-azure-arm-template. Terraform is a popular software for this which can target multiple platforms. Writing such a template that provides a desired state of environment is also called “Infrastructure as Code”. Along with these technologies, PowerShell DSC (Desired State Configuration) is an important part of it. PowerShell DSC is also used by Azure Automation.

Builds of the Dev branch are deployed on the QA or Test environment for doing integration testing. This environment is usually provisioned initially once in the sprint. Since it has deployments of earlier features which may be required for testing the integration, it is not advisable to provision them repeatedly as part of the deployment process. We can view these on the timeline now.

Deployment

The Application is deployed on the provisioned servers. Automation of this part of the activity is possible through various tools like Release Management part of Azure Pipelines or open source tool Jenkins (www.dotnetcurry.com/devops/1477/jenkins-for-devops).

These tools may use protocols like http, https, ftp, WinRM, Bash etc. to copy the packages on target (provisioned) hosts and to execute the installation scripts. Provisioning and deployment can be executed as two steps in succession for Dev environment; whereas for test and production environments, they are not clubbed together.

provision-deployment-strategy

Figure 6: Provisioning and Deployment Strategy in an iteration

Adding Testing on the timeline

Test types and execution

Testing is an important part of DevOps. It is done at multiple stages. My observations from the processes followed by many organizations with whom I logically agree, are as follows:

  • Unit testing is done along with development as well as part of the build. This is done usually when the code is in feature branches but also continues when the code is built in any other branch.
  • Functional testing including regression testing is done on the test environment. Some system integration testing may also be done in the Test environment before code is branched into Release branch.
  • Nonfunctional testing like performance testing, penetration testing or system integration testing is done by a team which may exist outside the DevOps team. It is done on an environment outside the context of this timeline. This is usually done once the release branch is created.
  • UAT is done on the Release branch in the Test environment
  • Availability testing is done continuously after the application is deployed to production

This is not an exhaustive list of test types but a list of commonly used test strategies. One should understand here that although testing is an integral part of DevOps, it is not necessary that all the testing be done by the DevOps team only. It is observed in many organizations that there are specialized testing team as part of the QA who do complex testing like nonfunctional testing. These nonfunctional testing include performance testing, penetration testing and system integration testing. It is not physically and logistically possible to bring these super-specialists under the DevOps teams. Usually such testing is done as and when code from Dev branch is branched in to the Release branch.

Test Cases Creation – Planning the testing efforts

Testers not only have to run tests but much of their time is taken in understanding the application changes and write the test cases for new features. They do start this part of their work as soon as the sprint is started. It is recommended to write automated tests for unit tests and functional tests. The same automated tests can be run as part of build (unit tests) and release (post deployment functional tests). The same tests will also run as part of regression test suite in the subsequent sprints.

common-timeline-testing

Figure 7: Testing on a common timeline

Conclusion

There are many disciplines that contribute their efforts in DevOps.

In this article, I have shown that their activities and strategies of Agile engineering can be brought on the same timeline so that each team member playing any role in DevOps team knows what she / he is supposed to do during the iterations. This strategy is one of the many that can be evolved.

I have put my observations from many organizations I am consultant for, and taken the best of them as practices that can be followed by a team. One of the essential things is not to take software engineering tasks out of context, and timebox agile management through a framework like Scrum.

This article was technically reviewed by Gouri Sohoni.

This article has been editorially reviewed by Suprotim Agarwal.

Absolutely Awesome Book on C# and .NET

C# and .NET have been around for a very long time, but their constant growth means there’s always more to learn.

We at DotNetCurry are very excited to announce The Absolutely Awesome Book on C# and .NET. This is a 500 pages concise technical eBook available in PDF, ePub (iPad), and Mobi (Kindle).

Organized around concepts, this Book aims to provide a concise, yet solid foundation in C# and .NET, covering C# 6.0, C# 7.0 and .NET Core, with chapters on the latest .NET Core 3.0, .NET Standard and C# 8.0 (final release) too. Use these concepts to deepen your existing knowledge of C# and .NET, to have a solid grasp of the latest in C# and .NET OR to crack your next .NET Interview.

Click here to Explore the Table of Contents or Download Sample Chapters!

What Others Are Reading!
Was this article worth reading? Share it with fellow developers too. Thanks!
Share on LinkedIn
Share on Google+

Author

Subodh is a Trainer and consultant on Azure DevOps and Scrum. He has an experience of over 33 years in team management, training, consulting, sales, production, software development and deployment. He is an engineer from Pune University and has done his post-graduation from IIT, Madras. He is a Microsoft Most Valuable Professional (MVP) - Developer Technologies (Azure DevOps), Microsoft Certified Trainer (MCT), Microsoft Certified Azure DevOps Engineer Expert, Professional Scrum Developer and Professional Scrum Master (II). He has conducted more than 300 corporate trainings on Microsoft technologies in India, USA, Malaysia, Australia, New Zealand, Singapore, UAE, Philippines and Sri Lanka. He has also completed over 50 consulting assignments - some of which included entire Azure DevOps implementation for the organizations.

He has authored more than 85 tutorials on Azure DevOps, Scrum, TFS and VS ALM which are published on www.dotnetcurry.com.Subodh is a regular speaker at Microsoft events including Partner Leadership Conclave.You can connect with him on LinkedIn .


Page copy protected against web site content infringement 	by Copyscape




Feedback - Leave us some adulation, criticism and everything in between!

Categories

JOIN OUR COMMUNITY

POPULAR ARTICLES

C# .NET BOOK

C# Book for Building Concepts and Interviews

Tags

JQUERY COOKBOOK

jQuery CookBook