A majority of IT professionals in the software industry have misconceived ideas about DevOps.
DevOps is considered as something that can be ‘implemented’ by following ‘some practices’ by any team.
Some think of a DevOps engineer as a designation in the organization. Others feel that it is an invasion of the development team on the operations team. Another school of thought is that DevOps is an automation of activities carried out by the operations team.
Are you keeping up with new developer technologies? Advance your IT career with our Free Developer magazines covering Agile Development, Patterns, C#, .NET Core, MVC, Azure, Angular, React, and more. Subscribe to the DotNetCurry (DNC) Magazine for FREE and download all previous, current and upcoming editions.
And there are many more notions similar to these. Alas, none of them grasp the essence of DevOps!
In this article, I will try to provide some clarity about the concept of DevOps.
Also Read: Microsoft's DevOps Story
What is DevOps?
To understand the concept of DevOps, we first need to have a clarity about Agile development.
Editorial Note: If you are new to Agile Development, make sure you read these tuts to understand its benefits.
Agile development seeks to provide early and rapid (frequent) delivery of software that is under making.
This has two benefits.
The Customer can start using at least a part of the software, in a short time. This provides a sense of satisfaction to the customer that there is some return on the investment (ROI).
It also provides an opportunity, for the customer and the team that is developing the software, to review the features of the software that are ready for use. These frequent reviews and related feedbacks make it possible to take corrective actions, incorporate changes in requirements and fix observed bugs, at the earliest and at the minimum cost.
By using agile development, most of the products and projects that develop custom software, stand a bigger chance to succeed.
Agile development is not a concept that can be implemented by any team. The team needs to become agile to do Agile development. Just like a fighter aircraft which is designed to be agile versus a passenger aircraft not designed to be agile; the team that is doing Agile development, needs to be designed to be agile.
The benefits of Agile development are accrued to the customer only when the software is developed, tested thoroughly and then deployed to production.
One of the issues faced by some of the agile teams is that the development process is made quite agile, but the process of build and deployment to environments for testing and production, are not designed to be agile.
The team that is developing software needs to embrace Agile practices for these operations as well.
Some of the impediments to Agile Development are:
- There is a team that is doing development. Let us call it the ‘Development Team’. The team that is doing build and deployment is a different team. Let us call it the ‘Operations Team’. The two teams may not be located at the same place. They may be insulated from each other.
- Development Team and Operations Team communicate only in a formal and indirect channel. Formal emails and approvals may be necessary to run a build or do deployment in certain environments.
- These teams are not in sync with each other regarding their goals, schedules and planned activities. The Operations team may have their schedule of build and deployment, which may not be in line with the Agile development practices. This can increase the time for delivery.
- The build and deployment activities may not be sufficiently automated to give the fastest results possible. Manual processes are slow to implement and are also error prone.
- Testing team may have their own schedule for cycles of testing. Those may not be in sync with the development and release cycles.
- Tools used by Development team, testing team and operations team may not integrate very well.
- Development team desires to use the latest technology and do the development in the shortest possible time. Testing team considers that maintaining quality is only their job and may not consider the rapid delivery to be of prime importance. Operations team always resists any changes because, sometimes, those may lead to crashes and service stoppages. These mindsets pull the teams in different directions.
Do these inhibitions sound familiar to you?
This list may not be comprehensive, but is thought provoking to the extent that you start wondering why the benefits of Agile development should not be passed on to the customer and the end user, as early and rapidly, as the development phase.
That is why we extend the agile team to become a DevOps team.
The DevOps team is a team that has representation from developers, testers and operations.
This team has a common goal - to provide early, rapid and high-quality product to the customer. Their mindset is aligned, they do not consider team members with external roles and use well integrated tools to achieve the common goal.
Let us have a look at each of these factors.
Image 1 – Four pillars of DevOps
DevOps - Mindset Changes
DevOps team is an integrated team comprising of members of Development, Testing and Operations. All the team members have same schedule and work in-sync to provide earliest and rapid delivery to the customers.
Most importantly, all the team members have the same mindset.
The success criteria for the team consists of early and most frequent delivery of usable software by the customer.
This criterion makes productivity, quality and rapid delivery as the goals for every team member. To adopt the mindset of DevOps, the team members have to break their earlier mindsets.
For eg: Development team members have to now also consider quality and operations as responsibilities owned by them. Testers should now work in line with the schedules of development and release promises.
Similarly, team members who were part of the Operations team have to now find different innovative ways to provide delivery of the software as rapidly as the software gets developed, without losing the reliability of the deployed software.
The biggest impediment to forming a DevOps team is the resistance to change the original mindsets of each role and create a new aligned mindset for the entire team. It is a misconception that every team member in the team has to be skilled in development, testing and operations. What is desired is that the team should have sufficient team members from all these streams and that they have common mindset, so that as a team, they can provide the most rapid increments of the product to the customer.
Communication and Collaboration
Like any other modern development team, the DevOps teams also depends heavily upon seamless and clear communication.
It is desirable to have co-located teams so that such communication is not dependent upon the vagaries of Internet and other devices. The team should be able to have a formal and informal channel of communication by which the synchronization of tasks between team members doing development, testing and operations, will be achieved.
Direct communication and collaboration is recommended to have unambiguous and continuous communication. Collaboration tools can be used as desired but should not be the impediments in communication.
Teams should be able to describe, detail out and decide on the schedules for the creation of environments, needs of provisioning, testing cycles, branching and merging as per the decided strategy etc. This list is not comprehensive but will give you some idea about how collaboration can improve the time - from development to deployment.
Interoperation of DevOps and Agile
Image 2 – Relationship between Agile – DevOps – ALM
DevOps shares most of the values with Agile. It does extend some of these values to suite the unique needs of the DevOps teams, and defers to a certain extent from those values. Let us go through some of them.
Individuals and Interactions Over Process and Tools – DevOps gives equal importance to Process and Tools. The team has to achieve not only development but also rapid deployment. Continuous Integration has to be extended to Continuous Deployment and if possible, Continuous Testing. This extension cannot be easily achieved without appropriate processes and tools.
Working Software Over Comprehensive Documentation – DevOps is in line with this value.
Customer Collaboration Over Contract Negotiation – DevOps extends this value by providing a set of concrete opportunities and tools for the customer to collaborate while creating the product backlog, as well as post deployment.
DevOps covers the post deployment feedback as well, as issues raised by the customer. In this regard, it goes towards ITSM.
Responding to Change Over Following a Plan – DevOps allows team to not only develop the code, but also build and deploy it in line with the business needs and feedback received.
We can consider DevOps as extending Agile to cover Operations also.
Naturally, DevOps team will also have to follow the practices of Agile development and agility in requirements management, as well as efforts management.
Agile development practices like continuous integration, test driven development, code reviews, pair programming are extended by operations practices like continuous deployment, automated provisioning, collecting application health data, feedback from client etc. Sprint planning now should consider plans of testing, provisioning and deployment too. The tasks for those activities should be included in the sprint plan and monitored through the task board and burndown chart.
Editorial Note: If you are new to concepts like CI, TDD, Pair programming and such, read www.dotnetcurry.com/visualstudio/1338/agile-development-best-practices-using-visual-studio.
DevOps does not need automation just for the sake of it. What it essentially needs is the rapid delivery to the customer.
Besides actual coding, it is a necessity to rapidly and frequently create environments, provision those with necessary software, deploy software and test the deployed software.
This may be needed to be done many times in a day in case of initial environments like Dev and Test environments. This frequency can only be achieved if the DevOps team adopts automation to execute these activities.
Manual execution of the same may not be as quick, as required. Another advantage of automation that has been defined properly, is repeatability without any errors.
Let us define the activities which need to be automated.
1. Build should be triggered as and when the code changes in the repository due to Commit – Push or Check-in
2. Activities of the build should be executed in a specified order
3. Build should run unit tests as configured
4. A release should be created that should link the created build output with the deployment scripts
5. Environment should be created and provisioned, if necessary
6. Build output should be deployed to Dev environment where smoke tests can be run.
7. Automated tests should be run as the final bit of the deployment
8. Necessary emails for notification and authorization requests should be sent
9. Deployment to subsequent environment should happen automatically, albeit after manual approval for those.
Image 3 – Automation in DevOps
These automation tasks are possible only with some appropriate tools.
Let us have a look at those tools.
Tools for Automation
Tools that automate various activities and facilitate continuous delivery is also an essential part of DevOps.
There are a number of tools for different activities. Many a times, these tools form a toolchain for activities in a serial order.
The following table summarizes the most popular tools.
Many of these tools are industry leaders in their own field of work.
For example, Jenkins for build automation, Selenium for automation of testing and Octopus for deployment, are industry leading softwares.
Although these are excellent tools in their field, they do not integrate with other tools seamlessly. It takes a lot of efforts and pains to integrate those with each other.
For example, to integrate Jira with Jenkins for build, and Octopus for deployment, may require efforts from the development team. To reduce the pain of such integration, many organizations desire to have integrated set of tools that are integrated by design.
Microsoft TFS, Microsoft VSTS, Rational Team Concert (RTC), HP ALM are such suites of tools that are integrated by design. These tools provide almost all the tool chain under one roof. Needless to say, these are much more convenient to enterprises.
Many of these tools are offered as installable on the hardware that is owned by the organization using it. Microsoft Visual Studio Team Services (VSTS) is an Azure cloud offering which works as Software as a Service (SaaS). It is installed, maintained and updated frequently by Microsoft in a multi-tenant model.
Since the organizations do not have the expenses of hardware, maintenance of software and basic administration, this offering works out to be quite attractive.
Dos and Don’ts for a DevOps Team
Here are some pitfalls for the DevOps team to avoid and some best practices:
• Focus only on one of the aspects of DevOps – Tools, Automation or Culture. DevOps has these equally important facets. Looking at only one of them and ignoring others will not give the team the desired benefits.
• Make operations team common to all development teams – I have seen many companies keeping the same operations team, as was earlier. They then try to improve communication between development team and operations team. This may not give the desired level of collaboration because of the adherence to old mindsets and not having shared goals.
• Choosing speed over quality – Early and Rapid delivery is a goal that DevOps team should desire to achieve, but not at the cost of quality. The customer cannot get any value from the deployed product if that product has bugs in it. Although no one can say with 100% surety that a software has no bugs, we should ensure that at the least there are no known bugs at the time of deployment. DevOps team should take care of that by testing in multiple stages.
• Ignoring Database – DataOps – Coding is not the only way to create deliverables. Design, implementation and versioning of database is equally important for the success of an application. DevOps team should provide equal importance for the activities related to database development.
• Forgetting Security – In the hurry to deliver software early, DevOps team should not take shortcuts in ensuring the security of the application. Any vulnerabilities and loopholes in the security can result in disasters in the long run.
• Not involving all concerned persons and stakeholders early on – I have observed many teams involving the testing and operations team members at a very later stage in the development cycle. These team members may not be able to plan, complete their activities in time or may not be able to execute those activities, with sufficient quality. It is my strong suggestion to involve all the team members and stakeholders, early-on in the development process.
• Creating a separate special “DevOps Team” – The entire team that is delivering software is a DevOps team and every team member of that team is equally accountable for development, build, deployment and testing of that software. It is not that a separate team is to be formed for these activities. If an organization has a post of DevOps Engineer or has a separate DevOps Team, then that organization has not really unraveled the mystery of DevOps.
• Not understanding that Scrum ≠ Agile ≠ DevOps – Many organizations feel that if they do a daily standup meeting, they have adopted Scrum and that means they have become agile. A team can become agile only if it has agile values, it follows agile principles and practices agile development and agile management. Scrum is only one part of becoming agile, that is of agile management. Extending this reasoning, being agile does not mean that the team is a DevOps team. When the agile team is extended for operations and testing, only then it becomes a DevOps team.
• Failing to look at the larger picture of ALM – Some organizations and many software professionals believe that developing software is an end in itself. I need to remind them that software is only one of the ways that businesses can achieve higher productivity, quality and in general profitability.
Software supports business and not vis-à-vis. DevOps team should not forget that besides DevOps practices, there are other practices of ALM like requirement management, efforts management, risk management etc. which are equally important for the business. Organizations should look at this larger picture of ALM, should know why they are developing software and remain a part of this frame.
In this article, I have tried to demystify the term DevOps. As observed in this tutorial, the Four pillars of DevOps are:
Adoption of mindset of a team that is well integrated with members from development, testing and operations, equally accountable and passionate about all activities towards the goal of early and rapid delivery.
Seamless Communication and Collaboration between all team members.
Automation to reduce the time to deliver software with high frequency and
Tools that facilitate such automation.
When all these four pillars are of equal strength, only then the promise of early and rapid delivery can be fulfilled by the DevOps team!
This article was technically reviewed by Suprotim Agarwal.