If there is one thing that’s needed for a garden, it’s the right soil. If the dirt has too many rocks or too much clay, it’s not good for growing much. (Unless of course you’re doing hydroponics.) So we begin our discussion of Software Gardening by discussing good soil, which in our case isn’t a software development technique at all, but is dependent on project management.
In the last issue Software Gardening: Software is Not a Building I explained that software development, contrary to how many people portray it, is not like constructing a building. Let’s continue down that discussion a bit more. When buildings are constructed, an architect or designer meets with the customer to discuss what they need and how they want it to look. They take notes of the meeting, then disappear and start work on blueprints, the plans for the building. Sometimes the land is already purchased so those blue prints include site plans. These are details of how the building will fit into the existing landscape. Other times, the land is purchased afterwards and then the site plans are added. Drawing blueprints could take weeks or months depending on the size of the building.
This article is published from the DotNetCurry .NET Magazine – A Free High Quality Digital Magazine for .NET professionals published once every two months. Subscribe to this eMagazine for Free and get access to hundreds of free .NET tutorials from experts
Once this the blueprints are ready and approved by the customer, the project is put out for bid. A General Contractor runs the project and has several sub-contractors, plumbers, electricians, painters, carpet layers, etc. All get a copy of the blueprints and submit their cost and time estimates to the General Contractor, who rolls everything up and submits one overall cost and time estimate.
You might think that once the bid has been awarded that construction will start shortly afterwards. Sometimes it does, but government permits must be secured and the bigger the project, the longer that takes. Actual construction may begin with site work, which prepares the land. You may have to dig a hole for a foundation or in the case of an office tower, underground parking. Concrete is poured. For a small project, subfloors built and walls are framed, the base for the roof is built. On a large office tower, steel girders are erected and eventually each floor starts to take shape. Eventually the electrical, plumbing, and other subcontractors do their job. Along the way, government inspectors may check things over to make sure the work meets legal building codes.
During all this, the General Contractor is managing the project. Each step must be complete before the next begins. For example, the electrician and plumber can’t come in before the subfloor is done and the walls studded out. The painter has to wait for sheetrock. The carpet layer has to wait for the painter, etc.
In the end, construction projects are generally managed using waterfall project management. It works for the construction industry, but is death to a software project. We can’t wait for one step in the process to be completed before moving on to the next.
This graphic shows the typical Waterfall process when applied to software. Each step must be completed before you can fall down to the next step. Notice the circular arrow in the graph. Typically when using Waterfall, a programmer compiles the code then throws it over a wall for someone to manually test it. When the code fails (and it often does), it is thrown back to the programmer. This circular process continues for some time. I have seen this happen so often that it seems like an end-less loop of back-and-forth between the programmer and the QA person.
Here in the US, we’re transitioning to new government mandated healthcare, commonly called Obamacare. Recently, the government run website, healthcare.gov went online and was a disaster. The site couldn’t handle the load put on it and frequently crashed. It couldn’t communicate with other government sites. Not all insurance information was available to the user. It didn’t have a good user experience at all. It was a disaster. This was a project that took years and hundreds of millions of dollars. The US Congress is holding hearings into the problem, trying to pin the blame on one individual. But it’s not the fault of any one person. It’s the fault of the system that used waterfall methodologies to manage the project.
By now, you know where I’m going with this. The answer is to use Agile methodologies to manage the project. If Waterfall is bad soil, Agile is good soil. With Agile, you do design, planning, coding, testing, and yes, even deployment, as one process. You don’t complete analysis before design. You don’t complete design before coding. You don’t complete coding before testing. And you don’t complete testing before deployment. Everything works together.
Agile has been around a long time, but as an industry, we pinpoint its birth to the signing and publication of the Agile Manifesto. The Agile Manifesto was written in February, 2001 when people that had competing ideas of how to deal with Waterfall came together at Snowbird, Utah to ski and talk about common ideas. In the end, four main concepts were common with each competing idea.
- Individuals and Interactions over processes and tools - This says that people are more important over how we develop and what tools we use to develop the software.
- Working software over comprehensive documentation - Don’t spend all your time writing documentation that becomes outdated before you need it. Deliver real, working software, and do it often.
- Customer collaboration over contract negotiation - Don’t waste time trying to define every little thing in the contract. Keep the customer informed on what you’re doing. This is generally done daily in a stand-up. Then at the most, every two weeks in a sprint review, where you show off the work you did in the two weeks before and actually deliver it to the user so they can test it.
- Responding to change over following a plan - Business needs change. Prepare for this change. Embrace it. Accept it.
There are many Agile methodologies you can choose from. Scrum seems to be the most widely accepted methodology. No matter which methodology you choose, there are some common things that run through most Agile methodologies.
First, Agile teams are self-organizing. This means that they take it upon themselves to figure out who will do which tasks. There is often no difference between developer, QA, and other workers. They are all team members and are working to the same goal: release at the end of the Sprint. Typically, work is done in a series of sprints, generally one or two weeks long. At the beginning of the sprint, the business owner of the project presents the team with a list of tasks to complete during the sprint. Each team member picks a task to work on. The ones not picked are returned to the backlog. If someone finishes their task, they pick another task. Tasks are small units of work that make up user stories. User stories describe how a user sees a particular function.
Each morning, the team meets in a stand-up or scrum that shouldn’t last longer than about 15 minutes. Each person working on tasks answers three questions. 1) What did you do yesterday? 2) What will you do today? 3) Is there anything stopping you from getting stuff done today? If you’re using Scrum, there is a Scrum Master on the team. His job is to remove things blocking work from getting done. He is not a project manager in the traditional sense. He doesn’t manage the project. That’s done by the team with input from the business owner.
There is no throwing the code over the wall. An important part of Agile is good Continuous Delivery techniques where software is automatically compiled and tested. Integration tests should also be automated as should deployments (you do test deployments, don’t you?). I’ll have much more to say about Continuous Deployment in future columns.
At the end of the sprint, there is a Retrospective. In this meeting, the new working features are demonstrated to the Business Owner. A discussion about what right and wrong in the Sprint is also an important part of the meeting. Also, the working software is released, possibly to production.
Now that you’ve had a very high-level introduction to Agile, you may be wondering if it will really work. Let’s circle back to the healthcare.gov web site. After the failed release, three software engineers in California took it upon themselves to do something right. They looked at some of the healthcare.gov code to see how to do access the database, integrate with other sites, etc and over a period of a couple of weeks, working nights and weekends, came up with thehealthcaresherpa.com. While is doesn’t have all the functionality of healthcare.gov, but it has much of it and it has a better user experience.
But what about taking things the other way? Can we apply Agile to construction? Turns out, we can. Prior to the 2002 Winter Olympics in Salt Lake City, the primary north-south freeway, Interstate 15 (I-15), through the entire Salt Lake valley was rebuilt. Every bridge, overpass, on-ramp, off-ramp, everything about this 17 mile stretch of freeway was rebuilt. In some cases, the interchange to other freeways needed to be redesigned too. Using standard, waterfall methodologies, the project would have taken over ten years. They had to do it in about four years. The people working on the project designed a small part of the new freeway and started construction on it while the next section was being designed. They called this “Design-Build”. We call it Agile. And it worked. The project came in both under budget and under schedule.
So, you want to bring Agile to your team. How do you do it? First, you need some buy-in from management. This is needed because when the big boss says, “How far along are you”, he generally wants to hear “we’re 60% done”. He won’t hear that. He’ll hear, “We’ve completed 60% of our story points”. That’s doesn’t mean you’re 60% done. But keep in mind that the team is self-organizing.
You need to pick an Agile process. Scrum, and Xtreme Programming (XP) are two favorites but there are others. (Note that Kanban is technically a release management technique, not project management, but many teams find success using it.) Pick one that will work well for your environment. If you frequently need to jump between projects, Scrum may not be for you.
Once you have your system, go all in. Get an expert to guide you. Get the proper training. Far too often I hear of teams that read a couple of books and adopt a couple of Agile ideas, then fail and blame it on Agile, saying that it doesn’t work. If you’re going to do it, do it right.
And finally, keep in mind that the importance of good soil. By using the right soil and applying software gardening techniques, your applications can be lush, green, and vibrant.