5 Practices to Make Any Project More Agile

A core foundational concept in Agile is to get feedback from real users early enough to be able to address any issues that arise.  In traditional Waterfall, we assume we can do enough work up front that there will be no issues at the end, but reality shows that this almost never happens. Instead, at the very end of the project, we do User Acceptance Testing and discover issues that we either rush to patch or we plan a follow up project to address them.

Even if your project cannot adopt an Agile methodology, there is a better way to do Waterfall that still lets you take advantage of all the experience you have doing Waterfall projects and requires very little change within your organization.

There are 5 practices we strongly recommend because they can be done by any project following any kind of methodology. We have in practice with real projects seen them dramatically improve the results of Waterfall projects, and in fact have found historical evidence that some of these practices were recommended in the past for Waterfall efforts. These 5 practices are:

• Divide the work into increments
• Plan, estimate, and prioritize each increment
• Synchronize work including vendors
• Demo each increment
• Retrospective each increment

This is what as known as Incremental Waterfall with the addition of the Agile practices of work synchronization, demonstrations of progress, and multiple retrospectives. For Waterfall projects, these practices should not require much change to current processes. Some of the phase gates may need to be passed each increment, such as detailed requirements or design each increment, so some additional time should be allowed for in the schedule. The time is made up later when better quality leads to shorter test cycles and involvement of users means much less (if any) late rework. Big picture work such as overall planning, high level requirements, architecture, and solution selection can be done at the beginning because they are common across all increments and typically do not change much throughout the project lifetime.

In general the approach for a Waterfall project will be a relatively short up front planning, requirements, and architecture phase followed by a set of increments for implementation. Each increment will include the work of detailed planning, requirements, design, implementation, and testing. The complete release process may take place at each increment if desired, or may be delayed until the end of the project.

Each practice is described in more detail below.

Divide the work into increments

An increment is a working subset of the overall solution. This practice would mean examining a project to determine the best way to divide it into 2 or more incremental deliveries of working solution. Note that these deliveries do not have to be general availability, but rather could be collected in a holding area until the whole solution is complete. But each increment will go through a full test cycle including especially user acceptance testing.

The criteria for determining what goes into each increment should include business value (the most valuable work should be done earliest), risk (we should use feedback from real users to drive out risk as early as possible), and the ability to deliver a working subset of the overall solution (minimum viable product). This practice by itself has dramatically improved traditional Waterfall projects by providing feedback well before the end of the project.

In fact, this practice was proposed for Waterfall projects in 1970 by Dr. Winston W. Royce based on his previous long time experience with large Waterfall projects (Managing the Development of Large Software Systems, Proceedings IEEE Wescon August 1970, pages 1-9). His experience and research showed that there would almost always need to be at least a second increment of a Waterfall project to refine the solution based on lessons learned, and so teams should just plan for it. We see the same realization at most of our large customers, where large Waterfall projects have typically been followed by another improvement or fix-it project. We are suggesting to formalize this practice with multiple planned increments of any software project.

Plan, estimate, and prioritize each increment

No matter the length of the increment, the detailed work planning, refining of estimates, and prioritizing of work should happen not long before the start of the increment, instead of doing all of it up front for the whole project. This allows the flexibility to take advantage of lessons learned in one increment to apply to the planning for subsequent increments. It is also a good time to review project metrics to verify the work is going according to plan, and allows a planned time to make adjustments as needed rather than waiting for the end of the project to discover there are issues.

We expect high level planning at the start of the project for all except the most innovative and fast-changing work (which by nature cannot be planned much in advance). This practice is to wait to do detailed planning until just before you need it so that the plans are the most accurate based on the information you have to date. This practice avoids spending time doing detailed work that is likely to be changed later in the project. The less well-understood the work is, the more important this practice becomes.

Synchronize work including vendors

Everyone involved with the project should regularly synchronize their work, including a representative from each vendor. This synchronization can take the form of a daily standup. In some cases, that may be more frequent than needed, but this synchronization should take place at least weekly. Synchronization should be often enough to remove the risk of unnecessary delays due to unknown work blockages.

Note that this is not meant to be a long status meeting, but is rather a 15 minute or less structured ceremony where each person just states what he or she has recently completed, what work he or she is working on now, and anything preventing (blocking) the person from completing his or her work. The most important information to come from this meeting is typically the blockages. The work blockages can be addressed after the synchronization meeting.

It is very important that the vendor representatives be included. A big problem when working with vendors is lack of visibility into what they are working on. Communication can be difficult already when working with another company, so anything we can do to promote regular, meaningful communication will improve (sometimes greatly) the work we do with vendors. Our customers who have worked this way with vendor representatives report a uniformly better experience than just handing work “over the wall” and expecting everything to be fine.

Demo each increment

At least once per increment, the working solution is demonstrated to at least the project sponsor and stakeholders. By far the better choice is to have real end users also attend the demonstration. This demonstration has two purposes: it provides a real deadline when work has to be finished so that it can be demonstrated, and it provides an early opportunity for users to provide feedback on whether or not the solution meets their needs.

Note that this does not replace formal user acceptance testing, but rather is a demonstration of progress and an opportunity for feedback well before the end of a project. Instead of providing status through reports showing perhaps 85% of requirements are completed (for example), we demonstrate working solutions as a measure of progress. If the solution is not complete enough to be demonstrated, we count it as zero percent complete.

We treat solutions as zero complete or 100% percent complete and do not try to determine an increment in between. This is because with any kind of knowledge work, it is not actually possible to truly determine how much is left to do. We can only know how much has been done. We all know of cases where the solution was 90% complete but that last 10% took 90% of the resources. We prevent that in this practice by only counting a 100% solution as complete and anything else as zero.

The demonstration provides a good opportunity to review the progress of the project to get an idea of whether it is on schedule, ahead, or behind. The outcome of the demonstration is that either the work to date is acceptable as is, or defects or change requests are created to fix problems that are detected at the demonstration. The work to fix the defects or change requests can be planned into subsequent increments of the project so that these problems do not exist in the released version of the product.

Retrospective each increment

Most companies have some kind of retrospective or lessons learned at the end of a project. Unfortunately, that is too late to do any kind of good for that team. Instead, at the end of each increment, the team will review the past increment, and in a non-judgmental way discuss what went really well, what did not go so well, and what they will do in the next increment to improve as a team.

We suggest the team start the retrospective by reciting a statement such as “We acknowledge that everyone has done the best job they were able to do in the circumstances that existed during this increment” and suggest the team end the retrospective with expressions of gratitude and thanks to other members of the team or even people outside the team who helped in some special way (it is nice to let those people know in person that they did a great job if they do not happen to be at the retrospective). This helps set a non-judgmental tone to the ceremony. The most important thing is to avoid blaming and finger pointing, to make the ceremony constructive.

The outcome of the retrospective is a small set of specific action items the team will take to improve the coming increment. We recommend starting with just 1 or 2 action items so as to not overwhelm the team with improvement work while they are focused on delivering a product. Often the outcome of the retrospective are suggestions that become habits over time, as the team incorporates better and more efficient ways of working together.