Project Managers vs. Scrum Masters: Agile Project Management Matures

There is a lot of resistance in the agile community against project management. Many (though not all) proponents of agile methods feel that project management is the root of all evils. To understand this perspective, we need to look at how agile methods were evangelized in their early days.

Even though you can trace the roots of these methods to the 1930s, most of them were developed in the 1990s with the name “agile” being adopted in 2001. Early proponents of these methods had to make bold claims and controversial statements to be noticed, as the new methods they were proposing were outside of mainstream development management processes. Of note, is that these early agile proponents came (almost exclusively) from the technical world, being software architects and developers. These development team members felt that traditional processes were causing them endless grief and lots of waste on high-change or highly-complex projects. The person enforcing these traditional methods was the project manager; thus, the project manager was put forth as the representation of all that was going wrong on traditional projects.

To be fair, in the early 1990s, very few people had heard of these new methods, and when facing a troubled high-change project, project managers were baffled with why their traditional management methods were failing. Business executives, determined to make things better, pushed the project managers to apply the traditional methods even more vigorously, often to the point of complete project dysfunction. This just further reinforced the image among the technical team members that project management was the root of their problems.

Since project management is a key role on most projects, the agilists created new roles that they refused to call “project managers” as an expression of their rebellion against traditional project management. These new roles took on names like Scrum Master, Development Manager, Delivery Manager, and several other titles that all encompassed all or part of the traditional project management role. Nevertheless, many in the agile community will strongly deny that Scrum Masters perform some of the job of a project manager.

Now that project managers have more tools available to them such as agile project management techniques, they have a different set of options for responding to high-change projects. Still, many people in the agile world recall the early days of the agile movement when technical team members often saw project managers as the enemies. They are shocked to find that project managers have now joined the agile movement and are looking to apply agile principles to traditional project management practices.

To overcome this criticism of project management, we first need to look at good (or even excellent) project management when we are assessing its applicability to a situation. There are a lot of incompetent or mediocre project managers running around out there, just as there are a lot of incompetent Scrum Masters running around with the ink still drying on their certifications. To be fair to everyone, let's look at what each role should look like.

An excellent project manager does not try to attack all problems with the same tool; rather, he (or she) analyzes the situation, figures out what problems should be dealt with at all, then prioritizes them, and the chooses the appropriate tools. This is all done in cooperation with the sponsor and other stakeholders, as well as the senior project delivery team members -- after all, if you don't get buy-in from the stakeholders, sponsor, or team, then the approach is just not going to work. (At this level, it is very similar to agile.) The key skills here are collaboration, communication, and negotiation.

Excellent project managers will chose techniques (or even methodologies) to address specific problems. For example, if requirements are fuzzy or stakeholders are having trouble understanding the solution being proposed or a technical solution is experimental and high-risk, then an iterative, incremental approach is best. If the requirements are very stable, stakeholders in alignment, and technology is low risk (such as building yet another package of custom reports for a system for which you have developed custom reports many, many times before), then a more linear approach (a.k.a. "waterfall") may be the most efficient way to deliver the project. (With the complex solutions we seem to face more and more these days, projects tend to resemble the first example far more often than the second.)

Another key point is one of leadership, communication and collaboration styles. Excellent PMs will understand that they need to vary these styles based upon the needs of the individual team members and the nature of the situation. Excellent PMs choose skilled team members, help everyone converge on a common vision of the goal, facilitate a consensus on the approach, gains agreement (signoffs, etc.) and then let's the team do what they know how to do best. A PM trying to tell a Java programmer how to do his or her work is a recipe for disaster -- the PM should focus on overall risk, communications with stakeholders, financial tracking and management, delivery strategy (alignment between interdependent projects, alignment with key business dates), and helping the team stay productive by taking project issues off of their hands.

Senior (and "enlightened") PMs tend to take a more collaborative approach to working with their teams. These PMs work alongside the others as part of the team -- NOT "leading" the team -- and act as team members with an equally-important but separate role. When I manage projects (agile, traditional, or somewhere in between) I see my job as serving the team, not running it. The team members can delegate things to me that they can't or don't want to deal with such as finding support for a product they are using, escalating on a group that is not providing materials on time, shielding the team from interfering stakeholders, etc.

The PM role (when done properly) pretty much covers the job of a Scrum Master. They do the same things: manage scope (backlog), build schedules/plans (iteration plans, release plans), manage change (backlog maintenance, updating release plans, etc.), track and report on progress (burndown, velocity, etc.), track and manage quality, manage risk, manage the overall delivery process, etc.

In Scrum, a Product Owner is responsible for the contents and the prioritization of the contents of the backlog (the overall project scope). When looking at real-world situations, however, you find that you are often lucky to get an hour a day with the Product Owners as they are busy running their existing businesses. In many situations, it is the Scrum Master who makes the backlog updates, guides the Product Owner through the agile process (especially since many POs have had no formal training on their role), and influences the prioritization decisions by explaining some of the impacts/tradeoffs of prioritization decisions (along with the technical team members). These can be seen as some aspects of "managing." Note that managing does not mean "owning." But even that explanation does not go far enough. If you are a Scrum Master of a project with an external customer (meaning a separate company) then someone needs to manage the scope of the contractual commitments. Depending upon how your contract was written, that may require a one-page change order at the start of each iteration detailing the stories (scope) that is being authorized for the upcoming iteration. You cannot ignore the legal obligations of the agreement, and to ignore the paper trail authorizing scope changes leaves you open to claims of negligence and litigation. (I've seen it before in some of my troubled project recovery work. There are lots of troubled agile projects out there where newly-minted Scrum Masters failed to take basic financial management or scope management steps --largely because they were not mentioned in their Certified Scrum Master training -- and left their company open to lawsuits.)

On all agile projects, the team members (including the technical members, the Scrum Master, PM, sponsor, and others) need to all work together to come up with a plan that works for everyone. There are lots of technical and business tradeoffs that need to be balanced during the planning process, and everyone needs to have their perspective included. However, at the end of the day, someone takes notes, someone builds the "plan" that gets posted and communicated outwards (if relevant) -- this person is usually the PM or Scrum Master, as the technical team members are busy sizing stories, doing design, etc. and the Product Owner doesn't have the skills (nor the motivation) to do this. As for the iteration scope (sprint backlog), the data to build it and update it comes from the team, but the work to do the initial build or update is an administrative task that usually gets assigned to the Scrum Master -- unless you have a tool like VersionOne that makes the updates quick and easy for the team. Even though there are good tools out there, most people I've seen still end up using Excel spreadsheets for their backlogs.

Tracking and reporting progress is, again, an administrative task that the development team likes to get off of their plates. The building of reports and communicating that information usually falls to the Scrum Master so that the technical work can progress most efficiently. Yes, the team does get together to review metrics and discuss progress; however, in most large companies, the teams have almost no say in what gets reported. There are standard financial reports to prepare (Sarbanes Oxley, anyone?), project portfolio management tools to feed, and many, many information stakeholders to satisfy. To say that the team can decide what to report and in what format only works in theory or in smaller companies. Try that in a major bank, utility company, or (gasp) a government department.

And when you are trying to do agile within large, complex and bureaucratic organizations, you'll find Excellent PMs have a leg up on Scrum Masters. Scrum Masters get little or no formal training (there are significant variances between the offerings of Scrum trainers) on subjects such as mandatory earned value reporting for agile projects, adding governance structures to deal with compliance and audit requirements (e.g. FDA), and much more that you will find on real-world projects that make our projects less agile than the theoretical model people are trained in, yet are nevertheless constraints we have to live with every day. This is where an Excellent PM will bring all of the other management skills and experience that go beyond agile to the table to help the project navigate through the complexity.

To manage complex projects, Scrum Masters need more than just the governance-type issues above. There are other traditional management skills needed to make a successful project such as negotiation, interpersonal conflict resolution (when team members don't get along with each other), communication strategy (for stakeholders – called “chickens” in Scrum parlance), political awareness... and the list can go on and on.

Agilists who think that project managers can only prevent agility from working reveal that they have too narrow of a focus. Perhaps in a truly agile organization with appropriate agile-friendly processes and agile-trained stakeholders, this may work; however, from what I have mentioned above, in large, complex organizations, where agile is an experiment or only used in some pockets of the business (which is very, very common) the PM can help by working to create an environment (bubble) where agility has a chance of succeeding while the PM protects the team from outside non-agile forces.

In conclusion, I think that I have shown the complexities and subtleties of the Scrum Master role when employed in large, complex, real-world projects. Classically-trained PMs are often the ones who will be governing a Scrum project in a large corporation -- whether there is a Scrum Master assigned to the team or not -- to help the agile projects fit in to the complex governance structures in place (often for very good reasons) in these environments. Do I think there are opportunities for removing some of this governance? Yes, of course -- that is one of the first steps I take in recovering troubled projects using agile methods -- the bureaucracy is often holding back progress; yet, some of it is there for a good reason.

I don't care whether you call the role a project manager, solution manager, delivery manager, Scrum Master, technical lead, project coordinator, or some other name -- the work of project management still needs to get done on complex projects. The new agile project management movement is one very good way to address this need.

-----

Kevin Aguanno, PMP, IPMA-B, MAPM is an Executive Project Manager specializing in troubled project recovery using agile methods. He is the author of over one dozen books on agile, is an award-winning trainer specializing in teaching “disciplined agile” to project managers, and is the thought leader behind the new Certified Agile Project Manager qualification available from the Project Management Association of Canada (www.PMAC-AMPC.ca)