Strategies for Successful Development

Process and technology for small teams building real software

The World > Your Project > Your Process > Your Tools

Posted by S4SD (Seth Morris) on 2008/10/11

This is something I’ve been wanting to write about for a while; I might as well use a short answer I wrote on LinkedIn as a jumping-off point.

In the Scrum Practitioners group, Aqueel Khan asked about baseline implementation before starting an agile project: 

I know there’s no cookie cutter template for every team in every organization. But there has to be "something" which needs to be implemented as the foundation of a Scrum/Agile/XP project.

Gunther Verhyen gave a fabulous description of the raw process requirement for Scrum1, suggesting that the process has no procedural prerequisites.

So I added some comments about beginning any process, with specifics that apply to Scrum:

Gunther’s answer is pretty much definitive, so let me add something outside of the Scrum process that you need for company and team success: know what you aren’t managing with Scrum.

It gets very easy to take a simple and effective process like Scrum and assume it is larger than your project, larger than your product, and larger than your company; the result is to assume Scrum (or UML, or … or …) will solve problems it isn’t designed to address.
Don’t let people try to shoehorn product planning (market research, customer interviews, experimental prototypes, …), engineering R&D (technology evaluation, build/buy analysis, large-scale architecture and refactoring plans, …) into a project Scrum.

It is very easy to take the simplicity and success of Scrum, combine it with the increased day-to-day involvement of groups that are traditionally "above" the implementation phase, and find your organization start to assume that the Scrum process is one never-ending chain of sprints that somehow encompasses all product work. It isn’t intended to be that and trying to use it that way will weigh down your implementation.

Make sure the Scrummaster(s), the heads of technology, the heads of product, and the heads of sales/support/customer-interaction all know what not to try to inject into the Scrum.

If you want to keep this in a Scrum framework, you can do so by creating backlogs for groups, as well as projects/products, by keeping a firm hand on the project backlog to prevent creep-in from larger-than-project items, by ensuring that projects are running with larger sprint lengths to work on larger items (I know, many poorly-trained Scrum fans find this type of "planning" anathema, but successful teams have it whether they recognize it or not), and keeping a sense of humor about the whole process.

Or, just remember that nothing inherent in the Scrum process would lead a company to cancel a project; clearly, something outside the process has to exist to do larger things. Keep that separate from the Scrum so neither interferes with the other.

[LinkedIn Scrum Practitioners Group question thread]

Overall, this is a problem that wheedles its way into every methodology. In fact, it’s one of the measures of when a successful methodology becomes a "religion": when people insist that it solves problems it doesn’t.

Scrum is about software development at the low, development team level. It manages projects by breaking off chunks that meet certain requirements (size, testability, and completeness-to-ship) and managing the day-to-day implementation to finish that chunk on schedule. It does not address anything else.

Scrum can be used to manage other, non-coding projects and is quite successful with any pile of work to be done that can be broken down into linearly-independent units. For what it’s designed to do, Scrum is great.

Unfortunately, Scrum suffers from too much adherence by too many people with too little understanding of how it fits into their overall problem. Scrum doesn’t say anything about what to build, when to build it, or what to build next. It doesn’t address training, delivery, internal or external documentation, or requirements gathering.

Most agile practitioners eschew any kind of overall architectural design (believing it always leads to if "big design up front"), yet senior developers invariably practice a form of it—it is one of the features that makes them senior. Because it isn’t part of the day-to-day development, Scrum doesn’t address larger-scale architecture—or even small-scale design2,3. Because it isn’t part of Scrum, many houses embracing Scrum try to avoid doing it at all, believing that Scrum—a deliberately short-sighted process—somehow replaces future-looking planning.

The Agile Manifesto tried to avoid some of this. It specifically called out that "things on the right"—which includes process, tools, documentation, and planning—have value. It claims that other things have more value when they come into conflict.

This is a process of criteria ordering. It is one of the first pieces of work any team should do and one of the premier jobs of technical management, all the way up to the director or VP level. Criteria ordering is relatively simple and incredibly valuable and the Agile Manifesto authors are to be commended in their work on it and the way they tried to bring it to the forefront.

But given who they are, who the initial signators of the manifesto were, they really should have known the damage they were doing and they should know how little the have done to ameliorate it.

Every major agile process, and especially Scrum, has done a great job trying to bring the left-side criteria into play by a) ignoring the right-side items, or b) diminishing the value the right-side items bring. This isn’t bad with people who know what they’re doing, but it is devastating when people embrace the methodology as larger than their problem.

Let’s set Scrum aside to show the situation. Back in the dark ages (1999 or so), the religion was UML, especially the Rational approach to UML. Agile development was still a guerrilla phenomena and the disdain for "big design" was partially in reaction to UML and it’s more narrow-minded adherents. To make matters worse, UML was usually tied to one of a few big, multidimensional tools that did very good jobs at some things and very poor jobs at others, but integrated them well enough to force some team members to use a poor tool to make other members happy.

Like other methodologies—waterfall planning, OOAD, BRD-as-product design, even the relatively benign CRC—people heavily invested in the process came to believe that the model in the process was somehow able to model the world outside of software development (a kind of reversal of the Conant-Ashby theorem). 

When this happens, you find several fallacies and errors creeping in to daily life:

    • Items that are not explicit in the process are disregarded (design, buy-in from other groups, redesign in the face of change, real measurement instead of believing the schedule, etc.)
    • Work that isn’t part of the process gets shoehorned into the process somewhere ("Well, considering those user requests is really part of bug tracking, so we’ll just have the bug database manage it.")
    • Some data or functionality in a tool is partitioned off and used in a way that doesn’t make sense. Sometimes fields are used to mean something entirely different just because they hold the same kind of data.
    • Business processes that have nothing to do with software are driven in the development process even when it doesn’t work4.

I am as guilty both of doing these and of letting other people do them on my watch. They are seductive and there are times to do the wrong thing in favor of the greater project.

I’ve done them. But you don’t have to.

When things seem amiss, ask yourself some questions:

  1. Is this thing we’re doing something inside the model of development that the process envisions?
  2. Is this process, as we’re implementing it, designed to support this business process?
  3. Is this working for us? Have we reduced work, shortened schedules, increased participation, eliminated political maneuvering, or increased quality?
  4. If we’re responding to something outside the process’ model (a change in the real world, usually, as opposed to something in the virtual world of our plans and hopes), what are the parts that are not about this process and have we made sure they’re being addressed in their own way?

Once you see the problem, it isn’t usually hard to address (although the die-hard adherents of any belief system will require some convincing). Just remember the formula:

The Real World > The Project > Your Process > Your Tools

Items on the right cannot contain items on the left. Models on the right are unlikely to be good models of anything on the left.

1) The minimum view of Scrum:
    3 roles: Scrummaster, Product Owner, and Team Member
    3 ceremonies: Sprint Planning, Daily Scrum, and Sprint Review
    3 artifacts: Product Backlog, Project Backlog, and Burndown

2) Also, any enshrinement of design into Scrum would have made Scrum less applicable to non-coding projects and would have left some people confused over the agile principles to value "working software over comprehensive documentation" and to value "responding to change over following a plan."

3) Unfortunately, the stigma associated with design is felt more strongly by more senior developers, who have more invested in appearing to be closely allied with the current development religion. The people most hurt are the junior developers, who are deprived of the seniors’ intuitive expectations as to how a project will "look" when it is done.

4) When a process works for something, just keep doing it. Change the names of things if you have to.

Listening to: Cat Stevens – Tea for the Tillerman – Hard Headed Woman



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: