Strategies for Successful Development

Process and technology for small teams building real software

Archive for October, 2008

The World > Your Project > Your Process > Your Tools

Posted by S4SD (Seth Morris) on 2008/10/11

This is something I’ve been wanting to write about for a while; I might as well use a short answer I wrote on LinkedIn as a jumping-off point.

In the Scrum Practitioners group, Aqueel Khan asked about baseline implementation before starting an agile project: 

I know there’s no cookie cutter template for every team in every organization. But there has to be "something" which needs to be implemented as the foundation of a Scrum/Agile/XP project.

Gunther Verhyen gave a fabulous description of the raw process requirement for Scrum1, suggesting that the process has no procedural prerequisites.

So I added some comments about beginning any process, with specifics that apply to Scrum:

Gunther’s answer is pretty much definitive, so let me add something outside of the Scrum process that you need for company and team success: know what you aren’t managing with Scrum.

It gets very easy to take a simple and effective process like Scrum and assume it is larger than your project, larger than your product, and larger than your company; the result is to assume Scrum (or UML, or … or …) will solve problems it isn’t designed to address.
Don’t let people try to shoehorn product planning (market research, customer interviews, experimental prototypes, …), engineering R&D (technology evaluation, build/buy analysis, large-scale architecture and refactoring plans, …) into a project Scrum.

It is very easy to take the simplicity and success of Scrum, combine it with the increased day-to-day involvement of groups that are traditionally "above" the implementation phase, and find your organization start to assume that the Scrum process is one never-ending chain of sprints that somehow encompasses all product work. It isn’t intended to be that and trying to use it that way will weigh down your implementation.

Make sure the Scrummaster(s), the heads of technology, the heads of product, and the heads of sales/support/customer-interaction all know what not to try to inject into the Scrum.

If you want to keep this in a Scrum framework, you can do so by creating backlogs for groups, as well as projects/products, by keeping a firm hand on the project backlog to prevent creep-in from larger-than-project items, by ensuring that projects are running with larger sprint lengths to work on larger items (I know, many poorly-trained Scrum fans find this type of "planning" anathema, but successful teams have it whether they recognize it or not), and keeping a sense of humor about the whole process.

Or, just remember that nothing inherent in the Scrum process would lead a company to cancel a project; clearly, something outside the process has to exist to do larger things. Keep that separate from the Scrum so neither interferes with the other.

[LinkedIn Scrum Practitioners Group question thread]

Overall, this is a problem that wheedles its way into every methodology. In fact, it’s one of the measures of when a successful methodology becomes a "religion": when people insist that it solves problems it doesn’t.

Scrum is about software development at the low, development team level. It manages projects by breaking off chunks that meet certain requirements (size, testability, and completeness-to-ship) and managing the day-to-day implementation to finish that chunk on schedule. It does not address anything else.

Scrum can be used to manage other, non-coding projects and is quite successful with any pile of work to be done that can be broken down into linearly-independent units. For what it’s designed to do, Scrum is great.

Unfortunately, Scrum suffers from too much adherence by too many people with too little understanding of how it fits into their overall problem. Scrum doesn’t say anything about what to build, when to build it, or what to build next. It doesn’t address training, delivery, internal or external documentation, or requirements gathering.

Most agile practitioners eschew any kind of overall architectural design (believing it always leads to if "big design up front"), yet senior developers invariably practice a form of it—it is one of the features that makes them senior. Because it isn’t part of the day-to-day development, Scrum doesn’t address larger-scale architecture—or even small-scale design2,3. Because it isn’t part of Scrum, many houses embracing Scrum try to avoid doing it at all, believing that Scrum—a deliberately short-sighted process—somehow replaces future-looking planning.

The Agile Manifesto tried to avoid some of this. It specifically called out that "things on the right"—which includes process, tools, documentation, and planning—have value. It claims that other things have more value when they come into conflict.

This is a process of criteria ordering. It is one of the first pieces of work any team should do and one of the premier jobs of technical management, all the way up to the director or VP level. Criteria ordering is relatively simple and incredibly valuable and the Agile Manifesto authors are to be commended in their work on it and the way they tried to bring it to the forefront.

But given who they are, who the initial signators of the manifesto were, they really should have known the damage they were doing and they should know how little the have done to ameliorate it.

Every major agile process, and especially Scrum, has done a great job trying to bring the left-side criteria into play by a) ignoring the right-side items, or b) diminishing the value the right-side items bring. This isn’t bad with people who know what they’re doing, but it is devastating when people embrace the methodology as larger than their problem.

Let’s set Scrum aside to show the situation. Back in the dark ages (1999 or so), the religion was UML, especially the Rational approach to UML. Agile development was still a guerrilla phenomena and the disdain for "big design" was partially in reaction to UML and it’s more narrow-minded adherents. To make matters worse, UML was usually tied to one of a few big, multidimensional tools that did very good jobs at some things and very poor jobs at others, but integrated them well enough to force some team members to use a poor tool to make other members happy.

Like other methodologies—waterfall planning, OOAD, BRD-as-product design, even the relatively benign CRC—people heavily invested in the process came to believe that the model in the process was somehow able to model the world outside of software development (a kind of reversal of the Conant-Ashby theorem). 

When this happens, you find several fallacies and errors creeping in to daily life:

    • Items that are not explicit in the process are disregarded (design, buy-in from other groups, redesign in the face of change, real measurement instead of believing the schedule, etc.)
    • Work that isn’t part of the process gets shoehorned into the process somewhere ("Well, considering those user requests is really part of bug tracking, so we’ll just have the bug database manage it.")
    • Some data or functionality in a tool is partitioned off and used in a way that doesn’t make sense. Sometimes fields are used to mean something entirely different just because they hold the same kind of data.
    • Business processes that have nothing to do with software are driven in the development process even when it doesn’t work4.

I am as guilty both of doing these and of letting other people do them on my watch. They are seductive and there are times to do the wrong thing in favor of the greater project.

I’ve done them. But you don’t have to.

When things seem amiss, ask yourself some questions:

  1. Is this thing we’re doing something inside the model of development that the process envisions?
  2. Is this process, as we’re implementing it, designed to support this business process?
  3. Is this working for us? Have we reduced work, shortened schedules, increased participation, eliminated political maneuvering, or increased quality?
  4. If we’re responding to something outside the process’ model (a change in the real world, usually, as opposed to something in the virtual world of our plans and hopes), what are the parts that are not about this process and have we made sure they’re being addressed in their own way?

Once you see the problem, it isn’t usually hard to address (although the die-hard adherents of any belief system will require some convincing). Just remember the formula:

The Real World > The Project > Your Process > Your Tools

Items on the right cannot contain items on the left. Models on the right are unlikely to be good models of anything on the left.

1) The minimum view of Scrum:
    3 roles: Scrummaster, Product Owner, and Team Member
    3 ceremonies: Sprint Planning, Daily Scrum, and Sprint Review
    3 artifacts: Product Backlog, Project Backlog, and Burndown

2) Also, any enshrinement of design into Scrum would have made Scrum less applicable to non-coding projects and would have left some people confused over the agile principles to value "working software over comprehensive documentation" and to value "responding to change over following a plan."

3) Unfortunately, the stigma associated with design is felt more strongly by more senior developers, who have more invested in appearing to be closely allied with the current development religion. The people most hurt are the junior developers, who are deprived of the seniors’ intuitive expectations as to how a project will "look" when it is done.

4) When a process works for something, just keep doing it. Change the names of things if you have to.

Listening to: Cat Stevens – Tea for the Tillerman – Hard Headed Woman



Posted in Opinion | Leave a Comment »

The No-Spec Spec: Purpose

Posted by S4SD (Seth Morris) on 2008/10/09

This is part of the series The No-Spec Spec

"Purpose" Description

The Purpose document is the user- and client-focused The purpose document is fairly close to an MRD (Market/Marketing Requirements Document). If you have a good MRD, you could call it the Purpose and move along, but the chances are that a team that can’t get a good spec also can’t get a good MRD.

The Purpose document may be somewhat longer than the Fit and Identity pieces, depending on the level of detail and size of the project. It should answer:

  • Why will a user use the product?
  • What problem will the user expect the product to solve?
  • What business process (or user process/goal) is the user trying to accomplish when using the product?
  • If the customer is different from the user, what is the customer’s goal?

These seem very basic, but they form the reality check for functionality. Late in a project, these are the ultimate test for any change in requirements. Referring to these can prevent most late-project schedule- or quality-damaging changes and almost all developer gold-plating.

This is a simple document. Hopefully the product managers can produce it in 30 minutes or less and hopefully it contains less than two pages of text. It may be longer if the Purpose requires images or background information to explain1, but the critical user and client goals should be relatively short.

Change rules:

    If the Fit changes, this and all other documents need to be rewritten. The Fit may indicate that the user, customer, or business problem has changed.
      The Fit may change without affecting the Purpose if the business model or the business environment changes but the user doesn’t. This is a good thing. It means that even though the project relates to the company/team in a different way (and presumably the old way was wrong), but the problem the project was trying to solve was correct. The need still has to be filled (and all the work up until now is still valid).

    If the Identity changes, the purpose needs to be re-examined. The Purpose is the how to the Identity’s what—and to some degree to the Identity’s why. If the product as a whole is different, that probably indicates that the goal of the product has changed.
      If the Identity changes and the Purpose doesn’t, that indicates that the Identity was incorrect to begin with, but again the problem the project is solving was identified correctly.

If the Purpose of the project changes, you need to re-evaluate each of the:
    Assumptions: What is true about the new problem or value-add that wasn’t true before?
    Environment: Does the new Purpose indicate a change in where, when, or under what conditions the product will be used?
    Capability: Is the new Purpose part of a different larger framing context?
    Behavior: What is the new functionality? This is the item almost certain to change if the Purpose does.

1) Examples of projects that might require additional detail in a Purpose document would be financial software that should include background information on the behavior of the financial instruments, kiosk software that may require screen mock-ups to indicate process flow, or software that cannot assume literacy in the localized language and requires special interface elements.

Listening to: King Crimson – In the Court of the Crimson King – Moonchild/The Dream/The Illusion


Posted in Dev Process | Leave a Comment »