Strategies for Successful Development

Process and technology for small teams building real software

Timeline Postmortems

Posted by S4SD (Seth Morris) on 2008/09/12

I wrote this article in 2004 for StickyMinds.com, the website of STQE (Software Testing and Quality Engineering) magazine*. It was reprinted in the April, 2005 issue of Projects & Profits magazine.

* The magazine is now called Better Software.


Timeline Postmortems

by Seth Morris

Abstract

We want to use project postmortems to improve our software process, but few teams do them. And fewer teams reliably learn from them. You can introduce postmortems to your team easily with timeline postmortem process. If you are already doing postmortems, a timeline-based approach may improve your results.

This process:

· Takes little time (a few hours or less of team time)
· Has a high degree of engineer acceptance
· Provides immediate practicable feedback into your development process
· Increases team cohesion and rapport
· Reduces finger-pointing

Postmortem Goals and Problems

You’ve got to accentuate the positive
Eliminate the negative
Latch on to the affirmative
Don’t mess with Mister In-Between

— lyrics by Johnny Mercer

Inevitably, on projects, something goes wrong. Ideally, something else goes right. On the next project, you want to prevent the one from reoccurring and reproduce the other. This is the goal of postmortems.

However, most teams don’t do postmortems. And the teams that do don’t always get value from them. Frequently, postmortems turn into "the blame game" or whitewash mistakes. A bad postmortem can create dissension and institutionalize failures.

A good postmortem builds team rapport, rewards project successes, and improves the next project after it. Good postmortems are rare, but they don’t have to be. Whether you’re introducing postmortems to a team that doesn’t use them, rescuing a failed postmortem process, or improving a successful one, a few simple steps will help.

The complete timeline postmortem process takes 3-4 hours of team time plus a little preparation for the moderator. The process produces results that are immediately applicable and focuses on low-hanging fruit. With it, you introduce incremental change that the team can adapt to rather than reinvent the entire development process.

In keeping with this philosophy, the process elements can be separated and introduced one at a time.

Process Outline

The basic steps of a timeline postmortem are:

1. Create the project timeline. This is a record of the “official truth” of what happened when, at a granularity that allows team members to see the entire project as a whole.

2. Walk through the timeline, identifying successes, failures, and "themes" of the project.

3. Group successes, failures, and themes by project phase. You will focus on process change one phase at a time. Don’t redesign the delivery phase if you haven’t made it past design; it’s just spinning wheels and it’s unlikely to be successful.

4. Adjust your software process to address the failures and recreate the successes.

5. Only implement process changes that affect the immediate future. Keep constant, public feedback on new process to inform the next postmortem. This also assures team members that their concerns will be heard and allows them to “disagree and commit” for the time being.

6. Revisit the postmortem when your subsequent projects enter a new phase. Finalize your notes on the ending phase and reiterate the process changes, if any, in the new phase.

Process Elements

If you’re mixing and matching, the key elements of the process are:

  1. Separate "what happened" from interpretation, and stay away from fuzzy "why" questions.

Creating a timeline of project events allows the team to talk about events and process without pointing fingers. Getting agreement on the timeline first, without any analysis, keeps the later discussions from devolving into "did not/did too" exchanges.

In addition, the timeline becomes the "official truth" for the team. Shared history increases "team-lyness." Just the act of creating the timeline can resolve esprit issues after a difficult project.

  1. Give equal attention to each part of the project. Postmortems can easily focus on the pain of the final phases of a project because that’s the most fresh in the team’s minds. A good timeline will remind the team of successes and failures from the beginning of the project, which are the most important to address while the next project is beginning.
  2. You want to do your postmortem right after the project is over, but you can use the timeline process before that if a project is stalled or failing. Waiting until another project is beginning gives some useful perspective.

On failing projects, creating a timeline sets up the recommitment meeting. It implies that any issues are part of the past and focuses attention on the results rather than the difficulties.

  1. Always have an impartial moderator/facilitator. This moderator only asks questions and writes down answers the team agrees on. The moderator should not be contributing opinions.

If your company doesn’t have someone impartial, hire an outsider.

If you have to fake impartiality, create a clear distinction between providing facilitating and providing your opinion. One good approach is to use standing and sitting: when standing, only ask for information and repeat what the team says. Sit down and wait a second before giving your opinions. Have someone ready to interrupt if you step out of role.

  1. Describe events and proposed process in terms of behavior, not intent. You can use the TOTE for this
          What is the trigger? (Initial Test)
          Who does what? (What is the Operation)
          What causes it to end? (Completion Test)
          What happens next? (After completion, Exit to where?)
  2. Address technical decision making as well as traditional development process. Be sure to discuss technical decisions in terms of criteria and results. This is an area to watch for finger-pointing and dogmatism.
          You can address dogmatism (and, to a degree, finger-pointing) by using clear project and development criteria. If you have clear criteria, put them on the wall during the postmortem. Actually, put them on the wall at the start of the project and just leave them there. Shared criteria makes for cohesive teams.

The Process

1. Preparation

To implement the process, you need some preparation:

· One impartial moderator

Too many postmortems are run by project managers or executive sponsors. The team members are unlikely to be honest about successes and failure in this environment. When the facilitator is not a manager, s/he may have a preferred software process methodology to push (dogmatism or “technical/process religion”), skewing the postmortem results.

If possible, get someone outside of the group—or even outside of the company—to lead the postmortem. The facilitator needs a basic understanding of development process, but doesn’t have to be highly technical or intimately familiar with the project. In fact, a degree of unfamiliarity will help.

»The moderator has final ownership of all postmortem deliverables, including the preparation.

· A (mostly) blank project timeline

Begin with some basic research.
      When was the project begun?
      When were major visible milestones, like first QA drop and final ship?
      Were there any personnel or company changes during the project?
      Major holidays?

» Put all of these on the project timeline to begin.

· Project phases

Use project phases to contextualize successes and failures. During implementation, focus on the current and upcoming phases.

Phase definitions will depend on your team’s development methodology.
      A typical waterfall project may have spec, design, development, and testing phases.
      An iterative project may have several iterations, with spec, design, and test in each one.
      Prefer to define phases that are generally present in your projects, rather than phases that will never occur again, but be willing to violate that rubric as needed. Be descriptive, not prescriptive.
      Phases may overlap or repeat. The QA group may be in one phase out of sync with development and it is likely that Product Management will have phases that differ from everyone else. Use phases that are useful to the groups they affect. This is the time for empirical, not theoretical, analysis.

During the meetings, the team may adjust the definition and dates for the phases. This is normal.
      The team is always right about this. Phases are a tool to help people group activities together; what the team considers related or grouped is related or group.

» Put a pass at the phases on the project timeline.

· Gathering the whole team

Get as much of the team as possible present for the first two meetings. This includes at least development, QA, and product management, as well as any support, release, documentation, or other groups as may be appropriate.

Managers and executives are useful in the first meetings, but be prepared to send them out of the room if people seem reluctant to talk around them.
      If you have to send a manager out of the room, do it openly in front of the team. Tell the manager that his or her presence is impeding communication. The team already knows this; acknowledging it increases rapport and openness. It may also be a useful lesson for the manager.
      If a manager refuses to leave, stop the postmortem. It will not be successful and any good ideas that come out of it, but do not come from the manager, are unlikely to be effected.

2. Timeline creation

» 60-75 minutes. Whole team.

The meeting goal is to establish the project timeline.

1. Put the timeline on a whiteboard where everyone can see. Leave plenty of room for adding events and activities.

2. Establish the major dates for key events: specs begun, estimates delivered first code fork, first delivery to QA, project completion (if possible), etc.

3. Get buy-in on the project phases.

4. Add project activities and events. Use language like, "During the design phase, what else happened?" Mark what features were affected when.

5. For each event or activity, ask "does everyone agree that this happened?" Watch the team when you say this and look for nods or head shakes. Agreement is more important than precision.

Keep this to an hour. An hour should be sufficient to get a level of detail appropriate to the length of the project.

Do not include any analysis in this meeting. No one should say "why" anything happened, or state any consequences of anything. When someone starts (and they will), interrupt them.

3. Project analysis

» 60-90 minutes, whole team.

You can run the analysis meeting right after the timeline meeting, but put at least a 10 minute break in between and make everyone stand up and leave the room. Get yourself a drink of water.

Lead the team through the project timeline from start to finish, phase by phase. Identify key themes, successes, and failures on this project. Start at the beginning of the project if you can; don’t let the team use up all the time on the phase most recent in memory.


Successes? Failures? Themes? Let’s define some terms:

Successes

A project success is something that reduced risk or improved the project’s result.

Failures

Failures increased risk or reduced the project’s success. Definitely include failures that are repeated from project to project. Those are the best to address.

Some teams don’t like to use the word "failure." This isn’t the time to use words like "challenge" or "opportunity." If you need to use a different word, make up something, like "lamppost."

Themes

Themes are neither successes nor failures. Instead, they provide context for successes and failures that can’t be considered part of the timeline. Often, a success on one project would be a failure on a project with different themes.

» If the half the team thinks something was a success and the other half thinks it was a failure, then it’s probably a theme.


Key points to the analysis meeting:

· Work from the timeline.

· Ask questions by phase, but be prepared for people to offer comments on other phases.

· Keep separate lists for Successes, Failures, and Themes. I use three flip charts taped next to the whiteboard, preferably on different walls.
      » If you use flip charts and whiteboards together, use whiteboard markers for both.

· Use behavioral questions only. "In this phase, what worked or didn’t work?" "What was different from previous projects?" Do not ask "why" questions.

· Successes, Failures, and Themes should all be described in terms of "When X, then Y, which caused Z." Ask for any parts you don’t get. If a team members offers "We didn’t follow the bug process," ask when and whether that was a Success or a Failure.

· Repeat every Success, Failure, and Theme before writing it down. Get agreement from the team before listing the item.

· Write down the exact words you repeated to get agreement. Do not "translate." Analysis, synthesis, and clean-up will happen later.

· Keep the meeting moving. Try to limit it to an hour.
» If you can’t get through in a reasonable amount of time, stop and either reconvene another day or finish the process on the phases you did complete. If that is useful, either convene another meeting on the rest of the project or abandon that and start with the next project, depending on scheduling and availability.

· Watch who doesn’t participate. Ask them directly if they have any comments. Language like "What worked or didn’t work for you?" is often effective. Be willing to wait in relaxed silence for an answer.
» Other good questions: 
      What did you do that wasn’t part of the official process? 
      What parts of the process did you skip or try to skip?
      What was a process step you trusted and didn’t have to worry about at all?
» Note that these questions all presuppose that there is an answer.

After the analysis meeting, deliver a document containing the timeline and the project analysis. Organize Successes, Failures, and Themes into project phases. Give the team a few days to digest it.
      This is the time for synthesis. Group Successes, Failures, and Themes into related sets and call out any that crossed many or all phases.
      Expect a small amount of feedback. This is not the stage where most team members will put in solo time.

4. Process design

» 60-90 minutes, all leads and manager plus all interested team members.

Get the leads and managers as well as interested individuals. This is not the meeting to force people who don’t want to participate.
      » Do try to get someone from every discipline.

1. Group Successes and Failures into any obvious categories: intra-team communication, scheduling, making technical decisions, etc.

2. For each category or item, identify the project phase to address it. Identify existing processes which affect it.

3. For each phase, develop new process or process changes to address the highest-value items.

When designing development process, keep in mind:

A process requires the TOTE:
· A cue to start
· An operation or behavior
· An end point
· Process(es) to switch to

» Write all of these down.
      Try to include a way to measure progress and any events that would call a recommitment or change meeting.

You may not be able to address every success or failure. Focus on phases and changes that will be immediately useful on current projects.

» Process will change
» No team follows every process it expects to
» No team follows every process the same way every time
Since you know this, don’t design in too much detail and let evolution happen. Evolution means something was getting in the way of work. Don’t fight it.

If the team has a split about a process, just pick one side. Let the team try it one way and change if needed. Be willing to be wrong and be sure the team understands that.

5. Implementation

When the next project starts–if it hasn’t already–you’re ready to implement new process.

Only implement pieces that affect the project phase you’re on.
      For example, don’t spend time re-tooling your release process before the specs are written.
      When each project phase ends, review the previous project’s postmortem and implement process for the new phase.

Write down new processes and post them in a public place.
      » People will be confused and make mistakes. Always refer them to a written description of the process.

If something doesn’t work, acknowledge it and move on. Make a change if you have to, but if you can muddle through with a broken process it’s best to save the redesign for the next postmortem.

» Always remember that the process is not the product. You have software to ship.


No matter how successful your current project is, I hope your next one is even better.

And may you never mess with Mister In-Between.


Listening to: The Players – Pippin (Original Cast Album) – Finale -The Player

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: