Strategies for Successful Development

Process and technology for small teams building real software

Archive for the ‘Dev Process’ Category

The No-Spec Spec: Environment

Posted by S4SD (Seth Morris) on 2009/11/07

This is part 6 in the series The No-Spec Spec
A lot of teams that do produce specs could benefit from adding this one: it’s a hidden piece that gets shuffled between Marketing, Development, and Support, often winding up on the floor between them. It’s often a short document, but the thinking that goes into it can be subtle and it’s often a point of disagreement. Get it on paper as early as you can and let it change as it needs to1.
The Environment piece of the No-Spec Spec describes the context where the product is intended to operate. This can be pretty short; sadly, it’s rarely simple.
Although you should only include sections relevant to your project, Context (Environment) can include:
  • Who is the user?
  • What is the user trying to accomplish?
  • Is the user the same as the customer? If not, who is the customer and what are they trying to accomplish?
  • What is the timeframe the product runs in? Does it have an expiration date? Note that this is different from the product schedule. It is about how the business/regulatory/calendar environment affects the product. A game may need to ship in November for sales reasons, but that doesn’t mean playing it is tied to the calendar. Software for students, however, may be affected by school schedules in the target area.
  • Is there a time constraint on execution? (Only at night, only while the store is open, must finish before opening of business, etc.)
  • What hardware is the product running on?
  • What OS does the product require?
  • What other software is running on that hardware?
  • What other business processes does the product interact with?
  • What other products does the product interact with?
  • What kinds of aftermarket extensibility does the product expect? (Note that this related to, but is not dependent on, the Capability document)
  • What regulations control the product’s behavior?
  • What kinds of support does the user/customer expect as part of normal use?
  • What continuous billing or auditing does the product require?
In many companies, these answers are kept in different departments. In many companies, these answers are never written down.
Conceptually, this is a simple doc. The hard part comes when people try to pin the answers down2.
Some sample answers (no, these are not for the same hypothetical product):
User High-school or college student with a laptop or tablet that they take to class.
Some functions require a tablet.
(List tablet functions)
User’s goal Have one or more chosen songs play while enjoying a social time out
Customer: Owner of the jukebox
Customer’s goal: Continual income from users paying to play music
Timeframe: During school sessions (Sept-Dec and Jan-May in most places; list trimester schedule is appropriate)
Time constraints: Software updates and new database contents must download and install (or rollback) by start of business
Hardware: Server: (example servers)
Client: (example clients)
OS: Server: (list supported)
Client: (list supported)
Mobile: (list supported)
Other software: Server: SQL Server 2008 or Oracle (version)
Client: Software firewall, antivirus, SQL Server desktop, VPN, (list necessary support tools)
Mobile: SMS, email
Other business processes: Routine remote administration
Regular cash auditing
Regular removal of cash
Regular on-screen income reporting and income split
Reporting ad display and click-through
Other product interaction: Our server software (list)
Our billing and support tools (list)
Ad network (which ones?)
Third-party data feeds (list)
Extensibility: Full UI replacement
Added menu/toolbar items that launch third-party dialogs
Automation from other software
Regulations: HIPPA
Compulsory license reporting
Contractual license reporting
COPPA
Support: Quarterly db analysis and cleanup
Billing: Monthly metered billing. We need to connect and read the use meter by the 5th of each month.
This is a pretty simple document (although you want more detail than the example above!). You may be surprised how many arguments break out over defining the answers, though. Get through them.
Then the answers will change. When this happens, you need to re-evaluate a bunch of things. At a minimum, re-evaluate the Behavior and the Capability documents. Most of the time, very little will change, but when it winds up being a big change you’ll be glad you didn’t miss it. And if the Behavior or Capability do change, re-evaluate some non-spec documents. At least validate that technical design and test plans are still correct.
Pay special attention to the user and customer goals. These are the primary downstream consequences of the Fit, Identity, Purpose, and Assumptions. Make sure everything else in the document supports those and make sure they are re-validated if the upstream documents change.
This doc is no less important than the others, but it is usually easier. Enjoy having it: you will find that it helps in unexpected ways.

1) Make a sign. Put it up where everyone can see it. Put these words on the sign: “Don’t get it right; get it written.” Really. Do it.

2) One product I worked on could only define the user as “anyone with a credit card.” That was pretty far from accurate, but it was even farther from useful.

Listening to: Nimpf – Lonesome Town – Cyber Punk Fiction


Posted in Dev Process | Leave a Comment »

The No-Spec Spec: Assumptions

Posted by S4SD (Seth Morris) on 2009/07/05

This is part 5 in the series The No-Spec Spec

Most likely, the first time you write an Assumptions piece, it will seem easy. The second time, it will seem very hard. And the first time you have to apply it, it will take courage and make a lot of people angry.

It’s worth it. Especially the making people angry.

The Assumptions document is a place to write down what should make you rethink the project: what things outside of the project could change the requirements, change the deadlines, or even suggest cancelling the project. You want to start this list early, before you are in the day-to-day give-and-take that makes everything seem more—or less—fluid than it really is.

Assumptions come in a variety of categories, but they all represent something that has to stay true for the project to proceed as planned. For each assumption, identify what will be affected and what process or work is invalidated if the assumption proves false1.

Common areas of assumptions:

  • Technology
    • The 3rd party dependencies are available on time
    • The 3rd party dependencies work as planned
    • Internal dependencies (libraries, hardware design, etc.) are available and work
    • New versions of tools and libraries integrate smoothly
  • Team
    • The team can focus on the new project as planned (rather than being pulled onto other work)
    • Key personnel remain available
    • New hires can be filled on schedule
    • Outside contractors perform as expected
  • Corporate
    • The corporate structure continues to support the project
    • Necessary resources are available to support the project (no emergency or suddenly-critical project consumes all IT support or asset creation, for example)
  • Business
    • Competitors behave as expected (with respect to the project)
    • Business partners remain committed (and in business)
    • No unexpected developments that affect the project (e.g., new entries into the project space)
    • The regulatory environment remains as expected
    • Company legal issues (patent claims, etc.) resolve as (and when) expected

When you write your assumption list, don’t be general. Don’t say "3rd party tools work as expected." List them, list when and where you expect to apply them, and list what they impact. When they’re written most usefully, assumptions are digital: they are either true or not. Don’t accept "well, it’s mostly true, so we don’t have to consider changes." Down that road lies failure.

An assumption comprises:

  1. A statement about the project’s environment that you can test as true or not
  2. A list of disciplines that should re-evaluate the project and what they need to re-evaluate. This can be as complete as "is this still the right project to do."

Here are some examples:

Assumption If this Assumption changes, re-evaluate these:
Congress will authorize <specific regulatory change expected>2 – Legal: List of likely implications
– Sales: Begin managing customer expectations and bring the results back to Business
– Business: re-evaluate if the project is still viable
– Product: review and sign off on the entire functional requirement
Ad sales will be remain our main source of revenue – Product: re-evaluate the design
– Business: re-evaluate if the project still makes business sense
Hardware development and production will occur on-schedule – Product: Features dependent on the hardware
– Sales: Commitments and schedules
– Product: Relative priority and urgency of other projects
Legal issues with design <whatever> will resolve by the time we ship – Legal: Advisability of shipping while in litigation
– Product: Can features based on that design be removed?
– Development: Other designs
– Business: Schedule vs. Features vs. Risk
Existing codebase supports the features with minimal maintenance – Development: Estimates
– Development: Maintain or rewrite
– Product: Schedule
– Business: Importance of new features
Products in production continue to function with minimal intervention …etc.
…etc.  

It is important to note: "re-evaluate" is not the same as "check that what we said is still valid." Looking at the old plans and seeing if they can be made to work is how projects spiral slowly into failure, always chasing one more fix to react to the world. Re-evaluate is just that. Look at the product, the project, and the business goals and decide what is needed; then compare that to existing plans. The whole point of the term "assumption" is that it indicates a surprise. Whatever you did before could not have been based on the current conditions.

An assumptions list is very much like a risk list. It is similar, but neither completely replaces the other. The risk list identifies things that might happen to affect a project and lists amelioration plans; it focuses on the things you most want to prepare for—the things you most expect and which you can tweak back under control. A risk list is an attempt to create a proactive situation; assumptions stay reactive. The assumptions list calls out things you do not expect to surprise you. These are generally things you cannot control. You put something on a risk list to make sure you pay attention to it. An assumption is something you don’t pay attention to every day, but when it calls for your attention you have to respond fully.

  Risk Assumption

Changes:  

Often Occasionally, maybe never

Respond With:   

Small changes Complete re-evaluation

Control:   

Internal External

Scope:   

Usually smaller Usually larger

When one of the assumptions changes, you don’t expect to make a subtle change. You get people together, demonstrate what’s happened, and ask, "From the current plan, what is now wrong? Not how to fix it, but what needs to be replaced?" You can work out a new plan when you know if the project is still worth doing.

So what do you gain? Clearly, most assumptions wouldn’t go unnoticed during a project (although some do). The assumption list gives two direct benefits and one powerful indirect one.

  1. It serves as a commitment to respond fully when circumstances change. Because many assumptions are large and pervasive, it is common to try and keep moving while reacting. The team keeps working on something they know isn’t right and the project drifts while plans are half-made and tried on top of each other. Often, no one wants to call something broken and recommit4. The assumption list provides a visible, binary test for when to withdraw and re-evaluate.
  2. It reminds the team that the whole project is affected. It is very easy for one discipline to believe that a problem doesn’t affect them5. Often, a faulty assumption isn’t fully visible to every discipline, and it’s as easy for the discipline most directly affected to overlook it as it is for a discipline very remote to the assumption. Pulling out the assumptions list and showing someone how many groups are affected and how significant an impact is can help you get people to give up a problem to the group or to agree to devote their time to a situation they didn’t think was important.

    …and…

  3. Thinking about assumptions makes for writing better requirements, designs, and code.

1. This is where the dependencies between the elements of the No-Spec Spec can help you, if you’re using all the parts. Often an assumption supports one piece of the process: the behavior, the mission, etc. If that needs to change, it’s relatively easy to see what other pieces are affected.

2. I worked for E-Stamp back in ’99. Being in a business where business plans (and technical plans that follow) are dependent on congressional action is incredibly difficult.  My heart goes out to those who try and my hat goes off to those who succeed3.

3. Obviously, E-Stamp didn’t succeed.

4. Engineering is usually the last to admit that something isn’t worth fixing. Consider a broken third-party tool or library. Most engineering groups will muddle along with it for months, causing delays and introducing quality risk, rather than announce that they won’t make it work and choose another route. Doing so turns a technical problem into one affecting every part of the project, but the problems often creep up in small increments and engineers never like to throw out something they might be able to fix.

5. Again, it’s as common for Marketing to decide to ignore a problem as being "Engineering’s responsibility" as it is for Engineering to try and convince Marketing to do so. I’m amazed at how often major problems in development groups are overlooked because non-technical people saw it first.


Listening to: Herb Alpert & the Tijuana Brass – Classics, Vol. 1 – The Lonely Bull

 

Posted in Dev Process | Leave a Comment »

John Moore asks: Why is it that the best bugs are found at the end of a release?

Posted by S4SD (Seth Morris) on 2009/05/04

I came across this on a LinkedIn group:

Why is it that the best bugs are found at the end of a release? « John Moore’s Weblog

My summary of Moore’s answers:

  1. Features are delivered late and QA time is sacrificed to keep the date
  2. QA isn’t able to test whole-system issues like performance and scalability until the end
  3. QA spends too much of the up-front time doing planning instead of testing
  4. QA test plans don’t match the real world
  5. QA sticks to test plans robotically
  6. QA can’t find the bugs that customers can

Mr Moore has some interesting answers to a really good question; it’s worth a second consideration. Let’s start with the question itself. I notice three immediate, interesting things:

   The term "best bugs" is a fun one. It isn’t clear from Moore’s post what qualifies as a "best1". Is it most subtle? Most careless/amusing? Easiest to fix? Most impact on the product? It could be any of these and retain the value of Moore’s post; it begs the very interesting question: What did you think of when you saw the term "best bugs"?
 
    There is a presupposition of universality in the question: it isn’t scoped to one technology, language, problem domain or development methodology. Is it universally true that the "best bugs" are found at the end? Always? All of the "best bugs"? If this is true in your experience, or true in your experience with one team/company, do you think it speaks more to the nature of software development, to the current state of the industry, or to a specific individual’s (or team’s, or company’s)
methodology? Do you think any testing or development process methodologies or tools address this (or try to)?

For that matter, do you think it is an inherent property of software? Is it a bad thing? A good thing? How good? How bad?

    In the list above2, the first two items are properties of the project itself: they’re project management issues. The other 4 answers are all ways of blaming QA’s methodology3. None consider the role that developers, project planning, and the general nature of programming play in the process.

I don’t mean to imply that John Moore blames every QA team in the world for making this a universal problem; he’s explicit that these are only some of the reasons for an observed phenomena, although he does describe these are the causes he’s seen most often4.

So let’s add some reasons that don’t implicate QA; everyone should get in on the fun.

Product Design, Marketing, Sales, etc:

  • Product managers deliver incomplete specs and don’t finish the specs in a useful order; important features aren’t fully specced until very late in the lifecycle, leaving those features to be (re)written quickly at the end.
  • Product management adds new features throughout the project, leading to features that don’t have the support of a full process of design, review, test planning, and inspection.
  • Product managers never paper tested the design and product owners change the requirements when they actually see what they asked for, leading to quickly-turned-around rework.
  • Sales or Marketing promise customers that they can see specific features early without the dev process being considered, restricting the ability either to front-load risk or front-load low-hanging fruit.

Development:

  • Developers front-load risk and believe they can do the "easy stuff" at the end, without any planning.
  • Developers take the easy stuff first and don’t give high-risk items attention until late in the process.
  • As schedules tighten, development cuts corners on code inspection and documentation, leading to less well-thought-out work.
  • Under the threat of a deadline, developers are likely to give less-tested code to QA, figuring they can save time on unit testing by risking the entire QA team’s time on a dead build.
  • Work that requires technology new to the team (new languages, new APIs, etc.) has much higher risk and can’t be delivered until later in the project, when the developers have had some time to learn.
  • Floundering developers are left on their own, causing their high-risk code to ship at the end or having it rewritten by more-senior developers at the end.
  • "Cowboy" features and fixes are slipped in at the end, when developers are buttoning up features.
  • Towards the end of a project, developers are focused on the next project and don’t give the current work their best effort.

Executive Oversight:

  • Any particular discipline in the process is likely to be pulled off on a random firedrill without the rest of the project understanding the full impact—often because no one but the executive could know how bad it is and the executive is in denial.
  • Features that are pulled out for a future release become must-have features for the current cycle, often causing them to push ahead with incomplete (or no) planning.
  • Project schedules are interleaved with no relationship between timeframe, risk, and importance—such as the most critical project delivering last, but taking resources from higher-risk projects that deliver first, causing those projects’ inevitable failures to impact the critical project and lead to last-minute attempt to catch up.

General Life in Software:

  • QA find the easiest and most-visible bugs first; the subtle stuff takes time to discover, reproduce, and document.
  • Simple bugs, particularly in the user interface, mask more interesting bugs in underlying code.
  • Early in a project, QA is only likely to get code that developers have very high confidence in. As deadlines near, code has to be given to QA ASAP instead of sitting on a dev box until someone has time to go through it again.
  • Interesting bugs are likely to cross functional boundaries, meaning they can’t be discovered until several functional areas are delivered (and working well enough to see them pipelined).
  • At the end of a project, when deadlines are tight and the team is pushing hard, exhaustion overcomes the team, leading to bugs in the more complicates systems.
  • At the end of a project, when deadlines are tight and the team is pushing hard, exhaustion overcomes the team, causing subtle bugs to take longer to fix and generate more false-fixes, further increasing their apparent complexity.

In short, everyone is susceptible to basic, human behaviors that lead to the "best bugs finish last" situation. It is fundamental to human nature and mitigating it is the #1 goal of every development methodology ever. It’s easy—and currently chic—to say that agile methods are focused on eliminating the "big ball of failure at the end of the project," and they can do a pretty good job of it, but don’t fall into the trap of thinking older (or newer) methodologies embrace failure. They have the same goal and make some reasonable attempt to reach it.

If I had to guesstimate, I’d say that development—in the roles of coders, inspectors, and estimators—is is responsible for the plurality of the last-minute "best bugs," with product management (in their role as "constant revisers of too-incomplete specs") being close behind. But really, all of these are a symptom of the fundamental problem of too many people trying to do too hard a job on too little time and with too little information. In other words, it’s all a symptom of human beings producing software in an entrepreneurial world.

We see it, we face it, we mitigate it, we plan for it, and we deal with it when it—inevitably—happens.

It’s what we do and it’s why we keep coming up with new ways to do it.

Thoughts?

Technorati Tags: ,,,

1) Moore’s post does say that these bugs are the "most interesting, and sometimes the most critical."

2) The list doesn’t match the bulleting or order in the original post; I have reordered them to group related items and I have flattened his nicely-nested bullets.

3) This could easily turn into a "developers versus QA" situation; let’s agree not to do that.

4) I certainly do believe that these are the causes Moore has seen most often. I do not believe these are the causes Moore has most often been exposed to. As a developer with a passion for SDLC and development process, I have realized that I am likely to see faults in QA’s methodology (without necessarily understanding the methodology well enough to be right) long before faults in areas I’m closer to, like dev process, coding, or design. I assume everyone, including Moore, has similar biases.


Listening to: Herb Alpert & the Tijuana Brass – Classics, Vol. 1 – Tijuana Taxi

 

Posted in Dev Process | 1 Comment »

The No-Spec Spec: Purpose

Posted by S4SD (Seth Morris) on 2008/10/09

This is part of the series The No-Spec Spec

"Purpose" Description

The Purpose document is the user- and client-focused The purpose document is fairly close to an MRD (Market/Marketing Requirements Document). If you have a good MRD, you could call it the Purpose and move along, but the chances are that a team that can’t get a good spec also can’t get a good MRD.

The Purpose document may be somewhat longer than the Fit and Identity pieces, depending on the level of detail and size of the project. It should answer:

  • Why will a user use the product?
  • What problem will the user expect the product to solve?
  • What business process (or user process/goal) is the user trying to accomplish when using the product?
  • If the customer is different from the user, what is the customer’s goal?

These seem very basic, but they form the reality check for functionality. Late in a project, these are the ultimate test for any change in requirements. Referring to these can prevent most late-project schedule- or quality-damaging changes and almost all developer gold-plating.

This is a simple document. Hopefully the product managers can produce it in 30 minutes or less and hopefully it contains less than two pages of text. It may be longer if the Purpose requires images or background information to explain1, but the critical user and client goals should be relatively short.

Change rules:

    If the Fit changes, this and all other documents need to be rewritten. The Fit may indicate that the user, customer, or business problem has changed.
      The Fit may change without affecting the Purpose if the business model or the business environment changes but the user doesn’t. This is a good thing. It means that even though the project relates to the company/team in a different way (and presumably the old way was wrong), but the problem the project was trying to solve was correct. The need still has to be filled (and all the work up until now is still valid).

    If the Identity changes, the purpose needs to be re-examined. The Purpose is the how to the Identity’s what—and to some degree to the Identity’s why. If the product as a whole is different, that probably indicates that the goal of the product has changed.
      If the Identity changes and the Purpose doesn’t, that indicates that the Identity was incorrect to begin with, but again the problem the project is solving was identified correctly.

If the Purpose of the project changes, you need to re-evaluate each of the:
    Assumptions: What is true about the new problem or value-add that wasn’t true before?
    Environment: Does the new Purpose indicate a change in where, when, or under what conditions the product will be used?
    Capability: Is the new Purpose part of a different larger framing context?
    Behavior: What is the new functionality? This is the item almost certain to change if the Purpose does.

1) Examples of projects that might require additional detail in a Purpose document would be financial software that should include background information on the behavior of the financial instruments, kiosk software that may require screen mock-ups to indicate process flow, or software that cannot assume literacy in the localized language and requires special interface elements.


Listening to: King Crimson – In the Court of the Crimson King – Moonchild/The Dream/The Illusion

 

Posted in Dev Process | Leave a Comment »

The No-Spec Spec: Identity

Posted by S4SD (Seth Morris) on 2008/09/24

This piece of the No-Spec Spec defines the what to the Fit document’s why. It is generally owned by the product or program manager and should answer the question "Does this feature/process/change belong in this project?"

It is amazing to me how often the answer to that question is "no" and the company doesn’t discover it until after the feature, process, or change has been implemented.

Again, this should be short. It can be as short as one sentence (such as McConnell’s Vision Statement: "A description of the highest-level objectives of the project") or it could stretch to a page or two, with one section (the "is not" list) potentially growing continuously.

These are the parts you may want to include.

Vision Statement – the key objectives

This is straight from McConnell and the SPSG: What are the objectives of the product and the project? This can include all of the company’s business goals, engineering’s technical goals, the user’s business needs, etc. Keep this to simple statements, bullet lists, etc., and avoid "fluffy" words.

There is all sorts of good advice out there on goal definition. SMART—Specific, Measurable, Attainable, Realistic, and Time-bound—for example, is popular, simple, and useful. I’ll just add/highlight a couple that are often missed.

  • Initiated and maintained by the company or team

    The hope that "our major competitor loses market share each quarter through the new year" is a great wish, but not a practicable goal.

    Initiated by the team means that the ability to make a change (release software, fix bugs, upgrade technology, …) exists within the organization. Outside resources may be necessary; if so, the organization should have the ability to go get those resources.

    Maintained by the team means that the goal has to stay within the organization’s power. This may mean coming into conflict with the company’s history and maturity around follow-through. If only internal causes threaten maintaining the goal then it still meets this criteria (although the chance of success may drop).

  • Has an explicit test for achievement

    If you want to "Improve performance of the system," you need to include a way to test that kind of performance. You don’t have to state how much the project wants to improve performance, unless that is a known goal, but you do need to indicate what kind of measurable performance. Even semi-subjective measurements can be acceptable ("most users in the survey group agree that it feels faster" is a fine goal), but goals that don’t have a way to test are open to misinterpretation.

    I worked on one project to update the base drive image copied onto all new units a customer shipped. We spent weeks tuning the OS install, tweaking the application setup, and researching the data most likely to be useful pre-loaded. Then, being engineers, we went about making the process repeatable: writing scripts to recreate the base image, writing queries to isolate the necessary data, and so forth. The product manager came to me and expressed concern over the time it was taking, especially since he had seen it basically running in the dev pit. I explained and showed him the project plan (which he had seen, but had not discussed in detail) and he asked me a great question: What was our deliverable? I responded that we were giving him a process that produced his drive image reliably and that could accommodate changes to the drive image in days instead of weeks.

    He replied that the image was updated once every 3-4 years and all he needed was a hard drive to stick in a duplicator. The last couple of weeks of the project were useless to him and the repeatability we were building was probably going to be outdated by the next time they tried to do this. Throughout the project, we described the project as "Recreate the source image." Stated as a testable goal from the product manager’s point of view, it should have been "Have a hard drive to duplicate on new boxes." We would have been done weeks earlier.

The Is/Is-Not List – project scope and boundaries

Invariably, someone wants to add something to a project. It’s a normal and often useful instinct: this train is rolling, let’s throw another bag on instead of having to start a whole second train. Some of these will be great ideas; some will be feature creep or attempts to change the product direction.

The Is/Is-Not list works to counter this by listing what is explicitly included in the product and project, by explicitly listing exclusions, and by keeping a hard and clear line between the two.

At the start of the project, you probably know what the project is to some reasonable degree:
– …is the follow-on to version 3.2
– …is minor UI enhancements to almost every screen
– …is the new data layer (formerly code-named "Skippy")
– …is data integration with SQL Server 2008
– …is new charting modules for for the Analytics module

Note that the Is/Is-Not doesn’t have to be as well-structured as the Vision should be. Is/Is-Not should contain the colloquial shorthand your teams use. Feel free to use code names if people like them, or those "cute" names that pop up at every company now and then. If everyone knows what "finally implement Special Sauce" is, giving it another, more detailed name doesn’t do anyone any good at this stage. This isn’t a spec: it’s a list of the areas that are being opened up for change and the kinds of change that are in scope.

At the start, the Is-Not list may be very short, but in practice it is often as long as the Is list.
– …is not implementing the UI redesign hanging in Marketing’s conference room
– …is not a rewrite of the Analytics module flow
– …is not refactoring the GUI engine
– …is not the "Auto-Advise" feature

List the projects, proposals, and pipe dreams that float around often enough and have active champions that you know they might try to shoe-horn into this project1.

As the project proceeds, every time a large change is rejected, add it to the Is-Not list. You don’t want to add every little change or bug, but do list things people feel passionate about—those are the ones that will keep coming back if you don’t do something with them.

In addition to cutting off discussion on revisiting changes, the Is-Not list helps engineers keep their creep and gold-plating closer to scope. But the major value is in giving feature advocates a place to "park" their ideas2. The Is-Not list is more visible and less cluttered than a product backlog, a feature database, or whatever you’re using to plan future work. It ties the proposal to a project and gives the project postmortem/lessons learned a place to see what was excluded and make recommendations to include specific things in the next project.

As features are cut, especially partially- or completely-finished features, definitely add them to the Is-Not list.

Using the Identity document

This doc should be short enough to post on a wall somewhere, but even if it isn’t you should put the project vision everywhere you can: add it to the footer in templates for project docs, put it on the wall, whatever. The Is list shouldn’t be nearly as detailed as a functional spec, and not as detailed as most MRDs, but it is a good start for project planning; development and QA can often start to rough out some plans from it (a good place to start Sliding Window Scheduling).

When the project is closing down, the Is/Is-Not list comes back into the forefront as a check that the project did what it intended to and no more. The Is-Not list also serves to suggest changes that might be important enough to implement soon.


1) If you do decide to make a change and include them, just edit the doc and tell every one. One of the wonderful things about docs is how easy they are to change. Keep a history if you like (highly recommended, actually), but nothing is carved in stone unless you have a very, very specialized plotter.

2) This is similar to the "parking lot" often used in classrooms and at meeting. It has a great effect on people who have a visible indication that their request has been heard and caused action, even if it isn’t being enacted immediately.


Listening to: Steely Dan – A Decade of Steely Dan – Deacon Blues

 

Posted in Dev Process | Leave a Comment »

The No-Spec Spec: Fit

Posted by S4SD (Seth Morris) on 2008/09/20

This is part of the series, The No-Spec Spec.

The Fit document is the easiest to create, the hardest to describe, and the most pervasively important piece in the No-Spec Spec. The project’s Fit tells you why you’re doing the project at all.

This is a job for the highest level authorizing the project, or at least the highest level you can get. On a large initiative this would be the executive sponsor, although a head PM or someone in Sales could do it. On an internal feature or a drill-down piece of a larger project it could be a PM or a member of the development organization.

It should be short. Half a page is just fine; more than a page suggests that someone is going off into inappropriate detail.

What goes in the Fit

Answer these questions:

  • How is this project related to previous projects, strategically?
  • What business or organizational purpose does the project meet?
  • What future plans will this project support?

If this is a large product initiative, the Fit should tie the project to the company’s goals, strategy, and business purpose. If you’re on an internal project, the Fit should describe how the organization’s mission within the company relates to the project. This is the place to reiterate that internal tools are never the product; even for an internal tools project, the Fit should reach all the way to the organization and company’s purpose.

Put another way, Fit is about what is larger than the project and about how the project fits with that. This is an exercise in "chunking up" and seeing the larger environment that contains the project.

A project’s Fit shouldn’t change. If it does, something has fundamentally altered in your whole purpose. Reevaluating the project and its priority becomes the #1 job.

Using the Fit

The Fit is the ultimate criteria: it is the background which allows the project and product to exist. Pin it up somewhere. Any time someone is off in the weeds, walk them over to the printout and ask them if what they’re doing is described on that paper.

It’s amazing how often looking at the reason for a project reminds people to back off, simplify, and propose features instead of trying to sneak them in.

Example Verbiage

None of these are complete Fit statements, but they give a good idea of the kinds of things that wind up in the Fit document.

A rev of enterprise product:

This product update will meet specific demands from 3 of our high-profile enterprise clients:
     <bullet list of demands and clients>
Meeting these on-schedule will raise our visibility in those companies and help the sales team reach new departments in the companies.
Increasing sales within these companies will raise revenue and increase the value of these clients as references.

A shrink-wrap product rev:

This project will get the product to back-of-the-box feature parity with competitor A, version N.
     <bullet list of feature checklist items>
Other aspects of the project should be listed as well, but not intermingled.

An internal feature from engineering:

This feature will reduce development time and number of bugs on all projects that use this module; it will increase QA and Operation’s visibility into the behavior of the system. Specifically, it will (a) reduce the maintenance time on future products by reducing ramp-up time when a developer has to use this module, (b) increase the granularity of logging, and (c) resolve several long-standing bugs that have hampered new feature implementation.
<List the bugs and what resolving them will do for the team overall>

A drill-down piece of a larger project:

This piece will enable features A and B by servicing requests from several features/modules, returning data they need to do synchronous processing. It will maintain state for several business objects and abstract all data storage details of these objects from its clients.
<List the client modules/objects and the business objects this module owns>

Summary

As you can see, the Fit piece is about something that includes or contains the project. It describes how the project serves the purpose of something larger than itself; it says why the project exists. Absolutely everything on the project should tie back to the Fit.


Listening to: Pink Floyd – The Piper at the Gates of Dawn – Interstellar Overdrive

 

Posted in Dev Process | Leave a Comment »

The No-Spec Spec

Posted by S4SD (Seth Morris) on 2008/09/18

Articles in this series:
     The No-Spec Spec: Overview
     The No-Spec Spec: Fit
     The No-Spec Spec: Identity
     The No-Spec Spec: Purpose
     The No-Spec Spec: Assumptions
     The No-Spec Spec: Environment
     The No-Spec Spec: Capability
     The No-Spec Spec: Behavior


Specifying without a spec

There are a lot of reasons why teams don’t get—or write—good specs: fast-changing product definitions, overloaded product managers, no time in the schedule for technical specifications, lack of training on specifying, communication barriers preventing getting information from other departments, etc. Some groups even eschew specs as inappropriate to agile development (although that actually means they have a too-narrow definition of both "agile" and "specification").

Over the course of training teams to create specifications, I’ve come to a humbling realization: some teams/companies just shouldn’t be writing specs yet. In the CMM parlance, they’re maturity level 0 teams/companies and don’t have the ability to produce an artifact as formal as a spec.

Other teams could, but a senior manager or respected developer whose identity is bound up in being "seat of the pants" or in "cowboy development" keeps the culture from supporting the people trying to implement some minimal process. This is often the case in companies very tied to the "cult of the programmer."

So here’s an alternate approach, based on multiple small artifacts. The main goals are:

  • Severable: The process can be implemented piecemeal, although some pieces are better with others finished as a prerequisite
  • Practicable: Each piece is useful on its own
  • Low-cost: No piece requires extensive time to produce
  • Pervasive: Every discipline, from executive sponsorship down, can be involved under their own initiative
  • Guerrilla-ready: No piece requires significant input from multiple disciplines
  • Produces living documents: Each stage artifact is open to revision and the consequences of that revision are clearly and immediately visible
  • Recursive: the process can be applied to whole projects and recursively re-applied to pieces of the project down to code-level design.
  • Supports a sliding-window:  You only have to specify the pieces you are working on next and can increase the level of detail as information becomes available

The No-Spec Spec is designed around Robert Diltz adaptation of Gregory Bateson’s notion of logical levels. Project artifacts are sorted into logical levels with an eye towards what sets the environment for what. If something fundamental changes, every artifact and decision below it needs to be re-evaluated. Changes don’t often propagate the other way, though, unless the change is due to some out-of-band change that should affect a higher level.

Here are the parts of the No-Spec Spec:

Level Describes…
Fit How the product/project fits with the company’s business

This is not usually included in specs. Making this explicit solves many problems that arise over trade-offs, the importance of functionality, the flexibility of schedules, etc.
 

Identity What the product is in simple language. This forms the core mission of  the project.

This may be included in an MRD, but companies that have trouble producing specs rarely produce MRDs. Those that do produce good MRDs rarely carry their value all the way down to implementation.
 

Purpose The business need/problem the product/project addresses. The value proposition for the project.

This is the user-centered version of Identity (and, to a lesser extent, Fit). This is the place to start talking about use cases and other customer-centric design methodologies that have become standard.
 

Assumptions Assumptions that underlie the project. If any assumptions are invalidated, everything in the project needs to be reexamined.

These should include assumptions about the business environment, competitors, technology, regulatory environment, team and company composition, other projects in the company, and anything else that would require re-examining the project from top to bottom.

There is a close relationship between assumptions and risks, but risks are those things you prepare for and monitor for change. Assumptions are generally completely out of your control and are black or white: they don’t have a risk level, although you may be able to estimate a current risk level that an assumption will be invalidated.
 

Environment What is the environment for the product? Is this for an office? For home? A kiosk in a bar? A PDA? A server product in a Fortune 100 IT room?

What is the hardware and software it is expected to run on (note that this is not the same as hardware requirements, but expected environment)?

What is the user’s experience before using the software? What are they going to do after?
 

Capability The breadth of the project. This includes elements of the product roadmap, the range of applicability (versions, add-ons, integration, etc.)

This is where canceled featured go to resurrect them in a subsequent version. For those of you using SCRUM1, this is not the same as a product backlog. This is for product planning outside the SCRUM process.

1 The project religion of the month when this was written
 

Behavior The main areas of functionality and what they do. This is the place to go into user stories/use cases.

Keep this to a level of detail appropriate to the rest of the artifacts. An Identity document describing a 20-developer, 6-month project as a whole should not be coupled with a Behavior document listing every error screen and log file format. Save that for another set of documents; otherwise, the Behavior artifacts are likely to hold up the rest of the work.
 

In general, change flows downhill and the rate of change increases at each lower level. A change in Fit means the entire project needs to be reexamined. A change in Environment means the capability is likely to change, which will in turn affect Behavior, but an environmental change is unlikely to affect Purpose. If it does, then the project Identity most likely changed without anyone noticing.

If any level is changing faster than the level above it, the project has a serious problem and is likely to fail. If the Assumptions don’t stabilize, you won’t ever get clear Behavior; if the Purpose keeps changing, then the bulk of the project is premature. Track how often these docs rev and note whether they are making major revisions and whether they are making changes in response to another change or if they are spinning their wheels.

Despite the fact that change flows (and grows) downhill, you don’t need—or want—to prepare these in order. Most of these documents capture and explicate existing information. If the executive sponsor doesn’t have time to write a Fit statement, you can and should still go ahead and define your Assumptions, Purpose, Behavior, etc. You will only need to change the other documents if the Fit surprises you—and you should reexamine what you’ve prepared anytime you are surprised by something more fundamental.

I’ll go into each of these levels and give examples in later postings. Let me know what you think.


Listening to: Blood, Sweat & Tears – Greatest Hits – You’ve Made Me So Very Happy

 

Posted in Dev Process | Leave a Comment »

Timeline Postmortems

Posted by S4SD (Seth Morris) on 2008/09/12

I wrote this article in 2004 for StickyMinds.com, the website of STQE (Software Testing and Quality Engineering) magazine*. It was reprinted in the April, 2005 issue of Projects & Profits magazine.

* The magazine is now called Better Software.


Timeline Postmortems

by Seth Morris

Abstract

We want to use project postmortems to improve our software process, but few teams do them. And fewer teams reliably learn from them. You can introduce postmortems to your team easily with timeline postmortem process. If you are already doing postmortems, a timeline-based approach may improve your results.

This process:

· Takes little time (a few hours or less of team time)
· Has a high degree of engineer acceptance
· Provides immediate practicable feedback into your development process
· Increases team cohesion and rapport
· Reduces finger-pointing

Postmortem Goals and Problems

You’ve got to accentuate the positive
Eliminate the negative
Latch on to the affirmative
Don’t mess with Mister In-Between

— lyrics by Johnny Mercer

Inevitably, on projects, something goes wrong. Ideally, something else goes right. On the next project, you want to prevent the one from reoccurring and reproduce the other. This is the goal of postmortems.

However, most teams don’t do postmortems. And the teams that do don’t always get value from them. Frequently, postmortems turn into "the blame game" or whitewash mistakes. A bad postmortem can create dissension and institutionalize failures.

A good postmortem builds team rapport, rewards project successes, and improves the next project after it. Good postmortems are rare, but they don’t have to be. Whether you’re introducing postmortems to a team that doesn’t use them, rescuing a failed postmortem process, or improving a successful one, a few simple steps will help.

The complete timeline postmortem process takes 3-4 hours of team time plus a little preparation for the moderator. The process produces results that are immediately applicable and focuses on low-hanging fruit. With it, you introduce incremental change that the team can adapt to rather than reinvent the entire development process.

In keeping with this philosophy, the process elements can be separated and introduced one at a time.

Process Outline

The basic steps of a timeline postmortem are:

1. Create the project timeline. This is a record of the “official truth” of what happened when, at a granularity that allows team members to see the entire project as a whole.

2. Walk through the timeline, identifying successes, failures, and "themes" of the project.

3. Group successes, failures, and themes by project phase. You will focus on process change one phase at a time. Don’t redesign the delivery phase if you haven’t made it past design; it’s just spinning wheels and it’s unlikely to be successful.

4. Adjust your software process to address the failures and recreate the successes.

5. Only implement process changes that affect the immediate future. Keep constant, public feedback on new process to inform the next postmortem. This also assures team members that their concerns will be heard and allows them to “disagree and commit” for the time being.

6. Revisit the postmortem when your subsequent projects enter a new phase. Finalize your notes on the ending phase and reiterate the process changes, if any, in the new phase.

Process Elements

If you’re mixing and matching, the key elements of the process are:

  1. Separate "what happened" from interpretation, and stay away from fuzzy "why" questions.

Creating a timeline of project events allows the team to talk about events and process without pointing fingers. Getting agreement on the timeline first, without any analysis, keeps the later discussions from devolving into "did not/did too" exchanges.

In addition, the timeline becomes the "official truth" for the team. Shared history increases "team-lyness." Just the act of creating the timeline can resolve esprit issues after a difficult project.

  1. Give equal attention to each part of the project. Postmortems can easily focus on the pain of the final phases of a project because that’s the most fresh in the team’s minds. A good timeline will remind the team of successes and failures from the beginning of the project, which are the most important to address while the next project is beginning.
  2. You want to do your postmortem right after the project is over, but you can use the timeline process before that if a project is stalled or failing. Waiting until another project is beginning gives some useful perspective.

On failing projects, creating a timeline sets up the recommitment meeting. It implies that any issues are part of the past and focuses attention on the results rather than the difficulties.

  1. Always have an impartial moderator/facilitator. This moderator only asks questions and writes down answers the team agrees on. The moderator should not be contributing opinions.

If your company doesn’t have someone impartial, hire an outsider.

If you have to fake impartiality, create a clear distinction between providing facilitating and providing your opinion. One good approach is to use standing and sitting: when standing, only ask for information and repeat what the team says. Sit down and wait a second before giving your opinions. Have someone ready to interrupt if you step out of role.

  1. Describe events and proposed process in terms of behavior, not intent. You can use the TOTE for this
          What is the trigger? (Initial Test)
          Who does what? (What is the Operation)
          What causes it to end? (Completion Test)
          What happens next? (After completion, Exit to where?)
  2. Address technical decision making as well as traditional development process. Be sure to discuss technical decisions in terms of criteria and results. This is an area to watch for finger-pointing and dogmatism.
          You can address dogmatism (and, to a degree, finger-pointing) by using clear project and development criteria. If you have clear criteria, put them on the wall during the postmortem. Actually, put them on the wall at the start of the project and just leave them there. Shared criteria makes for cohesive teams.

The Process

1. Preparation

To implement the process, you need some preparation:

· One impartial moderator

Too many postmortems are run by project managers or executive sponsors. The team members are unlikely to be honest about successes and failure in this environment. When the facilitator is not a manager, s/he may have a preferred software process methodology to push (dogmatism or “technical/process religion”), skewing the postmortem results.

If possible, get someone outside of the group—or even outside of the company—to lead the postmortem. The facilitator needs a basic understanding of development process, but doesn’t have to be highly technical or intimately familiar with the project. In fact, a degree of unfamiliarity will help.

»The moderator has final ownership of all postmortem deliverables, including the preparation.

· A (mostly) blank project timeline

Begin with some basic research.
      When was the project begun?
      When were major visible milestones, like first QA drop and final ship?
      Were there any personnel or company changes during the project?
      Major holidays?

» Put all of these on the project timeline to begin.

· Project phases

Use project phases to contextualize successes and failures. During implementation, focus on the current and upcoming phases.

Phase definitions will depend on your team’s development methodology.
      A typical waterfall project may have spec, design, development, and testing phases.
      An iterative project may have several iterations, with spec, design, and test in each one.
      Prefer to define phases that are generally present in your projects, rather than phases that will never occur again, but be willing to violate that rubric as needed. Be descriptive, not prescriptive.
      Phases may overlap or repeat. The QA group may be in one phase out of sync with development and it is likely that Product Management will have phases that differ from everyone else. Use phases that are useful to the groups they affect. This is the time for empirical, not theoretical, analysis.

During the meetings, the team may adjust the definition and dates for the phases. This is normal.
      The team is always right about this. Phases are a tool to help people group activities together; what the team considers related or grouped is related or group.

» Put a pass at the phases on the project timeline.

· Gathering the whole team

Get as much of the team as possible present for the first two meetings. This includes at least development, QA, and product management, as well as any support, release, documentation, or other groups as may be appropriate.

Managers and executives are useful in the first meetings, but be prepared to send them out of the room if people seem reluctant to talk around them.
      If you have to send a manager out of the room, do it openly in front of the team. Tell the manager that his or her presence is impeding communication. The team already knows this; acknowledging it increases rapport and openness. It may also be a useful lesson for the manager.
      If a manager refuses to leave, stop the postmortem. It will not be successful and any good ideas that come out of it, but do not come from the manager, are unlikely to be effected.

2. Timeline creation

» 60-75 minutes. Whole team.

The meeting goal is to establish the project timeline.

1. Put the timeline on a whiteboard where everyone can see. Leave plenty of room for adding events and activities.

2. Establish the major dates for key events: specs begun, estimates delivered first code fork, first delivery to QA, project completion (if possible), etc.

3. Get buy-in on the project phases.

4. Add project activities and events. Use language like, "During the design phase, what else happened?" Mark what features were affected when.

5. For each event or activity, ask "does everyone agree that this happened?" Watch the team when you say this and look for nods or head shakes. Agreement is more important than precision.

Keep this to an hour. An hour should be sufficient to get a level of detail appropriate to the length of the project.

Do not include any analysis in this meeting. No one should say "why" anything happened, or state any consequences of anything. When someone starts (and they will), interrupt them.

3. Project analysis

» 60-90 minutes, whole team.

You can run the analysis meeting right after the timeline meeting, but put at least a 10 minute break in between and make everyone stand up and leave the room. Get yourself a drink of water.

Lead the team through the project timeline from start to finish, phase by phase. Identify key themes, successes, and failures on this project. Start at the beginning of the project if you can; don’t let the team use up all the time on the phase most recent in memory.


Successes? Failures? Themes? Let’s define some terms:

Successes

A project success is something that reduced risk or improved the project’s result.

Failures

Failures increased risk or reduced the project’s success. Definitely include failures that are repeated from project to project. Those are the best to address.

Some teams don’t like to use the word "failure." This isn’t the time to use words like "challenge" or "opportunity." If you need to use a different word, make up something, like "lamppost."

Themes

Themes are neither successes nor failures. Instead, they provide context for successes and failures that can’t be considered part of the timeline. Often, a success on one project would be a failure on a project with different themes.

» If the half the team thinks something was a success and the other half thinks it was a failure, then it’s probably a theme.


Key points to the analysis meeting:

· Work from the timeline.

· Ask questions by phase, but be prepared for people to offer comments on other phases.

· Keep separate lists for Successes, Failures, and Themes. I use three flip charts taped next to the whiteboard, preferably on different walls.
      » If you use flip charts and whiteboards together, use whiteboard markers for both.

· Use behavioral questions only. "In this phase, what worked or didn’t work?" "What was different from previous projects?" Do not ask "why" questions.

· Successes, Failures, and Themes should all be described in terms of "When X, then Y, which caused Z." Ask for any parts you don’t get. If a team members offers "We didn’t follow the bug process," ask when and whether that was a Success or a Failure.

· Repeat every Success, Failure, and Theme before writing it down. Get agreement from the team before listing the item.

· Write down the exact words you repeated to get agreement. Do not "translate." Analysis, synthesis, and clean-up will happen later.

· Keep the meeting moving. Try to limit it to an hour.
» If you can’t get through in a reasonable amount of time, stop and either reconvene another day or finish the process on the phases you did complete. If that is useful, either convene another meeting on the rest of the project or abandon that and start with the next project, depending on scheduling and availability.

· Watch who doesn’t participate. Ask them directly if they have any comments. Language like "What worked or didn’t work for you?" is often effective. Be willing to wait in relaxed silence for an answer.
» Other good questions: 
      What did you do that wasn’t part of the official process? 
      What parts of the process did you skip or try to skip?
      What was a process step you trusted and didn’t have to worry about at all?
» Note that these questions all presuppose that there is an answer.

After the analysis meeting, deliver a document containing the timeline and the project analysis. Organize Successes, Failures, and Themes into project phases. Give the team a few days to digest it.
      This is the time for synthesis. Group Successes, Failures, and Themes into related sets and call out any that crossed many or all phases.
      Expect a small amount of feedback. This is not the stage where most team members will put in solo time.

4. Process design

» 60-90 minutes, all leads and manager plus all interested team members.

Get the leads and managers as well as interested individuals. This is not the meeting to force people who don’t want to participate.
      » Do try to get someone from every discipline.

1. Group Successes and Failures into any obvious categories: intra-team communication, scheduling, making technical decisions, etc.

2. For each category or item, identify the project phase to address it. Identify existing processes which affect it.

3. For each phase, develop new process or process changes to address the highest-value items.

When designing development process, keep in mind:

A process requires the TOTE:
· A cue to start
· An operation or behavior
· An end point
· Process(es) to switch to

» Write all of these down.
      Try to include a way to measure progress and any events that would call a recommitment or change meeting.

You may not be able to address every success or failure. Focus on phases and changes that will be immediately useful on current projects.

» Process will change
» No team follows every process it expects to
» No team follows every process the same way every time
Since you know this, don’t design in too much detail and let evolution happen. Evolution means something was getting in the way of work. Don’t fight it.

If the team has a split about a process, just pick one side. Let the team try it one way and change if needed. Be willing to be wrong and be sure the team understands that.

5. Implementation

When the next project starts–if it hasn’t already–you’re ready to implement new process.

Only implement pieces that affect the project phase you’re on.
      For example, don’t spend time re-tooling your release process before the specs are written.
      When each project phase ends, review the previous project’s postmortem and implement process for the new phase.

Write down new processes and post them in a public place.
      » People will be confused and make mistakes. Always refer them to a written description of the process.

If something doesn’t work, acknowledge it and move on. Make a change if you have to, but if you can muddle through with a broken process it’s best to save the redesign for the next postmortem.

» Always remember that the process is not the product. You have software to ship.


No matter how successful your current project is, I hope your next one is even better.

And may you never mess with Mister In-Between.


Listening to: The Players – Pippin (Original Cast Album) – Finale -The Player

 

Posted in Dev Process | Leave a Comment »