PeterSchuh.com

The two-headed, three-armed beast is slain. (See my previous post if you don’t get this reference.)

An interesting question arose the week before we began UAT. How do we decide when UAT is done?

There were several proposed answers to this.

• When all the test cases pass.
• When there are no open issues remaining.
• When there are no critical open issues remaining.

Or, my personal favorite:

• Starting Friday afternoon, put all the VPs in a room and let them decide. Repeat on a daily basis until the VPs all decide we’re done.

Isn’t that a lot like how they select the pope?

I pulled a page from agile (at least, from my book on agile) for the answer to this:

UAT is done when the users say they can and will begin using the delivered product in their daily work.

Sometimes I earn my pay. And sometimes people even listen to me.

UAT lumbered along for four weeks as the PM reported daily progress against existing issues and the new issues that had been uncovered. The users regularly said “it’s getting closer but we can’t trust the data yet.”

Sometimes things work out just about right - and this was one of those times. When it was clear things weren’t going according to plan, everyone ignored the arbitrary deadline. No one pushed the users to accept a deliverable that made them uncomfortable. The users and the team, meanwhile, communicated meaningful status on a daily basis. Everyone agreed on the goal, and could sense whether we were moving toward or away from that goal.

Finally, earlier this week, we got the word: “We’re ready to use it. UAT is over. And here’s the three things that still need to be resolved in a follow-up release.”

Again, the way it should be. I know there are sound reasons why you cannot always let users decide when they are ready to accept the delivered product and end UAT, but everything goes so much smoother when you can.

At present, I am watching a release cycle lumber into its fourth week of user acceptance testing (UAT). This release is the first tangible deliverable after five months of work by an out-sourced vendor.

The release plan - as developed by the vendor - included one week of QA (stomped) and one week of UAT (clobbered). Our project manager, to his credit, questioned those durations as inadequate the moment he saw them.

When a team codes for five months and throws that work product over the fence for testing, the team cannot efficiently field each issue that gets tossed back. Every new issue is likely to force a context-switch from what was built in month one to what was built in month four and back again.

Additionally, five months of development can result in layers of unreleased code, built one on the next, greatly increasing the number of bugs hidden behind two, three, or more other bugs. This is how big UAT transforms into a bad metaphor. It’s the repeated digging to find an issue, commission a fix, test the fix, resolve the issue, dig again, hit another issue, etc. When bugs are buried one behind the other the entire exercise becomes an exhausting and duration-consuming beast. And we’re living with that.

None of this is meant to fault the vendor (well, except for aggressive optimism on the release timeline). First, the vendor is very good at what they do - and what they do is different than what we do. Second, the nature of the work is not amenable to an agile approach (I know this from experience). While there are lessons to learn from this experience, there are no buildings being toppled nor people waiting for a two-ton-heavy-thing to drop on them. It’s not a crisis.

But it does remind me of a huge benefit that comes from agile development (in general) and short iterations (in particular). The team that releases code frequently has much greater control over the tesing, acceptance, and delivery process. Each short iteration keeps testing (and the issues that fly out of testing) to a manageable size. Finally, each small work product ensures that everyone’s head is in the same game, since the team that just completed an iteration’s functionality is immediately involved in its testing and review.

Scott Ambler started a minor brouhaha a couple weeks back when he rolled out his first draft of an Agile Process Maturity Model (APMM). I’m not stepping in to evaluate the proposal or the rebuttals because I believe that level of the discussion largely misses the point.

I’d prefer to rebut the whole conversation by reiterating the agile maturity model that I proposed in my book:

The team has decided to start implementing agile development.

The team has successfully implemented its first agile practice and has posted a copy of the Agile Manifesto in inch-high letters on a common wall near the team’s location.

The team has adopted a second practice while collecting metrics and holding review meetings to measure its level of agility.

The team is regularly adopting additional agile practices and actively monitoring its level of agility.

The team has not checked its level of agility for months and is altogether focused on adopting and adapting any practice that will result in better communication, higher quality code, and a system that most accurately meets the needs of the business.

My point? Maturity models are good for selling something, They’re not nearly so useful when your focus is (1) maintaining a healthy team and (2) delivering valuable software. That’s because maturity models simply cannot account for the myriad environmental factors, technical challenges and orgainzational obstacles that teams face. The team that focuses on the characteristics that most directly drive its situation will win out - every time - over a team that evaluates and corrects itself using a borrowed yardstick built from industry standards.

I must thank Costco for making me realize there’s a whole category of topics I need to keep an eye out for around oxymoron process (or, in this case, oxymoron policy).

As I’m sure some readers have witnessed, Costco really applies the hard sell on their executive membership, jumping you at the register with high tech scanners that tell you how much cash back you would get if you upgraded from a regular membership. So, last year, my wife and I broke down and took the bait, since we do buy quite a bit of our building supplies as well as other items from Costco.

(Note to self: if someone repeatedly jumps in your face to try to sell you something, no matter how good it sounds, just say no.)

So, over the last few months my wife and I have earned quite a bit of cash back on our purchases, This cash back isn’t really cash; it comes in the form of Costco script which must be used within six months. That’s fine, my wife shops at Costco often enough so she’ll just use the script coupon that we received in the mail the next time she goes to Costco.

Wrong.

The coupon is written out in my name, even though we are both on the account. Since it’s written in my name, only I can use the coupon at Costco. This happened today. I called Costco this evening and asked a manager WTF. The manager confirmed that this is Costco’s policy and it is to protect me in case I divorce my wife - so she couldn’t go to Costco and use the coupon with my name on it. (Really. You can’t make this crap up.)

Now that’s just a moronic policy (since my wife and I still share an active account at Costco). But here’s what makes it an oxymoron policy. My wife purchased everything today with our Costco Amex. That Costco Amex is in my name. And - wait-a-minute - my cash back coupon comes printed on the statements for that very same Costco Amex.

Guys, I could have just divorced my wife. I don’t give a damn about the stupid coupon. What about the hundred thousand dollar Costco Amex credit limit that you would have happily let my newly-divorced wife max out?

Again, I did speak to a “manager” about this, who did say it is company policy. I’ll probably follow up with Costco corporate tomorrow.

The Backstory

The other day, I was having a conversation with several people about culling a 200+ item backlog for a legacy application that is being sunset (and the successor of which is already live in some capacity). A business person was arguing that we needed to take the entire backlog into a meeting to vet all 200 stories individually to determine which still need to be implemented. There was a shorter, 70 item list that the business met around each week to prioritize. They hadn’t looked at the other 130 stories in months.

The application did have about a dozen stakeholders who’s business units could be affected either positively or negatively by the prioritization decisions that came out of the meeting. At the same time, to sift through more than 200 stories and decide which stay and which don’t, we were looking at four hours of meetings with several very expensive attendees. I argued that we should take the shorter list into the meeting.

I was in a position only to advise on this one, and I didn’t get my way. What ensued was four hours of - you guessed it - tedium. The majority of those stories had little or no intrinsic meaning to the deciders in the room. We spent a surprising amount of time guessing why something was on the list, how many years it had been there, who really wanted it, and why. Tellingly, the end result was a roughly prioritized list of about 70 stories.

My Take

Prioritization should not be a mind-numbing meeting (or, worse, series of meetings). It should be speedy, efficient and sometimes painful. But that’s the point, isn’t it? We’re prioritizing so we can make the best decisions possible with limited resources (which include our own time). Here’s what I would have done (and, for that matter, what I have done) in a situation like this:

1. Find a way to shrink the list to the shortest size possible. Throw out anything that hasn’t been discussed in two months or that has already been slapped with a lowish priority.
2. Publish that list to all concerned parties and make it clear that anything not on the list will not be prioritized.
3. Wait for people to scream. If they scream, add whatever item they believe is missing to the list.
4. Identify the product’s vision owner (this may be someone other than the product owner or customer) and make sure this person’s vote counts extra.
5. Gather together the stakeholders who matter and prioritize.
6. Publish the prioritization.
7. If anyone screams, register the concern and confirm that it is not based on a point overlooked by the prioritization group. If it was something the group overlooked and it has a measurable impact on the prioritized list, adjust the list accordingly.

Please read the next four sentences before letting the dogs loose on me.

I love wikis. Really. And I am not a SharePoint fan. Really.

Now, I only need four more paragraphs to explain.

First, a wiki is supposed to help us make information maintainable. One way we do this is by centralizing and restricting topics to one per page. In order to review and revise a topic in its entirety, all one needs to do is edit the topic page. All the links into the page automatically send readers to the latest revisions.

Second, a wiki is meant to make information more accessible. Topics can be linked to directly, from both within and outside the wiki. By linking to referenced topics at every opportunity, we allow the reader to enter into the conversation at any level of granularity. The reader can then follow links across topics to gather the necessary background information and focus on his or her particular area of interest.

Third, a wiki is an evolving conversation. Pages are created and revised by numerous authors to reflect the current understanding of and thoughts around a given topic. Many pages never, truly, leave a draft status. The goal is not to collectively polish. The goal is to communicate in a collaborative, efficient and effective manner.

Fourth, a wiki is NOT a forum where: (1) each page is a document instead of a topic, (2) the pages rarely link to one another, and (3) each page carries enough information to be consumed outside of the context of wiki itself.  If your wiki looks like this, then what you’ve constructed is a poor-man’s SharePoint (a wiki anti-practice). And your SharePoint wiki is not likely to thrive, since it has none of the benefits of a wiki (maintainability, accessability, consersational tone) and all of the drawbacks (clunky editing, clunky formatting, clunky or nonexistant permissions). If you want a navigatable hierarchy of polished, stand alone pages that do not converse with one another then you’d be better off using SharePoint and Microsoft Word (or open-source alternatives).