Posts Tagged “UAT”

The two-headed, three-armed beast is slain. (See my previous post if you don’t get this reference.)

An interesting question arose the week before we began UAT. How do we decide when UAT is done?

There were several proposed answers to this.

  • When all the test cases pass.
  • When there are no open issues remaining.
  • When there are no critical open issues remaining.

Or, my personal favorite:

  • Starting Friday afternoon, put all the VPs in a room and let them decide. Repeat on a daily basis until the VPs all decide we’re done.

Isn’t that a lot like how they select the pope?

I pulled a page from agile (at least, from my book on agile) for the answer to this:

UAT is done when the users say they can and will begin using the delivered product in their daily work.

Sometimes I earn my pay. And sometimes people even listen to me.

UAT lumbered along for four weeks as the PM reported daily progress against existing issues and the new issues that had been uncovered. The users regularly said “it’s getting closer but we can’t trust the data yet.”

Sometimes things work out just about right – and this was one of those times. When it was clear things weren’t going according to plan, everyone ignored the arbitrary deadline. No one pushed the users to accept a deliverable that made them uncomfortable. The users and the team, meanwhile, communicated meaningful status on a daily basis. Everyone agreed on the goal, and could sense whether we were moving toward or away from that goal.

Finally, earlier this week, we got the word: “We’re ready to use it. UAT is over. And here’s the three things that still need to be resolved in a follow-up release.”

Again, the way it should be. I know there are sound reasons why you cannot always let users decide when they are ready to accept the delivered product and end UAT, but everything goes so much smoother when you can.

Comments 1 Comment »

At present, I am watching a release cycle lumber into its fourth week of user acceptance testing (UAT). This release is the first tangible deliverable after five months of work by an out-sourced vendor.

The release plan – as developed by the vendor – included one week of QA (stomped) and one week of UAT (clobbered). Our project manager, to his credit, questioned those durations as inadequate the moment he saw them.

When a team codes for five months and throws that work product over the fence for testing, the team cannot efficiently field each issue that gets tossed back. Every new issue is likely to force a context-switch from what was built in month one to what was built in month four and back again.

Additionally, five months of development can result in layers of unreleased code, built one on the next, greatly increasing the number of bugs hidden behind two, three, or more other bugs. This is how big UAT transforms into a bad metaphor. It’s the repeated digging to find an issue, commission a fix, test the fix, resolve the issue, dig again, hit another issue, etc. When bugs are buried one behind the other the entire exercise becomes an exhausting and duration-consuming beast. And we’re living with that.

None of this is meant to fault the vendor (well, except for aggressive optimism on the release timeline). First, the vendor is very good at what they do – and what they do is different than what we do. Second, the nature of the work is not amenable to an agile approach (I know this from experience). While there are lessons to learn from this experience, there are no buildings being toppled nor people waiting for a two-ton-heavy-thing to drop on them. It’s not a crisis.

But it does remind me of a huge benefit that comes from agile development (in general) and short iterations (in particular). The team that releases code frequently has much greater control over the tesing, acceptance, and delivery process. Each short iteration keeps testing (and the issues that fly out of testing) to a manageable size. Finally, each small work product ensures that everyone’s head is in the same game, since the team that just completed an iteration’s functionality is immediately involved in its testing and review.

Comments No Comments »