The Map Is Not The Territory And A Wish List Is Not A Map (2)

The Limits of Prototyping in Agile Development

Central thesis of why software development is hard, and Part 1

So pretty soon in the late 90s it became common to find out what humans wanted out of computing systems by giving them a simulation of the system to play with, a prototype, which could come in all kinds of fidelities, from hand-sketched screens to, as the technology got better, clickable wireframes, to full front-ends. The craft of a UXer at the time was to be able to execute all these prototypes well with the tools available. The art was to fit prototyping at the proper fidelity into the software process such that you could find out the most, in order to decrease the risk of making the wrong thing as much as possible with the least resources. Sometimes the schedule would allow for a lot of time before development and you got to call that a “discovery phase”, sometimes you had to fit it into the Agile cycles somehow.

In a few engagements I even got both, so my team could make some outlandish mid-fidelity prototypes in Axure to run through with users and really elicit some deep thinking about their problems in the field we were working in, but then also do a broad test of the half-finished system mid-way through development to see if we were getting it right. The art there was to put the right user stories at the top of the backlog so you would have an unfinished but testable system halfway.

This is a notion that could fail in fun and unexpected ways. Like every map leaves something out of describing the territory, prototypes can’t be complete. For one effort, as we were in the UK, we put T&C stories at the bottom of the backlog, so to do after mid-way testing, because surely we didn’t need them to test the rental funnel? The prototype ended up failing testing in Germany because the test subjects insisted on thoroughly checking the T&Cs. And as I once had to explain to a group of stakeholders on another platform, we can’t prototype even more thoroughly to find all contingencies because by then you have basically just built the thing for a lot of money.

So prototypes are an answer, not the answer to dealing with the fallout from the rule

Humans can not accurately describe what they want out of a software system until it exists.

The reason that knowing when to use prototypes, and which, is an art and not a craft is because Agile doesn’t actually know how to deal with product design. Check the original principles: they do talk about design in one spot, but it is a given software developers just take one next step at a time and then check if it was the right one with the business people, and that is the full extend of the thinking about what Agile makes. How it is decided what that step is, and how to make sure you end up with a coherent system across the multiple touch-points at the end, is left as an exercise to the reader. So when these Agile edicts were translated into repeatable and teachable processes like Scrum or Kanban, fitting in designing the experience became a matter of how the team or department wanted to organize, and the UX field has been struggling with that since.

Especially when the development field when through a long phase of demonizing Big Design Up Front and deciding instead software creation was supposed to be about jumping right in and asking in tiny steps if what was made was right, with a lot of bright people advocating you could go from a two-wheel kick-scooter to a Porsche SUV in small cyclical increments, of which the first stage got called MVP and rushed out. And if the market was only ready for a Porsche, well, you’d better hope you found that out by having some really deep kick-ass user interviews and conversations about that scooter MVP, or some other channel, because you’d never find out from sending that MVP out on the web and checking the numbers. Quant doesn’t give qual answers.

User research through prototyping made a resurgence, but flattened to be a repeatable and teachable process, called Design Sprint, to be only about asking people on the street what they want with half sketches, in a cycle that is only allowed to last a week. The rest of the knowledge to create a success has to come from… hunches from the product manager? Marketing? In my last job it was edicts by stakeholders, when it should have been customer service. Pulling all these signals together is the synthesis-between-departments glue UX Research and Product Design should really be bringing now and are often not empowered to do or can’t because they are stuck in cycles.

As Joanna Weber writes in this brilliant article about why Agile and Lean are such difficult fits in organizations that are vast and actually have to be trustworthy, coherent, and good: “If Scrum only worked for as long as there were waterfall systems in place to support it, we need to replace both with something that both acknowledges and improves that reality.“

And the reality is rule nr 1 above, and that

Humans can not accurately predict how long any software effort will take beyond four weeks. And after 2 weeks it is already dicey.

So that replacement has to stay incremental in nature and show a lot to users at every step. It’s a tough situation and it has been true for years: we still do not have a repeatable, teachable process to make great software systems that span multiple touch-points and are a joy to use and maintain. We have to navigate between speed of incremental delivery and allowing time for thinking for design.

Right now there are roughly three fundamental ways in which design fits into Agile of various forms, Sprint 0 (which can be Big Design Upfront), Sprint Ahea d, and Parallel Tracks. Of these, Sprint 0 and Sprint Ahead are the ones I am finding the most, with Parallel Tracks, that could combine research and design into a very strong customer experience proposition, seeming the least popular, mostly because “devs want their designer embedded for synergy and speed”.

That should change, though. While UX Research and Product Design currently have an employment and cvredibility crisis, I recently did some prototyping with new tools that make me think there’s a whole new direction to go here. But this is already too long, so I will describe my ideas for the future next week.

The Map Is Not The Territory And A Wish List Is Not A Map (1)

“Where have all the task decompositions gone?” I was talking to a very experienced Head of UX about the state of our vocation when she asked that. I had to agree I have not seen one in years either. A task decomposition is when you take a task and divide it into steps, and then divide those into smaller steps, until you reach some granularity that makes sense for why you are doing this, like making screens or coordinating robot movements.

We used to do them all the time in UX, mostly when the field was still called HCI, to make sure we understood what the human was doing before we taught the computer to help them with it. There were many notation systems for them, and you could write PhD’s on comparing these notation systems and then inventing new ones.

Also something I haven’t seen in a decade is a specification full of descriptions of features that a system SHOULD and MUST and COULD have. The were called Functional Requirements, and while they often tried not to impose a view on how the system should look to users, you could tell how desperate the writer was trying to convey their needs in something else than fuzzy human language when they invariably started to use Word’s shapes tool to mock up screens—and then write the word SUGGESTION underneath so as to not offend their designers.

TDs and FRs are a relic from the waterfall period when you did a lot of design and understanding up front to make sure you were making the right thing for people before you committed the programming resources to make it. They were intrinsically incomplete in the same way a map always leaves things out of its description of the actual terrain, and expensive to make, and of limited use because:

Humans can not accurately describe what they want out of a software system until it exists.

Bit of an issue.

Rule nr 1 is and was true all the time. You’d computerize a workflow of paper files in a shop or local government and at the end it would turn out that there were all these exceptions being made by clerks and admins using different color pens or writing in the margins that all the workers understood, but nobody above them working with IT did. The exception would be so important you’d have to retool the whole thing down to the tables in the database, and the project would be late and expensive.

When Agile originally said to deliver value frequently, it wasn’t to unlock money from customers cycle after cycle—that wasn’t even really possible until we started putting everything on the instantly monetizable web. It wasn’t for investors either, they will happily wait years for a return if the projected return is big enough. Agile wants frequent releases so you can show the results to humans fast and get feedback and then correct, instead of finding out when you deliver the whole thing after two years that rule nr 1 above always holds. It’s only around the time Lean Startup came along that every iteration wasn’t just to correct the course but also had to deliver some new mini-feature every time.

So if you want to replace Agile Scrum or Kanban with something, you have to deal with the fact that 40 years of trying to first find out how people work, and then making wish lists in all kinds of notations of how that work is to be done by computers, never was really successful and often a total failure.

Still, adding functionality bit by bit as you explore what is needed comes with things you should be aware of:

The resulting system is kludged together cycle after cycle, unless you take some choice time between cycles to refactor huge chunks. This is why every seven years a software team wants to just start over, they can’t take doing archeology in all those cycles of hacks anymore and don’t feel they can add any more functionality without watching the tower of hacks fall over.
It’s actually not faster than Waterfall. It just decreases the risk of ending up with ~~garbage~~ a sub-optimal product-market fit.

But, but, but, if ~~wish lists~~ specifications didn’t work because making software in itself changes the work the software is supposed to help out with, what about prototypes? That worked, right? Yes, with a list of caveats including that Agile actually doesn’t know when to use them and that AI-derived UX is deeply changing that game in the last 6 months, but I’ll discuss that in part 2.

The Two Rules Of Software Creation From Which Every Problem Derives

Scrum has been having a bad time for the last ten years, and thus so has Agile. My favorite article on this is truly exhaustive about all the problems we have encountered in the last two decades trying to deliver software of any kind using these methodologies. (I know nothing about the writer, this could totally be a Milkshake Duck experience; some algorithm just recommended this insanely long post to me one day and I went “Uh huh. Uh huh. Uh huh. Uncharitable but I can see where they are coming from. Baby and bathwater but yeah. Uh huh.” It’s been on my open tabs for 4 months now waiting for me to write this.)

Thing is, I remember the before times. Waterfall, short or long. I delivered projects in those systems. In fact, the first time I encountered Agile in a company, I was new and young and thus stupid enough to not ask what that was about and thus very confused until I looked up where it came from. I remained very confused how you get from the statements in the Agile manifesto (seriously, check them out again) to the rituals of Scrum like stand-ups, retros, pointing and everything else that makes programmers so angry they only get to program 50% of their day and have to talk to other people otherwise, until I actually did some work in Agile Scrum and understood what it was trying to do.

None of the critics are offering real alternatives, just modifications of Scrum to fit Design or Product management in. I don’t think anyone wants to go back to Waterfall, but they can’t really explain why not.

I can. With two rules (which may become more as we discuss them).

It’s the two rules that actually are behind every statement in the agile manifesto. The manifesto unfortunately doesn’t name them really; the people behind it were so steeped in the problems of software delivery—and what they thought would fix it—that they posited their statements without saying why each of these things are necessary to deliver good software. (Unfortunately, necessary but not enough for success, but that we found out in the next decades.)

They are

Humans can not accurately describe what they want out of a software system until it exists.
Humans can not accurately predict how long any software effort will take beyond four weeks. And after 2 weeks it is already dicey.

That’s it. Every other problem that you have to solve in software delivery rolls out of these two major issues, I think. I may be wrong, you may need a third or fourth.

Am I going to spend time proving these two correct? No. There’s enough literature out there, most of it documenting failed software efforts in Waterfall in the 60s, 70s, 80s, 90s, to support them, and I am not going to go over that. What I will, in the next few days, is go over their implications, how they led to current Agile practices, and how they can not be ignored when you want to make things better.