The Rest of the Company Can’t Handle Agile

Medium for some reason thinks I continue to want to read about the demise of Agile project management methodologies for making software. My daily email has headline after headline by Product Managers and Engineering Leads writing why Agile—by which they usually actually mean Scrum, a framework of Agile principles turned into a repeatable, teachable methodology—is over.

(I am wondering if, for balance, some other designer who is heavily into Waterfall is getting a ton of headlines about why Agile is the best thing ever and will never die.)

The articles always discuss the shortfalls the authors think comes with Scrum with its short cycles and loads of coordination, but I have barely ever seen discussions of what I consider the actually most painful problem with Agile: the rest of the company can’t handle it. To put it as short as I can:

All our best methodologies to create successful software successfully—Agile, Lean UX, Outcome-oriented delivery—say that every software team needs to be in constant learning and adaptation mode, which means that delivery is now in an R&D model. But R&D is fundamentally unpredictable and most companies are not set up to live that way.

It becomes impossible to plan marketing for a launch, to budget commercials, to have an announcement for a conference, to set up your suppliers, to keep relationships with your partners, to project the ROI to finance and stakeholders, or in any way to hit your internal targets when every week or two the direction, delivery time, or the projected features of the digital product can change.

Inside anything but a small start-up, functions and departments depend on each-other and end up depending on what features and forms are on the website, and yet the software industry at all levels advocates not committing to both a time and scope of work beyond 4 weeks because predictions that include both time and scope beyond 4 weeks overwhelmingly turned out to be wrong in the last 80 years. That’s a really tough message to accept when you depend on that software. Every other discipline can deliver to time and scope, but somehow software can’t? Indeed it can’t, actually, but to truly internalize that you have to have been a software engineer who repeatedly has blown past deadlines you committed to with full confidence just a few months ago.

(Software, but see also residential bathroom remodels. I have never heard of one of those being on time and budget and on-plan either.)

It’s difficult for agencies

Now try being a digital agency, whose model depends on promising clients certain functionality by a certain date. The dance I have seen account managers do with clients to first tell them user and market research may uncover things they absolutely do not want to hear, and then that delivery of what they agree to make might have a large margin of error and thus cost (and never downwards), has been a sight to behold. But in the end, companies use agencies to lower their risk and so they will insist the statements of work eventually specify time and scope of what will be delivered — because who wants to spend money and not know what they are getting when? Most agencies will thus sign everyone in their ranks up to doing fake Agile: everything made in little chunks but with a fixed end date, leading always to overtime.

The best way I ever saw this managed when I worked agency-side was how an account manager, after a lot of discussion with the client and a number of smaller engagements where we had proven ourselves, negotiated that the client would only purchase weeks of time of a certain team. We got away from billing for finished pieces, features, screens, separate deliverables; instead we agreed together on what outcome the client was after, settled on an approach, and would follow design and delivery for it, with a lot of touchpoints to exchange progress and feedback and agree on course-corrections as we learned. We continued to work hard to keep the trust that allowed us to stay within this model that got us away from endless negotiations about milestones and estimates, while staying true to the Agile principle of committing long-term only to time or only to scope, not both—in this case, time.

And businesses that have been around for a while

Inside companies that make their software in-house, this boundary between the Agile and the more traditional parts can get especially painful if it that company has a centralized software group to make the websites and apps that other divisions depend on. This is the model you see most often in companies older than the Internet, who had to learn how to make software later; the company tries to control the unpredictability of creating software by keeping it isolated and concentrated in one place.

This boundary between the Agile software delivery side and the departments relying on features is usually a beleaguered Product Manager who is constantly trying to figure out what the priorities really are this month while also trying not to over-promise anything. They have to handle increasingly pressing questions from department heads about why this thing they need is not live on the website yet and when are we enabling this new product category in the CMS, in between having to convince stakeholders about what the latest tests and data uncovered, to suddenly have management believe your findings (without crediting you) and change direction and expecting results yesterday while the budget stays the same because budgets are set for the year. The Product Manager feel they aren’t really in charge of their product, the other departments feel they have no control over their future, and the C-suite doesn’t understand why everything is so slow and everyone is so defeated and hires another COO to clean things up.

The solution: pushing the Agile boundary up and out

I honestly believe that the only solution long-term to this is the opposite of having a specific IT department, but instead to push software-creation deeper into the company. As I evangelized inside one of the place set up this dysfunctional: “Do you think the business group inside Facebook that manages the friendslist writes up their feature priorities for the year and then submits them to a some central programming group? Sitting there, just hoping their features get prioritized over the needs of the calendaring business group, perhaps shouting louder on every call with the central programmers to get what they need? Of course not, inside Facebook everyone whose outcomes depend on what is on the website gets to program that little piece of the website with their own team — last I heard 7000 teams could push changes to the website. Do you think Spotify’s playlist recommender is a separate business division begging for time from the central Spotify programming teams? Of course not, the Playlist business group makes their own Playlist features. Yet here we are in this platform division of [this company I was working at] trying to juggle a backlog of stories listing what 5 different internal product divisions need from our web platform and asking the VP above them to please set the priority so we can exist without always having at least 4 knives in our back.”

It takes tons of coordination to keep a large digital service coherent when so many teams can push things to the web and apps, but we have structures and procedures for that, and at least it lets all the teams in the company chart their own course. But yes, it does mean a lot more people throughout such a legacy business have to learn about making software. It means that a lot of people who signed up 20 years ago to do, say, production chemistry or business admin, now due to their successful careers in management need to lead software efforts for their division and learn very quickly what difference is between an MVP and an MVT. And lot of people are instead more comfortable leaving that to a separate software group and then bitching at them.

A stunning amount of workplace friction, if not outright toxicity, comes directly from this misalignment between traditional corporate structures and Agile ways of working. It remains to be seen whether AI-assisted software engineering is going to change the unpredictability of making software or the unpredictability of what users will use; all signs are that AI will deliver more of everything per cycle — more screens, more prototypes, more code — but no AI prompting will compensate for not knowing what the market wants or what bugs hide in back-end integrations with your new code. It will remain necessary for software system creators to show their work often, to users, to stakeholders, in order to get feedback whether they’re on the right path. You can call it something else than Agile if you want, but the principles won’t change, and thus neither will the need to do the hard work inside the rest of the company to align everyone around it.

The Map Is Not The Territory And A Wish List Is Not A Map (2)

The Limits of Prototyping in Agile Development

Central thesis of why software development is hard, and Part 1

So pretty soon in the late 90s it became common to find out what humans wanted out of computing systems by giving them a simulation of the system to play with, a prototype, which could come in all kinds of fidelities, from hand-sketched screens to, as the technology got better, clickable wireframes, to full front-ends. The craft of a UXer at the time was to be able to execute all these prototypes well with the tools available. The art was to fit prototyping at the proper fidelity into the software process such that you could find out the most, in order to decrease the risk of making the wrong thing as much as possible with the least resources. Sometimes the schedule would allow for a lot of time before development and you got to call that a “discovery phase”, sometimes you had to fit it into the Agile cycles somehow.

In a few engagements I even got both, so my team could make some outlandish mid-fidelity prototypes in Axure to run through with users and really elicit some deep thinking about their problems in the field we were working in, but then also do a broad test of the half-finished system mid-way through development to see if we were getting it right. The art there was to put the right user stories at the top of the backlog so you would have an unfinished but testable system halfway.

This is a notion that could fail in fun and unexpected ways. Like every map leaves something out of describing the territory, prototypes can’t be complete. For one effort, as we were in the UK, we put T&C stories at the bottom of the backlog, so to do after mid-way testing, because surely we didn’t need them to test the rental funnel? The prototype ended up failing testing in Germany because the test subjects insisted on thoroughly checking the T&Cs. And as I once had to explain to a group of stakeholders on another platform, we can’t prototype even more thoroughly to find all contingencies because by then you have basically just built the thing for a lot of money.

So prototypes are an answer, not the answer to dealing with the fallout from the rule

Humans can not accurately describe what they want out of a software system until it exists.

The reason that knowing when to use prototypes, and which, is an art and not a craft is because Agile doesn’t actually know how to deal with product design. Check the original principles: they do talk about design in one spot, but it is a given software developers just take one next step at a time and then check if it was the right one with the business people, and that is the full extend of the thinking about what Agile makes. How it is decided what that step is, and how to make sure you end up with a coherent system across the multiple touch-points at the end, is left as an exercise to the reader. So when these Agile edicts were translated into repeatable and teachable processes like Scrum or Kanban, fitting in designing the experience became a matter of how the team or department wanted to organize, and the UX field has been struggling with that since.

Especially when the development field when through a long phase of demonizing Big Design Up Front and deciding instead software creation was supposed to be about jumping right in and asking in tiny steps if what was made was right, with a lot of bright people advocating you could go from a two-wheel kick-scooter to a Porsche SUV in small cyclical increments, of which the first stage got called MVP and rushed out. And if the market was only ready for a Porsche, well, you’d better hope you found that out by having some really deep kick-ass user interviews and conversations about that scooter MVP, or some other channel, because you’d never find out from sending that MVP out on the web and checking the numbers. Quant doesn’t give qual answers.

User research through prototyping made a resurgence, but flattened to be a repeatable and teachable process, called Design Sprint, to be only about asking people on the street what they want with half sketches, in a cycle that is only allowed to last a week. The rest of the knowledge to create a success has to come from… hunches from the product manager? Marketing? In my last job it was edicts by stakeholders, when it should have been customer service. Pulling all these signals together is the synthesis-between-departments glue UX Research and Product Design should really be bringing now and are often not empowered to do or can’t because they are stuck in cycles.

As Joanna Weber writes in this brilliant article about why Agile and Lean are such difficult fits in organizations that are vast and actually have to be trustworthy, coherent, and good: “If Scrum only worked for as long as there were waterfall systems in place to support it, we need to replace both with something that both acknowledges and improves that reality.“

And the reality is rule nr 1 above, and that

Humans can not accurately predict how long any software effort will take beyond four weeks. And after 2 weeks it is already dicey.

So that replacement has to stay incremental in nature and show a lot to users at every step. It’s a tough situation and it has been true for years: we still do not have a repeatable, teachable process to make great software systems that span multiple touch-points and are a joy to use and maintain. We have to navigate between speed of incremental delivery and allowing time for thinking for design.

Right now there are roughly three fundamental ways in which design fits into Agile of various forms, Sprint 0 (which can be Big Design Upfront), Sprint Ahea d, and Parallel Tracks. Of these, Sprint 0 and Sprint Ahead are the ones I am finding the most, with Parallel Tracks, that could combine research and design into a very strong customer experience proposition, seeming the least popular, mostly because “devs want their designer embedded for synergy and speed”.

That should change, though. While UX Research and Product Design currently have an employment and cvredibility crisis, I recently did some prototyping with new tools that make me think there’s a whole new direction to go here. But this is already too long, so I will describe my ideas for the future next week.

The Map Is Not The Territory And A Wish List Is Not A Map (1)

“Where have all the task decompositions gone?” I was talking to a very experienced Head of UX about the state of our vocation when she asked that. I had to agree I have not seen one in years either. A task decomposition is when you take a task and divide it into steps, and then divide those into smaller steps, until you reach some granularity that makes sense for why you are doing this, like making screens or coordinating robot movements.

We used to do them all the time in UX, mostly when the field was still called HCI, to make sure we understood what the human was doing before we taught the computer to help them with it. There were many notation systems for them, and you could write PhD’s on comparing these notation systems and then inventing new ones.

Also something I haven’t seen in a decade is a specification full of descriptions of features that a system SHOULD and MUST and COULD have. The were called Functional Requirements, and while they often tried not to impose a view on how the system should look to users, you could tell how desperate the writer was trying to convey their needs in something else than fuzzy human language when they invariably started to use Word’s shapes tool to mock up screens—and then write the word SUGGESTION underneath so as to not offend their designers.

TDs and FRs are a relic from the waterfall period when you did a lot of design and understanding up front to make sure you were making the right thing for people before you committed the programming resources to make it. They were intrinsically incomplete in the same way a map always leaves things out of its description of the actual terrain, and expensive to make, and of limited use because:

Humans can not accurately describe what they want out of a software system until it exists.

Bit of an issue.

Rule nr 1 is and was true all the time. You’d computerize a workflow of paper files in a shop or local government and at the end it would turn out that there were all these exceptions being made by clerks and admins using different color pens or writing in the margins that all the workers understood, but nobody above them working with IT did. The exception would be so important you’d have to retool the whole thing down to the tables in the database, and the project would be late and expensive.

When Agile originally said to deliver value frequently, it wasn’t to unlock money from customers cycle after cycle—that wasn’t even really possible until we started putting everything on the instantly monetizable web. It wasn’t for investors either, they will happily wait years for a return if the projected return is big enough. Agile wants frequent releases so you can show the results to humans fast and get feedback and then correct, instead of finding out when you deliver the whole thing after two years that rule nr 1 above always holds. It’s only around the time Lean Startup came along that every iteration wasn’t just to correct the course but also had to deliver some new mini-feature every time.

So if you want to replace Agile Scrum or Kanban with something, you have to deal with the fact that 40 years of trying to first find out how people work, and then making wish lists in all kinds of notations of how that work is to be done by computers, never was really successful and often a total failure.

Still, adding functionality bit by bit as you explore what is needed comes with things you should be aware of:

The resulting system is kludged together cycle after cycle, unless you take some choice time between cycles to refactor huge chunks. This is why every seven years a software team wants to just start over, they can’t take doing archeology in all those cycles of hacks anymore and don’t feel they can add any more functionality without watching the tower of hacks fall over.
It’s actually not faster than Waterfall. It just decreases the risk of ending up with ~~garbage~~ a sub-optimal product-market fit.

But, but, but, if ~~wish lists~~ specifications didn’t work because making software in itself changes the work the software is supposed to help out with, what about prototypes? That worked, right? Yes, with a list of caveats including that Agile actually doesn’t know when to use them and that AI-derived UX is deeply changing that game in the last 6 months, but I’ll discuss that in part 2.

If culture eats strategy for breakfast, OKRs are just the cheap juice everyone gulps down first

Six hundred years ago, in the early 2010s, I was working a contract—a heritage company that needed their website to be responsive—where I met my first digital transformation consultant. A few weeks in I found out their day-rate was literally twice mine, so I asked them what they actually did. The answer was that after all the stakeholder fluff, their actual work was to find and look every function involved in the digital side of the company, and recommend how to align everyone’s incentives for a good outcome. I nodded sagely and had no idea what that actually meant; until then I had worked for large companies with long histories and already aligned missions, or tiny research teams that were making it up as they were going along. Well, I’ve worked for very different companies since then, and I have seen the pain of mismanaged alignments, usually in large companies that don’t take the time to define themselves.

There’s this moment burned in my brain from when I was in a 1-on-1 with an organizational advisor about some of the issues that were massively, massively frustrating trying to getting good design delivered in the company. I had pulled up an org chart that depicted two departments that were supposed to work together to make good things for customers but were instead just barely pushing a few new features out.
The advisor pointed to the topmost leaders of each department and told me how they advised both of these people and thus was cross-organizationally aware.
And soon confidently added: “They have the same OKRs, right, so that is how they stay aligned.”
I managed to not fall off my chair.

Gentle reader, if, for example, the Customer Service and the Product departments share an OKR of lowering customer complaints, but CS is culturally and financially incentivized for high throughput of calls, then CS may invest in a CRM to record the customer issues but they will not really work with the caller to find the root issue and then log it exhaustively. They will just report that 80% of all issues are password-related and put in some more voice-over messaging to send callers to the chat bot on the website, while the User Research department will have to spend a ton of money to find out what customers already desperately want to tell the company on the phones every day about what they are trying to do that doesn’t even need a password. The OKR is indeed empowering every department to chart their own course—separately.

Sales and Digital Delivery may share an OKR to increase new conversions, but if the management of Digital incentivizes rapid fail-fast online revolving experimentation, while Sales is leaning into the reliable, trustworthy, solid aspects of the established brand in their outreach, the user really will end up, at best, unnerved by the difference between what they are told and the rawness of what they use, and the designers and content strategists in Delivery trying to bridge this chasm into one experience will burn out in no time from being yelled at by Sales. By focusing only on a measurable quarterly outcome that can be pursued independently, the OKR is doing nothing to align where it counts and chewing up the people in-between.

These examples are made up, by the way, I can’t write about what I have actually seen fail.

The literature about OKRs all say things like “Objectives and Key Results (OKRs) provide a framework for businesses to execute and achieve their desired strategies through simple, collaborative goal setting” and that those create internal alignment, but the alignment ends up being only about which measurements to game this quarter or year. OKRs say nothing fundamental about what kind of relationship the company wants to have with their customer beyond “let’s make money this specific way right now”. It’s not a framework that includes a vision of how that specific money-making thing fits in the long-term relationship between the company and the customer, what the values and standards are of what the company considers acceptable to offer to customers. That’s the company culture and it needs to be set and maintained separately. Of course, that activity doesn’t generate an immediate return, so that’s a non-starter these days except in some very committed companies.

Some might right now say “wait, no, a good OKR absolutely defines a quality level: if the experience or product or marketing is below a certain quality level then you won’t make your key result, and therefore the OKR implicitly aligns the product and marketing and experience departments to this particular level; they have to talk to each-other to reach that ambitious result.”
To which my answer is: look around you at the disjointed mediocre experiences from all these companies who are supposedly all-in on OKRs. Without a company culture defining a baseline of expected quality that can only be achieved when groups work together, a culture maintained from above, departments will just do what they can themselves because creating that alignment horizontally by themselves is just too hard. The alignment will stop at the boundaries of departments committing to separate numerical targets. Every department will try to achieve the key result they settle on with the tools they control: one department may go all-in for quality in their space while the other goes for gaming and dark patterns. While everyone is also loudly complaining about internal waste and separation, of course.

OKRs supposedly communicate and align strategy. Culture eats strategy for breakfast, especially if the strategy comes in as a two-line description of a numerical target. Between all the merging and slicing of companies by current venture capitalism, the corporate cultures that unify departments have been totally eroded. When the main product companies actually are tasked to produce is a higher stock price, everything else becomes secondary to that, including even keeping customers alive, even if that was a guiding principle for absolutely decades. All that is left is departmental culture, and it only takes one group deciding on a different course from the others to make the experience disjointed to the customer.

There’s no free breakfast. Aligning takes work in very large companies, and it has to come from the top, looking at what is being put out, holding it up to the company standard you have taken time to define, doing the leg-work until you know who made what and why so you can shape teams and align incentives and create the right communication channels until everyone is making their part of the same thing. It’s so seductive as management to think you just need to write a couple of goals and target numbers in an OKR format and you can then just throw it down the pyramid to “empower” departments and everything will be alright and everyone will innovate and make their best stuff. And they will make their best stuff—within their constraints, expectations, and reward structures. Aligning those is where the real management work is.