The folly of a "global" product development process

Can’t there be just one single development process for all aspects of a product system?

===

Note: to agree on the definition of “process”: “a series of actions or steps taken in order to achieve a particular end”.

Phase-gate. Agile. Lean. Design Thinking. Build-Measure-Learn… So many options to choose from for your development processes. And yet, most companies (still) usually develop tangible products within all-too-linear phase-gate processes, software using some variant of Agile, and services usually in an ad-hoc manner; with Lean and Design Thinking methods and tools sprinkled on top of all kinds of project management and product development processes to make sure that nothing is missing from the “must-have” great ideas of management.

Wouldn’t it be easier, some ask, to have a single “global” product development process, which can be used across aspects and domains? In that case, why not take care of everything using, e.g. phase-gate, or Agile/Scrum?

One size cannot fit all

A “global” product development process that can address all kinds of components, i.e. mechanical, hardware, embedded software, application software, services, is a “Holy Grail” of management and, actually, not very realistic. The motivation for such an all-encompassing one-size-fits-all process is often to establish measurability of development progress across domains; an aspect that is not the biggest contributor to customer value.

However, the reality of each aspect/domain of a product system is that it is different by nature. For example, procuring tangible prototypes is typically subject to long supply-chain lead times, in contrast to software prototypes. Modifying software is typically less expensive than hardware, and it can also be highly automated, compared to tangible product components. And validating the concept of an offline service product requires the effortful ramping-up of service delivery resources, and the acceptance that these very resources have an inherent variability that prevents really repeatable experiments.

So: phase-gate for everything? Agile for everything? A mix of both? Wait; why do we even need to cement the world using a single approach?

We don’t. There is a way to make it work: periodic synchronization between value streams, even if (actually, especially because) these have different attributes.

Most companies’ processes are out of sync

It is, in fact, far more realistic to strive for a set of domain-specific processes that share common concepts and are interlinked, with each having its own process content. These processes would need to be company-specific to address each company’s own setup, “gotchas”, organizational learning, culture, and peculiarities.

However, things definitely get tougher when the various types of components are developed in processes that are unrelated to or even incompatible with each other. Specifically, “incompatible” in the sense that integration of components and their aspects is arduous and unsynchronized. This is what, in my view, is currently happening in companies struggling with software-enabled hardware/tangible product components: the software world is colliding with the rest of classical product development (and vice versa), both in content and in management mentality.

For example, some processes can achieve new increments every so many hours with little human interaction (e.g., software with continuous integration), whereas others can only do so every so many weeks or months after much deliberation (e.g., greenlighting regulatory aspects after much back-and-forth between people inside and outside of the company). When each component of the system reaches its “whatever” milestone “whenever”, it’s tough to align the “micro-batch” of a sprint in terms of product or project metrics.

Synchronize and (attempt to) integrate

Things are easier when all development processes share common principles and periodic expectations of alignment to de-risk interfaces. That’s why larger (a euphemism for “overkill”) Agile frameworks, such as SAFe, introduce the notion of Product Increments (i.e., integration events).

When each aspect of the system is timeboxed according to its own time-scale and expected to deliver an increment at predictable points in time synchronized with the other aspects, people know what to strive for in the near future. When higher-level backlog alignment takes place, people also know what to strive for in the middle term. And when, e.g., higher-level requirements and insights (call them “Epics”, if you insist on terminology) are linked to the product portfolio strategy (or other strategies), people also know what to strive for in the long run.

In contrast, the reality in most companies is that each development process is either nigh-arbitrarily takted or not takted at all. For example, mechanical components might be developed using an untakted phase-gate with planned milestones/gates (usually, accompanied by heavily-planned Gantt charts) that are months or years apart, and movable anyway through changes and politics. Meanwhile, software is probably being developed using Scrum in sprints of some weeks. And other intangible aspects, such as regulatory approvals, might be “developed” in whatever way works to address the regulatory agencies’ own workload and responsiveness (or lack thereof).

Of course, integrating these three aspects can be a nightmare, especially when a company culture promotes a siloed view of development: i.e., when “whatever” happens “whenever”, in order to satisfy not only the end result of product development, but also internal criteria with scant connection to customer value. This means that each aspect (hardware, software, service, etc.) is building up its own arsenal of invisible “just-in-case” inventory of deliverables while waiting for the other aspects, and therefore has to later do rework (or discard work) with a very high probability. This is not exactly a setup that many would call compatible with the ideas of Lean.

Compare this to having a cascade of established release cadences. Instead of things getting done “whenever” you would be able to attempt to integrate every so many weeks. The integration, if possible, might be only conceptual for some aspects, as not everything within a product system starts maturing at the same time, or matures with the same speed. However, the rest of the system’s stakeholders would at least be informed about the latest state—instead of, for example, having regulatory aspects being run in “submarine mode”.

Furthermore, in case you cannot integrate at all at a specific sprint target date, you could then run a system-wide retrospective to figure out why (and then perhaps run a kaizen event)—and feed the insights into the next product increment and the sprint cascades into its separate aspects.

Requirements, requirements, requirements

“But wait!”, some say; “I get how Agile fits software, where teams seem to focus on shipping first, and fixing later. But how could I ever trust Agile in developing aspects of a system that require zero defects, such as human safety or other regulatory aspects? Surely, using a phase-gate with clearly-defined gate acceptance criteria to signify done-done-done is the only way possible!”

Frankly, the argument that Agile doesn’t match the development of such aspects is a red herring. All kinds of aspects, including, for example, human safety or regulatory approvals, represent requirements. Much like any other requirement, these also have to be fulfilled with a certain aspiration of quality; in this case, the aspiration is clear-cut and non-negotiable: zero defects. The work to satisfy these requirements and, e.g., assure human safety, needs to happen regardless of with Agile or phase-gate processes/methods.

In fact, as much as I like Agile, I would even dare say that the choice of process is a red herring by itself; it’s more a result of consultants trying to position themselves and sell what they know best, rather than a reflection of actual client needs. Or, it’s a result of people trying to use a specific process for its own sake; in other words: Deft.

Even in a phase-gate, once-unknown requirements are ultimately also discovered and fulfilled through an iterative and incremental process of talking to stakeholders, uncovering the known and unknown unknowns, and working in tight loops with those who can deliver insights towards a solution that hits the mark.

Skip the red herrings

Agile methodologies, such as Scrum, simply help put a structure on top of that pragmatic and experimentation-based way of working, which—let’s face it—is the only way to develop anything when there is uncertainty. And uncertainty is pretty much the only certain thing when developing anything novel, especially when attempting to innovate.

So, don’t get stuck on debates regarding the “ideal” “global” development process to impose on every single value-stream within a product system—if there is one at all, it must be one huge beast of a contrived mega-process, a miserly compromise trying to satisfy (supposedly) everyone, and achieving the exact opposite.

Instead, choose the right type of process for each aspect/domain of the product system, and then customize each one to suit your business context. Even better: collaborate to design each process. And, if striving for agility, make sure that some kind of timeboxing and synchronization takes place between processes, even if some of the processes, such as those that have to do with regulatory aspects, are “stiff” and supposedly not amenable to any form of Agile.

You might not end up with a global process to rule them all, but you might be surprised at how well things work out when there is synchronized, iterative, and incremental work on fulfilling requirements across the entire product system.