The Theory of Constraints as Applied to AI Integration

AI tools and processes are a rapidly evolving technology. This creates a difficult situation for companies. The changes are far too disruptive to ignore, but must also be integrated real-time into existing workstreams with minimal disruption to existing and future commitments while also adapting to the effects on morale, culture, and organizational efficiency.

It is common for individuals to feel the power and potential of AI but without it manifesting as tangible improvements in hard metrics like revenue growth, time-to-completion stats, as well as soft metrics like career growth and employee satisfaction.

The primary reason for this lies in the Theory of Constraints developed by Eliyahu Goldratt which can be summarized as:

Local improvements everywhere do not automatically translate into the global improvement of the organization.

This is because organization output, like an engineering system, is driven primarily by bottlenecks. In a coffee shop, if the cash register is the limiting constraint, no improvements to any other part of the system will make a difference. Not better customer service, not better food or coffee, not faster wifi, not cleaner bathrooms.

Any improvement that is not at the bottleneck has no effect and often actually makes things worse! Broad mandates to “Improve productivity using AI!” can actually have a negative effect on organizational efficiency if done so without awareness of organizational constraints.

For example, if the engineering department is the bottleneck (the most scarce resource), then organizational throughput is limited by developer output. Excess capacity in marketing and product departments actually make the bottleneck worse because it takes away productive capacity from the bottleneck via effects like requests for estimation.

This makes the bottleneck even smaller, reduces organizational throughput, and gives the upstream work centers even more spare capacity which creates more requests for estimates which makes the bottleneck even worse and so on. In one case study at Microsoft, an innocuous policy to provide time-estimates within 24 hours caused a 40% reduction in bottleneck capacity!

Even downstream work centers can affect the bottleneck because they may require decision touchpoints with engineering.  This creates more overhead, more meetings, more phone calls, more email, more blocked decisions and so forth.

At this point, the mandate to “increase productivity” has drastically reduced total throughput. Everyone is mad at the bottleneck and the bottleneck is mad at everyone else.

The solution? “Increase productivity even more!”.

Breaking out of the cycle requires internalizing the unintuitive realization that a company where everyone is busy optimizing their own productivity is incredibly inefficient if it comes at the expense of the bottleneck. In order to have maximum throughput, all other parts of the system that are not the bottleneck must have spare capacity.

Without organization trust or awareness of this dynamic, managerial mandates can be interpreted as “Everyone stay busy!” which makes the problem worse and generates organizational dysfunction and resentment as everyone optimizes for their own performance at the cost of the bottleneck.

Unique Issues with AI

The Toyota Production System used in car manufacturing and the Agile Scrum systems used in software engineering systems both revolve around the idea of visualizing “Work In Progress” in order to figure out where the bottleneck is. (Car parts or work tickets end up piling in front of it.)

This allowed the bottleneck to be constantly tracked and improved upon.

Integrating AI creates new unique challenges! In some domains AI can perform at superhuman levels where specifying what to build takes about as much time as actually building it. In other domains, it can fail completely. The non-intuitive boundary between these tasks is known as the “jagged frontier”. This frontier is constantly changing and can make keeping up with and integrating AI capabilities a full time job.

Imagine a car assembly line where one work unit could suddenly 10x its output and another had a 50% chance to either 5x or halve its output.

Integrating these new capabilities with a positive effect on margin, revenue, culture, and morale requires a bottleneck-aware high trust culture.

Solutions

A unique issue that arises with AI capability is that all roles are shifted towards a more technical role. When everyone can create their own tools, they are engaged in the process of software engineering whether they know this or not. Instead of rediscovering best practices from first principles it is better to draw on the experience and battle-tested processes of the industry.

As a silicon valley engineer with over a decade of experience working at large and small companies I am uniquely qualified to debug systemic bottlenecks at both the macro organization level (mapping the value stream through interviews & shadowing) as well as providing mentorship and feedback at the level of raw code. The primary focus would be to evaluate and improve broad organizational AI literacy.

Examples of questions I’d focus on discovering the answer to:

Value Stream Mapping

• How does value creation move throughout the organization?
• Where does it pile up and create bottlenecks?
• How does communication flow through the various work units?
• Are there disruptive deadlocked cycles where people feel blocked by decisions from each other?
• How much overhead is created for each work unit by their upstream and downstream units?
• Do work units have projects that can absorb spare capacity to provide organizational value without affecting the work-in-progress of other units?

AI Tooling Integration

• How much work is de-facto software engineering?
• Are the right software development practices in place?
• What mix of internal and external software and AI tools should be used?
• What are the costs and opportunities of internal vs external?
• How to evaluate and integrate new tools given the speed at which the industry cahnges?
• Should there be production customer facing software systems such as dashboards?
• Would there be on-call rotations and service level agreements?
• Would the value add justify engineering roles?

Cultural

• Are there opportunities to mentor and be mentored within the company?
• What are the effects of AI on employee morale?
• Do the existing roles and responsibilities make sense or have they been disrupted by AI?
• Are there on and off ramps for new roles and are they clear enough so that everyone knows what is expected of each other?
• Is there enough organization trust to create “spare capacity” or does work accrue at the bottleneck as a convenient scapegoat?