Where software architecture is going — and why a first reading of the trend leads enterprise investment straight into a trap.
The industry is coalescing on a single idea: an AI operating layer that sits across the enterprise and runs the agents. Read it the obvious way, and you invest in the layer itself — the gateways, the protocols, the generic interface. That is the wrong thing to own.
This post is the second reading.
The unit of software has shifted from the application to the task
For most of the last two decades, software was a set of applications. The unit is changing. The unit is now the task — and there are two ways to execute one.
Deterministic systems run on train tracks. The path is fixed in advance. A person clicks, fills a form, calls an API, and the system executes a predefined workflow. The human supplies precise inputs; the machine follows the route it was given.
Goal-seeking agents drive off-road with a GPS. The path is dynamic. The agent interprets intent, assembles the context it needs, calls tools, evaluates the result, and adjusts. The human sets the goal and approves the actions; the system plans and executes toward the objective.
Embedding an agent has gone from novelty to default — it is now table stakes, not a differentiator.
Most of the execution still ends in failure
The agents are being embedded. They are mostly not working.
A large and fast-rising share of companies have abandoned most of their AI initiatives in the past year, and the typical organization scraps a big portion of its proofs-of-concept before they ever reach production. Only a small minority report meaningful ROI from agents specifically.
The reflex is to read this as a capability gap — the models are not good enough yet. That reading is wrong, and it sends money toward bigger models when the failure is somewhere else entirely.
This is an organizational problem wearing a technology costume.
The blocking issue isn’t raw model capability
Ask the people running these programs what stops a pilot from reaching production, and the answer has shifted. Enterprise leaders increasingly name non-deterministic output — not raw capability — as the number-one barrier to production readiness.
The crux is not that the model is wrong. The crux is that we cannot tell ahead of time when it will be wrong, and our regression tests don’t catch it. A deterministic system fails the same way twice; you write a test, the test holds, you move on. A goal-seeking agent can take a different path on the same input and fail in a way no prior test anticipated.
That is why the spend is moving. Evaluation and observability tooling has become one of the hottest budget lines in the category.
Retrieval is safe; action is a cliff
There is a safe zone and a danger zone, and the line between them is sharp.
Retrieval is the safe zone. Ask a question across systems, pull the relevant context, summarize it. The risk is low, the work is read-only, and horizontal agents work fine here. This is where most early wins live.
Action is the danger zone. The moment an agent is asked to do something — write a record, move money, change a configuration — the calculus changes. Whose credentials does it use, and what happens when it makes a mistake in a production system at 2 a.m.? A horizontal agent has no answer to that question, because the answer is specific to the system, the role, and the consequences.
The architecture is horizontal; the defensibility is vertical.
Where the value actually lives
Split the stack into what is free and what is expensive, and the investment thesis falls out of it.
The free wiring is a commodity. Protocols — MCP, A2A, AG-UI — are standardizing rapidly. Tool and data connectors, basic agent runtimes: the same. Most successful deployments have already standardized on open protocols like MCP. Connecting to many systems is table stakes, not a moat.
The expensive defensibility is contested ground. Context engineering — knowing what to put in front of the model and when. Evaluation and observability — knowing whether it worked. Governance and guardrails — keeping it inside the lines. Underneath all of it sits deep knowledge of a specific industry’s workflow, which is what makes an agent trustworthy rather than merely functional.
Two traps
Two designs look like the right place to build. Both are traps.
Trap #1: the gateway as infrastructure
The first trap is to build the gateway as infrastructure — a toll booth that every request passes through. A toll booth is judged on millisecond latency and five-nines reliability. That means deep-systems engineering against funded giants, in a market that is already saturated: LiteLLM, OpenRouter, Portkey, and a stack of open-source alternatives.
A gateway as infrastructure fights on the one axis where there is no edge. It is a race to the bottom.
Trap #2: the generic agentic interface
The second trap is to build the generic agentic interface — a chat panel and an approval queue. An interface by itself is a thin layer: a few weeks of frontend work for a small team. If it is that quick to build, platform vendors will ship it for free, bundled into the tools customers already pay for.
The value was never the interface; it was the domain underneath it.
Breadth versus depth
The two strategies are not points on a spectrum. They are opposites, and they differ on every axis that matters.
| Generic horizontal AI | Governed domain workspace | |
|---|---|---|
| Scope | Open-ended autonomous assistant | Scoped workflows with human checkpoints |
| Reliability | Probabilistic — “looks good enough” | Binary — it worked, or it didn’t |
| Governance | Low; relies on generalized model behavior | High; bounded by strict governance and audit logging |
| Defensibility | None; commoditized by platform vendors | Absolute; built on fragmented, regulated, deep industry knowledge |
The synthesis
The two traps are not dead ends. They are instincts pointed at the wrong target. Aim them correctly and they become the product.
The gateway instinct — control every request — becomes governance and audit. The interface instinct — give people one place to work — becomes a shared workspace. Apply both to a high-stakes domain, where the cost of a wrong answer is real, and the domain becomes deep vertical expertise. Governance and audit, multiplied by a shared workspace, multiplied by deep vertical expertise, is the Governed Domain Workspace.
These are not two separate products. They are the front and back of one product. The element that turns it from a feature into a business is the domain — the one asset platform vendors cannot copy.
Walled gardens of value
The destination is not a layer of the horizontal stack. It is a walled garden of value: an AI reasoning layer enclosed by domain rules, human approval gates, and binary success tests, sitting on a foundation of deterministic systems of record.
It blends deterministic workflows with agentic reasoning only where genuine judgment is needed. It focuses on document-heavy, approval-bound, regulated areas. It prioritizes auditability and role-based access over generic intelligence.
The money is not in owning a layer of the horizontal stack. The money is in owning the hard, high-stakes domain.