“AI infrastructure” is used as though it names one market. It does not. At AI Infra Signal, the 100-person room OpenStages ran in Sunnyvale on September 12, 2026, nine sessions covered GPU cloud economics, the data stack, web agents, reinforcement learning, cluster operations, causal world models, simulated-user evaluation, enterprise workflow ROI, and model routing. Read as a set, they sort into six layers — and almost every company in the room sat in exactly one of them. This is the map.
Layer 1 — Compute: how much useful work comes out of the hardware
Not how many GPUs you own, but how much of them produces output. Workload placement, scheduling, isolation between tenants, and reliability all decide what a fleet actually returns, which is why neoclouds — cloud providers built specifically around GPU capacity and AI workloads — compete on utilization rather than on raw capacity.
This is the layer where engineering and economics stop being separate conversations. A cluster running at half utilization has doubled the cost of every token it serves, and no amount of procurement fixes that. It is why the GPU cloud and investment sessions at AI Infra Signal — hosted.ai on neocloud economics, Llama Ventures on the investment lens — ended up describing the same constraint from two sides.
Layer 2 — Data: the constraint that surfaces once compute is solved
Storage throughput, memory capacity, and network behaviour, evaluated together with scheduling. A GPU waiting for data cannot turn its theoretical performance into useful output, and improving any one component usually just exposes the next bottleneck.
Teams tend to discover this layer in the wrong order — buying compute first, then finding the data path cannot feed it. The useful framing from the VDURA session was that training, inference, and data movement are one system, not three purchases.
Layer 3 — Agent execution: acting is a different problem from generating
An agent has to find current information, navigate a page, hold a session, and complete a sequence of actions — then learn from where the sequence went wrong. That needs search, page retrieval, browser sessions, multi-step execution, state, and execution traces. A text-generation stack provides none of it.
Two AI Infra Signal sessions framed the two halves of this: TinyFish on web access — helping an agent act — and XTrace on teaching agents through the mistakes they make during execution. Acting and learning from the attempt are the same infrastructure problem viewed from either end.
Layer 4 — Learning loops: RL turns training into a distributed systems problem
A reinforcement learning loop has to generate attempts, interact with environments, compute rewards, update a model, and evaluate the result. Each stage has different performance and reliability requirements, so coordinating them is an infrastructure question rather than only a research one.
Cohere's session put modern RL infrastructure in the program for exactly this reason. Related work on causal world models — Aether AI and UC San Diego — pushes on the same seam from the research side: the difference between recognising patterns and reasoning about how an action changes an environment raises questions about state, prediction, intervention, and feedback that the serving stack has to answer.
Layer 5 — Evaluation: static tests cannot check behaviour over time
A static test checks one answer. A simulated interaction explores how behaviour changes across different users, goals, and sequences of steps — which is the thing that actually breaks in production. That is why simulated-user evaluation showed up as an infrastructure topic rather than a QA topic.
The MatrAIx session at AI Infra Signal — researchers from Microsoft and MIT — made the case by scale: persona simulation as a way to test a product against many users at once rather than one golden path.
Layer 6 — Model access: routing became an operating decision
Once an application uses different models for different tasks, choosing between them is an operating decision with five inputs: quality, latency, cost, availability, and data requirements. A shared interface simplifies access; it does not decide which model fits a workload.
Infron AI demoed the access side of this at AI Infra Signal — multiple models behind one API. The evaluation side stays with the application, which is what makes this a layer rather than a feature.
What ties the six layers together
The system around the model. AI infrastructure creates value by making intelligence usable — supplying the right inputs, supporting execution, measuring results, and keeping the economics workable. Every layer above is one of those four jobs.
- Compute economics depend on useful utilization: scheduling, data movement, memory, and reliability all affect what a GPU fleet returns.
- Inference and data infrastructure have to be designed together, especially as context length and concurrency increase.
- Agents need support for complete tasks: web interaction, state, execution traces, evaluation, and recovery.
- RL, world models, and simulated users create new systems questions, with validation needed alongside capability gains.
- Enterprise AI is measured at the workflow level, where integration, retries, monitoring, and human review all count.
Go deeper
Frequently asked questions
What counts as AI infrastructure?
AI infrastructure is the system around the model rather than the model itself: the compute that runs it, the data path that feeds it, the execution layer that lets it act, the loops that let it learn, the evaluation that tells you whether it works, and the routing layer that decides which model handles a task. A company usually sits in exactly one of those layers.
Which AI infrastructure layer is the most crowded right now?
Model access and routing is the easiest layer to enter, because a shared API over several providers is a small surface. It is also the hardest to hold, since the decision that matters — which model fits a workload at what quality, latency, cost, and data requirement — stays with the application. Compute utilization and the data path are harder to enter and harder to displace.
