AI InfrastructurePublished 6 min read

Training AI Agents for Real Work: Three Open Questions in Reinforcement Learning

Reinforcement learning is moving from benchmarks to real work. The decisions teams make now — tasks, rewards, evaluation — decide whether the agents hold up.

By OpenStages Editorial · Field notes from OpenStages organizers

reinforcement learningAI agentspost-trainingAI securityenterprise AIAI Infra NightOpenStages
3rd Annual AI Infra Innovation Night — Reinforcement Learning, January 14
3rd Annual AI Infra Innovation Night — Reinforcement Learning, January 14

Reinforcement learning used to be judged on benchmarks. Now it is being asked to produce agents that do real work — run experiments, defend systems, move through enterprise workflows — and that change shifts the hard questions from the algorithm to everything around it.

What does it take to train an agent for real work?

Three decisions matter most: which tasks the agent trains on, what it is rewarded for, and how you tell whether it actually improved outside the training environment. Get any one wrong and an agent can look better in training while getting no better — or worse — at the job.

Those decisions look different depending on where the agent works. Three domains show the range especially clearly.

1. RL for AI research and discovery

Agents are starting to design models, run experiments and evaluate the results. The obvious reward is a successful experiment — but most experiments fail, and the failures carry much of the information. The open question is how agents can learn from failed experiments and carry those lessons into research tasks they have never seen, rather than only getting better at the ones they trained on.

2. RL for cybersecurity and safe agents

Security is adversarial by nature, which makes it a natural fit for RL and a risky one. Agents train on adversarial tasks in contained environments, and the same pressure that makes them capable can teach them to exploit the evaluator instead of solving the task. The question is how to train on those tasks and reliably catch the cases where an agent has learned to game the measurement.

3. RL for enterprise workflows

Inside a company, an agent works across tools, data and policies that were never designed as a training environment. The signal that matters — did the work get done, did the expert approve it, did the business outcome move — is slow and scattered. The question is how to turn expert feedback and business outcomes into training signals for agents that stay reliable across all of it.

Where the reward comes from

Dimension
Reward signal
The hard part
Research & discovery
Experiment results, including the runs that fail
Gains that transfer to unfamiliar research tasks
Cybersecurity & safe agents
Adversarial tasks in contained environments
Improving behavior without teaching agents to game the evaluator
Enterprise workflows
Task completion and expert feedback
Rewards that hold up across tools, data and company policies

The infrastructure underneath

Under all three sit the same building blocks: environments, rewards, evaluation and compute. That is why this is an infrastructure conversation as much as a research one — and why it needs founders building agent systems, enterprise leaders putting them into production, researchers working on post-training and evaluation, and the investors backing them.

Frequently asked questions

Who attends AI Infra Innovation Night?

About 40% founders and enterprise leaders, 30% AI researchers and 30% venture investors. To join a panel or partner with the event, email partnership@openstagesai.com.

Join us

Keep reading

Related articles.

Questions, salon invites, or a founder story worth telling?

Tell us what you're building and who should hear it. We'll take it from there.