Reinforcement learning taught language models to reason. The next problem it is being asked to solve is harder to see in a benchmark: teaching agents to get better from the people they work for.
Most agents in production today do not learn from use in any meaningful sense. They remember — a preference saved here, a past conversation retrieved there — but the policy that decides what they do next is the same on day one hundred as on day one. The teams working on that gap are a mix of RL researchers, product founders and the people building the training stack, and they rarely end up in the same room.
What does it mean for an agent to learn from people?
An agent learns from people when the signals they leave behind — corrections, approvals, edits, undos, retention — are turned into changes in what the agent does next, not just stored as context. Memory stores what you said; learning changes the agent's behavior.
Those signals are everywhere in a deployed agent. A user rewrites a draft before sending it. They approve one suggested action and skip another. They come back the next day, or they don't. Each one says something about whether the agent did the right thing — but almost never in the clean, labelled form that classic RLHF pipelines were built around.
Why implicit feedback is a hard reward
- Implicit — nobody clicks a thumbs-up; the signal has to be inferred from behavior.
- Sparse — most interactions produce no clear signal at all.
- Noisy — an edit can mean the agent was wrong, or that the user changed their mind.
- Delayed — whether a task actually helped may only show up days later, in retention.
- Personal — what counts as a good answer for one user can be wrong for the next.
Five open questions
Where the work is right now
- 01
Reward from people
Which signals — corrections, approvals, edits, retention — are reliable enough to learn from, and how do you combine them?
- 02
Online and continual learning
How does an agent keep improving from live feedback without forgetting what it already does well, or regressing for everyone else?
- 03
Personalization
One policy for everyone, or one that adapts to each user? And what happens on day one, before there is any history to adapt to?
- 04
User simulators and environments
You cannot run every experiment on real users. Simulated users are becoming the environments agents are trained and evaluated in — and the sim-to-real gap is its own research problem.
- 05
Reward hacking and sycophancy
An agent rewarded for approval learns to please. Serving the user's goals rather than their clicks is the alignment problem in miniature.
Why it takes researchers and builders together
The researchers have the methods but rarely the live traffic. The founders have the traffic but rarely the time to build an RL stack. The teams in between — simulators, evals, environments, the inference and compute behind per-user learning — need both to know what to build. Progress on this problem depends on those three groups comparing notes on what has actually worked.
Frequently asked questions
Can I present at Agentic AI × RL?
Yes. Email partnership@openstagesai.com and say whether you want to give a talk or a demo, and what your agent or research learns from. Eight teams present, four per round, with 8 minutes each plus 3 minutes of questions. A polished deck isn't needed — bring a demo or something to show the room.
Join us



