AI ResearchPublished 6 min read

Agents That Learn From the People Who Use Them: The Next RL Problem

Memory stores what you said. Learning changes what the agent does. The gap between the two is where the next round of RL work lives.

By OpenStages Editorial · Field notes from OpenStages organizers

reinforcement learningAI agentspersonalizationcontinual learningRLHFAgentic AI × RLOpenStages
Agentic AI × RL: Research Talk and Demo Night, Sunnyvale, December 10
Agentic AI × RL: Research Talk and Demo Night, Sunnyvale, December 10

Reinforcement learning taught language models to reason. The next problem it is being asked to solve is harder to see in a benchmark: teaching agents to get better from the people they work for.

Most agents in production today do not learn from use in any meaningful sense. They remember — a preference saved here, a past conversation retrieved there — but the policy that decides what they do next is the same on day one hundred as on day one. The teams working on that gap are a mix of RL researchers, product founders and the people building the training stack, and they rarely end up in the same room.

What does it mean for an agent to learn from people?

An agent learns from people when the signals they leave behind — corrections, approvals, edits, undos, retention — are turned into changes in what the agent does next, not just stored as context. Memory stores what you said; learning changes the agent's behavior.

Those signals are everywhere in a deployed agent. A user rewrites a draft before sending it. They approve one suggested action and skip another. They come back the next day, or they don't. Each one says something about whether the agent did the right thing — but almost never in the clean, labelled form that classic RLHF pipelines were built around.

Why implicit feedback is a hard reward

  • Implicit — nobody clicks a thumbs-up; the signal has to be inferred from behavior.
  • Sparse — most interactions produce no clear signal at all.
  • Noisy — an edit can mean the agent was wrong, or that the user changed their mind.
  • Delayed — whether a task actually helped may only show up days later, in retention.
  • Personal — what counts as a good answer for one user can be wrong for the next.

Five open questions

Where the work is right now

  1. 01

    Reward from people

    Which signals — corrections, approvals, edits, retention — are reliable enough to learn from, and how do you combine them?

  2. 02

    Online and continual learning

    How does an agent keep improving from live feedback without forgetting what it already does well, or regressing for everyone else?

  3. 03

    Personalization

    One policy for everyone, or one that adapts to each user? And what happens on day one, before there is any history to adapt to?

  4. 04

    User simulators and environments

    You cannot run every experiment on real users. Simulated users are becoming the environments agents are trained and evaluated in — and the sim-to-real gap is its own research problem.

  5. 05

    Reward hacking and sycophancy

    An agent rewarded for approval learns to please. Serving the user's goals rather than their clicks is the alignment problem in miniature.

Why it takes researchers and builders together

The researchers have the methods but rarely the live traffic. The founders have the traffic but rarely the time to build an RL stack. The teams in between — simulators, evals, environments, the inference and compute behind per-user learning — need both to know what to build. Progress on this problem depends on those three groups comparing notes on what has actually worked.

Frequently asked questions

Can I present at Agentic AI × RL?

Yes. Email partnership@openstagesai.com and say whether you want to give a talk or a demo, and what your agent or research learns from. Eight teams present, four per round, with 8 minutes each plus 3 minutes of questions. A polished deck isn't needed — bring a demo or something to show the room.

Join us

Keep reading

Related articles.

Questions, salon invites, or a founder story worth telling?

Tell us what you're building and who should hear it. We'll take it from there.