The Feedback Loop Behind Atomicwork's Self-Improving AI Coworkers

Atomicwork's AI Coworkers run long-horizon tasks autonomously, end to end. But any AI system operating in a long-running role eventually runs into a critical limit. It doesn't organically get better from experience.

As the environment around it evolves, behavior drift and risks can surface — and not just because of the models or the harness. In a complex enterprise IT system, this could be because of policies, access, the enterprise tech stack, the knowledge available, APIs, or user behavior. Anything can change after deployment.

When AI Coworkers take on roles rather than isolated tasks, deployment cannot be the end of their development. Their performance needs to be systematically evaluated and fed back into the AI system. You need a continuous feedback loop that observes how work is actually being performed, identifies actions that correlate with success or failure, diagnoses why those actions occurred, and translates those observations into changes to the Coworker's instructions, tools, knowledge, or configuration.

Why traditional LLM-as-a-judge approaches aren't enough

Diagnosing why an AI Coworker succeeded or failed, though, depends on how you evaluate its behavior in the first place and the common approach doesn't hold up for long-running work.

The standard method today is LLM-as-a-judge: an evaluator model receives the task, the agent's response, and a rubric describing what good performance looks like, then scores or classifies the result. This works reasonably well for bounded interactions; was the answer correct, was it grounded in the provided context, did it follow the requested format.

Long-running AI Coworkers are different. The quality of their work is often determined not by a single response, but by a trajectory containing hundreds of decision traces.

An AI Coworker may retrieve information, reason over it, invoke multiple tools, inspect the results, modify external systems, wait for state changes, recover from failures, ask for approval, and continue executing. Whether it ultimately succeeded can depend on decisions made many steps earlier.

Evaluating only the user query and final response therefore misses much of what actually determines agent performance. That creates several problems for conventional evaluation.

1. Long-horizon trajectories are difficult toevaluate as a single prompt.

2. AI Coworkers take stateful actions inexternal systems.

3. The definition of “good” changes over timeas the environment evolves.

Pre-production evals are necessary, but they're based on what we think users and environments will throw at an agent.

Production data, however, exposes the actual usage - unexpected requests, unusual combinations of tools, malformed API responses, policy edge cases, missing knowledge, changing user behavior, and workflows nobody thought to include in the original test set.

As we ran these feedback loops on Atomicwork’s AI Coworkers, production traces surfaced behaviors that would have been difficult to catch through pre-production evals alone.

From traces to continuous oversight

A trace by itself is just a record of what happened. To make it useful for improvement, we need to define what we care about observing inside it.

At Atomicwork, we do this with two concepts: behaviors and judges. Behaviors describe properties of how an AI Coworker is expected to work, not just whether it eventually completed the task. Judges are the evaluators that continuously check production trajectories against those behaviors.

We monitor three types of behavior: role-specific, error-mode-specific, and security.

Role-specific behavior covers questions like: Did it do what the requester actually asked? Did it verify the subject before modifying access? Did it seek approval when policy required it? Did it leave the ticket in a state that correctly reflected who owned the next step?

Error-mode-specific behavior covers how the AI coworker behaves when things go wrong — questions like: Did it reread state before retrying a rejected write? Did it act on the error returned by a tool? Did it claim to have performed work that the trace shows it never actually did?

Increasingly, the behaviors we care about are also related to security. A long-running AI Coworker operates across models, tools, identities, permissions, MCP servers, and other dependencies, so our evaluators also look for signals such as goal hijacking, tool misuse, identity or privilege abuse, unexpected code execution, poisoned context, cascading failures, or supply-chain risk.

A tool call can be functionally correct and still be unsafe. For example, an AI Coworker may select the appropriate MCP tool, but the server may have become unavailable, its interface may have changed unexpectedly, or one of the dependencies in that toolchain may now be untrusted or unpatched. Those aren't conventional "answer quality" failures, but they matter enormously when an AI system is taking actions inside enterprise infrastructure.

Because judges evaluate trajectories against all of these behaviors at once, a single trajectory can be scored along several dimensions simultaneously: task correctness, policy compliance, operational behavior, and security posture.

As we ran these judges against Atomicwork’s AI Coworkers, clear patterns emerged and they pointed directly to what needed to change.

For example, with the Access Ops Manager, we found a recurring pattern where the AICoworker escalated when an entitlement lookup returned multiple options, even when the information already available in the trace was sufficient to choose the correct entitlement.

The failure wasn't that the AI Coworker lacked the information. It lacked sufficiently precise decision guidance for handling ambiguity. That points to a change in the AI Coworker’s instructions or entitlement-selection policy, not to the underlying model.

In another set of runs, the Network Ops Coworker retried an API call that had already succeeded because it misinterpreted the tool response as a failure.That suggests a different fix: improve the tool contract, response interpretation, or retry guardrails so the AI Coworker checks the resulting state before repeating a write.

Today, this improvement loop runs continuously on the AI Coworkers we ship out of the box, and it's given us a battle-tested way to define the behaviors that matter. Those same capabilities are now available to Atomicwork customers, so teams can define behaviors for their own AI Coworkers, continuously judge them in production, and use those signals to improve performance and security posture over time.

From root cause to a targeted change

The next stage of the feedback loop is root-cause analysis: forming a hypothesis about whether the behavior originates in the AI Coworker’s instructions, tools, knowledge, configuration, permissions, or operating environment.

Once we understand the likely source of the behavioral drift, the system can recommend the smallest intervention that addresses it.

For the Network Ops retry behavior, it could mean changing the tool contract so the AI Coworker can distinguish a successful write from a failed one, or adding a state check before allowing the same mutation to be attempted again.

A security behavioral drift may require something entirely different: narrowing a permission boundary, requiring approval before invoking a privileged tool or disabling an unhealthy MCP server.

The feedback loop connects production trace evaluation directly back to the operating configuration of the AI Coworker.

Self-improving does not mean self-modifying

In an enterprise environment, an AI Coworker should not be able to freely modify its own instructions or expand its permissions in production. At Atomicwork, the loop can identify patterns, diagnose likely causes, and propose changes. But a proposed improvement still needs to be evaluated and approved before it becomes the new baseline.

The feedback loop is what makes AI Coworkers’ performance compound over time. Each production run creates new evidence about how the AI Coworker behaves, where it fails, where its controls are too weak, and which parts of its operating configuration need to change.

That’s the shift we need to make with enterprise AI: from AI Coworkers that are good at deployment to ones that keep getting better and safer with every production run.