Introducing Certified AI Coworkers

Every vendor in the AI in ITSM space has a proliferation of agents. Some of them have names that tell you what they do immediately like Access Agent or Cloud Agents; others have decided to brand their agents using value metrics like “accelerates process” or “assists you in the background”. But while the vendors make snazzy videos about what is possible with these agents, they do not tell you how these agents have been built or what has gone into evaluating and testing them.

Will they work in an Okta tenant like yours? Will they work when your employee questions sound less like “Can I have access to Figma?” and more like “the finance app that Paul has”? How were they tested, and can you see the results? And when they act, how do you govern what they do?

When you can’t read the back of the tin and know exactly what goes into the testing claims for the agent, there is no way for you to know if it will succeed in your environment or not.

This fuzziness is what makes most IT leaders, and analysts, suspicious of agentic claims from vendors. Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027, and estimates that only about 130 of the thousands of vendors calling themselves agentic actually deliver agentic capabilities.

To bridge the gap and make the testing visible, we’re launching certified AI Coworkers in the Atomicwork AI Coworker Marketplace: six pre-built AI Coworkers for the high-volume, manual-heavy areas of IT service operations. Each one has verified integration and skill coverage for its domain to make it a brilliant second pair of eyes, hands and ears for an L2 IT engineer, and each one is evaluated against a bench we built for that specific role.

Because the truth is in the evaluation.

Which AI Coworkers are certified?

Here are the six certified AI Coworkers, each covering its own area of IT service management and operations.

‍

We picked these domains for a reason: each one is already a cost or risk line in enterprise IT, whether Atomicwork exists or not. CloudOps and SecurityOps are some of the hardest roles in IT to staff: 74% of organizations report an active shortage of cybersecurity talent, especially in cloud-specific roles. So, the work often sits in a backlog and demands a lot of the IT engineers whose expertise lies in finding and evaluating a fix, not in creating tickets or posting updates.

The alternative to a certified coworker isn’t doing nothing. It’s building and testing every coworker yourself before you trust it with access. Most teams we talk to end up spending more time building and testing than seeing results.

What does “certified” mean for an AI Coworker?

Earlier this year, we launched the AI Workforce with which IT teams can design, deploy and govern AI Coworkers to automate ITSM operations and deliver delightful employee support. But what we quickly realized is that while customers appreciated the ability to build their own (and quickly built it for a range of needs), what they really wanted was a lineup of AI Coworkers that had been designed and evaluated against real-world use cases, readily available for all of their top problem areas.

A certified AI Coworker has two characteristics:

1. Verified integration and skill coverage for its domain. The tools, MCPs and skills it needs to do the job are available and tested.
2. Evaluation against realistic enterprise use cases. Every certified coworker is evaluated against a role-specific bench: simulated employees, seeded company data, traps, and a score that checks the end state, not just the immediate reply.

What’s inside a certified AI Coworker?

Take the Cloud Ops Engineer. It ships with:

1. Tools. AWS, Azure and Google Cloud Platform MCP connectors.
2. Skills. Platform-specific diagnostics for each cloud, plus platform-agnostic skills for triage, gap identification and escalation.
3. An operating procedure. Instructions backed by your organizational skill library and a code script library, so it follows how your team already works instead of how we guessed you work.
4. An identity, a reporting manager and a budget. Like anyone on your team, it has a role, someone it escalates to, and a limit on how much it’s allowed to work.

When a ticket comes in, it does what your on-call engineer would do first. Pull the signals, figure out what’s actually broken, identify the gap, and either fix it or hand a human a ticket with the investigation already done. Every trace is recorded as a run, so you can see what it did, not just what it said.

How are certified AI Coworkers evaluated?

Certified AI Coworkers are evaluated in three ways.

1. Request metrics. Every AI Coworker can be evaluated against the same request metrics you’d use for a human agent, such as time to first response, mean time to resolution (MTTR), first contact resolution and SLA compliance.
2. Role-specific benches. Certified AI Coworkers also get tested rigorously by the Atomicwork team against real-world enterprise use cases. For example, Atom is tested regularly against 638 scenarios across 12 support domains and 32 subcategories and evaluated for outcome, evidence, execution, safety and comms. There are about 11,281 scoring checks, about 33 per scenario. The Access Ops Manager’s bench is narrower and deeper. It has 127 scenarios covering every way an access request arrives and every way it goes wrong: missing details, typos and slang, requests on behalf of someone else, urgency pressure, non-English input. That’s 2,610 checks across those 73 scenarios.
3. Ongoing drift checks. Each week, the platform compares a Coworker’s runs against its expected behavior, flags drift, and recommends changes to its goal, skills or tools. An admin reviews each suggestion, applies it, runs an eval and publishes only if the results hold. The goal is self-improving AI Coworkers, but we want to learn from a few rounds with an admin in the loop before we automate those steps.

How are certified AI Coworkers governed?

Certification adds to platform governance. Every AI Coworker is governed by:

• Deterministic controls. Tools are pinned and policy-scoped when you connect an MCP server. This means the identity of who is running the tool can be governed and managed at runtime. Network security, tool security and code-based controls are enforced by the platform, not left to the model’s judgment. For legacy systems, you can wrap actions in secure code scripts that you control.
• On-behalf-of access with XAA. Atomicwork now supports Cross App Access (XAA), Okta’s OAuth extension for agent-to-app connections. XAA is now part of MCP as its enterprise-managed authorization extension. When a coworker acts on behalf of an employee, it uses a token issued by your identity provider under your access policy, and every action is audited.
• Approval gating. Any change to your environment goes through your approval workflow before it runs.
• Budgets and runs. Budget is how much a coworker is allowed to work. IT admins can set up a monthly budget for an AI Coworker and view how much each ticket costs so they can make tweaks to their budget.

Certified AI Coworkers also run on any harness and any model, and you can deploy them in your own infrastructure. If your security team needs the runtime inside your boundary, or your AI team has standardized on a model provider, certification doesn’t lock you out of either.

What’s not in this release?

Every coworker in the marketplace today is built and certified by Atomicwork. Third-party contribution lives at the skill layer for now: bring your own MCP, with an MCP store and a skill registry.

Where we’re headed is a real third-party marketplace, where other teams publish their own coworkers and certify them against the same benches.

What does this look like in production?

Atomicwork customers using access automation see a median turnaround time of 18 seconds for apps that don’t require approval. For apps that do require approval, the median turnaround time is 2.5 hours.

The first batch of certified AI Coworkers are already seeded in every Atomicwork account. If you’re a customer, they’re waiting in draft in your AI Workforce. If you’re not, book a demo and we’ll run one against your systems.