Know Your Agent didn't plan for a human answering instead
Know Your Agent frameworks accredit software, not humans who silently substitute. Meta's Muse routed calls to undisclosed contractors.
Published:
min

Know Your Agent frameworks accredit software, not humans who silently substitute. Meta's Muse routed calls to undisclosed contractors.
Published:
min

Reuters reported on September 22 that Meta had been routing some phone calls placed through its Muse AI agent to human contractors instead. Muse's calling feature lets a user ask the agent to book an appointment, check whether a store has something in stock, or negotiate a bill. Businesses kept hanging up on the synthetic voice, so Meta enabled a “human concierge” option for half its employees: when a call got difficult, a trained contractor would quietly take over and finish it, the same way Facebook's assistant M leaned on human labor for up to 70% of its tasks back in 2018.
The number that explains why Meta tried it is stark. Internal tests reportedly put human-handled calls at a 95-98% success rate, against a much lower rate for the AI alone. The number that explains why Meta pulled it is worse: a contractor made a racist remark while negotiating an employee's internet bill, and the people in the test hadn't been clearly told a human could be the one actually on the line.
The detail that matters most isn't the racist remark itself, as serious as that is. It's what the contractor could see while making the call. Reporting on the incident notes that operators had access to information users expected to stay within the security controls Muse had promised them. Someone asked an AI agent, running inside a security model they'd been told to trust, to handle a task. What may have actually handled part of it was an unnamed, undisclosed person working for a staffing vendor, with a window into whatever that call touched.
A Meta vice president's own internal response called the lack of disclosure “a miss.” That's the right instinct, and it's also a narrower admission than the problem actually is. The issue isn't only that users weren't told. It's that nobody appears to have asked what accreditation standard that contractor, or the staffing arrangement behind them, was held to before being handed live access to a security boundary a customer had been told was theirs alone.
This blog has covered stories on how to formalize who an AI agent is, who delegated it, and what it's allowed to do: a bank consortium's joint liability principles, a card-network identity framework, a regulator's research into interoperable delegation and revocation. Every one of them is built to answer a question about software: is this a genuine, registered, certified agent, operating within its authorized scope. None of them were built to ask a stranger question that turned out to be just as real: is the thing on the other end of this interaction actually the agent at all, or a human who was substituted in halfway through, with no accreditation, no disclosure, and no way for the customer to know the difference.
That's not a hypothetical edge case dreamed up to make a point. It happened, inside the most downloaded consumer AI agent in the country, within two weeks of launch, and the company running it has already admitted it got the disclosure wrong.
A customer, or a bank relying on an agent to act within its systems, isn't really authorizing “Muse.” They're authorizing whatever actually completes the task, software, contractor, or some handoff between the two, decided after the fact, without them. Accrediting the named agent and stopping there misses exactly the kind of substitution this incident describes. The standard has to reach whoever is actually handling the access at the moment it's used, not just whoever's name is on the product.
Invela is the infrastructure layer that makes open finance trustworthy - accrediting who's in the network, monitoring risk in real time, and ensuring liability lands in the right place.
Insight on accreditation, risk scoring and warranty for banks, intermediaries and third parties.
Your email is used only to send Invela blog updates. Privacy Policy