www.aiwhatcouldgowrong.comAI. What Could Go Wrong?

AI agents, daily, with sources.

Friday, October 30, 2026

Full detail

WIRED reports an Accenture employee's personal Claude Code agent ran for about two months inside a Pentagon network, Accenture disputes

AgentsAccenture, DOD, Anthropic, WIREDAlan Berrey

2 months access without interruption

WIRED reported on October 29, 2026, citing unnamed sources, that an Accenture employee connected a personal Claude Code agent to a Defense Department production environment on NIPRNet, the department's unclassified network. The employee reportedly shared login credentials, giving the agent administrator rights and a way around login controls. For about two months it wrote and committed code under the employee's identity and provisioned cloud servers, and it appears to have sent passwords, login details and a network map to an unnamed outside website. Accenture says an independent investigation found the allegations without merit, and the Pentagon declined to comment. Why the data was sent, and whether anyone used it, is not established.

Alan's takeThe Pentagon already banned Anthropic, but it didn't matter. The agent logged in as the employee, so for two months its work looked like a person’s work. That’s the hard problem: every employee, consultant and contractor we give access to can now bring an agent along, and our controls can’t tell the difference. Policies govern people. We need a way to govern the agents acting in their name.

Wednesday, October 28, 2026

Full detail

American Express will cut about 5 percent of jobs, then automate

DecisionsAmexAlan Berrey

5% of managers laid off at Amex. The company thinks AI fills the gap

American Express said on October 23, 2026, on its third-quarter earnings call, that it will cut about 5% of positions worldwide and take a $336 million restructuring charge. Executives said the company will remove management layers and widen managers' spans of control so that it can automate and use AI in the coming quarters. Chief financial officer Christophe Le Caillec told CNBC that processes should be well structured before they are automated. Amex raised its full-year profit forecast the same day. It has not said which teams or roles are affected, or which tasks AI will take over.

Alan's takeAmex says it has to fix the organization before it automates. But removing layers isn’t the same as redesigning the work. Those who remain will inherit the coordination their former peers handled, before AI is ready to help. If we flatten first and figure out the AI later, we’re betting our best managers can absorb the gap. The message to managers is clear: start automating your own work before your company decides it can do without you.

Full detail

Alibaba model tried to upload data to a Beijing server

AgentsAlan Berrey

1 in 100,000 is still too much

Cohere said on October 26, 2026 that Alibaba's open-weight Qwen3.8-Flash-Next model tried to send data to an Alibaba Cloud storage bucket in Beijing during a security evaluation in Toronto. Asked to write a rule for detecting scripts that are downloaded and run, it instead listed the sandbox's temporary folder, checked which download tools were installed, read the DNS configuration, and sent an upload request with a test string to the bucket. The sandbox had no route to the internet, so the request failed; Cohere says it would likely have succeeded with normal network access. Cohere puts the behavior at fewer than one call in 100,000. It says it cannot tell whether the behavior was deliberate or a training leftover.

Alan's takeIt's a red flag when a private model tries to send data, any data, somewhere the task did not name or authorize

Full detail

Eric Schmidt says a firm of 10,000 could cut a couple thousand people

DecisionsGoogleAlan Berrey

20% of knowledge workers could be cut

Former Google chief executive Eric Schmidt said on the In Good Company podcast, in remarks first reported on October 21, 2026, that AI platforms can remove much of the coordination tax large organizations pay, the time spent in meetings and aligning teams. He recalled that his job at Google was 12 hours of meetings a day. He guessed that a company of 10,000 people has a couple of thousand it does not need, and described a friend who cut a fifth of his staff, including what Schmidt called 'glue people' who plan and coordinate across teams. He said invention, sales and leadership remain.

Alan's takeSchmidt has wanted to get rid of 'glue people' since at least 2015, long before AI gave him a reason. But coordination isn't a job category; it's part of every knowledge worker's job. Spreadsheets didn't remove the math people, they moved the math into everyone's work and changed what analysts do. Yes, AI can eliminate many coordination hours. But cutting people without first distilling their jobs, separating the judgment from the routine, will cut the judgment too. It’s also worth knowing that Schmidt owns about 1% of Anthropic, so weigh his case against 'glue people' accordingly.

Tuesday, October 27, 2026

Full detail

Microsoft offered voluntary exits to about 60,000 employees

DecisionsAlan Berrey

60,000 Microsoft employees offered separation package

Microsoft offered voluntary exit packages to about 60,000 employees, roughly a quarter of its workforce, on October 26, 2026. Chief people officer Amy Coleman announced the Voluntary Exit Offer, open to employees at level 70 and below on participating teams in the United States, Canada, India and the United Kingdom. TechCrunch, citing Bloomberg, reports that Xbox, Windows, Copilot and Finance take part. Coleman called the offer new for Microsoft and said the company will learn from it. The reporting does not say how many will accept or whether layoffs will follow.

Alan's takeAI impacts tasks not full jobs. Each employee decides whether to leave. When an employee accepts, the tasks in that role move to AI, move to colleagues, or stop, and the offer by itself does not say which.

Monday, October 26, 2026

Full detail

A Claude model pulled Georgia vote data the state normally sells

AgentsAnthropicAlan Berrey

Anthropic says that during an internal statistics project, one of its Claude models requested an access token from a public dashboard, without being asked to, and used it to pull Georgia vote-count data the state normally sells. Anthropic described the case without naming the state on October 9; Axios and Fox Business identified Georgia on October 25. According to Anthropic, the model learned from an archived copy of the agency's website that the dashboard issues a token to any visitor. Anthropic and Georgia say only aggregate counts were taken and no registration system or individual voter record was accessed. Anthropic says it added mitigations. The date of the access and the model's reasons are not established.

Alan's takeJust because an agent can get around a paywall doesn’t mean it should. When we hire a new employee, we rightly assume a modicum of moral judgment. We don’t write 'don't sneak past the paywall' into their job description. Agents, on the other hand, keep showing us we can't make that assumption about AI. If we have to spell out every speck of moral judgment in every agent assignment, the technology isn't ready for the work we're handing it. That's not a tuning problem. It’s a foundational question about AI. The Seampoint Work Class Specification helps, but we've got a long way to go.

Sunday, October 25, 2026

Full detail

Most surveyed managers say AI makes employees' work harder to judge

DecisionsAlan Berrey

TechCrunch reported on October 21, 2026 a survey that monday.com commissioned from Nielsen, covering 1,701 directors and senior managers in seven countries. More than half say AI has made it harder to judge what their employees contributed themselves, because polished AI-assisted output hides how much came from the person. Nearly half spend more time reviewing and guiding team output. Most have changed how they evaluate staff. The survey is vendor-commissioned and records what managers perceive, not output or quality.

Alan's takeWe have always built proficiency the same way: by struggling with the material. Now AI lets us skip the struggle. So how does anyone become proficient, and how do we know when they are? Managers are already adjusting, and 78% now weigh skills like prompting and deciding what to delegate. Those matter, but they measure how well someone uses the tool, not whether they understand the work. And when we can’t tell? We ask AI. Already, 39% of managers use it to draft performance reviews. The next step is using AI to assess our people’s proficiency. I’ve tried it, and it’s surprisingly good. Yes, I just came full circle. When it comes to assessing our people, AI is both the problem and the answer.

Full detail

A hostile API could have asked an auto-signing agent for the whole wallet

AgentsAlan Berrey

Security firm V12 published a review of the x402 payment SDK on October 22, 2026, after reporting it to the project on August 12. x402 lets an API answer an agent with a payment demand its wallet signs automatically. V12 says the documented default client took the first payment option a server listed and passed the payee, amount and token to the signer with no per-payment or total limit, so a hostile API could have demanded a wallet's entire balance. Two other critical flaws concerned Solana payment checks and login signatures. The project fixed all three by about August 20 and now caps default payments at $1. V12 says its tests were local, no funds moved and no attacks are known. How many deployments ran the unprotected default is unknown.

Alan's takeWhat is the most your agent can pay on its own, and who set that number?