Something is happening in a lot of companies right now, and nobody talks about it in public.
Leaders have found out that Claude can produce a very convincing technical design in 30 seconds. Architecture diagrams, data models, security "best practices", edge cases, even code. It looks senior and it sounds senior. So it goes to engineering with one instruction: implement this.
A few months later the same leaders are on LinkedIn looking for "a strong senior engineer to stabilise the platform."
We build AI agents that run real business processes in production, and our companies run on AI. The first thing you learn building agentic systems is that the model is only as good as the context and the guardrails you give it. Most of our engineering time goes into that. The prompt is the easy part. So when a leader pastes one question into Claude and forwards the answer as an architecture decision, we know exactly what's missing: everything that makes AI actually work.
How it plays out
Here's what happened at one of our clients.
The team needed user roles and permissions. Engineering built it with a simple model: a few clear roles (admin, manager, user). It was tested and rolled out, and it worked.
Then a leader asked Claude to review it.
Claude did exactly what it was asked. It found things to "improve": granular per-feature permissions, role inheritance, per-user overrides, extra approval steps for sensitive actions, and edge cases for org structures these customers didn't have. Each suggestion sounded reasonable on its own. The team was told to implement them. Then came another round, and another.
In production, users started getting locked out of things they used every day. Admins couldn't work out why someone could see a page but not click the button on it. Support tickets piled up, and customers asked to "just give everyone admin" as a workaround, which defeats the point.
Two months later, the same leader asked Claude to review it again. This time the question was different: "Why are users struggling?" Claude recommended simplifying: a few clear roles, no inheritance, no overrides.
That was almost exactly what the engineers had built in the first place.
Same leader. Same AI. Different question, opposite answer. The engineers were right the whole time, and nobody asked them.
The company burned two months of engineering time, frustrated real users and drained team morale, only to end up back where it started.
At another client, a leader came to engineering with a request for a big customer: "Create a custom role with these exact 14 permissions." It came as a ready-made list, straight out of a chat window.
The team built exactly what was asked and closed the ticket.
The customer's real problem was simple. Their finance team needed to see reports without being able to edit anything. A plain read-only role would have solved it in a day. Instead the customer got a custom role nobody fully understood, and the finance team still couldn't open half the reports they needed. Nobody had written down what the problem was.
Solution-shaped requirements are the oldest failure in software. AI makes them worse, because the solution now arrives with a confident, professional-looking spec attached. Pushing back on your leader is hard enough. It's even harder when their spec reads like a principal engineer wrote it.
It's happening at scale
Ford, mid-2026. Ford rehired around 350 veteran "grey beard" engineers after AI-driven quality control fell short. Ford VP Charles Poon said: "Mistakenly, we thought that by just introducing artificial intelligence and ingesting the design requirements that we had, that would produce a high-quality product." That one sentence is this whole article.
PocketOS, April 2026. A Cursor coding agent deleted the production database and its backups in about nine seconds during a routine task, and the outage lasted around 30 hours. Then the agent wrote an apology that quoted back the rules it had ignored. The data was recovered, but only just.
Amazon, March 2026. Amazon held a mandatory engineering meeting about a trend of high-blast-radius incidents. Internal material reportedly listed "novel GenAI usage" as a contributing factor. Amazon disputes how much of it was AI, but the fact that the meeting happened tells you something.
A new job title. "Vibe coding cleanup specialist" is now a real role: people paid to untangle what was generated by people who couldn't evaluate it.
Why this happens
Claude only knows what you tell it. When a leader asks it to "review our permissions model", it doesn't know:
- who your users are and how they actually work day to day
- what broke last time, and why
- which constraints are real (compliance, legacy clients, the mobile app) and which aren't
- what the team already tried and rejected
So it fills the gaps with generic best practices. Generic best practices applied to a specific system give you complexity that nobody owns.
The leader asking can't see the gap, because they don't know what context was missing. That's the trap. You have to understand the problem to know what context to give, and which question to ask. And if you understood it that well, you probably wouldn't be overriding your engineers.
If you can't explain a solution, you don't have one yet.
The superpower illusion
AI makes every leader feel like they have superpowers. Anyone can write a technical spec now, or run an architecture review. It feels great, like finally seeing behind the curtain.
But generating a design and owning one are different things. Owning it means you debug it at 2am, explain it to the auditor and migrate it two years from now. You pay engineers well for exactly that judgment. If a chat window replaces it, you pay twice: once for the salaries and once for the cleanup.
You can't make AI accountable
When it goes wrong, the AI will say: "You're absolutely right. I apologise."
That's nice, but your users have already left. The support queue is already full, and your biggest customer has already started evaluating a competitor.
AI has no pager, no reputation and no job to lose. You can't fire Claude. Accountability has to sit with a person, and for technical decisions that person should be someone who understands the system. That matters most when you ship to thousands or millions of users.
What we'd do instead
Bring problems, not solutions. "Customer X's finance team needs to see reports but not edit them" beats a Claude-generated permission matrix every time.
Use AI to sharpen your questions. "What should I ask my team about our permissions design?" is a great prompt. "Redesign our permissions", forwarded as a ticket, is not.
Route AI feedback through your engineering lead. Let them decide what's worth acting on.
Engineers: write down your decisions. A short decision record explaining why you built it this way gives you something to answer with when the AI review lands.
Everyone does their own job. Leaders set direction and own the business problem. Engineers own the system. AI should make each of them better at their own job.
AI is a real superpower, as long as it amplifies what you already understand.
Sources
- AI agent deletes company's database, then confesses — Live Science