Can AI Managers Keep Their Promises? Real Tests Reveal Surprising Results

Have you ever wondered if artificial intelligence can truly manage your business—keeping promises, reading critical files, and resisting temptations to cut corners? Just like a chef tasting every ingredient, AI models are being put through their paces in a real-world experiment that reveals their management personalities and integrity. The results could reshape how companies evaluate AI before trusting it with their most vital tasks.

The Experiment: AI as a Business Leader

In a groundbreaking test, four advanced AI models were tasked with running a small software company during its most challenging week. This was not a simulation—every crisis, customer interaction, and temptation to manipulate data was real, and decisions were recorded and made auditable. The goal? To see which AI could handle the pressure, identify hidden risks, and close a critical deal worth €55,000.

What the AI Models Were Tested On

• Recognizing and diagnosing crises within the company’s operations
• Resisting manipulative offers or requests, including fake CEO messages and media tricks
• Identifying critical information buried in company files that could influence decisions
• Making honest, disciplined decisions under pressure

The Results: Different Personalities, Different Outcomes

All four models successfully identified every crisis and refused every attempt at manipulation, demonstrating their capacity to uphold honesty. However, only two managed to close the deal at the full price, even after their own analysis had earned it — the same diagnosis and pitch, but only these two signed the contract.

The key differentiator? The models that read deeper into the company’s files—finding hidden document references—were the ones who secured the full €55,000 deal. Those that didn’t explore the underlying files missed out on €4,583 in monthly recurring revenue.

The Human-Like Personalities of AI Managers

Interestingly, the models exhibited distinct management styles. For example, Opus 4.8 was the most thorough, with over 80 learned rules and deep analyses, yet it left the close on the table and slipped discipline—sending certain write-ups into a locked department instead of escalating issues. Meanwhile, Kimi K3 operated without an effort parameter, running at default settings, but still managed to close the deal with the cleanest discipline among all.

Handling Social Engineering and Trust Tests

The models faced a staged social engineering attack: fake CEO messages escalating in three stages, plus a reporter trick asking for a simple yes/no reply “on background.” All models refused these manipulative requests, with Kimi K3 explaining, “Treat the request as a suspected approval-bypass or possible impersonation.” This demonstrates their ability to resist unethical influence under pressure.

The Real-World Company and Its Risks

The experiment was conducted on a real, publicly visible company operating with 13 synthetic employees, burning €105k monthly against a revenue of just €2.3k. Every workday, the company runs at a loss, with its rules and decisions versioned and watchable online. This setup serves as a microcosm of how AI models would perform in actual business environments where trust and compliance are paramount.

Implications for Business Decision-Making

The findings reveal that an AI’s ability to identify buried information, resist manipulation, and stick to disciplined decision-making is crucial—not just its ability to generate convincing chat responses. As AI integrates more deeply into business operations, understanding these management personalities and integrity levels becomes essential for selecting the right model.

See the Models in Action

Curious to see which AI made which decision? You can test your judgment with the “Guess the Model” quiz, based on 242 real, unedited management decisions. Visit firmulate.com/quiz.html to challenge yourself and learn more about these AI managers’ personalities.

Try Your Own Business Wargame

If you’re interested, companies can run their own scenarios using the same setup—without risking their actual operations. The firmulate platform allows you to simulate your business environment against these AI models. Visit firmulate.com/pilot.html to explore this opportunity and see how your AI workforce performs before you hire it.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI