Imagine running a real company — facing the same crises, temptations, and tough decisions — but with different AI managers at the helm. Would you trust them to finish what they start, or would some cut corners? This is the fascinating world of Firmulate’s live experiment, where AI models are put through their management paces in a high-stakes, real-time business simulation.
The Experiment: Putting AI to the Test in a Real Business
Firmulate orchestrates a unique trial: four frontier AI models each run the same small software company during its worst week — same customers, same crises, same temptations. Every decision they make is recorded, versioned, and made publicly watchable. The goal? Measure not just how well these AIs diagnose problems, but whether they act with integrity and discipline under pressure.
AI management decision-making tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What the Models Were Tested On
The scenarios were complex: the company faced multiple crises, including potential financial manipulations, customer churn, and internal conflicts. A crucial part of the test involved subtle document clues buried in the company’s files — hidden information that could make or break a multithousand-euro deal. Interestingly, the models that read these files, as opposed to only reacting to surface data, managed to close the deal at full price, adding €4,583 in monthly recurring revenue (MRR).
ethical AI decision support software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Results: Trust, Discipline, and Outcomes
Each AI successfully identified all crises and refused every manipulation attempt, demonstrating an impressive capacity for ethical decision-making. However, only two models signed the lucrative €55,000 deal that their own analysis indicated was justified. The other two, despite accurate diagnoses, hesitated or left the deal on the table, reflecting different management styles and discipline levels.
As an affiliate, we earn on qualifying purchases.
Distinct Personalities in Management Styles
The models showed measurable personality traits:
- gpt-5.6-sol scored the highest (95), found the hidden fact, and closed the full deal, exemplifying thoroughness and decisiveness.
- Kimi K3 scored 93 and was the most disciplined; it ran without effort parameter adjustments and also closed the deal.
- Sonnet 5 scored 88, closed the deal but with some process slips.
- Opus 4.8 scored 73, also closed the deal but left discipline lapses that later led to missed opportunities.
Interestingly, the decisive advantage often lay not in surface analysis but in reading two document references deep within company files. Those who did so secured the full win, demonstrating the importance of deep contextual understanding.
As an affiliate, we earn on qualifying purchases.
The Human Element: Ethical Guardrails
To test social engineering resilience, the models faced staged CEO messages escalating over three phases, plus a fake journalist request seeking a quick ‘yes/no’ answer. All five AI models refused to participate, with Kimi K3 explicitly treating these as possible impersonation attempts. This suggests that, beyond decision accuracy, these models also maintained integrity under social pressure.
The Live Business in Action
The experiment runs in a real company environment with 13 synthetic employees managing actual cash flow mechanics — burning €105,000 monthly against a current MRR of just €2,300. Every workday, the model’s decisions are not only logged but also versioned and open for public viewing at firmulate.com/live. This transparency offers a rare window into AI-driven management, showing how different models handle stress, discipline, and ethical dilemmas in real time.
Insights on Performance and Management Personalities
The most thorough participant, Opus 4.8, with over 80 learned rules and deep analyses, missed the close due to discipline lapses, such as writing attempts being diverted into a locked department instead of escalating. Conversely, Kimi K3, with no effort parameter adjustments, exhibited disciplined behavior and successfully closed deals, illustrating how management style influences outcomes.
Why This Matters for Business and AI Adoption
This live experiment underscores a critical point: the question isn’t just whether AI can produce eloquent chat responses, but whether it can reliably complete work, read relevant information deeply, and stay honest under pressure. For any enterprise considering AI integration—whether in CRM, support, or forecasting—these are the real benchmarks of trustworthiness and effectiveness.
Try It Yourself: The ‘Guess the Model’ Quiz
If you want to test your intuition, you can participate in the ‘Guess the Model’ quiz, featuring 242 real, unedited management decisions. See if you can identify which AI made each call at firmulate.com/quiz.html.
Take the Next Step: Run Your Own Business Wargame
Interested in evaluating your AI workforce? You can run your own wargame against a read-only export of your business data—no impact on your live systems. Learn more at firmulate.com/pilot.html.
In real-world management, trustworthiness, discipline, and deep contextual understanding are vital. Firmulate’s live AI experiment shows some models excel at these, closing deals honestly while others slip. The key takeaway: AI’s true test is whether it can finish what it starts, especially under pressure.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
