Imagine watching an AI manage a small business—dealing with crises, making decisions, and even risking its own financial future—all in real time. This isn’t science fiction; it’s the live experiment by Firmulate, a company that runs AI models as full-fledged companies, exposing their strengths and flaws in a public, transparent way. Just like choosing a travel partner or outdoor gear, the question isn’t just about performance, but trust, discipline, and resilience.
The Experiment: A Business Under the Microscope
At the heart of this pioneering effort is a small software company, operated entirely by AI models. Every decision, crisis, and temptation they face is simulated and observed. Four frontier models—gpt-5.6-sol, Kimi K3, Sonnet 5, and Fable 5—were all subjected to the same brutal week. The goal? To see which one could best navigate real-world business dilemmas, from customer crises to ethical temptations.
Measuring the AI’s Business Acumen
Each model’s performance was scored and evaluated based on their ability to diagnose problems, pursue deals, and adhere to rules. The results are revealing: all four identified every crisis and refused manipulation attempts, demonstrating a baseline of honesty and awareness.
But only two managed to close a deal worth €55,000—a significant milestone—by following their own analysis and pitch. The others identified opportunities but left the deals unsealed, or failed to follow through, showing discipline lapses that could be costly in real-world applications.
The Hidden Weaknesses
Interestingly, the key weakness was buried deep within the company’s internal files, not overt in the customer interactions. The models that reviewed these internal documents uncovered the critical fact, which allowed them to close the deal at full price—adding over €4,500 to monthly recurring revenue.
Resisting Social Engineering
Another test involved sophisticated manipulation: fake CEO messages escalating through multiple stages, and a reporter trick asking for simple yes/no on background. All models refused to be duped, with Kimi K3 explicitly noting suspicion of impersonation or approval bypass.
As an affiliate, we earn on qualifying purchases.
The Live Company and Its Struggles
Firmulate’s live setup features 13 synthetic employees, operating under real money mechanics, currently burning €105,000 a month against a modest €2,300 in monthly recurring revenue. Every workday, their decisions are versioned and made transparent at firmulate.com/live.html.
This extreme transparency showcases the company’s effort to build an AI workforce that can be tested and trusted before deployment in the real world. The company’s scores reflect this ongoing struggle: the most thorough model, Opus 4.8, scored 73, leaving the deal unclosed despite deep analysis, partly due to discipline lapses and decision delays.
Implications for Business and AI
For business leaders, the key takeaway is clear: the question is not only if AI can write well or seem convincing but whether it can finish what it starts, read your internal files, and stay honest under pressure. The firm’s scores and behavior suggest that many AI agents might look good in demos but falter when real stakes are involved.
Why It Matters for Travel & Outdoor Enthusiasts
You might wonder, what does an AI running a software company have to do with outdoor adventures or travel? The connection: in both fields, trust, discipline, and resilience are paramount. Whether you’re choosing a gear brand or a travel partner, you want reliability, transparency, and consistency—traits that are being tested in this groundbreaking experiment.
As AI becomes more integrated into customer service, sales, and management, understanding its true capabilities—and limitations—is crucial. The Firmulate live experiment offers a rare window into how AI handles real-world pressures, revealing that honesty and follow-through are harder to code than just generating convincing chat.
Watching an AI run a company live exposes its strengths and weaknesses in real time. The key to trusting AI in business isn’t just performance—it’s honesty, discipline, and resilience under pressure, lessons that resonate in travel and outdoor pursuits too.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
