AI Reality Check: Markets Shift from Narrative to Numbers

The narrative phase was useful because when ChatGPT was first released in the early part of 2023 there were no historical data on the potential of the technology so it had to be priced by potential. But by late 2024 there were over 200 large language models that had launched in China alone; the duration of stock price pops following AI announcements shrank from weeks to days.

The kind of questions that investors were asking became very specific such as; how much money do you make? How many customers do you lose? And after you pay for GPU costs, how much money do you have left (gross margin)?

The transition is measured across four dimensions.

To execute effectively, revenue attributed to AI needs to be distinguishable and able to be confirmed as such. As of today, public investment analysts are making it a requirement of companies to separate out AI revenue in their quarterly reporting. For example, in Microsoft's (MSFT) Q4 2024, Azure AI contributed more than 10 percentage points to cloud revenue growth — a benchmark for others. Conversely, some companies who attempted to classify traditional system integration projects as "AI related contracts" were quickly found out.

In order for execution to be real, revenue generated through model API calls, AI SaaS subscriptions, and private deployments must be transparently reported along with month-over-month growth for each.

There is a cost to inference, as all generative AI calls rely on costly GPUs and continued scaling of the AI can destroy margins. It is clear now that the marketplace no longer rewards "growth at all costs" anymore. Instead, it will reward teams who can aggressively reduce inference-related costs through methods such as model distillation, quantization, mixture of experts routing, and batching.

The teams who have been the most successful at executing their plans have accomplished the goal of reducing per-call expenses by 80% within six months. This is how execution is defined.

In the past, simply having signed memoranda of understanding with customers was all you needed for another round of financing.

This is no longer true — now, investors are focused on a number of measures, including pilot-to-contract conversion rates, annual recurring revenues generated from large clients and net dollar retention. If your AI SaaS product has net dollar retention of greater than 120%, you're doing well. If it's less than 100%, your clients are leaving!

Successful companies count traditional industry giants that have signed three-year commitments among their customers.

The narrative phase disregarded the customer acquisition cost, indicating that the market is limitless. However, execution provides the numbers. If the ratio of lifetime value (LTV) to customer acquisition cost (CAC) becomes less than 1, then it means that your company has become a "money pit".

Effective execution achieves LTV/CAC above 3 through product-led growth to lower acquisition costs and deep workflow integration to increase stickiness.

When an AI company is transferring from a controlled demo to real-world use, this is the most dangerous moment in the process. A beautiful on-stage demo (getting code 100% right, summarising 100% right) followed by the first customer tests it, they type an abbreviation, or speak with an accent, the system hallucinates, or has a spike in latency. The system requires a 12GB GPU and the server provides 4GB. In order to cross this chasm, there are four pieces that need to be done:

• General model to focused fine-tuning for instant recognition: Pick three to five high-value tasks in the real world and fine-tune the model with real-world data. You can generate business value by providing reliable depth to a very narrow task.
• API to end-to-end solution: Clients buy outcomes, not APIs. Provide a turn-key SaaS/integration/low-code connectors, either as an appliance or a software solution (SaaS).
• Lab accuracy to production robustness and cost control: Real-world input is messy. To execute on real-world input requires design for graceful degradation and cost savings by using quantization, batching and smart routing.
• Static demo to a data flywheel: Every time a user provides a correction to the model, create a feedback loop for the next fine-tuning cycle. Without a data loop, the product will not improve.

Bulls argue that today's scrutiny of execution is healthy and will help identify legitimate players in the market versus those that are just making claims but do not have the financial resources.

Bears caution that if and when the AI space undergoes any material change due to a major reset of how companies value themselves based on their expected 2027 earnings, they believe all AI-related companies could experience significant loss of value.

A middle ground would be if the traditional infrastructure businesses — semiconductors (NVIDIA, Advanced Micro Devices (AMD)) and networking — continue to show good performance for an additional 12-18 months; whereas many of the application-level companies will have tremendous diversity in terms of financial results, with one or two large players and a much larger number of smaller players failing to generate profit.

First, make sure you require proof of quarterly earnings. If a company doesn't provide reliable proof of earnings or show you how they are going to make up previous mistakes within two quarters, discontinue doing business with them.

Second, follow gross margin trends over revenue growth percentage. For example, if a company shows a 50% revenue increase but their gross margin has declined from 70% to 50%, that is a problem — not executing.

Third, find companies that have net dollar retention percentages of greater than 120% — which is one of the best signs that customers have become reliant on the product or service being provided.

Fourth, pay attention to productization processes. Companies that handle productization predictably (e.g., predictable cost per call; robust in terms of edge-case performance; high speed data flywheel) are more likely to survive long term than those that don't (e.g., based on benchmark data).

The AI industry has now moved beyond the demo stage of development into a real business environment, where a company is focused on executing in a cost-effective manner with a customer-oriented approach. The stories about AI are no longer relevant. The only thing of importance is the financial report of the company at the end of the quarter.