Alex Karp is not known for pulling punches. The Palantir CEO, speaking at a tech conference in San Francisco yesterday, unloaded on the two titans of generative AI: OpenAI and Anthropic. His target? The pricing model that charges customers by the token—the basic unit of text that these systems process or generate.
“Something has gone completely wrong,” Karp said, his voice flat but his words cutting. “We’re seeing token costs that make no economic sense for enterprise deployment. Companies are being forced to choose: pay a fortune for a closed model, or switch to open-weight alternatives that give them control over costs.”
Karp’s criticism lands at a moment when the AI industry is grappling with a fundamental tension. On one side, OpenAI and Anthropic have locked in customers with impressive performance and user-friendly APIs. On the other, a growing chorus of CIOs and CTOs are balking at bills that, according to Karp, have tripled in some cases over the past six months.
The term “tokenmaxxing,” which Karp used with obvious disdain, has become inside baseball for the practice of optimizing prompts and workflows to minimize token consumption—a tax on innovation, in his view.
“Tokenmaxxing is a symptom of a broken market. You shouldn't have to warp your business logic to fit someone else’s meter.”
The Numbers Don’t Lie—Or Do They?
To understand Karp’s anger, you have to look at the raw numbers. OpenAI’s GPT-4o currently charges $5 per million input tokens and $15 per million output tokens. For a large enterprise processing millions of customer interactions daily, that adds up fast. Anthropic’s Claude 3.5 Sonnet sits at $3 per million input and $15 per million output—cheaper on input, but equally punishing on generation.
Now compare that to open-weight models like Llama 3.1 405B or Mistral Large. Once you’ve provisioned hardware, the marginal cost per token can be ten times lower. The trade-off? You handle your own infrastructure, security, and scaling. But for defense contractors, financial firms, and healthcare companies—Palantir’s core customer base—that trade-off often looks attractive.
Karp isn’t just complaining for the sake of it. He’s betting the house on open-weight models. Palantir’s AIP platform now integrates with Llama, Mistral, and Dbrx, giving clients the option to run models on-premise or in private clouds. “Our customers want sovereignty over their data and their budgets,” Karp said. “They don’t want to be at the mercy of a pricing change in San Francisco.”
OpenAI and Anthropic Push Back—Sort Of
Neither OpenAI nor Anthropic responded directly to Karp’s comments. But both have recently announced cost-cutting measures. OpenAI slashed prices on GPT-4o mini by 50% in June, and Anthropic introduced a batch processing tier that reduces rates by 40% for non-real-time workloads. These moves suggest they hear the grumbling.
But the core model remains unchanged: pay per token. For companies processing billions of tokens a month—Palantir’s clients often do—the difference between closed and open can mean millions of dollars annually. “It’s a pricing model that made sense when AI was a novelty,” Karp argued. “Now that it’s infrastructure, you can’t treat every prompt like a premium good.”
“Treating every token like a premium good is how you strangle the industry you’re trying to build.”
Karp’s critique also hits on a deeper philosophical divide. OpenAI and Anthropic have raised billions in venture capital, with investors expecting returns that justify sky-high valuations. They need to monetize aggressively. Open-weight model builders, by contrast, often have different incentives—sponsorship from tech giants, research grants, or simply the ethos that AI should be accessible.
What This Means for Enterprise AI
For years, enterprises have been told that the best AI comes from the biggest labs. Karp is calling that assertion into question—not on performance, but on economics. If a company can run a model that scores within 5% of GPT-4 but costs 80% less, the business case for staying proprietary weakens.
This is already playing out in procurement meetings. A VP of engineering at a Fortune 500 bank—who requested anonymity to discuss internal strategy—told me they recently evaluated switching from GPT-4 to Llama 3.1 for a fraud detection pipeline. “The output quality difference was negligible for our use case. The cost difference was seven figures annually. We made the switch in three months.”
Karp’s warning is that the trend will accelerate as more companies realize they’re overpaying for a status symbol. “It’s not about hating OpenAI or Anthropic,” he said. “It’s about math. You cannot build a sustainable industry on a pricing model that punishes usage.”
The Verdict
Karp is right about the math—but he’s also selling something. Palantir’s platform thrives when customers use open models, because Palantir provides the integration layer. So his critique is not entirely altruistic. That said, the underlying problem is real: token-based pricing creates a friction that discourages experimentation and penalizes success.
If you’re a startup, paying $50,000 a month for API calls might be acceptable when you’re cash-rich from a funding round. But when you need to hit profitability? That same bill becomes a razor. The market is already voting with its feet. Open model downloads are surging. Proprietary APIs are seeing slower growth.
Karp’s outburst may be the moment the industry admits that the emperor has no clothes—or at least that the clothes cost too much. The next move belongs to OpenAI and Anthropic: cut prices further, or risk losing the enterprise customers they fought so hard to win.
