The Price Is Wrong: What AI Actually Costs Per Point of Quality

Welcome to the Show

Let us play a game.

I am going to describe two AI models. They were given the same tasks, the same inputs, the same scoring rubric. A blind judge, itself an AI with no idea which model produced which output, scored them both on a scale of one to ten. Same judge, same criteria, same day.

One of these models costs forty four times more than the other. Your job is to guess which output came from the expensive one.

Here is why this matters. If you use AI for anything, professionally or personally, you are making pricing decisions every single day. Every time you pick a model, every time you hit send on an API call, every time you choose between the big model and the small one, you are spending money based on an assumption. The assumption is simple and almost universal. More expensive means better. The gap in quality roughly matches the gap in price. You get what you pay for.

That assumption is wrong. Not slightly off. Not a rounding error. It is wrong in a way that means most people using AI commercially are overpaying by a factor of ten or more on the majority of their workload. And I can show you exactly how wrong it is, because we measured it. Same tasks, same judge, same scale. Real numbers, not benchmarks from a press release.

Round One: The Flagship Versus the Budget

Here are your two contestants. Model A scored nine point zero out of ten, averaged across every task we threw at it. Model B scored eight point eight. That is a difference of zero point two on a ten point scale. Two hundredths of the total range. If these were students in a class, they would both have the same letter grade.

Now for the prices. Model A costs four point four cents per call. Model B costs two tenths of a cent per call. That is not a typo. Four point four cents versus zero point two cents. Model A is twenty two times more expensive than Model B.

Twenty two times the price. Zero point two points of quality. Ladies and gentlemen, we have our first pricing anomaly.

Model A is Claude Opus, the flagship from Anthropic. The most expensive commercially available model from one of the leading AI companies in the world. Model B is Claude Haiku, the budget option from the same company. Same family, same architecture, same safety training. The little sibling scores within spitting distance of the big one on real production tasks.

And the latency? Opus took sixteen point one seconds per call. Haiku took five point one. The cheap model is not just cheaper. It is three times faster. You pay twenty two times less and you get your answer three times sooner.

Let that sink in for a moment. In most markets, a twenty two times price difference represents a dramatic quality gap. A twenty two dollar bottle of wine versus a four hundred and eighty four dollar bottle, that is a gap you can taste. A twenty two dollar hotel room versus a four hundred and eighty four dollar suite, that is a gap you can feel. But in AI, that twenty two times multiplier buys you zero point two points on a ten point scale. Two percent. The gap between "very good" and "very slightly more good."

Round Two: The Free Competitor

Ready for round two? Because it gets worse. Or better, depending on which side of the invoice you sit on.

Model C also scored eight point eight out of ten. Same score as the budget model. Same blind judge, same tasks, same rubric. But Model C is not from Anthropic. Model C is Llama three point three, the seventy billion parameter model from Meta, running on Groq's infrastructure. And it costs zero. Not almost zero. Not rounds down to zero. Literally zero dollars per call on Groq's free tier.

It also responded in zero point eight seconds. Not five seconds. Not sixteen seconds. Under one second.

So let us update the scoreboard. The flagship model that costs four point four cents and takes sixteen seconds scores nine point zero. The free model that takes less than a second scores eight point eight. The price ratio is not twenty two to one. It is infinity to one. You cannot divide by zero, but you can certainly feel the absurdity. A free model, running on someone else's hardware, matches the second most expensive option on the market.

Zero dollars. Eight point eight out of ten. The bid from the back of the room that makes the whole auction house turn around.

Round Three: The One in Your Pocket

There is a model running on a laptop. Not a gaming rig. Not a workstation with four GPUs. A regular Apple Silicon laptop, the kind you might carry to a coffee shop. The model is Qwen two point five, seven billion parameters, running through MLX, Apple's machine learning framework. It downloads in a few minutes. It runs entirely offline. No internet required. No account. No API key. No terms of service. It sits on your hard drive and it answers when you ask.

It scored eight point four out of ten. That is zero point six below the flagship. On our ten point scale, the model that costs nothing, needs no internet connection, runs on hardware you already own, and could operate in a bunker during a blackout, lands within six percent of the most expensive commercial option available.

And there is another local model, Llama three point one with eight billion parameters, that scored eight point two. Still within eight percent of the flagship. Still free. Still offline.

The latency is worse. Seventeen to eighteen seconds for local models versus under one second for Groq. But latency is a different axis from quality, and in many use cases, you do not care whether the answer arrives in one second or seventeen. You care whether the answer is good. A background process that summarizes documents overnight does not need sub-second responses. A batch job that classifies ten thousand customer support tickets can take its time. The speed premium matters for interactive use, chat interfaces and real time assistance. For everything else, the model sitting on your own hard drive, answering for free, producing output within eight percent of the best model money can buy, starts to look less like a compromise and more like common sense.

The Paragraph That Was Worth More Than an Upgrade

Here is where the game takes a turn. Everything so far has been about swapping models. Cheaper model, similar quality, save money. That is a powerful finding, but it is also the obvious one. The counterintuitive finding is about something that costs nothing at all.

A classification task. Seven categories. The kind of thing AI should be good at. Put this piece of content into one of these seven buckets. The budget model, Haiku, the one that scored eight point eight on the general quality test, managed fifty percent accuracy. Half the time it got the category right. Half the time it did not. Coin flip territory.

The instinct here, and this is the instinct that costs people millions of dollars collectively, is to upgrade. Switch to Opus. Pay twenty two times more. Surely the expensive model will classify better.

That is not what happened. What happened was someone wrote a paragraph. Eight hundred and seventy tokens of domain specific context. A glossary, essentially. Here is what each category means. Here are the edge cases. Here is how to tell category three from category five when they overlap. One paragraph of human knowledge, the kind of thing an expert could write in ten minutes.

Haiku, with that paragraph, jumped from fifty percent to eighty percent accuracy. The same cheap model. The same price per call. The only thing that changed was the prompt. Eight hundred and seventy tokens of context, costing a fraction of a cent in additional input, doubled the accuracy of a model that was already cheap.

The cheapest possible fix. Not an upgrade. Not a migration. Not a new vendor contract. A paragraph.

The lesson is devastating for anyone selling AI on a per-call pricing model. The most cost effective way to improve AI output is not to buy a more expensive model. It is to write better prompts. Context beats compute. A few hundred words of domain knowledge, written by a human who understands the problem, outperforms a model upgrade that costs twenty two times more.

The Exception That Proves the Pricing

This episode would be dishonest if it ended there. The quality gap is not always small. There are real exceptions, and they matter, and pretending otherwise would be exactly the kind of one sided analysis we try to avoid.

Swedish. That same local model, Qwen two point five, the one that scored eight point four on English tasks, dropped to seven out of ten when asked to generate original Swedish text. It could clean up existing Swedish prose just fine. But creating Swedish from scratch, composing sentences in a language that makes up a tiny fraction of its training data, exposed the model's limits. Seven out of ten is not terrible, but it is noticeably below the threshold where the output sounds natural to a native speaker.

And there is a bigger exception. Coherence across long outputs. When a local model was asked to generate items one at a time, independently, it produced fifty four percent unique results. The rest were repetitions. The model kept circling back to the same ideas, the same phrasings, the same patterns. It had no memory of what it had already said.

A single call to a more capable model, with all the items in context at once, produced one hundred percent unique results. Fifteen times faster. One cent bought coherence that no amount of local model tweaking could achieve. Sometimes the expensive model is worth every cent. But the reason is specific. It is not "quality" in the abstract. It is coherence, the ability to hold a large context in mind and produce output that does not repeat itself.

And creative writing. Editorial judgment. The work that requires not just competence but taste. In these domains, the gap between cheap and expensive models is real and it is large. A model that can summarize a document, extract facts, classify content, and answer questions at eight point four out of ten will produce creative writing that sounds flat, repetitive, and mechanical compared to the flagship. The scoring rubric catches this, but a human reader catches it faster. This is the Director Principle, a pattern that emerged across dozens of experiments. Processing offloads beautifully. You can hand summarization, extraction, and classification to the cheapest model available and lose almost nothing. But judgment does not offload. The moment you need a model to make a creative choice, to decide what matters, to find the interesting angle, to write a sentence that makes someone lean forward, the cheap models fall short in ways that no amount of clever prompting can fix.

The Real Game

So here is the final scoreboard.

Processing tasks, the tasks that make up the majority of most AI workloads, summarization, extraction, classification, question answering, reformatting. The quality gap between a four point four cent model and a free one is six percent or less. The gap between a four point four cent model and a zero point two cent model is two percent. For most organizations, this means seventy to ninety percent of their AI spending is buying almost nothing.

Creative tasks, editorial judgment, long form coherence, generation in underrepresented languages. The quality gap is real. The expensive model earns its price here. But this is typically ten to thirty percent of the workload, not one hundred percent.

The game, the real game, is not choosing one model. It is building a system that uses the right model for each task. A chain. The cheap model handles the bulk of the work. When it fails, the expensive model catches it. When neither is needed, a paragraph of context in the prompt does the job that a model upgrade never could.

We built exactly this. Tier one, Claude Haiku at two tenths of a cent, handles everything first. Tier two, Groq Llama at zero cents, catches failures when the API is down. Tier three, local models at zero cents, handles everything when the internet is out entirely. The expensive model, Opus, is reserved for the ten percent of tasks where coherence and creativity genuinely matter.

The result is not a degraded experience with occasional upgrades. The result is a system where the baseline is already good enough, and the upgrade exists for the rare moments when good enough is not. The expensive model is not the default. It is the exception. And the total cost dropped by roughly ninety percent.

The Price Was Wrong

Here is what we learned from running the numbers instead of trusting the marketing.

The relationship between AI model price and quality is not linear. It is not even close to linear. It is a curve that rises steeply at first and then flattens into a near horizontal line. The first two tenths of a cent per call buy you eight point eight out of ten. The next four point two cents buy you zero point two additional points. That is a twenty two times price increase for a two percent quality improvement.

And the single most effective quality intervention we found was not a model swap at all. It was a paragraph of context written by a human in ten minutes. Eight hundred and seventy tokens. A rounding error in the cost column. A doubling of accuracy in the results column.

If you are building with AI right now, run your own version of this test. Take your actual workload. Send the same inputs to the expensive model and the cheap one. Score the outputs blind. Not by who made them. By whether they are good enough. The answer might save you a lot of money. Or it might confirm that you are one of the few people who genuinely need the flagship for every call. Either way, you will know. And knowing is better than assuming.

Because right now, for most people, the price is wrong.