A 27B Model Now Fits On A Laptop. Reprice Your AI.
PrismML released Bonsai 2 on September 17, squeezing a 27 billion parameter reasoning model into 5.9 GB while keeping 98 percent of its benchmark scores. For a founder paying per token, that is a pricing event, not a research result.
What PrismML actually shipped on September 17
A Caltech spinout called PrismML released Bonsai 2 27B on Thursday. It takes Qwen3.8 27B, a widely used open source model from Alibaba, and compresses it to 5.9 GB. That is a nine to ten times reduction in memory, and it fits on a normal laptop and possibly a high end phone.
The number that matters is not the size. It is that Bonsai 2 matches 98 percent of the original model's aggregate benchmark scores. The first Bonsai, released in March, hit 95 percent. Two percentage points of movement in six months.
The technique is ternary weights. Normally each weight in a model needs 16 bits. PrismML reduces each one to three possible values: plus one, minus one, or zero. Less to store per weight, dramatically less to store overall.
The company has raised $22.25 million in seed funding from Khosla Ventures, Cerberus Capital and Caltech. CEO Babak Hassibi is a Caltech professor who works on compression. Ion Stoica, the Databricks co-founder, advises.
Fine. Why should a founder with fourteen customers care?
Why a compression benchmark is a pricing event
Because most AI products are priced on an assumption that is quietly expiring.
The assumption goes like this. Intelligence is expensive, it lives in someone else's data center, and every user action costs you a metered API call. So you price to cover tokens, you watch gross margin like a hawk, and you say no to features that would be too chatty.
What PrismML's release says, along with similar work from Multiverse Computing and others, is that a large slice of that intelligence is now movable. Not all of it. A slice.
Stoica put the consequence plainly: intelligence at your fingertips, free because it runs on hardware the user already bought, and private because nothing leaves the device.
Treat that as a forecast, not a fact about today. Bonsai's first release has been downloaded more than 11 million times, and PrismML's smaller models another 2.6 million, so the direction is real. The question for a founder is narrower and more useful. Which of my calls still need to go to a frontier API, and which ones never did?
What fraction of your AI calls are actually hard?
Almost nobody knows their own answer to this, which is the whole problem.
The general pattern reported across production systems is lopsided. Routing work reported by teams like Inworld suggests that most requests in a typical AI product are not reasoning problems at all. They are classification, extraction, reformatting, short summarization, tagging, routing and validation. The hard reasoning sits in a minority of calls.
Teams that measure this and route accordingly report cutting inference spend substantially, often in the range of 40 to 85 percent, without users noticing a quality change. Treat the top of that range with suspicion, since vendors publish the flattering cases. The bottom of it is still a lot of money.
Here is the uncomfortable part. You cannot get any of that saving until you know your own distribution. And you cannot know your own distribution by guessing, because engineers systematically overestimate how hard their own product's calls are.
So measure. It takes about a week and it costs nothing.
The one week routing audit
Run this on production traffic, not on a test set you wrote.
Day one and two: log every model call with the prompt template name, input token count, output token count, latency and cost. If you already have this, skip ahead. Most teams find they have partial data at best.
Day three: bucket calls by template. You will likely find that three to five templates account for most of your volume.
Day four: for your highest volume template, sample two hundred real calls and answer one question for each. Would a small model plausibly get this right? Be strict. If you are not sure, mark it hard.
Day five: take the calls you marked easy and actually run them through a small open weight model. Compare outputs side by side. Count real disagreements, not stylistic ones.
Day six and seven: write the number down. The percentage of your volume that a small model handles at your accuracy bar, and the dollars that represents per month.
That number is now a real input to your financial model rather than a vibe. You can map the before and after in a spreadsheet, in Notion, or in a planning tool like Foundra that keeps projections next to the assumptions driving them. What matters is that the assumption is written down where someone can challenge it.
Your AI co-founder is ready when you are.
Foundra turns everything in this article into an actual plan. Validation, customers, pricing, launch. In one place, in your voice, in an afternoon.
Get started→$39/month. Cancel anytime.
What running on the device buys you besides money
Three things, and two of them are worth more than the cost saving at small scale.
Latency. No network round trip. For anything interactive, cutting a few hundred milliseconds changes how the product feels in a way no feature does.
Privacy. If the data never leaves the user's machine, an entire category of enterprise objection disappears. Any founder who has sat through a security review knows what "we do not send your data anywhere" is worth as a sentence.
Predictable cost. Your bill stops scaling with usage. That is a different business, and it prices differently. It is also the answer to the per-seat pricing squeeze that keeps eating AI products where a heavy user costs more than they pay.
The honest caveat is distribution. On-device means you now care about the user's hardware, the download size, and update mechanics. Those are real engineering costs that do not show up on an API invoice.
Where small models still lose
Do not read this as small models everywhere.
Long context is the clearest gap. When a task needs to hold a large document set in working memory and reason across it, size still helps.
Multi-step agentic work is the second. Errors compound. A two percent quality gap on a single call becomes something much worse over a twelve step chain.
Novel reasoning is the third. Anything truly unusual, where the model has to work out something it has not effectively seen, still favors the biggest thing you can afford.
And there is a subtlety worth knowing. Hassibi's own point is that compression always costs something, and his expectation is that larger models compress more gracefully because there is more redundancy to squeeze. PrismML plans several hundred billion parameter releases in the coming months. If that holds, the compressed-large category is where this gets interesting.
One more caveat from the same reporting: the harness around the model, the retry logic, the tool definitions, the validation, often matters more for accuracy than the model choice itself.
How to talk about this with investors and customers
With investors, the framing is margin structure, not technology. "Sixty one percent of our calls run on a model we host, which moved gross margin from 44 to 71 percent, and here is the quality regression suite that proves users did not notice." That is a slide. "We use small models" is not.
With enterprise customers, the framing is control. On-device or self-hosted inference answers data residency questions that otherwise require a six week security review. Say it early in the sales cycle, not in the security questionnaire.
With your own team, the framing is discipline. The rule is to route each step to the smallest model that provably clears your accuracy bar, with a clear escalation path when it does not. Provably is doing the work in that sentence. Without a regression suite, routing is just a way to make your product quietly worse to save money, and users find out before you do.
What to do in the next thirty days
Four things, in order.
Instrument your calls. Log template name, tokens, latency and cost per call. Everything downstream depends on this and most teams skip it.
Build a small regression suite. Fifty to two hundred real examples with known good outputs, per high volume template. This is the asset that makes every future model decision cheap.
Run the week long audit above on your single highest volume template. One template. Not all of them.
Then decide with a number in hand. If the easy share is under a quarter of your volume, note it and move on to something that matters more. If it is over half, you have found real money and a real moat for a product that has to be cheap to be good.
And set a calendar reminder for January. The next release in this family is supposed to be much larger, and the arithmetic will change again.
Frequently asked questions
What is model compression in plain terms? Shrinking the numbers a model stores so it needs far less memory to run. PrismML reduces each weight from 16 bits to one of three values, which is why a 27 billion parameter model fits in 5.9 GB.
Does a compressed model perform worse? Slightly. Bonsai 2 matches 98 percent of the original's aggregate benchmark scores. Whether two percentage points matters depends entirely on your task, which is why you need your own regression suite rather than a benchmark number.
Do I need to self-host to benefit from this? No. Several providers serve small open weight models cheaply through normal APIs. Self-hosting or on-device gets you the privacy and latency benefits, at the cost of real engineering work.
How much can routing actually save? Published figures cluster between 40 and 85 percent of inference spend, with the high end coming from vendors describing favorable cases. Measure your own distribution before you plan around any of it.
Is it too early for a seed stage company to care? If AI calls are a meaningful line in your costs, no. If they cost you $200 a month, spend the week on distribution instead and revisit when the number stings.
You just read the theory. Ready to build the thing?
Foundra is your AI co-founder. It turns an idea into a validated business plan, a go-to-market, and your first 10 customers. In an afternoon, not a semester.
$39/month. Cancel anytime. Works in 20 languages.