The Rule He Wrote Against Himself: Apple's On-Device Model, One Year On

Three Notes On The Same Afternoon

On the 18th of July 2026, three notes went into the same inbox within minutes of each other. Two of them were ideas. The third was a rule designed to stop the ideas from happening.

The first idea was a way to ask questions of your own notes. Mirror everything you have ever captured into the search index that already lives on every iPhone, then let the on-device language model pull a small set of relevant items out of that index and answer from them. Not a chatbot. A typed answer, grounded in evidence, entirely on the phone.

The second idea was smaller and more practical. Point the camera at an event poster, or share a screenshot of a booking confirmation, and have the phone read the text, understand what kind of thing it is, and file it. A poster becomes a date. A receipt becomes an expense. A barcode on a piece of rental equipment becomes an inventory row.

And then the third note, which reads like a man tying his own hands.

Apple Foundation Models guardrail: do not build custom model adapters, a generic on-device agent, or a Private Cloud Compute dependency until a measured problem requires one.

That note is the interesting one. It is a rule written by someone who could feel himself about to spend three weekends on something, and who wanted a version of himself from the future to be stopped at the door. It has sat there for 6 weeks, unexamined.

The question worth asking now, a full year after Apple opened this framework to developers, is whether the rule was right. And the answer is more interesting than a simple yes. It was right, it was right for one of the three reasons stated and for a completely different reason on another, and in the meantime Apple went and built two of the ideas it was guarding.

What Apple Actually Handed Developers

Start with what exists, because the shape of the gift determines everything else.

At its 2025 developer conference, Apple opened up the language model that had been sitting inside the operating system powering its own features. Writing Tools. Smart replies in Messages. The emoji generator. All of that ran on a model of roughly 3 billion parameters, living on the device, and until that summer it was completely sealed off. Developers had exactly 2 options before: call somebody's cloud over the network, or convert your own model into Apple's on-device format and spend months optimising it for the neural engine, which for any team without a dedicated machine learning engineer was simply not a realistic project.

The framework was the third path. There is a system model, Apple maintains it, Apple updates it, Apple optimises it, and any app can call it. No API key. No network request. No per-token cost. No model files to ship. On a phone in aeroplane mode with no signal at all, it still answers.

The engineering around it is genuinely thoughtful, and 2 pieces of it matter far more than the model's raw intelligence.

The first is guided generation. You define the shape of the answer you want as a data structure in your code, and the framework constrains the model so that what comes back is that structure, correctly formed, every time. Not a blob of text you then hope to parse. Not a request to please respond in a particular format followed by a prayer. The structure is guaranteed at generation time.

The second is tool calling, built on top of the same guarantee. You write an implementation of a small protocol, the model can decide to call it, and the framework prevents the model from inventing tools that do not exist or arguments that make no sense. Apple trained the model specifically on tool use to make this reliable, and the framework handles the messy orchestration of several tools being called in sequence or in parallel.

Underneath sits a session object that is deliberately append-only, because it is coupled to the model's memory cache. Filling that cache is slow, so the design goes out of its way to stop a developer from accidentally throwing it away and paying the cost again. That is the sort of detail that tells you the framework was built by people who had already been burned.

The Number That Governs Everything

There is one specification that decides more about what you can build with this than any other, and it is easy to miss because it sounds small and technical.

The on-device model shipped with a context window of 4,096 tokens. That is instructions, prompt, and output, all inside the same budget.

Sit with that for a moment, because it rules out an entire category of design. It means you cannot pour a document in and ask for a summary of the whole thing. It means you cannot hand the model 200 notes and ask it to find the pattern. It means the enormous, sprawling, context-stuffed prompts that people write against big cloud models are not just inadvisable here, they are impossible.

What it means positively is that this model is a component, not an assistant. It is very good at doing one small, well-defined job on a small piece of text, thousands of times, instantly, for free, without the network. Classify this note. Extract the date from this poster. Rewrite this sentence. Decide whether this item is a task or a thought. Tag this. Title that.

For a capture inbox with 241 open items, most of them untriaged voice notes in a mix of Swedish and English, that is not a limitation. That is precisely the job. Nothing in that inbox needs a model that can reason about all 241 items at once. It needs a model that can look at 1 item and answer 3 questions about it, correctly, in a guaranteed format, 241 times, in under a minute, offline, at no cost.

The intelligence was never the interesting part. The guaranteed structure was.

The Adapter Trap

Now to the first thing the rule forbade, which is the one where the rule was flatly, unambiguously correct.

The framework lets you train a custom adapter. The technique is a well-known one: instead of retraining the whole model, you train a small set of extra weights that ride alongside it and nudge its behaviour toward your domain. Apple ships a Python toolkit for this, producing adapters of a specific small rank, and they are fully compatible with the framework. You can optionally also train a second, smaller draft model to make inference faster.

Apple's own documentation on when to reach for this is unusually direct.

Use custom adapters only if you're comfortable training foundation models in Python.

They list the signals that justify it: you already have a suitable dataset, you need the model to become a subject-matter expert, you need it to follow a specific style or policy, prompt engineering is not getting you the accuracy you need, or your prompts have become so long that a specialised adapter would be faster. All reasonable. And they say plainly that you should exhaust prompt engineering and tool calling first.

Then come the costs, and they escalate.

You need a Mac with Apple silicon and at least 32 gigabytes of memory, or a Linux machine with a suitable graphics card. You need a specific entitlement from Apple before you can ship an adapter to the App Store, though not for local testing. You must not bundle the adapter inside your app; it goes out through a separate asset delivery framework, which is another moving part to operate.

And then the one that actually matters. Each adapter is compatible with exactly one version of the base model. When Apple updates the model, and Apple updates the model with the operating system, your adapter stops working and must be retrained.

Read that again in terms of what it means for a person, not a company. It means you have signed up for a recurring obligation you do not control the schedule of. Some morning in September, a point release lands, the base model version increments, and a thing you built in the spring silently stops behaving. You now need the 32 gigabyte Mac, the Python environment, the training dataset you had better have kept, and an afternoon. Every time. Forever. For a solo builder with 3 businesses and a newspaper to get out, that is not a feature. That is a subscription paid in weekends.

The rule was right. Not because adapters are bad, but because the maintenance shape is wrong for one person, and no amount of enthusiasm at the moment of building changes what September will feel like.

A Server You Are Invited To Distrust

The second thing the rule forbade was a dependency on Private Cloud Compute, and this one deserves a fair hearing, because it is one of the more impressive pieces of systems engineering anyone has shipped this decade.

The problem it solves is old. Some requests are too big for a phone. You want to send them to a server. The moment you do, the user has to trust the server operator, and no amount of promising helps, because promises are not verifiable and the operator can always change their mind quietly.

Apple's approach was to try to make the promise checkable. Every piece of software running on their inference servers has its measurement published in an append-only, tamper-evident log. The binary images themselves are published for inspection. A device will refuse to send a request to a server whose software is not in that log. And they released a research environment that lets anyone with a reasonably specced Mac boot the actual server software in a virtual machine, inspect it, verify the log's consistency, and poke at it, including a virtualised version of the secure enclave, which had never been available for outside research before.

Then they put money behind it, up to one million dollars for arbitrary code execution on a node, and 250 thousand dollars for getting at a user's request data.

The mechanism has a clear ancestor, and the parallel is worth stating because it explains why the design is clever rather than merely expensive. The same trick rescued the web's certificate system a decade ago. Certificate Transparency did not make certificate authorities honest. It made dishonesty visible after the fact, by requiring every certificate to appear in a public log that anyone can watch. That turned out to be nearly as good as honesty, because an authority that can be caught behaves like an authority that cannot cheat.

The inference log does the same thing for server software. Apple can still ship something malicious. What Apple cannot do is ship it to one person, quietly, without it appearing in a log that researchers are monitoring.

The honest limits are real and worth stating, because a one-sided account of this would be useless. Verification here is inspection, not proof. Most of the stack is not open source. Outsiders cannot fully reproduce the exact production builds. The whole edifice rests on the hardware and the secure enclave behaving as documented, and a compromised supply chain sits underneath every guarantee the log can offer. What the log promises is that you can see what shipped. It does not promise that what shipped is free of flaws.

Why The Dependency Rule Was Right Anyway

Here is where the guardrail was right for a reason that has nothing to do with any of that.

None of the trust engineering is the problem. The problem is that a dependency on server-side inference, in this design, is a dependency you do not control and cannot inspect from your own code. Whether a given request runs on the device or goes to a server is a decision the system makes. Device eligibility varies by model and by memory. Feature availability has varied by region, and regulatory pressure in Europe in particular has meant that Apple's intelligence features have arrived on different schedules in different places. A person building tools for himself in northern Sweden is downstream of every one of those decisions and party to none of them.

So the rule holds, but restate it accurately. It is not "the server cannot be trusted". By the standards of this industry the server is unusually trustworthy. It is "a personal tool should not have a hard dependency on a capability whose availability is decided by somebody else's product strategy and somebody else's regulator". That is a much more portable principle, and it applies to far more than Apple.

The correct shape is the one that degrades. The on-device path is the floor and it always works. Anything better is a bonus that appears when the device and the region and the operating system version all happen to line up, and nothing breaks when they do not.

Then Apple Built Two Of The Ideas

At the 2026 developer conference, the framework got an update, and this is where the story turns from a man being careful into a man being unexpectedly rewarded for being careful.

The framework gained built-in tools. Two of them are backed by Apple's existing vision technology: one that reads text out of an image, and one that reads barcodes. They are now first-party tools the model can call directly, with the same guarantees about not hallucinating the call.

That is the poster idea. That is the receipt idea. That is the equipment barcode idea. Not a thing to be assembled from parts, but a tool the model can invoke as part of a single structured request. The version of that feature written in July would have involved wiring the vision framework up by hand, managing the handoff into the model, and defining the output structure. The version available now is considerably less work.

The other addition is the one that lands directly on the first note. There is now a search tool backed by Spotlight, the system-wide index, designed specifically for retrieval: pull a small set of relevant local items, feed them to the model, answer from them.

That is the "ask my captures" idea, described almost exactly as it was written down, shipped by the platform. The note proposed mirroring capture items into the system index and letting the model retrieve a small evidence set from it. The framework now provides that retrieval step as a supported tool.

The server-side model also grew, to a context window of 32 thousand tokens, roughly 8 times the on-device budget, which changes what is possible on the server side without changing anything about the floor.

The Genre Of Rule He Wrote

There is a general pattern here, and it is worth naming because it recurs constantly for anyone building on top of a large platform.

Writing a rule that forbids you from building something is, underneath, a bet. The bet is that the gap you have identified will be closed by somebody else before your need becomes acute. Every time you decline to build a thing, you are wagering that waiting costs less than building.

Most of the time that bet is bad, which is why builders distrust it. Platforms are slow, they build for the median user, and the thing you need is usually too specific for them to ever ship. Waiting for the platform is how projects die of politeness.

But the bet gets much better when 2 conditions hold. The first is that the thing you want is obviously wanted by thousands of other developers, which means the platform has a real reason to build it. Retrieval from local data and reading text out of images are both squarely in that category. The second is that the version you would build yourself carries recurring maintenance rather than a one-time cost. A weekend project that then needs a weekend of upkeep every quarter is not a weekend project, and the true price only becomes visible around month 9.

Both conditions held here. The rule was a good bet, and it paid within a year.

The failure mode of this reasoning also deserves naming, in fairness. If you apply it to everything, you build nothing, and you spend your life reading release notes about features that would have been useful 2 years ago. The rule had a release condition built into it, and that condition was the right kind: not a date, not a mood, but a measured problem. Something must actually be observed to be too slow or too inaccurate before the heavy option unlocks. That clause is what separates a useful guardrail from simple procrastination with better vocabulary.

What Is Actually Worth Building Now

So, concretely, one year on, with 241 open items sitting in an inbox.

The lowest-risk, highest-value thing is triage, and it needs none of the forbidden pieces. A small on-device call, per item, using guided generation to return a fixed structure: what kind of thing is this, what topic does it belong to, how urgent does it sound, does it name a project. That fits inside 4,096 tokens with enormous room to spare, because the input is one short note. It costs nothing per call. It runs offline. It never needs an adapter, because the job is classification against a known set of labels and prompt engineering is entirely sufficient for that. And the output is structurally guaranteed, which means the code that consumes it does not need error handling for malformed responses, which is quietly the largest saving of all.

The second thing is the retrieval feature, and now it is much cheaper than it looked in July, because the hard part is a supported tool. The remaining work is the mirroring: getting capture items into the system index in a form worth retrieving, with sensible titles and content, so that a query returns the 5 right notes rather than 40 vaguely related ones. That work has nothing to do with the model. It is data hygiene, and it would improve ordinary search on the phone even if no model were ever involved.

The third is the image path, which is now largely a matter of wiring a share sheet to a request and deciding what structure you want back from a poster.

None of these require training anything. None require an entitlement. None break in September.

The Note That Can Be Closed

The guardrail note can come off the pile, and it should be closed as confirmed rather than deleted, because the reasoning inside it turned out to be worth keeping.

Adapters remain the wrong tool for a solo builder, and the reason is not difficulty but recurrence: they must be retrained on every base model version, and that is a standing obligation with no end date. A generic on-device agent remains a bad idea for a different reason, which is that a 4,096 token budget makes an agent a poor fit and a component a good one. And a hard dependency on server-side inference remains unwise, though the trust engineering behind it is better than almost anything comparable, because availability is a decision made by other people in other rooms.

The 2 idea notes, though, get much shorter. They no longer describe things to be built from scratch. They describe things to be wired up, using tools that now come in the box, against a model that is free and always present and unfashionably small.

And there is a closing thought that generalises past Apple entirely. The most valuable thing this framework offers is not intelligence. Plenty of things are intelligent and most of them cost money and require a network. What it offers is a small, fast, free, offline function that reliably turns messy human text into a correctly shaped data structure. That is a boring description of a genuinely rare capability, and it is the one worth designing around. The rest is weather.