What I learned putting macOS 27's on-device model behind a coding agent

macOS 27 ships fm, a command-line tool for the on-device model for Apple Foundation Models. Run fm serve and you get the OpenAI Chat Completions API on a local socket. I built fm-mcp on top of it, so that Claude Code and Codex can hand the model small jobs: summarising a log, pulling fields out of a document, sorting text into labels, reading a screenshot.

Because the API looks familiar, I expected it to behave like any other model endpoint. Mostly it does. These are the places it didn't, each measured on a real Mac, and what each one changed in fm-mcp. The aim throughout was an agent that delegates only what the model gets right, since an agent that hands off too much is worse than one that hands off nothing.

The context is about 8,000 tokens, and the answer comes out of it#

Prompts up to 7,883 tokens worked. At about 8,200 the request failed with "The session's transcript exceeded the model's context size." The answer shares that budget, so the input limit is 8K minus however long you let the answer run. For extract, that leaves 6,518 tokens of input, which is where the "about 3,500 words" in its description comes from.

Logs cost more than twice as many tokens as prose#

I measured characters per token with fm count-tokens:

So the usual "about four characters per token" guess undercounts a log by more than half. fm-mcp counts real tokens before it sends anything (fm count-tokens -q takes about 0.08 s). summarise splits text that won't fit; extract and classify refuse it with a message saying how long it was. It's also why the summarise description says it takes about 30,000 words of prose, but only about 1,000 lines of a dense log.

Only one setting limits the answer, and a cut answer looks finished#

fm serve accepts max_tokens and ignores it. I asked for a 600-word essay with a cap of 30: with max_tokens the answers ran 587 to 699 tokens, with max_completion_tokens exactly 30.

A capped answer still says finish_reason: "stop", so the only sign it was cut is that completion_tokens equals the cap. Codex found this one for me: summarise overflowed the context on a 22,000-token changelog, because a part summary meant to stay under 400 tokens ran past 1,500. fm-mcp now sends max_completion_tokens on every request and treats an answer that used the whole cap as suspect.

Ask for a field the text doesn't have, and the answer can run away#

Structured output with a JSON Schema works, within limits: no $ref, no nullable type arrays like ["string", "null"], and pattern fails outright. The real trouble starts when you ask for a field the text doesn't contain. The model fills the string with junk until the context overflows. Three of my first four tries took about 200 seconds and ended in that overflow error. The fourth returned "not available', 1, 2026-03-03, ".

Worse, fm serve keeps generating after the client gives up, so every request after it waits behind the runaway. Telling the model "use NOT_FOUND if missing" didn't help; it copied another field's value, or junk.

What worked was giving every field an explicit way to be empty:

"total": {
  "title": "InvoiceTotal",
  "anyOf": [{"type": "number"}, {"type": "null"}]
}

On an invoice with four fields present and two missing, that came back exactly right in 5 runs out of 5, about 1.4 seconds each. Two details matter. anyOf needs a title on the property, or the request is rejected. And the titles must be unique and specific: with short titles like name and qty inside array items, 6 runs of 6 ran away; with ItemName and ItemQty, 4 of 4 were right. fm-mcp rewrites every schema it's given this way before sending it.

A missing field that sounds like something in the text takes it#

With that rewrite, missing fields unrelated to the text (a tracking number, a phone number, an office address) came back null in 20 runs of 20. A missing field close in meaning to something that is in the text is different. On an invoice with no purchase order number, purchase_order took the line items, and tax_id took the supplier's name.

Two prompt changes helped: a line saying most of the text belongs to no field, and "Null unless the text states it." at the end of every field's description. Over 30 runs each, alternating between the two versions, purchase_order went from null in 1 run to 20, and the fields that were present went from 25 runs right to 30. That's better, not fixed: on short invoices the missing PO number still took the supplier's name in 10 runs of 25. So the extract description tells the agent to ask only for fields the text is likely to have.

Never ask the model to reformat a number#

Amounts written like $1,280.00 often came back null from a number field. The obvious fix, telling the model to leave out currency signs and commas, made things worse in a way that's harder to see. It left out the decimal point too: 12,450.50 came back as 1245050 in 7 runs of 8. Other wordings turned 126.45 into 12645 and USD 3,000 into 3. A null at least admits it doesn't know; those are confident wrong numbers.

Asked for the same amount as a string, the model copied it exactly every time it finished an answer, 46 times out of 46. So since 0.1.1, fm-mcp asks the model for number fields as text and converts them itself. It accepts one plain amount (a currency code or sign, digits with thousands commas, decimals) and turns anything else into null.

When a flat answer runs away, the rest of it is usually fine#

Capping the answer turns a 200-second runaway into a fast failure, but fm-mcp was still throwing the whole answer away. Looking at the raw capped answers showed why that was wasteful: on a flat schema, the field the model got stuck on is simply missing, and every other field is complete. That held in all 12 capped answers I checked. So 0.1.1 keeps the finished fields, sets the stuck one to null, and adds a note naming it. The note matters, because now and then the stuck field is one the text does contain, such as a receipt's total.

On invoices with two fields the text lacked, "got stuck" errors went from 13 calls in 25 to none. A rescued call still takes about 15 seconds, the time it takes to hit the cap.

A list of choices in the schema isn't a list the model can see#

classify asks for {"label": …} with the labels as an enum in the JSON Schema, which guarantees the answer is one of them. I assumed that also told the model what the choices were. It doesn't seem to. On 200 news articles from the AG News dataset, with four topics, it got 49% right and almost never picked "Science and technology". On 204 sentences labelled with six emotions, it got 29%, with a strong pull towards "anger".

One sentence in the prompt, "The labels are: …", took those to 64% and 58%. That fix is in fm-mcp 0.1.2. It's still a small model: fine for sorting support tickets into clear categories, weak at telling joy from love, and no substitute for a classifier trained on your data. News articles also drew a refusal I hadn't seen before, "The model refused to answer.", about one time in six.

Plain vision reads text better than the OCR tool#

fm has a dedicated OCR mode, fm respond --tool ocr. Sending the image to fm serve as an ordinary vision request, with the prompt "Return all the text in this image exactly as written, line by line", did better on my five test images: a screenshot, small text, two columns, a receipt, and handwriting-style text.

The OCR mode garbled small text, separated receipt prices from their items, and sometimes started with "Here is the text…". Vision kept the layout (Flat white 4.80) and added nothing. PNG, JPEG, HEIC, TIFF, GIF and BMP all worked. PDF isn't supported.

One request at a time, for the whole Mac#

Three requests sent at once to one fm serve finished at 5.6, 8.2 and 11.0 seconds. Three separate fm serve processes, which is what three agent sessions would start, queued the same way: each extra request added about 2.6 seconds, whichever process sent it. Nothing failed; requests just wait. More servers add no throughput, so the tool descriptions tell the agent that calls queue, and that a long summary takes a minute or more.

Smaller things#

• No log probabilities. logprobs: true returns HTTP 200 and no logprobs field, so there's no way to get a confidence score. classify instead constrains the answer to the given labels with an enum: on 20 support messages with 4 labels, all 20 answers were valid labels and 18 were the expected one.
• Don't trust the error type. The safety filter sometimes refuses harmless text. And input far over the limit (about 14,000 tokens) came back as a safety error rather than a context error, which is another reason to count tokens first.
• Long socket paths fail silently. Past macOS's 104-byte limit, fm serve keeps running, prints nothing and never creates the socket. fm-mcp checks the length, and fm-mcp doctor reports it.

What this means for delegation#

A small model is useful to a coding agent only if the agent knows where its edges are. That's why fm-mcp has no general "ask" tool, only four narrow ones; why each tool description states its limits in plain words; and why fm-mcp install adds a short guide telling the agent when to delegate and when to keep the work. Most of what's above ended up as a sentence in one of those descriptions.

If you have a Mac with macOS 27 and Apple Intelligence on:

brew install yrangana/tap/fm-mcp
fm-mcp install
fm-mcp doctor

How I measured: on one Apple Silicon Mac running macOS 27 and 27.0.1, between 6 and 11 October 2026, sending requests straight to fm serve or through fm-mcp. Run counts are small, usually 4 to 50 per variant, and the model varies a lot from run to run, so I alternated between variants rather than running one batch after another. Every number here is recorded, with its date, in the project's AGENTS.md.