Microsoft Made On-Device AI a Supported Product: What Foundry Local GA Changes

Microsoft moved Foundry Local to general availability at Build 2026 in June, and it is worth more attention than it received. It is a cross-platform on-device inference runtime with production SDKs, model management, and integration with Windows ML so that it can pick up hardware-matched execution providers for whatever silicon is in the machine.

Per Microsoft’s Foundry Local documentation, the pitch is straightforward: inference runs on the device, data does not leave it, there is no per-token cost, and it works offline.

None of that is new as a capability. People have been running models locally with llama.cpp and Ollama for three years. What is new is that the largest desktop operating system vendor now ships a supported product that says so.

Why Vendor Support Is The Actual News

For anyone who has tried to get local AI approved inside an organization, the blocker was rarely technical.

It was that “we will run an open-source inference runtime from GitHub on employee laptops” is a hard sentence to get through a security review. Not because the software is bad, but because procurement and IT departments are built to evaluate vendors, support contracts, and update channels, and a community project does not present those surfaces.

A generally available Microsoft product with documentation, SDKs, and an update path presents all of them. That does not make local inference better. It makes local inference a thing you can put in a proposal without spending your credibility.

The practical effect is that the conversation described in how to ask your company to approve an AI tool gets meaningfully easier, because “runs locally” stops sounding like a hobbyist argument.

What It Does Not Change

Three things, because the announcement language invites overreading.

It is a runtime, not a product you use. Foundry Local runs models for applications. It is developer infrastructure. On its own it does not rewrite your email any more than a database engine writes your invoices.

It does not make the models better. The same open small models are the ones being served. A local 1.7 to 4 billion parameter model in 2026 is competent at rewriting and structurally limited on long-form generation, and a supported runtime does not change that curve. We laid out the honest boundary in is local AI good enough for everyday work.

It does not settle where any given application sends your text. An application can use a local runtime for one feature and a cloud API for another, and most will. “Built on a local runtime” and “your text never leaves the machine” are different claims, and only the second one is a privacy guarantee.

That distinction is the one worth carrying into vendor conversations. Ask which operations run on device and which do not, and get the answer in writing. How to audit an AI vendor in 2026 covers the phrasing that produces contractual answers.

The Hardware Question Got Less Important

Foundry Local’s Windows integration acquires hardware-matched execution providers, which means it uses the NPU or GPU when one is present and falls back when one is not.

That reflects a broader 2026 shift. The 2024 framing was that on-device AI required a specific class of machine with a specific accelerator. The 2026 framing across the industry is that small models run acceptably on ordinary CPUs, and accelerators make them faster rather than possible.

For writing specifically, this was always true. A 1.7 billion parameter model at four-bit quantization streams a rewrite on a normal work laptop CPU in a couple of seconds. That is why Wrivio’s local engine runs llama.cpp in-process against a small GGUF and asks for no special hardware: the task does not need it. The hardware comparison is in NPU versus CPU versus GPU for local AI writing.

What To Actually Do With This

If you are trying to get local AI into an organization, this is a citation. Microsoft shipping a supported on-device runtime is evidence that the approach is mainstream rather than fringe, and that is the argument most security reviews are actually resolving.

Before:

Hi, I’d like to request approval for an AI writing tool. It runs locally so there shouldn’t be any data concerns. Let me know if that’s OK.

After:

I am requesting approval for Wrivio, a text rewriting tool, for drafting client correspondence.

In local mode the model runs in-process on the laptop and makes no network calls during a rewrite. Client text is not transmitted. This is the same on-device pattern Microsoft now supports through Foundry Local.

The tool also offers a cloud mode. I am requesting local mode only, and can confirm the setting is fixed.

Happy to run through it with security. Can I get a decision by 20 August so I can use it on the Henderson work?

The second version answers the questions a reviewer is actually going to ask: what leaves the machine, what the alternative mode does, and what you are committing to. It also gives a date.

A Wrivio Context for internal approval requests could say:

Rewrite this as a professional internal request. Lead with what is being requested and why. State the specific technical fact that answers the reviewer’s main concern. Keep every product name, date, and commitment exactly as written. Do not add hedging, apologies, or claims about security that are not in the original.

Press Ctrl+Shift+Space, paste the draft, and check the diff. Watch for the rewrite strengthening a security claim you did not make, which is the specific failure that gets you in trouble later.

Common Questions

Is Foundry Local something I install to get AI writing?

No. It is a runtime for developers to build on. You would use an application that runs on it, or an application that embeds its own local engine, as Wrivio does.

Does a Microsoft local runtime mean my data is private?

It means inference can run on device. Whether a specific application keeps your text on device depends on that application, so ask which operations are local and which are not.

Do I need a Copilot+ PC or an NPU for this?

No. Small models run acceptably on ordinary CPUs, and accelerators make them faster rather than possible. For rewriting, a normal work laptop is sufficient.

Does this replace llama.cpp and Ollama?

It competes with them for developer attention and adds vendor support. Those projects remain widely used, and llama.cpp in particular underpins a large share of local inference including Wrivio’s own engine.

Download Wrivio for Windows to run rewrites through a local model that makes zero network calls, on the laptop you already have.