Why Local AI Slows Down On Battery, And What To Do About It

You download a local model, try it plugged in at your desk, and it is quick enough to use. Then you take the laptop to a meeting, press the hotkey, and the same rewrite takes four times as long.

Nothing is broken. Windows is doing exactly what you asked it to do the last time you thought about power settings, which was probably never.

Local inference is one of the few desktop workloads that is genuinely CPU-bound for seconds at a time. Most software you run is bursty, so power management barely shows. A model generating tokens is a sustained load, which is precisely the case power management exists to suppress.

Battery Mode Caps The Clock Speed You Are Paying For

The single biggest factor is the processor power state. On battery, Windows lowers the maximum processor state and becomes far more reluctant to enter boost clocks. A chip that sustains 4.2 GHz on mains may sit near 2 GHz unplugged.

Token generation in engines like llama.cpp scales close to linearly with clock speed, so halving the clock roughly doubles the time per rewrite. That is the whole effect, and it is usually most of the gap you are feeling.

Windows exposes this as the power mode slider in Settings, under System and then Power and battery. “Best power efficiency” is the aggressive setting. Moving to “Balanced” while you actually need the speed recovers most of it.

Efficiency Cores Are Not Doing You A Favour Here

Modern Intel and AMD laptop chips split cores into performance and efficiency types. On battery, the scheduler prefers the efficiency cores, which is right for a browser tab and wrong for a model decode loop.

If your rewrite lands on efficiency cores it will be slow in a way no clock-speed setting fixes, because the cores themselves are narrower. This is also why the slowdown can feel inconsistent: it depends on what else is running and where the scheduler put you.

There is no clean per-application control for this in Windows. The practical lever is the same power mode setting, which changes the scheduler’s bias along with the clock ceiling.

Thermal Limits Bite Sooner Than You Expect

A laptop on a desk with a fan curve tuned for mains power behaves differently to the same laptop warm in a bag five minutes ago. Sustained inference heats the package quickly, and thermal throttling is indistinguishable from power throttling if you are only watching the clock.

The tell is timing: power limits apply immediately and consistently, while thermal limits appear after the first ten or twenty seconds of load and get worse across consecutive rewrites.

If your second and third rewrite are markedly slower than your first, you are thermally limited, and the fix is airflow rather than settings.

Smaller Models Degrade More Gracefully Than Big Ones

A 1.7B model on constrained cores is still usable. A 4B model on the same cores crosses from “slightly annoying” to “I will just do this myself”, because the per-token cost is higher and the gap compounds across a whole paragraph.

This is the practical argument for keeping a smaller tier installed even if you prefer a larger one at your desk. Wrivio ships two Apache 2.0 Qwen3 tiers for this reason: Standard at 1.7B, needing roughly 1.1GB of disk and 1.7GB of RAM, and Best at 4B, needing roughly 2.4GB and 3GB. On battery, Standard is often the one you actually want.

If you have not compared them on your own text, benchmarking a local model on your own writing takes about ten minutes and settles the question better than any leaderboard. The related question of how much RAM a local model needs matters here too, because swapping to disk under memory pressure looks exactly like a slow processor.

Or Just Switch Engines For The Afternoon

The honest answer is that sometimes the right move is not to fight the hardware. If you are on battery in a meeting and you need three quick rewrites, the cloud path finishes them in a second each and costs you nothing in battery beyond a small network request.

Local mode exists so that your text never leaves the machine when that matters. It does not have to be the only mode you ever use. Deciding which tasks should stay local up front means you can switch without re-litigating it every time.

A Wrivio Context for quick battery-mode rewrites could say:

Rewrite this as a short, plain, professional message. Keep it under 80 words. Keep every name, date, figure, and commitment exactly as written. Do not add pleasantries or a closing line I did not write.

Press Ctrl+Shift+Space, paste the draft, and check the diff. A tighter length constraint means fewer tokens to generate, which on a throttled processor is the difference between waiting and not noticing.

Common Questions

Will running local AI drain my battery faster?

Yes, while it is generating. Sustained CPU load is expensive, though a rewrite lasting a few seconds is a rounding error next to a video call; the practical cost only shows up if you are running dozens of long generations away from power.

Is there a Windows setting that fixes this completely?

No single setting does, but moving the power mode off “Best power efficiency” while you work recovers most of the clock-speed loss. Thermal throttling needs airflow rather than configuration.

Does an NPU avoid the problem?

In principle yes, because NPUs are built for sustained low-power inference, but runtime support is still uneven and most local text tools run on the CPU today. See NPU versus CPU versus GPU for local AI writing for where that actually stands.

Should I just use a smaller model on battery?

That is usually the best trade. A 1.7B model on throttled cores stays usable where a 4B model stops being worth the wait, and for rewriting the quality gap is smaller than the size difference suggests.

Download Wrivio for Windows to keep both a small and a large local model installed, and switch between them depending on whether you are plugged in.