How to Run a Local LLM on Windows: A 2026 Setup Guide

Running an AI model on your own Windows PC used to mean compiling C++ and chasing driver flags. Now it’s an installer and a download bar. Two free tools cover almost everybody, and once the model lands on your SSD it answers with no account and no bill.

The part nobody prints on the download page is memory. Your graphics card’s VRAM, or plain system RAM if you have no discrete card, decides which models run at all and whether they feel quick or sluggish.

#Why Run an LLM on Your Own PC?

Four reasons come up again and again, and privacy is the loudest one. A local model reads your prompt on your machine and writes its answer there too, so a draft contract or a private journal entry never leaves the drive. Cloud tools like Claude and ChatGPT Projects are more capable, but they see everything you type.

Offline access is the second. On a plane, in a basement lab, or on hotel Wi-Fi that blocks half the internet, a local model keeps answering.

Then there’s the bill. A local model has no subscription and no message cap, so you can throw a thousand throwaway prompts at it while you grind through something dull, and the only thing you spend is electricity.

Last, it’s yours. Nobody retires your model, rewrites its personality overnight, or moves the good version behind a higher tier.

#What Your PC Needs to Handle It

LM Studio sets a modest bar. According to the LM Studio system requirements, AVX2 instruction set support is required on x64 chips, at least 16GB of RAM is recommended, and at least 4GB of dedicated VRAM is recommended. ARM machines built on Snapdragon X Elite are supported as well.

Ollama draws its line somewhere else. Its Windows documentation states that you need Windows 10 22H2 or newer, NVIDIA driver 551.61 or newer on NVIDIA cards, and either a ROCm v7 capable or Vulkan capable driver stack on AMD Radeon. Set aside at least 4GB of disk for the binary before any model arrives.

Notice what appears on neither list: an NPU.

The neural chip in a Copilot+ laptop handles webcam effects and live captions, not language models. Our breakdown of what an NPU actually does explains the split, and NPU vs GPU vs CPU maps which chip takes which job.

Microsoft’s own bar for the Copilot+ badge is an NPU rated at least 40 TOPS, plus 16GB of RAM and a 256GB SSD, according to its Copilot+ PC comparison page. Useful sticker. Wrong spec for this particular job.

#Set Up LM Studio, the Point-and-Click Route

Start here if you’d rather not open a terminal. The whole flow is five steps.

1. Download the Windows installer from the LM Studio site and run it like any other app.
2. Open LM Studio and go to the Discover tab, which is the built-in model browser.
3. Pick a model whose file size sits under your available VRAM, then start the download.
4. Wait. First downloads are measured in gigabytes, so this is the slow part.
5. Switch to the chat tab, load the model you just pulled, and type a prompt.

If the app warns that a model may not fit, believe it. Loading a file bigger than your VRAM either spills into system RAM and crawls, or fails outright.

One quiet advantage of the graphical route: LM Studio shows the file size next to every quantization before you commit to a download. That single number saves more grief than any tutorial, because it turns an abstract question about parameters and precision into a concrete comparison against the VRAM figure you already know.

#Set Up Ollama, the Terminal Route

Ollama trades the browser for a command line and an API, which is what you want if you plan to script against the model.

1. Run OllamaSetup.exe. It needs no administrator rights and installs into your home directory by default.
2. Open Command Prompt, PowerShell, or your terminal of choice.
3. Type ollama run followed by a model name. The first run downloads the weights, then drops you into a chat prompt.
4. Leave it running. Ollama sits in the background and serves an API at http://localhost:11434.

Two flags are worth knowing early. Launching the installer as OllamaSetup.exe /DIR="d:\some\location" puts the program somewhere other than your system drive, and setting an OLLAMA_MODELS user environment variable moves the model files themselves. Weights pile up fast, so pointing them at a roomy drive on day one beats moving them at 200GB.

Its documentation notes that the uninstaller won’t clean up a relocated model folder, so remember where you put it.

#How Much Memory Does a Model Actually Need?

Models ship quantized, meaning each weight is stored in fewer bits than the original training format. Hugging Face’s GGUF format reference describes Q4_K as 4-bit quantization that works out to 4.5 bits per weight, against 16 bits for a half-precision original.

That ratio is the whole trick. Multiply the parameter count by roughly 4.5 bits and you get a usable estimate of the file size, which is why a 7-billion-parameter model at 4-bit lands near 4GB instead of the 14GB its unquantized version would demand.

Rough guide to which model class fits which memory budget, derived from the bits-per-weight arithmetic above

Treat these as starting points rather than guarantees. Context length, other apps holding VRAM, and the specific quantization all shift the real ceiling, and the honest test is simply loading a model and watching whether it stays on the GPU.

No discrete GPU at all? Models still run on the CPU using system RAM. They’re slower, sometimes badly so, but a small model on a machine with 16GB is a perfectly reasonable place to begin, and it costs nothing to find out whether the speed actually bothers you before you go shopping for a graphics card.

#Picking Your First Model

Stick to a well-known family for the first download. Llama, Mistral, Phi, Gemma, and Qwen all publish quantized builds that both tools can load, and all of them have enough community write-ups that troubleshooting is easy.

Specific version numbers age badly. A model that tops the charts this quarter gets replaced next quarter, so treat any named version in a tutorial, including this one, as a starting hint rather than a recommendation.

Start one size below what your hardware allows. A 7B model that responds instantly gets used daily; a 14B model that stutters gets abandoned in a week. Once the workflow sticks, scale up, and if you’re shopping for a machine to grow into, our guide to the best laptop for running LLMs covers what to buy.

#Where Local Models Still Fall Short

Smaller models get facts wrong more often than the big cloud ones. That gap narrows every year, but it hasn’t closed, and a 7B model will confidently invent a citation that a frontier model would catch.

They also can’t see today’s web. Ask about this morning’s news and you’ll get training data instead.

Speed is the third limit. On a CPU-only machine, watching tokens appear one at a time tests your patience in a way that no benchmark chart conveys. Local models win on privacy and cost, not on raw capability.

Knowing that ahead of time makes the whole thing more satisfying. If you want the wider hardware picture first, what an AI PC is sets the context.

#Bottom Line

Install LM Studio if you want a working local chatbot in the next twenty minutes, and Ollama if you’d rather have a command and an API to script against. Either way, pick your model by the memory you actually have, not by the benchmark leaderboard, because a 4-bit model that fits in VRAM beats a smarter one that thrashes. Start at 7B and scale up only once that feels limiting.

#Frequently Asked Questions