Ollama is a tool that lets you download and run open-weight large language models on your own computer with a single command. Running locally keeps…
See what Ollama does and why you'd run a model locally.
Ollama makes running an open-weight model on your own computer as easy as one command to pull it and one to chat. It handles downloading the model, fitting it to your hardware, and serving it — no cloud account or API key required.
Why do this? Privacy: your data never leaves your machine, which matters for sensitive documents. Cost: no per-token fees once you have the hardware. Offline and control: it works without internet and you decide exactly which model runs. The trade-off is that you're limited by your own hardware and usually run smaller models than the largest cloud offerings.
Understand how quantization lets real models run on normal machines.
A full-precision model is often too big for a normal computer's memory. Ollama uses quantized model files — weights stored in lower precision (like 4-bit) — which shrink the memory needed several-fold with only a small quality cost, so a capable model can fit in the RAM or VRAM you have.
The practical limit is memory. Smaller models (a few billion parameters, quantized) run comfortably on a modern laptop; larger ones need more RAM or a GPU. Ollama offers many models and sizes, so you pick one that fits your machine and your quality needs.
Call a local model from the command line and from code.
After installing Ollama, you pull and run a model by name, and can chat in the terminal immediately. Just as important, Ollama serves a local API on your machine, so your applications call the model over HTTP exactly as they would a cloud model — and it offers an interface compatible with common client libraries, so existing code often works with a changed base URL.
This means you can prototype against a cloud API and switch to a local model, or vice versa, with minimal code changes.
ollama pull llama3.2 # download a model ollama run llama3.2 # chat in the terminal # from an app, call the local endpoint: POST http://localhost:11434/api/generate { "model": "llama3.2", "prompt": "Summarize this text..." }
One command downloads the model, another starts an interactive chat. For apps, Ollama listens on a local port, so you POST a prompt to the local endpoint and get a response — no external service involved.
Decide when local models win and avoid the common traps.
Reach for local models when data privacy is paramount, when you want zero per-token cost for high volume, when you need offline operation, or for development and experimentation. Prefer cloud APIs when you need the very strongest models, effortless scaling to many users, or you'd rather not manage hardware.
Many teams use both: a local model for sensitive or high-volume simple tasks, and a cloud model for the hardest queries.
Watch for: expecting a small local model to match the largest cloud models — pick tasks that fit its ability; choosing a model too big for your memory so it runs painfully slowly or won't load; and forgetting that serving many concurrent users from one machine doesn't scale like the cloud. Match the model size to both your hardware and the task.
Ollama runs open-weight LLMs on your own machine with a couple of commands, giving privacy, no per-token cost, and offline use, bounded by your hardware and generally smaller models than top cloud services. Quantized model files let capable models fit in normal memory. Ollama serves a local API (often client-compatible), so apps call it like any model. Choose local for privacy, cost, and offline needs; cloud for the strongest models and easy scaling — many teams use both.
You want a private assistant over confidential notes on your laptop. Explain why a local Ollama model fits, how you'd pick a model that runs well on your hardware, and one type of query you might still send to a cloud model instead.
What does Ollama do?
Ollama runs open-weight models on your own machine, handling download, hardware fit, and serving a local API.
Why can capable models run on ordinary hardware with Ollama?
Quantization shrinks memory requirements several-fold, letting capable models fit in a normal machine's RAM or VRAM.
How do applications use a model running in Ollama?
Ollama serves a local endpoint, so apps call the model like any API — often just by pointing existing code at the local base URL.
When is a local model a better choice than a cloud API?
Local wins on privacy, cost, offline use, and development; cloud wins on top-end capability and effortless scaling.