Run a local model API and connect it to a simple app workflow.
You install Ollama on your OS, verify the service is running, and confirm the CLI responds — using a concrete install-and-check walkthrough on a laptop running macOS, Linux, or Windows.
A step-by-step guide to installing Ollama and confirming it runs on your machine.
Why this matters: Every later module depends on Ollama being installed and healthy — this is the foundation everything else builds on.
Your laptop is about to run a full AI model — no cloud account, no API key, no data leaving your machine. But how does a raw model file become something you can actually talk to?
is a tool that downloads AI (the files that hold a trained AI's "knowledge") and runs them as a on your computer.
You control it through a — a Command-Line Interface, meaning you type commands in a terminal window instead of clicking buttons.
Once Ollama is running, any app on your machine can send it questions and get answers — all offline, all private.
Picture a laptop running Ollama — three pieces work together every time you use it.
ollama pull, ollama run) you type to control everything.In this module you install the first piece and confirm the other two are alive. That's the entire goal.
# You open a terminal BEFORE installing Ollama and type: $ ollama --version # What you actually see: zsh: command not found: ollama # On Windows PowerShell: # 'ollama' is not recognized as an internal or external command
$ollama --versioncommand not foundBefore you install, the terminal has no idea what ollama means — it's not on your PATH yet.
This is the exact error you'll see if you skip the install step or if the installer didn't finish correctly. Keep it in mind — you'll recognise it later if something goes wrong.
The terminal prints command not found: ollama (macOS/Linux) or 'ollama' is not recognized (Windows). The shell can't find the program because it was never placed on the PATH by an installer.
# After installing Ollama, open a fresh terminal and run: $ ollama --version ollama version 0.32.0 # Now check the service is listening: $ ollama list NAME ID SIZE MODIFIED # (empty — no models downloaded yet, but the service answered)
ollama --versionollama listNAME ID SIZE MODIFIEDA version number means the CLI is installed. An empty-but-structured table from ollama list means the background service is running and ready to accept commands.
You haven't downloaded any models yet — that's fine. The table being empty is expected at this stage.
It prints a table header (NAME, ID, SIZE, MODIFIED) with no rows — because no models have been downloaded yet. The service is running; it just has nothing to list. If you see a connection error instead, the service isn't started (see the failure modes block).
Ollama is installed, the service is running, and the CLI responds. Your machine is now a local AI server — but it has no models yet.
Think of it like a DVD player with no discs: the hardware works, but you haven't loaded anything to play.
ollama --version and ollama list — one checks the CLI, the other checks the service.)In the next module you'll run ollama pull llama3.2 to download your first model, then ollama run llama3.2 to open a live chat with it — and you'll see exactly what the CLI prints at each step and why.
Go to ollama.com in your browser. The site detects your OS and shows the right button — click it.
.zip — unzip it and drag Ollama to Applications.OllamaSetup.exe — double-click and follow the wizard.On macOS and Windows, Ollama adds a small icon to your menu bar / system tray when it's ready. On Linux, the install script prints "Ollama is running" when done.
Open Terminal (macOS/Linux) or Command Prompt / PowerShell (Windows). Type:
ollama --version
You should see a version number like ollama version 0.32.0. That confirms the CLI is installed and on your PATH (meaning the terminal can find it).
Run one more command:
ollama list
If the service is running, you'll see a table header (even if it's empty — no models yet). If the service is not running, you'll see a connection error — that's the failure mode we cover next.
Three problems account for almost every failed install. Here's what each one looks like and how to fix it.
ollama list prints Error: could not connect to ollama app, is it running?. Fix: on macOS open the Ollama app from Applications; on Linux run ollama serve in a terminal.ollama --version still says 'ollama' is not recognized. Fix: close and reopen the terminal — the installer updated PATH but the old window didn't pick it up.ollama --version returns nothing. This happens when the download was interrupted mid-stream. Fix: re-run the install one-liner from the official site.If you asked an AI assistant to write your install steps, check these four things before you trust them.
ollama --version actually prints on your machine?ollama list or equivalent) — not just the install step?You run `ollama pull llama3.2` to download a model, then `ollama run llama3.2` to chat with it — seeing exactly what the CLI prints at each step and why each command is needed.
How to download and run a local LLM using two Ollama CLI commands — pull and run.
Why this matters: These two commands are the entry point to everything else in the lesson — no model running means no API, no chat, no local AI.
Decision this forces: Which model to pull (size vs. speed trade-off: e.g., llama3.2 3B for fast laptops vs. larger variants for better quality).
In module 1 you installed and verified the service was running — but no model was on your machine yet.
This module adds the missing piece: downloading a and chatting with it, all from the same you already know.
The driving question here is: which model should you pull, and what actually happens when you do?
A model file stores everything the AI needs to generate text: learned weights and prompt-reading rules.
Model size is measured in parameters (billions of numbers: "3B" or "7B"). More parameters mean better answers, but larger files and slower responses.
Ollama uses — compression that shrinks files by lowering weight precision. This trades minimal quality loss for smaller downloads and faster speed.
A 3B model runs comfortably on typical laptops. A 70B model may crawl or fail without sufficient (GPU memory) or RAM.
Imagine you just finished installing Ollama on your laptop and you open a terminal. You want to ask a local AI a question — privately, no internet needed after the download.
Step 1 — you run ollama pull llama3.2. Ollama contacts its model library, downloads the quantized 3B model file (about 2 GB), and saves it locally. You see a progress bar in the terminal.
Step 2 — you run ollama run llama3.2. Ollama loads the model into memory and opens an interactive chat prompt (>>>). You type your question and the model replies — all on your machine.
Two commands, one model, zero cloud. That's the whole flow.
# Run this in your terminal after Ollama is installed ollama pull llama3.2 # What you will see: # pulling manifest # pulling 966de95ca8a6... 100% ▕████████▏ 2.0 GB # pulling tokenizer... 100% # success
ollamapullllama3.2This command downloads the llama3.2 model file to your machine — you only need to run it once. The progress bar shows the download; "success" means the file is saved and ready.
Ollama checks the local copy first. If the model is already present and up to date, it prints something like "already up to date" and exits immediately — no re-download.
ollama run llama3.2 # Terminal output: # >>> Send a message (/? for help) >>> Why is the sky blue? # Scattering of sunlight by air molecules causes shorter # blue wavelengths to spread across the sky... >>> /bye
run>>>/byeAfter pulling, this command loads the model and opens a live chat session in your terminal. You type at the >>> prompt and the model replies. Type /bye to exit.
Ollama automatically pulls the model before starting the chat — so it still works, but you wait for the download first. The pull step is useful when you want to download ahead of time (e.g., before going offline).
Drag to see how parameter count affects speed and hardware needs. llama3.2 (3B) is the sweet spot for most laptops.
Three failure patterns show up most often — here's what each looks like and what causes it.
ollama serve in a separate terminal, then retry.llama3.2 (3B) instead of a 7B or 13B variant.llama3.2 works; llama-3.2 or Llama3.2 may not. Check spelling at ollama.com/library.ollama list to confirm the model appears with the correct name and size. If the size looks wrong (e.g., 0 B), the download was incomplete — pull again.| Option | Response speed on CPU | Answer quality | When to choose | Cost | Complexity |
|---|---|---|---|---|---|
| llama3.2 (3B) | Fast — replies in seconds | Good for most everyday tasks | Your main laptop, learning, quick experiments, or offline demos with no GPU. | Free | Low — 2 GB download, runs on any modern laptop |
| llama3.2 (7B or larger) | Slow on CPU-only machines | Better reasoning and longer answers | You have a GPU with 8+ GB VRAM or 16+ GB RAM and want noticeably better reasoning. | Free | Medium — 4–8 GB download, needs more RAM/VRAM |
You explore the two key endpoints (`/api/generate` and `/api/chat`) that Ollama serves on `http://localhost:11434`, reading the JSON request and response shapes so you know exactly what to send and what comes back.
A guide to the two HTTP endpoints Ollama exposes — /api/generate and /api/chat — including the JSON shapes you send and receive.
Why this matters: Knowing these endpoints lets you connect any program you write to your local model, moving beyond the CLI to real API-driven builds.
Decision this forces: Which endpoint to use: `/api/generate` for single-turn prompts vs. `/api/chat` for multi-turn conversations.
ollama run llama3.2 in module 2, where did the model actually live — on a remote server, or on your own machine? Also, what command downloaded it first?Answer: ollama pull llama3.2 downloaded the model to your machine. ollama run llama3.2 started a local chat session. Everything stayed on your laptop after the pull.
That local model is still running right now. The chat window is just one way to talk to it. Ollama also opens a web address on your machine. Any program — a Python script, a browser, or a tool you build — can send it messages directly. That web address is the , and it's what this module is about.
A (Representational State Transfer Application Programming Interface) is a standard way for programs to talk to each other over a web address. You send a request to a URL. You get a structured reply back.
When Ollama starts, it opens a local web server at http://localhost:11434. Here, localhost means "this machine". It does not mean the internet. 11434 is the port. Think of it as the door number on the building.
Every request you send is an . It is a type of web request that carries data in its body. You send it to one of Ollama's . These are specific paths like /api/generate or /api/chat.
The data you send and receive is formatted as . JSON means JavaScript Object Notation. It is a readable text format that uses key-value pairs, like {"model": "llama3.2"}.
Ollama exposes two main for talking to a . Each one expects a different JSON shape and is designed for a different use case.
/api/generate — single-turn. You send one prompt string and get one completion back. There is no memory of previous turns. Good for one-shot tasks like summarising a paragraph or filling a template./api/chat — multi-turn. You send an array of messages (each tagged with a role: system, user, or assistant), and the model sees the full conversation history. Good for chatbots and anything that needs context from earlier turns.Both endpoints accept a "stream" field. When "stream": false, Ollama waits until the model finishes and sends one complete JSON object back — the simplest shape to start with.
import requests response = requests.post( "http://localhost:11434/api/generate", json={ "model": "llama3.2", "prompt": "What is the capital of France?", "stream": False } ) print(response.json())
requests.post(...)json={...}"stream": Falseresponse.json()This sends a single prompt to Ollama and prints the raw JSON response. Before you run it, predict: what key in the response object will hold the model's actual answer text?
The key is "response". The full JSON looks like:
{"model": "llama3.2", "response": "The capital of France is Paris.", "done": true, ...}
The model's text lives at response["response"]. The "done" field is True when the model has finished generating.
import requests response = requests.post( "http://localhost:11434/api/chat", json={ "model": "llama3.2", "messages": [ {"role": "user", "content": "What is the capital of France?"}, {"role": "assistant", "content": "The capital of France is Paris."}, {"role": "user", "content": "What is its population?"} ], "stream": False } ) print(response.json()["message"]["content"])
"messages": [...]{"role": "user", ...}{"role": "assistant", ...}response["message"]["content"]This is the delta from Stage 1: the endpoint changes to /api/chat, and instead of a single "prompt" string you now send a "messages" array carrying the full conversation history.
The response shape also changes: the model's reply is now at response["message"]["content"] (not response["response"]) — a common trip-up when switching endpoints.
With history: the model answers about Paris's population (~2.1 million city / ~12 million metro) because it sees the prior context.
Without history (just the population question alone): the model has no idea which city you mean and will either ask for clarification or guess — /api/generate and a bare /api/chat call with one message both lose the context.
Slide to see which endpoint fits as the conversation grows longer.
ConnectionRefusedError: [Errno 111] Connection refused. Fix: run ollama serve. Or check that the Ollama app is open before making any API call.response["response"] on a /api/chat reply gives KeyError: 'response'. The chat endpoint puts the text at response["message"]["content"] instead."stream": False, Ollama sends back a stream of partial JSON lines. Calling response.json() on that raises JSONDecodeError. The body is not a single valid JSON object.http://localhost:11434. Do not use a remote host. Do not use port 8080 or 8000./api/generate vs /api/chat. AI tools often mix them up.["response"] for both endpoints. Verify it uses ["message"]["content"] for /api/chat."stream": False is present if the code calls .json(). Streaming and .json() do not mix.Next up — module 4 puts all of this into practice. You'll send a fully worked curl request. Then you'll make a Python requests call to http://localhost:11434/api/chat. Every field will be explained. The response will be parsed step by step.
You send a fully worked `curl` request and then a Python `requests` call to `http://localhost:11434/api/chat`, with every field explained and the response parsed step by step — using the same llama3.2 model from Module 2.
You send a POST request to the Ollama chat endpoint using curl and Python, then parse the JSON response to extract the model's reply.
Why this matters: This is the bridge between having a local model running and actually using it from code — the skill every subsequent module builds on.
Decision this forces: Whether to use streaming responses (stream: true) or wait for the full reply (stream: false) — and when each is appropriate.
Module 3 introduced two endpoints Ollama serves on your laptop. The chat endpoint is http://localhost:11434/api/chat, and you reach it with an request (POST = "send data to the server").
You send a body (a structured text format, like a labelled form) containing the model name, a list of messages, and a streaming flag. Ollama reads that form and returns the model's reply, also as JSON.
This module shows you exactly how to fill in that form — first with curl in the terminal, then with Python — and how to pull the reply text out of the response.
Every request to /api/chat needs three things: the model name, a messages list, and a stream flag.
model — the exact name of the model to use, e.g. llama3.2 (the same one you pulled in Module 2).messages — a list of turns in the conversation. Each turn has a ("user" for your words, "assistant" for the model's words) and a content string.stream — true to receive words as they are generated (like watching someone type), or false to wait for the whole reply at once.You can also add a role message at the top of the list to give the model standing instructions (e.g. "Reply only in bullet points"). For now, a single user message is enough to get a reply.
curl http://localhost:11434/api/chat \ -X POST \ -H "Content-Type: application/json" \ -d '{ "model": "llama3.2", "messages": [ {"role": "user", "content": "What is the capital of France?"} ], "stream": false }'
-X POST-H "Content-Type: application/json"-d '{ ... }'"stream": falseThis is the simplest possible request to the Ollama chat endpoint. Run it in your terminal while Ollama is running, and you will get back a JSON object containing the model's full reply.
Setting stream: false means curl waits silently until the model finishes, then prints everything at once — the easiest shape to read and parse.
The response contains: "model" (echoes back "llama3.2"), "message" (an object with role: "assistant" and the reply in content), "done" (true when finished), and timing fields like "total_duration". The reply text lives at response.message.content.
import requests url = "http://localhost:11434/api/chat" body = { "model": "llama3.2", "messages": [{"role": "user", "content": "What is the capital of France?"}], "stream": False } response = requests.post(url, json=body) data = response.json() print(data["message"]["content"])
import requestsrequests.post(url, json=body)response.json()data["message"]["content"]This does exactly what the curl command did, but in Python. requests.post() sends the POST request; .json() converts the raw response text into a Python dictionary you can index like any other dict.
The reply text is nested at data["message"]["content"] — you go into the message object, then grab its content field.
Python prints the model's reply text — something like "The capital of France is Paris." — because data["message"]["content"] holds exactly the assistant's reply string.
import requests url = "http://localhost:11434/api/chat" body = { "model": "llama3.2", "messages": [ {"role": "system", "content": "Reply in exactly one sentence."}, {"role": "user", "content": "Explain what a REST API is."} ], "stream": False } response = requests.post(url, json=body) # TODO: print only the reply text (not the whole dict) print(___)
{"role": "system", "content": "..."}response.json()["message"]["content"]This is a small variation of Stage 2: a system message has been added, and the user prompt has changed. Your job is to fill in the one missing line that prints the model's reply text.
print(response.json()["message"]["content"])
Changed lines vs Stage 2:
curl: (7) Failed to connect to localhost port 11434. Ollama is not running. Fix: open a terminal and run ollama serve (or check that the Ollama app is open).the response JSON contains {"error": "model 'llama3.2' not found"}. The model name is misspelled or was never pulled. Fix: run ollama pull llama3.2 first, then retry.stream: False and try to call .json() on a streaming response. Fix: set "stream": False until you are ready to handle chunks deliberately.If you use an AI assistant to write your Ollama request code, run this checklist before trusting it:
http://localhost:11434/api/chat — not /api/generate (different response shape) and not a cloud URL.llama3.2 character-for-character — a wrong name gives a silent 404-style error in the JSON body.true and breaks a simple .json() parse.data["message"]["content"] — not data["response"] (that's the /api/generate shape, not /api/chat).With these checks passing, you have everything you need for the next module: building a short Python script that takes live user input, sends it to llama3.2, and prints the reply — turning these two lines into a real interactive app.
You build a short Python script that takes user input, sends it to the local llama3.2 model via the Ollama API, and prints the reply — completing the loop from user → app → local LLM → app → user with every line explained.
You build a short Python script that sends user input to a local llama3.2 model and prints the reply, with every line explained.
Why this matters: This is the payoff module — you turn the raw API knowledge from earlier modules into a real, runnable app you can extend.
requests call to /api/chat in Module 4, what three fields did the JSON body always contain? Write them down, then check below.Answer: model (which model to use), messages (the conversation list), and stream (whether the reply streams piece by piece).
In Module 4 you typed that request by hand with curl and then with a raw Python snippet. Now you'll wrap that same call inside a clean function. A real app can use it. That's the whole goal of this module.
A here means one round trip: your script collects a question, sends it to the llama3.2 model running locally, and prints the answer.
The service listens at http://localhost:11434 — it's already running on your machine from Module 1. Your script just needs to talk to it over that address.
Every message you send must carry a : either "user" (what the human typed) or "system" (background instructions the model reads first). The model uses these roles to understand who is speaking.
Keeping the call inside a single Python function means you can reuse it, test it, and extend it without rewriting the same lines every time.
import requests URL = "http://localhost:11434/api/chat" def ask(user_message): body = { "model": "llama3.2", "messages": [{"role": "user", "content": user_message}], "stream": False } response = requests.post(URL, json=body) return response.json()["message"]["content"]
requests.post(URL, json=body)"stream": Falseresponse.json()["message"]["content"]This is the smallest working wrapper around the Ollama . It sends one user message and returns the model's reply as a plain string.
Setting stream: False tells Ollama to wait until the full answer is ready before sending anything back — simpler to handle than chunks.
It returns a plain Python string — the model's reply text. response.json() parses the JSON body into a dict; ["message"] gets the assistant's message object; ["content"] pulls out the text inside it. Example: "The capital of France is Paris."
# Continuing from Stage 1 — ask() is already defined above def main(): print("Chat with llama3.2 (type 'quit' to exit)") while True: user_input = input("You: ") if user_input.strip().lower() == "quit": break reply = ask(user_input) print(f"Model: {reply}\n") if __name__ == "__main__": main()
while True:input("You: ")user_input.strip().lower()if __name__ == "__main__":This adds the interactive shell around the ask() function from Stage 1. The while True loop keeps asking for input until the user types quit.
The if __name__ == "__main__" guard means main() only runs when you execute the file directly — not when another script imports it.
You: Hello
Model: <whatever llama3.2 replies>
The f-string prints "Model: " followed by the reply string, then a blank line (\n) to separate turns.
import requests URL = "http://localhost:11434/api/chat" SYSTEM = "You are a concise assistant. Reply in two sentences or fewer." def ask(user_message, system=SYSTEM): body = { "model": "llama3.2", "messages": [ {"role": "system", "content": system}, {"role": "user", "content": user_message} ], "stream": False } response = requests.post(URL, json=body) return response.json()["message"]["content"]
{"role": "system", "content": system}def ask(user_message, system=SYSTEM):The only change from Stage 1 is that messages now has TWO items: the first, then the user message. The model reads them in order, so the system instruction shapes every reply.
Passing system as a default argument means you can override it per call — useful when you want different personas or constraints without rewriting the function.
The model will give a much shorter answer — two sentences at most — because it reads the system instruction before your question and follows it. In Stage 1 there was no system message, so the model could reply at any length. CHANGED LINES: the messages list now has {"role": "system", "content": system} prepended before the user message — that's the only structural change.
You now have a working Python app. It takes user input, sends it to llama3.2 via the Ollama , and prints the reply. It can also use an optional to control the model's behavior.
Here's what the complete three-stage script does end to end:
ask() wraps the API call and returns the reply string.main() loops, reads input, calls ask(), and prints the result.messages shapes every reply the model gives.But what happens when the service isn't running, the model name is wrong, or your machine runs out of memory? The final module walks you through the five most common Ollama problems: service not running, model not found, slow inference, connection refused, and out-of-memory errors. It shows you exactly how to fix each one.
ConnectionError: HTTPConnectionPool … Max retries exceeded — Ollama isn't running. Open a terminal and run ollama serve before starting your script.KeyError: 'message' — the response JSON doesn't have the shape you expected. Print response.json() to see the actual dict. A common cause is a typo in the model name. For example, "llama3" instead of "llama3.2" returns an error object instead."llama3.2" — AI tools often guess a different version.stream is set to False if the generated code doesn't handle streaming. Missing this causes a partial or garbled reply.messages list. Order matters. A system message placed after the user message is ignored by most models.response.json() once to confirm the shape matches what you expect.You work through the five most common Ollama problems — service not running, model not found, slow inference, connection refused, and out-of-memory errors — with a symptom → cause → fix pattern, revisiting the install and model-size concepts from Modules 1 and 2.
A practical repair kit for the five most common Ollama setup problems, using a symptom → cause → fix pattern.
Why this matters: When your local LLM app breaks — and it will — this module tells you exactly what to look at and how to fix it in minutes.
Decision this forces: When to switch to a smaller/quantized model vs. when the hardware is simply insufficient for the chosen model.
Answer: your script posts to http://localhost:11434/api/chat and targets the llama3.2 model — the same one you pulled in Module 2.
That loop — user → script → → model → script → user — is now complete. But in the real world, something in that chain breaks. This module is your repair kit.
Every Ollama issue follows a three-step pattern: spot the symptom, find the cause, apply the fix.
The five most common problems are: the service not running, wrong model name, slow inference, refused connection, and out of memory.
Each section below names the exact error you see, explains why it happens, and gives the fix.
You run your Module 5 script and something goes wrong. Here is how to read each symptom and respond.
curl: (7) Failed to connectollama serve in a terminal (or restart the Ollama app on macOS/Windows). Then re-run curl http://localhost:11434 — it should return Ollama is running{"error":"model 'llama3' not found, try pulling it first"}llama3 but the pulled model is named llama3.2. Names must match exactly.ollama list to see every pulled model's exact name, then update your script to match.offloading layers to CPUollama pull llama3.2:1b, or shorten the context window in a .ConnectionRefusedError: [Errno 111] Connection refusedhttp://localhost:11434. Run ollama serve if the service is down. Check Ollama logs with journalctl -u ollama (Linux) or the app's log panel (macOS/Windows).CUDA out of memory (CUDA is NVIDIA's GPU software layer); the process exits silently.llama3.2:1b) or reduce the context length in a Modelfile with PARAMETER num_ctx 1024import requests # Wrong port — simulates a common typo (1143 instead of 11434) url = "http://localhost:1143/api/chat" payload = { "model": "llama3.2", "messages": [{"role": "user", "content": "Hello"}] } response = requests.post(url, json=payload) print(response.json())
requests.post(url, json=payload)"http://localhost:1143/api/chat"response.json()This script has a deliberate typo in the port number. Run it and predict what happens before reading the answer.
Python raises: ConnectionRefusedError: [Errno 111] Connection refused. Nothing is listening on port 1143, so the OS immediately rejects the connection. The fix is to change the URL to http://localhost:11434/api/chat.
import requests # Corrected URL — port 11434 is Ollama's default url = "http://localhost:11434/api/chat" payload = { "model": "llama3.2", "messages": [{"role": "user", "content": "Hello"}], "stream": False } response = requests.post(url, json=payload) print(response.status_code) # 200 means success print(response.json()["message"]["content"])
"stream": Falseresponse.status_coderesponse.json()["message"]["content"]This is the corrected script from Stage 1 — only the port number changed. A status code of 200 confirms the server accepted the request.
If you still get an error after fixing the URL, check the Ollama logs: run journalctl -u ollama --no-pager | tail -20 on Linux, or open the Ollama app's log panel on macOS/Windows. The logs show the exact model name the server tried to load and any memory errors — this is the fastest way to confirm your API call is correctly formed.
Change line 7: "model": "llama3.2" → "model": "llama3.2:1b". That is the only change needed. The URL, payload structure, and stream flag stay the same. The :1b tag selects the 1-billion-parameter quantized variant, which is smaller and faster than the default.
| Option | Inference speed | Output quality | Setup effort | When to choose | Cost | Complexity |
|---|---|---|---|---|---|---|
| Switch to smaller/quantized model | Fast — fits in VRAM | Reduced — smaller model knows less | One pull command | When your hardware is fixed (a laptop or a single GPU) and you need usable speed now. | Free; trades some quality for speed | Low — one pull command |
| Upgrade hardware / add GPU | Fast once installed | Full quality preserved | Hardware + driver install | When output quality is non-negotiable and you can invest in more VRAM (e.g. a dedicated GPU). | High — GPU hardware cost | High — hardware purchase and driver setup |
| Shorten context window via Modelfile | Faster — less memory used | Same model, shorter memory | One Modelfile parameter | When the model quality is acceptable but OOM errors appear only with long conversations. | Free; trades conversation length for stability | Low — edit one Modelfile line |
Drag to see how context length (the model's working memory size) affects VRAM pressure and the risk of an OOM crash on a typical laptop GPU.
Before running AI-generated Ollama code, check these four things.
http://localhost:11434 — AI tools sometimes hallucinate a different port or add a trailing slash.ollama list output — the AI may suggest a name you never pulled."stream": true without explanation, your script will hang.max_tokens; use num_predict instead.You now have a complete local LLM workflow and a repair kit. The capstone challenge asks you to build and debug a small app end-to-end so you can prove the whole loop to yourself.
Before looking at the summary, try to recall: what are the four commands you ran to go from a fresh machine to a working API call, and what does each one do? Then name the two endpoints and the one Python function you wrote to tie everything together.
Apply what you learned to Serving Local LLMs with Ollama.
What does Ollama do, in plain terms?
Ollama is a local model server: it pulls model weights to your machine and serves them through a REST API — no cloud, no fine-tuning, no browser involved. The cloud-service distractor confuses Ollama with hosted APIs like OpenAI. The fine-tuning distractor confuses it with tools like LoRA trainers. The browser-extension distractor is simply unrelated.
You are building a customer-support chatbot that must remember what the user said two messages ago. Which Ollama endpoint should you use, and why?
/api/chat takes a messages array, so you can pass the full conversation history and the model can refer back to earlier turns — exactly what a chatbot needs. /api/generate is designed for single-turn, stateless prompts; it has no built-in messages structure. The claim that /api/chat lacks system-prompt support is false — you include a system role message in the array. The two endpoints are not interchangeable for multi-turn use.
You run this Python snippet and want to know what it prints:
import requests
r = requests.post('http://localhost:11434/api/generate', json={'model':'llama3.2','prompt':'Hi','stream':False})
print(r.json()['response'])
Assuming Ollama is running and the model is pulled, what does the last line print?
For /api/generate with stream set to false, Ollama returns a single JSON object and the model's reply text lives in the 'response' field — so r.json()['response'] prints just the reply text. Printing r.json() would give the full object. Streaming chunks only appear when stream is true. The 'content' field name belongs to /api/chat's message object, not /api/generate.
Your Python app throws a 'connection refused' error when it tries to reach http://localhost:11434. What is the most likely cause and the fastest fix?
A 'connection refused' error on port 11434 almost always means nothing is listening there — the Ollama service is simply not running. Starting it with 'ollama serve' (or relaunching the desktop app) fixes it in seconds. A model-size RAM problem would show a different error after the connection succeeds. Port 11434 is Ollama's default and is not OS-reserved. A stale requests library would not cause a connection-refused error.
You have a laptop with 8 GB of RAM and inference with your current model is very slow — responses take over 30 seconds. Name TWO actions you could take to improve speed, and explain the trade-off of each.
Speed problems on low-RAM machines come from the model not fitting comfortably in memory, forcing slow disk swapping. A smaller or quantized model uses less RAM and fewer compute operations per token. Shortening context length reduces the memory footprint per inference call. Both help speed but cost quality or history depth — recognizing that trade-off is the core decision skill from the troubleshooting and model-selection modules.