Install Ollama, pull an open model sized for your memory (Qwen2.5-Coder 7B on 8 GB, Qwen3.5 9B on 16 GB), check that its OpenAI-compatible API answers at localhost:11434, and connect it to Cline in VS Code as an OpenAI-compatible provider. It costs nothing per request and works offline, but small models are weaker than hosted ones, so give them small tasks.
Tools you need
The 10 steps
Check your computer
Open your system information and note the memory (RAM). A rough rule from the model file size: the model needs its file size plus 2 to 4 GB for context, the system and your apps. Close your browser tabs before you start. With 8 GB you can run small models; 16 GB is comfortable for a 7 to 9 billion parameter model; 32 GB or more is needed for the 24 to 30 billion parameter coding models.
Install Ollama
Download Ollama for your system from ollama.com and install it. Open a terminal and check it responds:
ollama --versionPull a model sized for your memory
On 8 GB use a small coder model; on 16 GB use a larger one; on 32 GB try an agent-tuned coding model:
ollama pull qwen2.5-coder:7b # about 4.7 GB, Apache-2.0, fits 8 GB
ollama pull qwen3.5:9b # about 6.6 GB, Apache-2.0, 16 GB laptop
ollama pull devstral-small-2 # 24B, about 15 GB, needs about 32 GBThe first download takes a while. Pick one model, not all of them.
Talk to it in the terminal
ollama run qwen2.5-coder:7bAsk it to write a function that reverses the words in a sentence, then to explain it. Type /bye to leave. Note how long a reply takes: this is your baseline.
Check the API your tools will use
Ollama serves an OpenAI-compatible API on your computer. Check that it answers:
curl http://localhost:11434/v1/modelsYou should see your model listed. LM Studio does the same at http://localhost:1234/v1 if you prefer a desktop app.
Connect Cline in VS Code
Install Cline in VS Code, open its settings and choose the API provider OpenAI Compatible. Set the base URL to http://localhost:11434/v1, the API key to any text such as ollama, and the model ID to the exact name you pulled, for example qwen2.5-coder:7b.
Give it small, specific tasks
Local models work best on one function, one file or one explanation. Try: Add input validation to this function, Write a test for this file, Explain what this regular expression does. Avoid asking it to build a whole app in one go.
If it ignores instructions or cuts off
Agents send long prompts. If answers stop mid-way or the model ignores your rules, raise the model's context window in Ollama's settings, close other apps to free memory, or switch to a smaller model with a shorter task.
Measure what it can and cannot do
Run three tasks of different sizes: a small function, a bug fix in one file and a change across two files. Write down the time, whether it worked first time and how much you had to correct. This shows you where local is good enough.
Build a hybrid setup
Use the local model for explanations, small edits, tests and anything private. Use a free hosted route from the free LLM APIs for larger changes. Keep personal and confidential code on the local model, because some free hosted endpoints log or train on prompts.
Make it yours
- Use LM Studio instead of Ollama and compare speed.
- Connect OpenCode or Goose to the same local model.
- Try two models on the same three tasks and tabulate the results.
- Add Playwright or Context7 as MCP servers to the local agent.
Troubleshooting
| Problem | Fix |
|---|---|
| The model is very slow | It may be too large for your memory and swapping to disk. Choose a smaller model or a smaller quantised version and close other apps. |
| Cline says it cannot connect | Check Ollama is running, that the base URL ends in /v1 and that the model name matches the output of ollama list exactly. |
| Replies are low quality | A 7B model is limited. Make the task smaller, give it the relevant code, or hand that task to a larger hosted model. |
| My laptop gets hot and loud | Local models use the processor heavily. Plug in the charger, use a smaller model and take breaks between tasks. |
Frequently asked questions
Can I run an AI coding assistant offline for free?
Yes. Ollama or LM Studio runs open models on your laptop with no key and no per-request cost. Connect them to an agent such as Cline, OpenCode or Goose.
Which local model is best for coding on 16 GB of memory?
Qwen3.5 9B and Qwen2.5-Coder 14B are practical choices, both Apache-2.0. The 24 to 30 billion parameter coding models such as Devstral Small 2 need about 32 GB.
Is a local model as good as ChatGPT or Claude?
No. Small local models are much weaker on large or complex tasks. They are useful for private code, offline work and small edits, and a hosted model handles the bigger jobs.
Does a local model send my code anywhere?
No. It runs on your computer, which is the main reason to use it for private or confidential code.
Learn this with mentors
Programmes for builders and engineers with admissions open: live classes, mentors and project work.
