Local LLM coding agent: Ollama and LM Studio setup

A local LLM coding agent is a coding CLI, such as opencode, Aider, Goose, Qwen Code, Crush or Codex, pointed at a model running on your own machine through Ollama or LM Studio instead of a hosted API. You start a local server (Ollama listens on http://localhost:11434, LM Studio on http://127.0.0.1:1234), pick a model that supports tool calling, raise its context length well above the default, and tell the CLI to use that endpoint. It works, it keeps your code on your machine, and it costs nothing per token. It is also slower and less capable than the top hosted models, so most people end up with a hybrid setup.

What “local” actually means here

A coding agent is two parts: the CLI that reads files, runs commands and edits code, and the model that decides what to do next. Going local replaces only the second part. The CLI still runs the same loop; it just sends its requests to a server on localhost instead of a provider’s API.

The two common local servers both speak an OpenAI-compatible API:

Ollama’s docs note that OpenAI clients require an API key value but Ollama ignores it, so any placeholder works for a local server.

The setting everyone misses: context length

Before choosing a CLI, fix this. Agents send a lot of context: a system prompt, tool definitions, file contents and command output. A local server with a small window will quietly cut most of it.

Ollama’s default context window depends on your VRAM: 4k tokens under 24 GiB, 32k between 24 and 48 GiB, and 256k at 48 GiB or more. Aider’s docs add that Ollama silently discards context that exceeds the window. Goose’s docs call a 4096-token default too low, and Ollama’s own Codex guide asks for at least 64k.

Raise it when you start the server, then check:

OLLAMA_CONTEXT_LENGTH=64000 ollama serve
ollama ps   # CONTEXT column shows the allocated length

The Ollama app has a context length slider in its settings that does the same. If ollama ps shows the model partly on the CPU in the PROCESSOR column, the context or the model is too big for your GPU and everything will slow down.

Which CLIs support local models

Here is what each tool’s own documentation confirms. We only list setups we could verify.

Codex

Codex has the shortest path. Its docs say it can run against a local “open source” provider such as Ollama or LM Studio when you pass --oss:

codex --oss                     # default local model
codex --oss -m gpt-oss:120b     # a specific model

Set the default provider in ~/.codex/config.toml with oss_provider = "ollama" (or "lmstudio"). Ollama also offers ollama launch codex, which sets up a profile for you. Inside Codex, /model switches between installed tool-capable models.

opencode

opencode takes any OpenAI-compatible server as a custom provider in opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": { "baseURL": "http://localhost:11434/v1" },
      "models": { "qwen3": { "name": "Qwen3 (local)" } }
    }
  }
}

For LM Studio, use http://127.0.0.1:1234/v1 as the base URL and the model identifier shown in LM Studio. opencode’s docs add a useful tip: if tool calls are not working, increase num_ctx in Ollama, starting around 16k to 32k.

Aider

Aider talks to Ollama directly. Its docs recommend the ollama_chat/ prefix:

export OLLAMA_API_BASE=http://127.0.0.1:11434
aider --model ollama_chat/<model>

For LM Studio or any other OpenAI-compatible server, set OPENAI_API_BASE and OPENAI_API_KEY, then use aider --model openai/<model-name>. A fixed context size goes in .aider.model.settings.yml as num_ctx under extra_params.

Goose

Goose lists Ollama and LM Studio as providers. Run goose configure, choose Ollama and enter a model name. It uses localhost:11434 unless OLLAMA_HOST says otherwise; LM Studio defaults to localhost:1234 with no API key. Goose’s docs are blunt about the main constraint: it uses tool calling extensively, so a model without it can only chat.

Qwen Code

Qwen Code supports any OpenAI-compatible endpoint, and its README names local models through Ollama or vLLM. Point the standard variables at your server:

export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=ollama   # any placeholder for a local server
export OPENAI_MODEL=<model>
qwen

The same can live in ~/.qwen/settings.json under modelProviders, or you can set it up interactively with /auth.

Crush

Crush supports ollama and lmstudio as provider types. Its README shows Ollama added with a base URL of http://localhost:11434/v1. Pick the model inside Crush with Ctrl+L.

When the local agent misbehaves

Most problems with a local coding agent come down to a handful of causes. Check them in this order:

• It forgets files or instructions mid-task. The context is too small and the server is truncating. Raise OLLAMA_CONTEXT_LENGTH (or num_ctx) and confirm with ollama ps.
• Tool calls fail or come back as plain text. The model may not support tool calling, or the context is too short for the tool definitions. Try a tool-capable model and a bigger window; opencode’s docs suggest starting around 16k to 32k.
• The CLI says the model does not exist. The name in the CLI must match the server’s identifier exactly: the tag from ollama list, or the identifier in LM Studio’s server panel.
• Connection refused. The server is not running, or the CLI expects a different address. Ollama is 11434, LM Studio is 1234, and some CLIs want the /v1 suffix while Aider’s OLLAMA_API_BASE does not.
• Everything is very slow. The model or the context does not fit on the GPU. ollama ps shows the CPU share; pick a smaller model or a shorter context.

Hardware reality

This is where expectations usually break. A local coding agent is a heavier job than local chat:

• The model must fit in memory. GPU VRAM on a PC, unified memory on Apple Silicon. When it does not fit, layers spill to the CPU and generation slows sharply.
• Context costs memory too. A 64k window on top of the model weights needs a lot more room than a 4k chat. This is why Ollama ties its default context length to VRAM.
• Small models struggle with tools. Agents depend on reliable tool calls and on following a long system prompt. Small models tend to produce malformed calls, loop, or stop early. Goose notes that some models, such as the native DeepSeek-R1, don’t support tool calling at all.
• Speed compounds. An agent makes many requests per task, each with a large prompt. A model that feels fine in chat can make a ten-step task take a long time.

A practical way to start: take the largest tool-capable coding model that runs fully on your GPU with at least a 32k to 64k context, confirm with ollama ps, and test it on a small, real task before trusting it with anything bigger.

Why go local: privacy and control

The main reason is that inference stays on your machine:

• Your prompts and code are not sent to a model provider.
• It works offline and in air-gapped environments.
• No per-token bill and no usage limits to run into.
• The model does not change under you; you decide when to upgrade.

Two caveats keep this honest. First, the CLI can still use the network: Codex with Ollama’s launch profile has web search, and some CLIs have telemetry. Check each tool’s settings if nothing at all should leave the machine. Second, running locally does not make the agent itself safer. It still runs commands and edits files, so the usual coding agent security practices apply.

Trade-offs against hosted models

The gap is smallest on contained tasks: writing a test, explaining a function, a small refactor, a commit message. It is largest on long, multi-step work across many files, where the agent has to keep a plan and recover from its own mistakes.

A hybrid setup that works

A split usually works better than an all-or-nothing switch:

1. Sensitive or offline work goes local. Code that must not leave the machine, client repos with strict terms, work on a plane.
2. Routine, contained tasks go local. Tests, docs, small fixes and explanations.
3. Hard tasks go to a hosted model. Architecture changes, tricky bugs, large refactors.
4. Review in a fresh context. Let a second agent check the first one’s diff, whichever model wrote it.

The simplest way to run this is two or more agents at once: one on a local endpoint, one on a hosted model, each in its own folder or git worktree so they do not edit the same files.

Running local and hosted agents side by side in AgentVera

AgentVera is a desktop app that runs many CLI agents side by side in one workspace, which fits the hybrid setup above. To be clear about what it does and does not do:

• It has no built-in Ollama or LM Studio integration. Each agent uses its own provider setup: goose configure for Goose, opencode’s own config and /connect, Aider’s environment variables or .aider.conf.yml, Qwen Code’s sign-in or API key, Crush’s settings (Ctrl+L).
• The “Model” field is passed to the CLI. For agents such as Qwen Code, Goose and Aider, what you type there goes to the CLI as --model. For Goose the hint reads: “If left empty, the model chosen with goose configure is used.”
• Agents can carry environment variables. Under Settings → Accounts → Accounts of other agents, an account added “With an API key” stores a “Variable name” and its value. For Qwen Code, OPENAI_BASE_URL is one of the suggested names, and any other name can be typed. This is part of account switching on the Pro and Team plans.
• Context tools reach beyond Claude Code. Context size is measured from each CLI’s own history, and auto-compact uses the CLI’s own compact command where it has one. With a small local window, that matters even more.

So the realistic picture is: you set up the local model in the CLI the way its docs describe, and AgentVera lets you run that agent next to a hosted one, watch both, and keep their work apart.