# Local LLM coding agent: Ollama and LM Studio setup

URL: https://agentvera.dev/en/blog/local-llm-coding-agent/ · Published: 2026-10-23

Run a local LLM coding agent with Ollama or LM Studio: which CLIs support local endpoints, the hardware it takes, privacy gains, trade-offs and a hybrid setup.

**In short:**

- Ollama serves an OpenAI-compatible API at localhost:11434 and LM Studio at 127.0.0.1:1234, and coding CLIs such as Codex, opencode, Aider, Goose, Qwen Code and Crush can point at them.
- Ollama’s default context is 4k tokens below 24 GiB of VRAM and silently drops overflow, so raise it with OLLAMA_CONTEXT_LENGTH; Ollama asks for at least 64k for Codex.
- Local models keep code on your machine with no per-token cost but are slower and weaker on multi-file work, so a hybrid split with a hosted model usually works best.

A local LLM coding agent is a coding CLI, such as opencode, Aider, Goose, Qwen Code, Crush or Codex, pointed at a model running on your own machine through Ollama or LM Studio instead of a hosted API. You start a local server (Ollama listens on `http://localhost:11434`, LM Studio on `http://127.0.0.1:1234`), pick a model that supports tool calling, raise its context length well above the default, and tell the CLI to use that endpoint. It works, it keeps your code on your machine, and it costs nothing per token. It is also slower and less capable than the top hosted models, so most people end up with a hybrid setup.

## What “local” actually means here

A [coding agent](/en/glossary/coding-agent/ "Coding agent") is two parts: the CLI that reads files, runs commands and edits code, and the model that decides what to do next. Going local replaces only the second part. The CLI still runs the same loop; it just sends its requests to a server on `localhost` instead of a provider’s API.

The two common local servers both speak an OpenAI-compatible API:

| Server | Default address | How you start it |
| --- | --- | --- |
| Ollama | `http://localhost:11434/v1` | The Ollama app, or `ollama serve` |
| LM Studio | `http://127.0.0.1:1234/v1` | Developer tab → “Start server”, or `lms server start` |

Ollama’s docs note that OpenAI clients require an API key value but Ollama ignores it, so any placeholder works for a local server.

## The setting everyone misses: context length

Before choosing a CLI, fix this. Agents send a lot of context: a system prompt, tool definitions, file contents and command output. A local server with a small window will quietly cut most of it.

Ollama’s default [context window](/en/glossary/context-window/ "Context window") depends on your VRAM: 4k tokens under 24 GiB, 32k between 24 and 48 GiB, and 256k at 48 GiB or more. Aider’s docs add that Ollama silently discards context that exceeds the window. Goose’s docs call a 4096-token default too low, and Ollama’s own Codex guide asks for at least 64k.

Raise it when you start the server, then check:

```bash
OLLAMA_CONTEXT_LENGTH=64000 ollama serve
ollama ps   # CONTEXT column shows the allocated length
```

The Ollama app has a context length slider in its settings that does the same. If `ollama ps` shows the model partly on the CPU in the `PROCESSOR` column, the context or the model is too big for your GPU and everything will slow down.

## Which CLIs support local models

Here is what each tool’s own documentation confirms. We only list setups we could verify.

| CLI | Local route | Where it is configured |
| --- | --- | --- |
| Codex | `--oss` with Ollama or LM Studio | Flag, or `oss_provider` in `config.toml` |
| opencode | Any OpenAI-compatible provider | `provider` section of `opencode.json` |
| Aider | `ollama_chat/<model>` or `openai/<model>` | Environment variables |
| Goose | Ollama, LM Studio and other providers | `goose configure` |
| Qwen Code | Any OpenAI-compatible endpoint | `OPENAI_*` variables or `~/.qwen/settings.json` |
| Crush | `ollama` and `lmstudio` provider types | Its provider config |

### Codex

[Codex](/en/agents/codex/ "Codex CLI") has the shortest path. Its docs say it can run against a local “open source” provider such as Ollama or LM Studio when you pass `--oss`:

```bash
codex --oss                     # default local model
codex --oss -m gpt-oss:120b     # a specific model
```

Set the default provider in `~/.codex/config.toml` with `oss_provider = "ollama"` (or `"lmstudio"`). Ollama also offers `ollama launch codex`, which sets up a profile for you. Inside Codex, `/model` switches between installed tool-capable models.

### opencode

[opencode](/en/agents/opencode/ "opencode") takes any OpenAI-compatible server as a custom provider in `opencode.json`:

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": { "baseURL": "http://localhost:11434/v1" },
      "models": { "qwen3": { "name": "Qwen3 (local)" } }
    }
  }
}
```

For LM Studio, use `http://127.0.0.1:1234/v1` as the base URL and the model identifier shown in LM Studio. opencode’s docs add a useful tip: if tool calls are not working, increase `num_ctx` in Ollama, starting around 16k to 32k.

### Aider

[Aider](/en/agents/aider/ "Aider") talks to Ollama directly. Its docs recommend the `ollama_chat/` prefix:

```bash
export OLLAMA_API_BASE=http://127.0.0.1:11434
aider --model ollama_chat/<model>
```

For LM Studio or any other OpenAI-compatible server, set `OPENAI_API_BASE` and `OPENAI_API_KEY`, then use `aider --model openai/<model-name>`. A fixed context size goes in `.aider.model.settings.yml` as `num_ctx` under `extra_params`.

### Goose

[Goose](/en/agents/goose/ "Goose") lists Ollama and LM Studio as providers. Run `goose configure`, choose Ollama and enter a model name. It uses `localhost:11434` unless `OLLAMA_HOST` says otherwise; LM Studio defaults to `localhost:1234` with no API key. Goose’s docs are blunt about the main constraint: it uses tool calling extensively, so a model without it can only chat.

### Qwen Code

[Qwen Code](/en/agents/qwen-code/ "Qwen Code") supports any OpenAI-compatible endpoint, and its README names local models through Ollama or vLLM. Point the standard variables at your server:

```bash
export OPENAI_BASE_URL=http://localhost:11434/v1
export OPENAI_API_KEY=ollama   # any placeholder for a local server
export OPENAI_MODEL=<model>
qwen
```

The same can live in `~/.qwen/settings.json` under `modelProviders`, or you can set it up interactively with `/auth`.

### Crush

Crush supports `ollama` and `lmstudio` as provider types. Its README shows Ollama added with a base URL of `http://localhost:11434/v1`. Pick the model inside Crush with `Ctrl+L`.

## When the local agent misbehaves

Most problems with a local coding agent come down to a handful of causes. Check them in this order:

- **It forgets files or instructions mid-task.** The context is too small and the server is truncating. Raise `OLLAMA_CONTEXT_LENGTH` (or `num_ctx`) and confirm with `ollama ps`.
- **Tool calls fail or come back as plain text.** The model may not support tool calling, or the context is too short for the tool definitions. Try a tool-capable model and a bigger window; opencode’s docs suggest starting around 16k to 32k.
- **The CLI says the model does not exist.** The name in the CLI must match the server’s identifier exactly: the tag from `ollama list`, or the identifier in LM Studio’s server panel.
- **Connection refused.** The server is not running, or the CLI expects a different address. Ollama is `11434`, LM Studio is `1234`, and some CLIs want the `/v1` suffix while Aider’s `OLLAMA_API_BASE` does not.
- **Everything is very slow.** The model or the context does not fit on the GPU. `ollama ps` shows the CPU share; pick a smaller model or a shorter context.

## Hardware reality

This is where expectations usually break. A local coding agent is a heavier job than local chat:

- **The model must fit in memory.** GPU VRAM on a PC, unified memory on Apple Silicon. When it does not fit, layers spill to the CPU and generation slows sharply.
- **Context costs memory too.** A 64k window on top of the model weights needs a lot more room than a 4k chat. This is why Ollama ties its default context length to VRAM.
- **Small models struggle with tools.** Agents depend on reliable tool calls and on following a long system prompt. Small models tend to produce malformed calls, loop, or stop early. Goose notes that some models, such as the native DeepSeek-R1, don’t support tool calling at all.
- **Speed compounds.** An agent makes many requests per task, each with a large prompt. A model that feels fine in chat can make a ten-step task take a long time.

A practical way to start: take the largest tool-capable coding model that runs fully on your GPU with at least a 32k to 64k context, confirm with `ollama ps`, and test it on a small, real task before trusting it with anything bigger.

## Why go local: privacy and control

The main reason is that inference stays on your machine:

- Your prompts and code are not sent to a model provider.
- It works offline and in air-gapped environments.
- No per-token bill and no usage limits to run into.
- The model does not change under you; you decide when to upgrade.

Two caveats keep this honest. First, the CLI can still use the network: Codex with Ollama’s launch profile has web search, and some CLIs have telemetry. Check each tool’s settings if nothing at all should leave the machine. Second, running locally does not make the agent itself safer. It still runs commands and edits files, so the usual [coding agent security practices](/en/blog/ai-coding-agent-security/ "AI coding agent security") apply.

## Trade-offs against hosted models

| | Local model | Hosted frontier model |
| --- | --- | --- |
| Code leaves the machine | No | Yes, to the provider |
| Cost | Hardware and power | Per token or subscription |
| Usage limits | None | Plan-based |
| Quality on hard, multi-file tasks | Lower, depends on model size | Highest available |
| Tool-call reliability | Varies by model | Strong |
| Context window | Limited by your memory | Very large |
| Speed | Depends on your hardware | Fast and consistent |

The gap is smallest on contained tasks: writing a test, explaining a function, a small refactor, a commit message. It is largest on long, multi-step work across many files, where the agent has to keep a plan and recover from its own mistakes.

## A hybrid setup that works

A split usually works better than an all-or-nothing switch:

1. **Sensitive or offline work goes local.** Code that must not leave the machine, client repos with strict terms, work on a plane.
2. **Routine, contained tasks go local.** Tests, docs, small fixes and explanations.
3. **Hard tasks go to a hosted model.** Architecture changes, tricky bugs, large refactors.
4. **Review in a fresh context.** Let a second agent check the first one’s diff, whichever model wrote it.

The simplest way to run this is two or more agents at once: one on a local endpoint, one on a hosted model, each in its own folder or git worktree so they do not edit the same files.

## Running local and hosted agents side by side in AgentVera

AgentVera is a desktop app that runs many CLI agents side by side in one [workspace](/en/features/workspace/ "Workspace"), which fits the hybrid setup above. To be clear about what it does and does not do:

- **It has no built-in Ollama or LM Studio integration.** Each agent uses its own provider setup: `goose configure` for Goose, opencode’s own config and `/connect`, Aider’s environment variables or `.aider.conf.yml`, Qwen Code’s sign-in or API key, Crush’s settings (`Ctrl+L`).
- **The “Model” field is passed to the CLI.** For agents such as Qwen Code, Goose and Aider, what you type there goes to the CLI as `--model`. For Goose the hint reads: “If left empty, the model chosen with `goose configure` is used.”
- **Agents can carry environment variables.** Under Settings → Accounts → Accounts of other agents, an account added “With an API key” stores a “Variable name” and its value. For Qwen Code, `OPENAI_BASE_URL` is one of the suggested names, and any other name can be typed. This is part of [account switching](/en/features/account-switching/ "Account switching") on the Pro and Team plans.
- **Context tools reach beyond Claude Code.** Context size is measured from each CLI’s own history, and auto-compact uses the CLI’s own compact command where it has one. With a small local window, that matters even more.

So the realistic picture is: you set up the local model in the CLI the way its docs describe, and AgentVera lets you run that agent next to a hosted one, watch both, and keep their work apart.

## Frequently asked questions

**Which coding agent CLIs work with a local LLM?** opencode, Aider, Goose, Qwen Code, Crush and Codex all document a way to use a local or OpenAI-compatible endpoint. Codex has a built-in --oss flag for Ollama and LM Studio; the others take a provider config, a model prefix or environment variables.

**Do I need a GPU to run a local coding agent?** Not strictly, but it helps a lot. Coding agents need tool calling and a long context, which means a fairly large model and plenty of memory. Without a capable GPU or a lot of unified memory, responses get slow enough that agent work becomes painful.

**Why does my local model ignore files or forget instructions?** Usually the context length is too small. Ollama’s default depends on VRAM and is 4k on machines with under 24 GiB, and it silently drops what does not fit. Raise it with OLLAMA_CONTEXT_LENGTH or a num_ctx setting; Ollama recommends at least 64k for Codex.

**Is a local model completely private?** Inference stays on your machine, so prompts and code are not sent to a model provider. The CLI itself may still have telemetry, web search or other network features, so check its settings if nothing should leave the machine.

**Does AgentVera support local models?** AgentVera runs the CLI agents that support them, such as opencode, Aider, Goose, Qwen Code, Crush and Codex, side by side. Each CLI keeps its own provider config; AgentVera passes the Model field through and can give an agent environment variables, but it has no built-in Ollama integration of its own.

---

**About AgentVera.** AgentVera is a desktop app for macOS, Windows and Linux, built by Solviera Teknoloji. It runs 18 AI coding agents (Claude Code, Codex, Gemini CLI, Copilot, Cursor, Antigravity, opencode, Hermes, Qwen Code, Kimi Code, Kiro, Amp, Droid, Goose, Vibe, Crush, Aider, Cline) side by side, per project, in real terminals with the user's own CLI installs and logins. The free plan has unlimited agents. Download: https://agentvera.dev/en/download/ · Pricing: https://agentvera.dev/en/pricing/