Local models

Use /connect to add models from an OpenAI-compatible endpoint. Embedder supports Ollama, LM Studio, vLLM, llama.cpp, and other compatible servers.

If Local models is missing from /connect, ask your Embedder administrator to enable Local models.

Connect a local endpoint

Start the model server

Ollama
LM Studio
vLLM
llama.cpp

OLLAMA_CONTEXT_LENGTH=32768 ollama serve
ollama pull qwen3:32b

Use http://localhost:11434/v1 as the base URL.

Load a model, set its context window, and start the server from LM Studio’s Developer view.Use http://localhost:1234/v1 as the base URL.

vllm serve Qwen/Qwen3-32B --max-model-len 32768

Use http://localhost:8000/v1 as the base URL.

llama-server -m qwen3-32b.gguf -c 32768 --port 8080

Use http://localhost:8080/v1 as the base URL.

Add the endpoint

Run /connect, choose Local models, and fill in:

Field	What to enter
Name	A short label, such as `ollama`
Server	The server you are running
Base URL	The server URL, with or without `/v1`
Model	Leave blank to discover models, or enter a model ID
Context window	The context limit configured on the server
API key	A bearer token, if the server requires one

Save the endpoint, use Test to check the connection, then choose the model in /model.

Set the context window

Match the context window to the server configuration. Setting it too high can cause the server to discard older messages before Embedder expects it. For Ollama, also set OLLAMA_CONTEXT_LENGTH before starting ollama serve.

Advanced configuration

/connect stores endpoints in ~/.embedder/local-endpoints.json. Edit this file when you need custom headers, multiple declared models, or other options not shown in the form.

~/.embedder/local-endpoints.json

{
  "version": 1,
  "endpoints": [
    {
      "name": "ollama",
      "baseUrl": "http://localhost:11434",
      "kind": "ollama",
      "apiKey": "${LOCAL_LLM_KEY}",
      "defaultContext": 32768,
      "models": [
        {
          "id": "qwen3:32b",
          "context": 32768
        }
      ]
    }
  ]
}

Endpoint options

Option	Purpose
`label`	Changes the label shown in `/model`
`headers`	Adds request headers
`timeoutMs`	Changes the connection timeout
`discover`	Enables or disables `/v1/models` discovery
`enabled`	Temporarily enables or disables the endpoint
`defaultOutput`	Sets the default output limit
`dropParams`	Removes unsupported request fields
`models`	Declares models explicitly

Keep secrets out of the file

Endpoint values support ${NAME} and ${NAME:-fallback} environment-variable references:

export LOCAL_LLM_KEY="your-key"
embedder

Typing a key directly stores it as plain text. If a referenced variable is missing, Embedder skips that endpoint.

Pin a local model

An endpoint named ollama with a model named qwen3:32b appears in /model as ollama/qwen3:32b. To select it automatically for a project:

.embedder/models.json

{
  "main": "local/ollama/qwen3:32b"
}

Project settings override ~/.embedder/models.json. A valid main pin is selected when the project opens. To show only local and subscription models, hide Embedder-hosted models.

Data and privacy

Prompts and responses travel directly between Embedder and your endpoint. The endpoint URL, API key, and custom headers remain in local configuration. Some features still use Embedder cloud services:

Account, team, and project information
System prompts and capabilities such as web search
Billing and usage information
Telemetry unless disabled by policy or EMBEDDER_NO_TELEMETRY=1

Troubleshooting

Local models is missing from /connect

Ask an Embedder administrator to enable Local models, then restart the session.

The connection test fails

Make sure the server is running, then check the base URL, HTTP or HTTPS setting, and API key.

A model does not appear

If the server does not support model discovery, enter the exact model ID in the Model field. Then open /model and select it.

The model forgets old turns

Reduce the declared context window to the value enforced by the server. For Ollama, also set OLLAMA_CONTEXT_LENGTH before starting the server.

The server rejects a request field

Add the field name to dropParams in local-endpoints.json.

Connect subscriptions

Connect GitHub Copilot or ChatGPT Codex for inference.

Get started

Core concepts

Complex workflows

Monitor and debug

Project tools

Customize

Integrations

Automation

CLI

Reference

API

Connect a local endpoint

Start the model server

Add the endpoint

Set the context window

Advanced configuration

Keep secrets out of the file

Pin a local model

Data and privacy

Troubleshooting

Connect subscriptions

​Connect a local endpoint

​Start the model server

​Add the endpoint

​Set the context window

​Advanced configuration

​Keep secrets out of the file

​Pin a local model

​Data and privacy

​Troubleshooting

Connect subscriptions

Connect a local endpoint

Start the model server

Add the endpoint

Set the context window

Advanced configuration

Keep secrets out of the file

Pin a local model

Data and privacy

Troubleshooting