> ## Documentation Index
> Fetch the complete documentation index at: https://docs.embedder.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Local models

> Connect Ollama, LM Studio, vLLM, llama.cpp, or another OpenAI-compatible model endpoint to Embedder.

Use `/connect` to add models from an OpenAI-compatible endpoint. Embedder supports Ollama, LM Studio, vLLM, llama.cpp, and other compatible servers.

<Note>
  If **Local models** is missing from `/connect`, ask your Embedder administrator to enable Local models.
</Note>

## Connect a local endpoint

### Start the model server

<Tabs>
  <Tab title="Ollama">
    ```bash theme={"system"}
    OLLAMA_CONTEXT_LENGTH=32768 ollama serve
    ollama pull qwen3:32b
    ```

    Use `http://localhost:11434/v1` as the base URL.
  </Tab>

  <Tab title="LM Studio">
    Load a model, set its context window, and start the server from LM Studio's Developer view.

    Use `http://localhost:1234/v1` as the base URL.
  </Tab>

  <Tab title="vLLM">
    ```bash theme={"system"}
    vllm serve Qwen/Qwen3-32B --max-model-len 32768
    ```

    Use `http://localhost:8000/v1` as the base URL.
  </Tab>

  <Tab title="llama.cpp">
    ```bash theme={"system"}
    llama-server -m qwen3-32b.gguf -c 32768 --port 8080
    ```

    Use `http://localhost:8080/v1` as the base URL.
  </Tab>
</Tabs>

### Add the endpoint

Run `/connect`, choose **Local models**, and fill in:

| Field          | What to enter                                       |
| -------------- | --------------------------------------------------- |
| Name           | A short label, such as `ollama`                     |
| Server         | The server you are running                          |
| Base URL       | The server URL, with or without `/v1`               |
| Model          | Leave blank to discover models, or enter a model ID |
| Context window | The context limit configured on the server          |
| API key        | A bearer token, if the server requires one          |

Save the endpoint, use **Test** to check the connection, then choose the model in `/model`.

### Set the context window

Match the context window to the server configuration. Setting it too high can cause the server to discard older messages before Embedder expects it.

For Ollama, also set `OLLAMA_CONTEXT_LENGTH` before starting `ollama serve`.

## Advanced configuration

`/connect` stores endpoints in `~/.embedder/local-endpoints.json`. Edit this file when you need custom headers, multiple declared models, or other options not shown in the form.

```json ~/.embedder/local-endpoints.json theme={"system"}
{
  "version": 1,
  "endpoints": [
    {
      "name": "ollama",
      "baseUrl": "http://localhost:11434",
      "kind": "ollama",
      "apiKey": "${LOCAL_LLM_KEY}",
      "defaultContext": 32768,
      "models": [
        {
          "id": "qwen3:32b",
          "context": 32768
        }
      ]
    }
  ]
}
```

<Accordion title="Endpoint options">
  | Option          | Purpose                                      |
  | --------------- | -------------------------------------------- |
  | `label`         | Changes the label shown in `/model`          |
  | `headers`       | Adds request headers                         |
  | `timeoutMs`     | Changes the connection timeout               |
  | `discover`      | Enables or disables `/v1/models` discovery   |
  | `enabled`       | Temporarily enables or disables the endpoint |
  | `defaultOutput` | Sets the default output limit                |
  | `dropParams`    | Removes unsupported request fields           |
  | `models`        | Declares models explicitly                   |
</Accordion>

### Keep secrets out of the file

Endpoint values support `${NAME}` and `${NAME:-fallback}` environment-variable references:

```bash theme={"system"}
export LOCAL_LLM_KEY="your-key"
embedder
```

Typing a key directly stores it as plain text. If a referenced variable is missing, Embedder skips that endpoint.

## Pin a local model

An endpoint named `ollama` with a model named `qwen3:32b` appears in `/model` as `ollama/qwen3:32b`. To select it automatically for a project:

```json .embedder/models.json theme={"system"}
{
  "main": "local/ollama/qwen3:32b"
}
```

Project settings override `~/.embedder/models.json`. A valid `main` pin is selected when the project opens.

To show only local and subscription models, [hide Embedder-hosted models](/integrations/model-providers#hide-embedder-hosted-models).

## Data and privacy

Prompts and responses travel directly between Embedder and your endpoint. The endpoint URL, API key, and custom headers remain in local configuration.

Some features still use Embedder cloud services:

* Account, team, and project information
* System prompts and capabilities such as web search
* Billing and usage information
* Telemetry unless disabled by policy or `EMBEDDER_NO_TELEMETRY=1`

## Troubleshooting

<AccordionGroup>
  <Accordion title="Local models is missing from /connect">
    Ask an Embedder administrator to enable Local models, then restart the session.
  </Accordion>

  <Accordion title="The connection test fails">
    Make sure the server is running, then check the base URL, HTTP or HTTPS setting, and API key.
  </Accordion>

  <Accordion title="A model does not appear">
    If the server does not support model discovery, enter the exact model ID in the **Model** field. Then open `/model` and select it.
  </Accordion>

  <Accordion title="The model forgets old turns">
    Reduce the declared context window to the value enforced by the server. For Ollama, also set `OLLAMA_CONTEXT_LENGTH` before starting the server.
  </Accordion>

  <Accordion title="The server rejects a request field">
    Add the field name to `dropParams` in `local-endpoints.json`.
  </Accordion>
</AccordionGroup>

<CardGroup cols={2}>
  <Card title="Connect subscriptions" icon="key" href="/integrations/model-providers">
    Connect GitHub Copilot or ChatGPT Codex for inference.
  </Card>
</CardGroup>