Ollama

ИИ и аудио

Run open language models locally with one command and a simple API.

Ollama downloads and runs open large language models on your own machine with a single command. It exposes a local HTTP API on port 11434 that tools like Open WebUI, editors and scripts can talk to, so your prompts and documents never leave your network. It uses an NVIDIA GPU when one is available and falls back to the CPU otherwise. Models such as gpt-oss, Llama, Gemma and Qwen are pulled from its library with ollama pull. It is the engine; for a chat interface in the browser, pair it with Open WebUI.

Лиценз

MIT

GitHub звезди

184 300

Официален сайт

https://ollama.com

Изходен код

https://github.com/ollama/ollama

Съвместимост

Архитектури

amd64, arm64

База данни

Stateless

Ресурси

4096 MB RAM · 4 CPU

Measured on 2026-10-22 on Ollama 0.34.2 in a Proxmox LXC (6 vCPU, 6 GB RAM) with an RTX 3090 passed through. The service idles at about 0.7 GB of RAM, because the model weights live in VRAM. gpt-oss:20b (a 13 GB download) loaded 100% onto the GPU and used about 13.7 GB of VRAM after a short prompt, up to roughly 16.5 GB in our longer guide tests. It generated about 118 tokens per second on a 146-token answer; the first request took about 48 seconds to load the model. Small models such as llama3.2:1b need only a few GB of VRAM, and Ollama can also run on CPU, much more slowly. Plan for 40 GB of disk or more once you pull a few models.

Галерия със снимки

Models
Running on GPU
API

Деплойване

# Official Docker method with an NVIDIA GPU. Needs the NVIDIA driver and
# nvidia-container-toolkit on the host. We run Ollama as a native install
# inside an LXC (see the guide), so we have not tested this exact snippet.
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ./ollama:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

Алтернатива на

ChatGPT API, Claude API, Google Gemini API