Local Models for Odysseus
Odysseus is the workspace. Connect a local engine — Ollama (easiest), Cookbook Serve, or LM Studio — then chat. No API key required for local setups.
Quick answer
Fastest path: install Ollama, run ollama pull qwen3.5:0.8b, then add http://localhost:11434/v1 in Odysseus Settings.
Three ways to connect a model
Pick one path. You can switch later — Odysseus is OpenAI-compatible for local servers.
| Path | Best for | How | Notes |
|---|---|---|---|
| Ollama | Most beginners — any OS | Install Ollama → pull model → add /v1 endpoint in Settings | Easiest path. No API key. Works with Docker via host.docker.internal. |
| Cookbook Serve | Built-in downloads + hardware picks | Cookbook → download → Serve → use in chat | Needs GPU passthrough in Docker (NVIDIA overlay). On Mac Metal, use native start-macos.sh. |
| LM Studio | You already use LM Studio | Enable local server → point Odysseus at :1234/v1 | Same OpenAI-compatible pattern as Ollama. Turn on LAN serve for Docker. |
Model recommender
Start smaller — a working tiny model beats a huge one that crashes.
System RAM
GPU VRAM
Recommended: qwen3.5:0.8b
Fastest on 8 GB RAM or CPU-only machines.
ollama pull qwen3.5:0.8bPath A — Ollama (recommended start)
Pick based on your RAM and GPU. When in doubt, start small.
| Model | RAM | VRAM | Notes |
|---|---|---|---|
| qwen3.5:0.8b | 8 GB | None (CPU) | Fastest start — good for testing Odysseus on low-end PCs |
| llama3.2:3b | 8–16 GB | Optional | Balanced small model for everyday chat |
| llama3.2 | 16 GB | 8 GB recommended | Better quality — needs more RAM or a GPU |
| mistral | 16 GB | 8 GB recommended | Strong general-purpose model |
| qwen2.5:14b | 32 GB | 12 GB+ | High quality — for powerful machines only |
# Install Ollama from https://ollama.com/download
# Then pull a model (pick one based on your hardware):
ollama pull qwen3.5:0.8b # 8 GB RAM, no GPU — fastest start
ollama pull llama3.2:3b # 8–16 GB RAM
ollama pull llama3.2 # 16 GB RAM, GPU helps
# Verify:
ollama listConnect in Settings — native
# In Odysseus → Settings → add model endpoint:
http://localhost:11434/v1Connect in Settings — Odysseus in Docker
# Odysseus in Docker + Ollama on host:
# Windows / Mac Docker:
http://host.docker.internal:11434/v1
# Linux Docker:
http://172.17.0.1:11434/v1
# If connection fails, let Ollama listen beyond loopback:
# OLLAMA_HOST=0.0.0.0:11434 ollama serve
# Keep Ollama running (system tray or: ollama serve)Need more hardware detail? GPU & RAM requirements
Path B — Cookbook Serve
Built into Odysseus: hardware-aware picks, download, then Serve. Different from Ollama — you do not need a separate Ollama install for Cookbook-served models.
# Cookbook = built-in model browser + Serve (llama.cpp path)
# 1. Open Odysseus → Cookbook
# 2. Pick a model that fits your VRAM / RAM (hardware-aware list)
# 3. Download, then click Serve — wait until it finishes
# 4. The model should appear in the chat dropdown
# Docker + NVIDIA: enable the official GPU overlay first
# (COMPOSE_FILE=docker-compose.yml:docker/gpu.nvidia.yml)
# See /install/docker and /gpu-requirements
# Apple Silicon: use ./start-macos.sh — Docker cannot use MetalPath C — LM Studio
Same idea as Ollama: expose an OpenAI-compatible /v1 endpoint and add it in Settings.
# LM Studio = OpenAI-compatible local server (common alternative)
# 1. Install LM Studio and load a model
# 2. Start the local server (enable "Serve on Local Network" if Odysseus is in Docker)
# 3. In Odysseus Settings, add an OpenAI-compatible endpoint, for example:
# Native Odysseus → LM Studio on same machine:
http://localhost:1234/v1
# Odysseus in Docker → LM Studio on host:
http://host.docker.internal:1234/v1
# Port may differ — check LM Studio's server panelFAQ