Connect
Use a local AI model
Relay can use a model that runs on your own PC or server through Ollama. The text Relay sends for each task then stays on your network.
When to choose local
- You handle sensitive client data and want nothing sent to an AI company.
- You'd rather pay once for hardware than per email.
Local models are usually slower and less capable than the large cloud models, so try one on real emails before relying on it. A PC with a recent graphics card (8 GB of video memory or more) makes a big difference.
Set it up
- Install Ollama from ollama.com/download on the PC that will run the model.
- Download a model, for example from a Command Prompt:
ollama pull llama3.1. Browse the Ollama library for others. - In Relay, open Settings > AI and choose Ollama (models on this computer) as the provider. In the setup wizard, Use a local AI does the same.
- Leave API address empty if Ollama runs on the same PC. If it runs on another machine, enter its address, for example
http://192.168.1.20:11434. - Enter the model name exactly as
ollama listshows it, then press Test connection.
Use it for decisions too
Set Settings > AI > Decision engine to Use my main AI, or run Kev. Then sorting emails also stays on your PC. If the local AI is down, Relay never falls back to a cloud service: it passes the email to a person and says why.
Other local servers
LM Studio, vLLM and similar tools offer an OpenAI-compatible address (often http://localhost:1234/v1). Choose Other: OpenAI-compatible API and enter that address.
Good to know
Keep Hide private details from the AI on anyway. It costs little and protects you if you ever switch back to a cloud model.