Getting started

Quickstart

Install, pull a model, talk to it. No terminal is required for any of this on macOS, where the app does the same four steps with a window; the commands are here because they are the shortest way to write it down.

The four steps

1 · Install

macOS and Linux:

curl -fsSL https://llamay.com/install.sh | sh

Windows, in PowerShell:

irm https://llamay.com/install.ps1 | iex

The script works out your platform, downloads the matching artifact, verifies it against the release's SHA256SUMS, installs the binary and sets up a service. It refuses rather than continuing if the machine has neither sha256sum nor shasum: an installer that skips the check is one that runs whatever it was handed.

LLAMAY_NO_SERVICE=1 before the script installs only the CLI. Packages, an app bundle and the portable archives are on the download page; from source it is go install github.com/AzmxAI/llamay/cmd/llamay@latest.

2 · Get a model

llamay has its own store and its own downloader; nothing else has to be installed. Five source forms are understood, and a path that exists always wins over a name — no store on the machine can change what a path means.

llamay pull qwen2.5:0.5b                        # an Ollama-style registry
llamay pull hf.co/ggml-org/tiny-llamas:q8_0     # Hugging Face, quant as the tag
llamay pull hf:Qwen/Qwen2.5-0.5B-Instruct-GGUF/qwen2.5-0.5b-instruct-q4_k_m.gguf
llamay pull https://example.com/model.gguf      # any URL serving a GGUF
llamay models                                   # what this machine has

name                                   size  architecture
all-minilm:latest                  43.8 MiB  bert (embedder)
bija-sovereign:latest             128.9 MiB  gpt2 (decoder)
gemma3:270m                       278.0 MiB  gemma3 (decoder)
qwen3:1.7b                          1.3 GiB  qwen3 (decoder)

Downloads resume — these files run to gigabytes and a transfer that restarts from zero after failing at 90% is one most people never finish. The store is content-addressed, so two tags of one model are one file on disk and a partial download can never be mistaken for a complete one. HF_TOKEN is sent to Hugging Face when it is set, which is what a gated repository needs. A model already pulled by Ollama is read too, as a courtesy rather than a dependency.

3 · Run it

llamay run -m qwen2.5:0.5b -p "The capital of France is"
llamay run -m azmx-code-q4.gguf -p "write a binary search in Go" -temp 0.2
llamay run -m azmx-one-q4.gguf  -p "extract the fields" -json

4 · Serve it

llamay serve

With no -m, serve takes the first decoder in your store, sorted by name so a restart serves the same one, and looks in Ollama's store as well as llamay's. It listens on 127.0.0.1:11435.

With no models at all it still starts, serving /healthz and POST /api/pull so you can pull a first one through it, and loads that model when it lands without a restart.

curl localhost:11435/v1/chat/completions -d '{
  "model": "qwen2.5:0.5b",
  "messages": [{"role":"user","content":"hello"}]
}'

The same server answers POST /v1/messages in Anthropic's shape and POST /api/chat in Ollama's, in Ollama's own conventions — newline-delimited JSON, streaming on by default, durations in nanoseconds — so an existing client can be pointed here by changing a URL. Every route is listed here.

Where to go next