Deploy lumnik

Get the hub running. The self-host path is one command; k8s/Helm is the cloud path.

Self-host (one command)

git clone https://github.com/icreated/lumnik.git && cd lumnik
./deploy/selfhost/up.sh --build

Run it with --build — it works for everyone; the plain pull path needs access to the (pre-launch, private) published image. The launcher:

• generates a gitignored, owner-only (mode 600) .env with fresh random secrets — Postgres, Keycloak admin, both crypto keys, and HUBADMIN_PASSWORD — and never overwrites an existing one (back it up — losing LUMNIK_SECRET_MASTER_KEY makes stored secrets unrecoverable);
• builds the hub image from source (--build) or pulls the published one, then starts hub + Postgres + Keycloak;
• waits for the hub health check (Flyway migrations run at boot — the first boot can take a minute);
• sets the per-install hubadmin password through the Keycloak admin API and prints the hub and Keycloak URLs.

The launcher, line by line

#!/usr/bin/env bash
# Stand up the self-host lumnik stack (hub + postgres + keycloak) with one command.
# On first run, generates .env with strong random secrets; pulls the published image
# (or --build compiles from source); waits for the hub to be healthy; prints access URLs.
set -euo pipefail

# repo root = two levels up from deploy/selfhost/up.sh
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
cd "$ROOT"

COMPOSE=(docker compose -f docker-compose.selfhost.yml)
BUILD=0
for arg in "$@"; do
  case "$arg" in
    --build) BUILD=1 ;;
    -h|--help) echo "Usage: $(basename "$0") [--build]   (--build compiles the hub from source instead of pulling)"; exit 0 ;;
    *) echo "unknown argument: $arg" >&2; exit 2 ;;
  esac
done

# 1) ensure .env with secrets (never overwrite an existing one)
if [[ ! -f .env ]]; then
  echo "==> no .env found — generating one with fresh random secrets"
  [[ -f .env.example ]] || { echo "!! .env.example not found — run this from the lumnik repo (the template is at the root)" >&2; exit 1; }
  umask 077  # any file created below (.env.tmp holding the real secrets) is owner-only from the start
  cp .env.example .env
  for key in POSTGRES_PASSWORD KEYCLOAK_ADMIN_PASSWORD LUMNIK_SECRET_MASTER_KEY LUMNIK_CRYPTO_SECRET_KEY HUBADMIN_PASSWORD; do
    secret="$(openssl rand -base64 32)"
    # fill the empty `KEY=` line; awk prints the secret literally (no shell/sed metachar issues)
    awk -v k="$key" -v v="$secret" 'BEGIN{FS=OFS="="} $1==k && $2=="" {print k"="v; next} {print}' .env > .env.tmp
    mv .env.tmp .env
  done
  chmod 600 .env  # owner-only — these are secrets (cp + redirect would otherwise leave it world-readable)
  echo "    wrote .env (gitignored, mode 600). BACK IT UP — losing LUMNIK_SECRET_MASTER_KEY makes stored secrets unrecoverable."
else
  echo "==> using existing .env"
  # Older .env files predate the per-install hubadmin password (C1) — grow one in place.
  if ! grep -q '^HUBADMIN_PASSWORD=..*' .env; then
    umask 077
    grep -q '^HUBADMIN_PASSWORD=' .env || printf 'HUBADMIN_PASSWORD=\n' >> .env
    secret="$(openssl rand -base64 32)"
    awk -v k="HUBADMIN_PASSWORD" -v v="$secret" 'BEGIN{FS=OFS="="} $1==k && $2=="" {print k"="v; next} {print}' .env > .env.tmp
    mv .env.tmp .env && chmod 600 .env
    echo "    added HUBADMIN_PASSWORD to .env"
  fi
  # Older installs bootstrapped Postgres as 'adempiere' (the pre-rename default); the
  # superuser name is frozen into PGDATA at first init, so an existing .env without an
  # explicit POSTGRES_USER must keep the legacy name — pin it, or pg_dump/pg_restore and
  # the healthcheck would target a role that doesn't exist in the old volume.
  if ! grep -q '^POSTGRES_USER=..*' .env; then
    if grep -q '^POSTGRES_USER=' .env; then
      awk 'BEGIN{FS=OFS="="} $1=="POSTGRES_USER" && $2=="" {print "POSTGRES_USER=adempiere"; next} {print}' .env > .env.tmp
      mv .env.tmp .env
    else
      printf 'POSTGRES_USER=adempiere\n' >> .env
    fi
    chmod 600 .env
    echo "    pinned POSTGRES_USER=adempiere (legacy volume — fresh installs use 'lumnik')"
  fi
fi

# 2) start the stack
if [[ "$BUILD" -eq 1 ]]; then
  echo "==> building the hub image from source and starting the stack"
  "${COMPOSE[@]}" up -d --build
else
  echo "==> pulling images and starting the stack"
  "${COMPOSE[@]}" pull || { echo "!! image pull failed — if the ghcr image is private run 'docker login ghcr.io', or use --build to compile from source" >&2; exit 1; }
  "${COMPOSE[@]}" up -d
fi

# hubadmin ships in the realm WITHOUT a credential (C1 posture, 2026-07-25): a known
# admin password must never survive an install. Set the per-install one from .env via the
# Keycloak admin API. Failure is fail-closed (hubadmin simply cannot log in) — warn,
# don't abort a healthy stack.
set_hubadmin_password() {
  local admin_user admin_pw hubadmin_pw tok uid name
  admin_user="$(grep '^KEYCLOAK_ADMIN=' .env | cut -d= -f2-)"; admin_user="${admin_user:-admin}"
  admin_pw="$(grep '^KEYCLOAK_ADMIN_PASSWORD=' .env | cut -d= -f2-)"
  hubadmin_pw="$(grep '^HUBADMIN_PASSWORD=' .env | cut -d= -f2-)"
  if [[ -z "$hubadmin_pw" ]]; then
    echo "  !! HUBADMIN_PASSWORD missing in .env — hubadmin stays locked (no password)"; return 0
  fi
  # Password rides stdin (--data-urlencode password@-), never argv — no `ps` leak.
  tok="$(printf '%s' "$admin_pw" | curl -fsS -X POST "http://localhost:8180/realms/master/protocol/openid-connect/token" \
      -d grant_type=password -d client_id=admin-cli \
      --data-urlencode "username=$admin_user" --data-urlencode password@- 2>/dev/null \
    | python3 -c 'import sys,json;print(json.load(sys.stdin).get("access_token",""))' 2>/dev/null)" || true
  if [[ -z "${tok:-}" ]]; then
    echo "  !! could not obtain a Keycloak admin token (check KEYCLOAK_ADMIN / KEYCLOAK_ADMIN_PASSWORD in .env) — hubadmin stays locked (no password)"; return 0
  fi
  # Realms imported before the rename still carry the legacy 'gardenadmin' user (a realm
  # import is one-shot) — heal it under whichever name this install has.
  for name in hubadmin gardenadmin; do
    # Non-fatal like the token call above — a lookup failure must keep the
    # warn-don't-abort contract (set -e would otherwise kill a healthy stack).
    uid="$(curl -fsS -H "Authorization: Bearer $tok" \
        "http://localhost:8180/admin/realms/lumnik/users?username=$name&exact=true" 2>/dev/null \
      | python3 -c 'import sys,json;u=json.load(sys.stdin);print(u[0]["id"] if u else "")' 2>/dev/null)" || true
    [[ -n "$uid" ]] && break
  done
  if [[ -z "$uid" ]]; then
    echo "  !! hubadmin not found in the lumnik realm — skipped"; return 0
  fi
  HUBADMIN_PW="$hubadmin_pw" python3 -c 'import json,os;print(json.dumps({"type":"password","temporary":False,"value":os.environ["HUBADMIN_PW"]}))' \
    | curl -fsS -X PUT "http://localhost:8180/admin/realms/lumnik/users/$uid/reset-password" \
        -H "Authorization: Bearer $tok" -H "Content-Type: application/json" -d @- \
    && echo "  $name: per-install password set (HUBADMIN_PASSWORD in .env)" \
    || echo "  !! $name password set failed — stays locked (no password)"
}

# 3) wait for the hub to be healthy (Flyway runs at boot)
echo "==> waiting for the hub to become healthy (this can take a minute on first boot)"
hub_cid="$("${COMPOSE[@]}" ps -q hub)"
for _ in $(seq 1 60); do
  status="$(docker inspect --format '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$hub_cid" 2>/dev/null || echo none)"
  if [[ "$status" == "healthy" ]]; then
    echo "==> hub is healthy"
    set_hubadmin_password
    echo ""
    echo "  Hub:       http://localhost:8080   (health: /q/health)"
    echo "  Keycloak:  http://localhost:8180/realms/lumnik"
    echo "  Next:      point lm at the hub — server http://localhost:8080, OIDC http://localhost:8180/realms/lumnik"
    exit 0
  fi
  sleep 3
done
echo "!! hub did not become healthy in time — inspect with: ${COMPOSE[*]} logs hub" >&2
exit 1

The full self-host README

# Self-host lumnik

Stand up the whole lumnik stack — hub + PostgreSQL (pgvector) + Keycloak (real OIDC) — on a single
host with Docker. It runs anywhere Docker runs: a laptop, or a VM on any cloud (GCP, AWS, Azure, Scaleway).

> "Deploy anywhere" needs a portable image on a VM, not per-cloud infrastructure-as-code. That's this.
> Managed-service / Terraform paths come later, driven by a real deployment's needs.

## Prerequisites

- Docker + Docker Compose v2 (`docker compose version`).
- The published hub image `ghcr.io/icreated/lumnik-hub` is **private pre-launch** — `docker login ghcr.io`
  authenticates you but grants **no access** unless your GitHub account was added to the package.
  The path that works for everyone: `--build` (compiles from source inside Docker — no image access
  and no local Maven/JDK needed, only Docker; the first build takes a few minutes).
- **For the natural-language layer (RAG chat, semantic search, analytic "ask"):** an
  [Ollama](https://ollama.com) running on the host, with the three models pulled. Without it,
  *structured* ingestion, canonical fusion, and the REST API still work — but the embedding model
  is unreachable, so RAG indexing **skips its chunks** (semantic search/chat return nothing) and
  "ask" has no model to call. The hub reaches the host Ollama via `host.docker.internal`
  (wired in the compose file); override with `OLLAMA_BASE_URL` in `.env` for a remote Ollama.

  ```bash
  ollama serve                      # if not already running
  ollama pull nomic-embed-text      # embeddings (RAG indexing)
  ollama pull llama3.2              # conversational RAG chat
  ollama pull qwen2.5-coder:7b      # text-to-SQL "ask" (4.7GB — the analytic model)
  # optional: ollama pull gemma4    # the offline faithfulness @Judge eval
  ```

## Quick start

```bash
./deploy/selfhost/up.sh --build   # build from source — works for everyone (the image is private pre-launch)
# ...or, if your GitHub account has access to the published image:
./deploy/selfhost/up.sh           # pull the image, generate secrets, start, wait for healthy
```

On the first run, `up.sh` creates `.env` with strong random secrets (`openssl rand`). When the hub is
healthy it prints the URLs:

- Hub — `http://localhost:8080` (health: `/q/health` — unauthenticated; the `/api/platform/*` API needs a token)
- Keycloak — `http://localhost:8180/realms/lumnik`

Then point `lm` at the hub (server `http://localhost:8080`, OIDC `http://localhost:8180/realms/lumnik`).

```bash
./deploy/selfhost/logs.sh hub     # tail logs (omit the service name for all)
./deploy/selfhost/down.sh         # stop the stack
./deploy/selfhost/down.sh --volumes   # stop AND delete all data (full reset)
```

## Secrets

`up.sh` writes `.env` (gitignored, mode `600` — owner-only) with four secrets: the Postgres and Keycloak
admin passwords and the two hub encryption keys (`LUMNIK_SECRET_MASTER_KEY`, `LUMNIK_CRYPTO_SECRET_KEY`).
**Back `.env` up** —
losing `LUMNIK_SECRET_MASTER_KEY` makes secrets stored by the hub unrecoverable. To pin a specific image
version instead of the latest `main` build, set `LUMNIK_VERSION` in `.env` (e.g. `LUMNIK_VERSION=1.0.0`
once a `hub-v*` release is tagged).

## Ports

By default the hub (`8080`) and Keycloak (`8180`) bind to `127.0.0.1` — reachable from the host only.
For a remote host, access them over an SSH tunnel, or see "Going public" below.

## Going public — the TLS façade (shipped)

To expose lumnik beyond localhost with one HTTPS origin, use the **TLS façade overlay**
(`deploy/tls/`): a single Caddy origin fronting the hub and path-proxying Keycloak under
`/realms` — one issuer for every device. A LAN install gets an internal CA certificate;
setting a public `LUMNIK_TLS_HOST` gets Let's Encrypt automatically.

Runbook: [deploy/tls/README.md](../tls/README.md) — also summarized on the
[deploy page](../../docs/deploy.md).

## Postgres bootstrap user

Fresh installs bootstrap Postgres as superuser **`lumnik`**. Installs created before the
rename run as the legacy `adempiere` — the name is frozen into the data volume at first
init, so `up.sh` pins `POSTGRES_USER=adempiere` into an older `.env` automatically and
everything (healthcheck, backup, restore) keeps working. No action needed either way.

## Backup & restore

Nightly-able backup (hot pg_dump + Keycloak store) and restore, including onto a
fresh machine: `backup.sh` / `restore.sh` here — full runbook at
[docs/operations/backup-restore.md](../../docs/operations/backup-restore.md).
Back up `.env` separately: it holds the master key.

## Residuals

The hub's internal DB-role passwords (`lumnik_app` / `admin` / `readonly`) use built-in defaults; Postgres
is not published outside the compose network, so it is internal-only. Making those configurable is a
tracked follow-up.

Stop / tail logs: deploy/selfhost/down.sh, deploy/selfhost/logs.sh.

One https origin (the TLS façade)

The base stack is loopback-only http. An optional overlay puts one Caddy origin in front of everything — the hub (API + the /app/ PWA) at /, Keycloak path-proxied under /realms/* — so every device sees one origin and one token issuer, which is what installing the PWA on a phone requires. One command layers it on:

LUMNIK_TLS_HOST=erp.example.com \
  docker compose -f docker-compose.selfhost.yml -f docker-compose.tls.yml up -d

A public LUMNIK_TLS_HOST gets a Let's Encrypt certificate automatically; a LAN IP or localhost gets Caddy's internal CA for local testing. Note the deliberate posture change: unlike the base compose, the overlay binds 443/80 on all interfaces by default — serving phones on the LAN is its reason to exist — and LUMNIK_TLS_BIND restricts it. Certificates, trusting the internal CA on a phone, and the change-of-issuer caveat are covered in the TLS façade README.

Point lm at the hub

Once the stack is healthy, wire the CLI to it. A context needs only the server and the OIDC issuer — lm login discovers the device/token endpoints from the issuer's .well-known document, so nothing else is hand-copied:

cd lm && go build -o lm ./cmd/lm          # build the CLI (Go 1.24+)

./lm config set-context local \
     --server http://localhost:8080 \
     --oidc-issuer http://localhost:8180/realms/lumnik
./lm config use-context local
./lm login                                 # prints a URL + device code — open it, sign in as integrator / integrator

The bundled realm ships two demo users. integrator / integrator (tenant 1) — change it before anything real. hubadmin (admin, tenant 11) ships with no password: up.sh generates one per install (HUBADMIN_PASSWORD in .env) and sets it through the Keycloak admin API — a known admin password never survives an install, and re-running up.sh heals older installs that still carry the historical default. (--oidc-client-id defaults to lm-cli, the bundled realm's client — set it only against your own IdP.) After lm login, every lm call is Bearer-authenticated; verify with:

./lm connector list

Kubernetes / Helm

The cloud edition (commercial) ships a Kubernetes/Helm path — hub + Postgres + Keycloak with real OIDC, sealed secrets, and a managed-PostgreSQL mode. It is not part of this repository.