Skip to content

Troubleshooting, by symptom

You have a symptom; this page names its family and hands you to the page that owns the cure. Detailed cause→fix tables live with their subjects — this is the front door, so you never have to know which page to search.

The stack won't cooperate

Deciding whether a restart is safe — and what each one costs, with the exact commands — is the on-call runbook.

Symptom First move
The hub won't start, or 500s everywhere curl -s http://localhost:8080/q/health, then the hub logs
lumnik.secret.master-key is too short at boot the env var is missing or < 16 chars — quickstart table
First boot seems stuck migrations run at first boot — give it a minute, watch ./deploy/selfhost/logs.sh hub
up.sh refuses to start, naming missing tools its preflight — the message shows the exact install line; run it, re-run up.sh (Windows walkthrough, step 3)
!! could not obtain a Keycloak admin token at the end of up.sh — then logins fail Keycloak was still finishing its first start (up.sh retries ~25 s, then warns) — re-run up.sh; if it persists, ./deploy/selfhost/logs.sh keycloak
Anything on a Windows PC the dedicated page: Deploy on Windows → Troubleshooting
https / certificate trouble the TLS façade — caddy logs, Deploy

Login and identity

Symptom First move
401 with tenant_id, subject/issuer or claim errors the claim wiring — BYO IdP troubleshooting has the full table
403 on every endpoint despite a valid login the groups claim doesn't carry lm_* role names — BYO IdP
403 {"error":"user is deactivated"} the identity was archived (SCIM departure) — SCIM
Login broke after moving to TLS or a new hostname the issuer changed and orphaned the JIT identity — BYO IdP
SCIM calls misbehave SCIM troubleshooting — the issuer-mismatch 404 is the classic
403 with an empty body a misconfiguration, not a feature — a required role is missing, not a scope. Roles & permissions tells all three 403 shapes apart
403 plain text missing role lm_integrator the third shape — the /api/platform/* gate, ahead of any endpoint-level check. Same page: Roles & permissions

Refusals that are features

If several read surfaces go quiet at once — the PWA list and chat, say, for the same user — suspect the scope boundary before anything else: it covers the whole read surface, not just the chat, so one missing scope:<tag> grant explains both. Scopes

Symptom What it is
403 {"error":"scope not granted"} the métier boundary, working — your user holds scope roles and this resource is outside them. Scopes
The chat says "I can't answer that…" or "I won't run this…" a guard preferred refusal over a guess; the message itself says what to do next
"The query returned no rows. That can mean…" zero-row honesty — verify the shown SQL before concluding there are none
409 {"error":"no schema discovered for scope '…'"} on an ask expected on a fresh scope: nothing was applied to it yet — apply an entity (the message says so), then ask again
The ask answers "I have no data declared for this scope yet…" although you have ingested into that scope the sibling state of the 409 above: a schema card exists for the scope but declares no tables, so guard 1 abstains. Look at the card itself — lm source schema --scope <scope> (needs lm_admin on top of lm_integrator). (no tables — run 'lm source rediscover --scope …') is this state; API error 404: No schema card found for scope: … is the 409 one. (Both the ask and this command live in the chat extensions — an open-edition build has neither)
429 on chat, search, or view-generate — not a connector run the hub's own inbound limit; the other 429 (a vendor throttling a connector, see Ingestion below) is a different gate — Rate limits

The ask and the model

Symptom First move
The ask/chat errors after ~20 s, with an opaque error id the model failed to load on the machine running Ollama — on Windows, update the NVIDIA drivers first (the classic), then the RAM note: Deploy on Windows, step 5
The very first ask after a restart is slow or times out the model loads into memory on first use (tens of seconds on CPU) — ask again; on a server, keep it loaded by setting OLLAMA_KEEP_ALIVE=-1 in Ollama's own environment (deploy/selfhost/README.md has the full prerequisite section)
Chat/search return nothing, ask has no model Ollama isn't installed or reachable — install it (Windows: step 5) and check OLLAMA_BASE_URL (Models). On a Linux host this is usually the bridge trap: the hub reaches Ollama at host.docker.internal, which on Linux resolves to the docker bridge IP (172.17.0.1 by default) and not to your loopback — so Ollama must be bound on the bridge, not on 127.0.0.1. Docker Desktop masks this, which is why it works on your laptop and not on the server. Full prerequisite list: deploy/selfhost/README.md in the repo

Ingestion

Symptom First move
A run failed, or rows are missing the dedicated recipe: Diagnosing a failed run
run scheduled looked fine, but lm run list says Failed "scheduled" only means queued — the ledger holds the verdict. Failed at 0/0 usually means the source didn't answer (is the source system up and reachable?) — Diagnosing a failed run
A run says Completed but IN=0 OUT=0 the connector found nothing to read — for file connectors, the path/mount doesn't exist where the hub looks (a missing Docker mount is the classic; the WOW walkthrough's ✔ checks catch it)
lm entity apply errors naming a relation that does not exist the entity fuses tables that its source connectors create — a source run above it never landed rows; check lm run list for the two symptoms just before this one
An applied entity never shows up in lm ask/chat no scopes: were declared on the manifest — undiscoverable by design, not a bug — Entities troubleshooting
429 Too Many Requests from a REST vendor lower default_rate_limit — REST troubleshooting
SFTP auth, host-key, or S3 403 errors CSV & files troubleshooting
Bearer token unavailable (env=…) — REST connectors only the secret was never set — lm secret set, quickstart table
jdbc run fails: password_env … unset jdbc does not consult lm secret — the variable must exist in the hub server's environment, not the laptop's — JDBC troubleshooting
A cursor seems stuck in the past lm endpoint reset-cursor <connector>/<endpoint> — next run is a full refresh

Outputs and apps

Symptom First move
A webhook never arrived lm outputs list — deliveries, failures first; ^R in :outputs replays one
The Fiches list on a phone looks empty it is never a blank screen — look at how it's empty: an amber box with a message is a refusal with its reason (no view applied for the scope, or 403 scope not granted — same family as above); plain grey "Aucune ligne" text means the view rendered and genuinely found zero rows — check the source data, not the scope. Views troubleshooting
The PWA shows an old version the service worker re-installs only when sw.js changes — the shell-cache note
lm view -f v.yaml (the list), lm view KEY -f v.yaml (one record) or lm view apply refused a manifest field/column mismatch, or a missing scope role — Views troubleshooting
lm process apply refused, or a declared process never fires a manifest/schema mismatch, or its trigger table broke at runtime — Processes troubleshooting
lm workflow apply refused an unknown field, a lifecycle collision, or dormancy declared without on.key — Workflows troubleshooting
A workflow alert didn't fire dormancy is observation-relative and needs on.key + dormancy.after — Workflows

Still stuck? Reading the logs tells you where every component writes, and how to raise the volume for the minutes you need it.