Skip to content

How lumnik works

Everything you configure is a YAML manifest. Apply it, then see it in lm or the TUI. Every apply is checked first: refused with the exact field named, or saved.

This page covers what happens when you apply. What happens when a row changes is How lumnik breathes.

The kubectl parallel

If you know kubectl, you know most of lm:

For the other kinds, the fields are on each kind's page, and the TUI's :new opens a valid example of the kind you pick in $EDITOR.

lm apply never reconciles. An apply stays as it is until you apply again. Nothing pulls the hub or your sources back toward the manifest. The only watchers are a Workflow's dormancy and due-date sweeps. They raise alerts, and never change the manifest or the data.

Eight kinds, three families

kind: apiVersion: to write What it declares Apply Owning page
Connector connectors.lumnik.io/v1 ingestion from one source lm apply -f Connectors
Entity — (none) one thing fused from N sources lm entity apply -f Entities
View apps.lumnik.io/v1 a read-only list for one scope lm view apply -f Views
App apps.lumnik.io/v1 a business app's title, scope, icon and theme lm app apply -f The PWA
Process apps.lumnik.io/v1 a reaction to a row landing lm process apply -f Processes
Workflow apps.lumnik.io/v1 a document lifecycle over a status column lm workflow apply -f Workflows
Export apps.lumnik.io/v1 a managed delivery of an entity's rows lm export apply -f Exports
SearchIndex search.lumnik.io/v1 which entity columns an optional Search provider indexes lm search apply -f Search
  • Ingest and fuse: Connector, Entity.
  • Apps: View, App, Process, Workflow, Export.
  • Search: SearchIndex.

Connector is the only kind applied with the bare lm apply -f; every other kind has its own prefixed apply.

Living with your objects

Kind List Read one Remove Also
Connector lm connector list lm connector get NAME (stored fields) lm connector delete NAME run, pause, resume, scope
Entity lm entity list lm entity get NAME (rows) lm entity delete NAME export, resolve, ambiguous
View lm view list lm view get NAME (manifest) lm view rm NAME lm view NAME renders it
App lm app list lm app get NAME (manifest) lm app rm NAME
Process lm process list — lm process rm NAME enable, disable
Workflow lm workflow list lm workflow get NAME (manifest) lm workflow rm NAME enable, disable
Export lm export list lm export get NAME (manifest) — run, runs, status, retry
SearchIndex lm search status lm search status NAME lm search rm NAME query, rebuild, cancel
  • An Export cannot be removed. Re-applying it without spec.schedule lifts its schedule.
  • No single command lists every kind; each kind keeps its own list.

Full command tables: CLI & TUI reference.

One lifecycle

flowchart LR
  write["write the YAML"] --> validate["lm validate<br/>(dry run)"]
  validate --> apply["lm apply<br/>(idempotent)"]
  apply --> observe["observe in the TUI"]
  validate -. refused .-> write
  apply -. refused .-> write

Write, apply, observe — every kind. A refusal at either check sends you back to the file.

Keep manifests in your repository and version them there. If a local file is lost, lm view get, lm app get, lm workflow get and lm export get print the stored manifest back.

The little languages inside the YAML

A manifest's fields are plain YAML, but several of them hold a small syntax of their own. Each one is documented in full on the page that owns it — this table is the map, not the manual.

Language One-line syntax note Where it shows up
JSONPath $ is the response root — $.data[-1:].id reads "the id of the last element of data" REST/GraphQL/SOAP pagination + promote fields — REST
cron 5 fields (UNIX crontab dialect), evaluated in UTC any connector's spec.schedule — Connectors Overview
glob * and ? only, in the final filename segment — no ** recursion CSV local and sftp transport.path (S3 selects by prefix + suffix) — CSV & files
date patterns dd/MM/yyyy — capital MM is month, lowercase mm is minutes; add H/h/m/s only when the value carries a time parse-date — Transformers
regex Java syntax; each hook says whether the pattern must match the whole value regex-extract / regex-replace / filter-where — Transformers
{{secret:NAME}} / {{env:NAME}} secret: reads the lm secret registry, then the hub's environment. env: reads the environment only. An unresolved name fails the run before anything is sent SOAP body/header templates — SOAP
${now} the moment of ingestion enrich-context — Transformers
Kotlin .kts the escape hatch: runs with all of the hub's rights. The operator puts scripts in the hub's hooks directory; a manifest only names them a csv-file connector's spec.mapping, a REST/GraphQL/SOAP endpoint's own mapping — Kotlin hooks

Refusability as a feature

Every apply runs the same checks as validate. A refusal names the field and the fix:

$ lm validate -f acme-nightly.yaml
✗ acme-nightly.yaml — 1 error(s), 0 warning(s)

  ERROR    spec.archive_path
          required field missing (required when after_process = move)
          → Set archive_path: /path/to/archive/

Fix errors above before running 'lm apply -f acme-nightly.yaml'.
Error: validation failed: 1 error(s)

More refusals and warnings: Connectors — Validate before apply.

See also