Skip to content

The WOW, step by step

Two heterogeneous sources — a SOAP Siebel and a CSV — fused into one canonical Customer, served over REST, and asked in natural language. Every file below is the real file the end-to-end test runs, so this walkthrough can never drift from what actually works.

1. A SOAP source (fake Siebel)

The connector manifest:

# Source 1 — the "legacy CRM": our fake Siebel over SOAP CustomerQueryPage.
# kind: soap is a request-shape on the HTTP engine (connector_type stays rest-generic).
# Lands canonical hub table connector.cust_siebel — the manifest fuses on its promoted columns.
apiVersion: connectors.lumnik.io/v1
kind: Connector
metadata:
  name: siebel-customers
  connector_type: rest-generic
spec:
  # fake-siebel.py runs on the HOST. From the hub container (self-host stack) reach it via
  # host.docker.internal; if you run the hub with `quarkus:dev` on the host, use localhost:8077.
  base_url: http://host.docker.internal:8077
  endpoints:
    - id: customers
      kind: soap
      path: /eai
      soap_action: CustomerQueryPage
      body: |
        <In><PageSize>{{page_size}}</PageSize><StartRowNum>{{cursor}}</StartRowNum></In>
      pagination:
        kind: soap
        page_size: 100
        last_page_path: "$.Body.Out.LastPage"
        response_path: "$.Body.Out.ListOfCustomer.Customer"
      incremental:
        mode: full
      target:
        kind: hybrid
        table: connector.cust_siebel
        unique_key: "$.CustCode"
        promote:
          - { name: customer_code, path: "$.CustCode", type: text }
          - { name: name,          path: "$.Name",     type: text }
          - { name: segment,       path: "$.Segment",  type: text }

2. A CSV source

# Source 2 — the "flat-file export": a CSV the second system spits out nightly.
#
# For csv-file connectors the hub table name is NOT configurable: it is
#   connector.t_<slug(metadata.name)>
# i.e. a hardcoded `t_` prefix + the slugified connector name. So:
#   metadata.name: cust-csv   ->   connector.t_cust_csv
# (that's the table the entity manifest fuses against). `target` only carries optional
# `indexes`/`tags` — there is no table_prefix knob for csv-file.
apiVersion: connectors.lumnik.io/v1
kind: Connector
metadata:
  name: cust-csv                     # -> connector.t_cust_csv
  connector_type: csv-file
  tags: [demo]
spec:
  target:                            # the target block itself is required (ConfigDef error if absent)
    tags: [demo]                     # tags is recommended: silences a warning + gives the RAG scope its filter
  transport:
    kind: local
    # Path AS SEEN BY THE HUB. On the self-host stack the demo folder is mounted at /demo
    # (docker-compose.wow.yml); with `quarkus:dev` on the host, use ./customer.csv.
    path: /demo/customer.csv
  parser:
    has_header: true                 # header row drives the column names (code, cname, email, ville)
  after_process: none                # leave the source file in place

Its data:

code,cname,email,ville
C-100,Jean Dupont,jd@x.fr,Paris
C-200,MARIE A.,m@x.fr,Lyon
C-300,Solo,solo@x.fr,Nice
C-500,Claire Martin,claire@x.fr,Lyon
C-600,Pierre Corneille,corneille@x.fr,Paris
C-700,Maurice Druon,druon@x.fr,Paris
C-800,Albert Camus,camus@x.fr,Lyon
C-900,Saint Exupery,exupery@x.fr,Lyon

3. Fuse into a canonical Customer

The entity manifest (match + merge) that unifies both sources:

# The hero entity: one Customer fused from the two legacy sources on customer_code.
# Siebel is listed first, so on a disagreement Siebel is the representative and the CSV
# value is preserved in the _conflicts ledger (nothing is silently dropped).
kind: Entity
name: Customer
description: "A customer, fused from the Siebel CRM and the nightly CSV export."
match: [customer_code]
scopes: [clients]               # the métier(s) whose chat can ask about this entity
columns:                        # grounding fed to the chat + human labels a kind:View renders
  customer_code:
    label: "Code client"        # shown by kind:View instead of the raw canonical name
  name:
    label: "Nom"
  segment:
    label: "Segment"
    description: "Commercial segment."
    values: [GOLD, SILVER, BRONZE]   # the domain — a filter on any other value is REFUSED, not run
  city:
    label: "Ville"
    description: "Billing city (free text, French city names)."
    example: "Lyon"
sources:
  - table: cust_siebel          # source 1 (SOAP/Siebel) — promoted columns
    map:
      customer_code: customer_code
      name: name
      segment: segment
  - table: t_cust_csv           # source 2 (CSV export) — header columns
    map:
      code: customer_code
      cname: name
      email: email
      ville: city

4. Ask it

lm ask --scope clients "combien de clients à Lyon ?"
# → an answer computed over a fused truth that lived in neither Siebel nor the CSV alone,
#   with the SQL that ran shown beneath it.

Act 2 — the data starts working for you.

5. See it (kind: View)

A declarative, read-only list over the fused Customer — reference:

# The first per-métier app surface: a list over the fused Customer entity (scope: clients).
# Read-only, card-backed — every column must exist in the clients SchemaCard or the render is refused.
apiVersion: apps.lumnik.io/v1
kind: View
metadata:
  name: clients-desk
spec:
  scope: clients
  table: connector.v_customer
  title: "Clients"
  columns: [customer_code, name, city, segment]

lm view -f docs/demo/wow-customer/clients-view.yaml --filter city=Lyon

Or skip writing the YAML — describe the view and let the AI write it, validated by the same gate (an unknown column can never slip through, and an unanswerable ask is refused with the reason):

lm view generate clients "les clients gold avec leur ville et segment" -o gold.yaml
cat gold.yaml            # the first line is the ready-to-run render command
lm view -f gold.yaml --filter 'segment=GOLD'

6. React to it (kind: Process)

When a new Lyon customer lands, emit a decision — delivery rides your existing webhook subscriptions (n8n here) — reference:

# The dragon's first rung, PROVEN LIVE: when a new customer from Lyon lands in the CSV,
# emit a decision event. The effect IS the event — delivery rides the existing webhook
# subscriptions (n8n/Make/Zapier), zero new code.
#
# Why `ville` and scope `demo` (not `segment`/`clients`): a process triggers on
# data.row.changed, born at the RAW write path — so on.table is the raw ingest table
# `connector.t_cust_csv`, and its columns are the CSV's own (code, cname, email, ville).
# `segment` is a Siebel field, absent from the CSV. The raw table lives in the `demo` scope
# card (the csv connector is tagged `demo`); the fused `clients` card names only v_customer.
apiVersion: apps.lumnik.io/v1
kind: Process
metadata:
  name: lyon-client-arrive
spec:
  scope: demo
  on:
    event: data.row.changed
    table: connector.t_cust_csv
  when:
    - ville=Lyon
  emit:
    type: decision.lyon-client-arrive
    message: "Nouveau client {cname} à {ville}"

lm process apply -f docs/demo/wow-customer/lyon-process.yaml
# or create it in the TUI: `:new` → Process — the same living example opens in $EDITOR

7. Decide (inbox → ack)

The human disposes — and the ack is itself an event, idempotent by construction:

lm inbox
# ID  QUAND  PROCESS             MESSAGE                       ENTITY_KEY
# 1   2m     lyon-client-arrive  Nouveau client Claire à Lyon  CUST-042
lm inbox ack 1
# acked.  (re-acking returns the same 200 — one outbox row, ever)

8. Declare the lifecycle (kind: Workflow)

The three rungs above act on rows. This one teaches the substrate what the rows mean: the states a document moves through, the aliases twenty years of Excel produced for them, and which are terminal — reference.

# Declare the document lifecycle of a column instead of teaching the chat its values by
# hand: states + labelled aliases (Excel reality is dirty: "en cours"/"encours" both mean
# the same thing) + a transition graph. provenance MUST be `declared` — v1 accepts no
# other value. `on.table`/`on.column` name the raw ingest column whose values this
# taxonomy explains; apply reports the data-truth confrontation: undeclared VALUES found
# in the data (reality, never refused — 'EN LITIGE' is one) and unobserved CODES declared
# but never seen.
apiVersion: apps.lumnik.io/v1
kind: Workflow
metadata:
  name: commande-lifecycle
spec:
  scope: demo
  provenance: declared
  # the sweep alerts a non-terminal commande observed unchanged past 30 days —
  # observation-relative, see docs/apps/workflows.md
  dormancy:
    after: 30d
  on:
    table: connector.t_commandes_csv
    column: statut
    key: num
  states:
    - code: BROUILLON
      initial: true
    - code: EN_COURS
      label: "En préparation"
      aliases: [en cours, encours]
    - code: TERMINE
      aliases: [fini]
      terminal: true
  transitions:
    - { from: [BROUILLON], to: EN_COURS }
    - { from: [EN_COURS], to: TERMINE }

lm workflow apply -f docs/demo/wow-customer/commande-workflow.yaml
# {"name":"commande-lifecycle","scope":"demo", … ,
#  "dataTruth":{"undeclared":[{"value":"EN LITIGE","rows":2}], … }}

Applying it doesn't hide the mess, it names it: EN LITIGE sits in the column and in no state, so the response reports it — and still returns 200, because reality doesn't ask the manifest's permission. The declared graph then lands in the scope's corpus, so the chat answers a lifecycle question from it instead of inventing one.

The manifest also declares spec.on.key (num) and spec.dormancy.after: 30d — a scheduled sweep watching every commande for the one thing the raw table itself cannot say: how long it has sat, unmoved, in a non-terminal state. Cold start is honest — nothing can be dormant before 30d elapses from lumnik's own first observation, never from the source's unknown history — and the alert never claims a source-absolute duration. Captured from a live replay with the threshold shortened to 1m so the sweep could fire without the real 30-day wait (the committed manifest still declares 30d):

lm inbox
# ID  QUAND  PROCESS                      MESSAGE                                                ENTITY_KEY
# 2   1m     workflow:commande-lifecycle  CMD-005 : observé immobile en « EN LITIGE » depuis …    CMD-005

CMD-002 and CMD-003 never alert — both resolve to the terminal TERMINE, which the sweep excludes. EN LITIGE — undeclared, and therefore never terminal — is watchable too: CMD-005 and CMD-007 sleep in it and alert exactly like any other stalled commande. See Dormancy for the full vocabulary.

Continue hands-on: the full by-hand replay — on the deployed self-host stack, real OIDC, lm login and Bearer-authenticated calls throughout (including the fake-siebel.py SOAP stub and the demo credentials).