SOAP connector (`kind: soap`)¶

Ingest from legacy SOAP services (Siebel, SAP PI, old middlewares) into the same hybrid JSONB + typed storage as every other source. SOAP is a request shape on the universal REST engine — the manifest stays connector_type: rest-generic; each endpoint opts in with kind: soap. Auth, rate-limiting, retry, and storage are all shared.

Full example — Siebel QueryPage¶

The real, validator-true manifest (the same file the tests exercise):

# Siebel CustomerQueryPage over SOAP, exposed as a canonical hub table.
# kind: soap is a request-shape on the HTTP engine (connector_type stays rest-generic).
apiVersion: connectors.lumnik.io/v1
kind: Connector
metadata:
  name: siebel-customers
  connector_type: rest-generic
spec:
  base_url: https://siebel.example.com
  auth:
    kind: basic
    username: siebel_svc          # literal username (the basic block reads `username`, not an env)
    password_env: SIEBEL_PWD      # env var name holding the password
  endpoints:
    - id: customers
      kind: soap
      path: /eai_enu/start.swe
      soap_action: "document/urn:crmod:...:CustomerQueryPage"
      body: |
        <CustomerQueryPage_Input>
          <PageSize>{{page_size}}</PageSize>
          <StartRowNum>{{cursor}}</StartRowNum>
          <ListOfCustomer><Customer><Id query="*"/><Name query="*"/></Customer></ListOfCustomer>
        </CustomerQueryPage_Input>
      pagination:
        kind: soap
        page_size: 100
        last_page_path: "$.Body.CustomerQueryPage_Output.LastPage"
        response_path: "$.Body.CustomerQueryPage_Output.ListOfCustomer.Customer"
      target:
        kind: hybrid
        table: ext.siebel_customers
        unique_key: "$.Id"
        tags: ["crm", "customers", "siebel"]
        promote:
          - { name: name, path: "$.Name", type: text }

Endpoint fields (`kind: soap`)¶

Field	Required	Meaning
`id`	yes	endpoint identifier (kebab-case)
`path`	yes	POST target, relative to `base_url`
`body`	yes	the inner `<soap:Body>` XML template (lumnik wraps the SOAP 1.1 envelope)
`soap_action`	no	sent as the `SOAPAction` header
`header`	no	inner `<soap:Header>` XML template
`pagination.kind: soap`	yes	see below — `kind: soap` endpoints require it, and vice-versa

Method is implicitly POST with Content-Type: text/xml.

Body templating¶

Placeholders resolved on every request:

{{page_size}} / {{cursor}} — pagination state (cursor = row offset, starts at the strategy's first row)
{{secret:NAME}} / {{env:NAME}} — both resolve from the hub's process environment (OS env var first, JVM system property as dev/test fallback). The DB-backed lm secret registry is not consulted here — export the variable where the hub runs.

An unresolved placeholder fails the run fast — a template is never sent with a literal {{...}} left inside.

Connector-level headers¶

spec.headers[] applies to SOAP endpoints too — the entries ride every page of the QueryPage loop. Applied before auth (auth-injected headers win on a name clash); Content-Type and SOAPAction are set by the engine, so don't redeclare them. Endpoint-level headers are refused at apply, exactly as for REST endpoints.

Pagination (`kind: soap`)¶

Offset arithmetic in the QueryPage style:

Field	Required	Meaning
`page_size`	yes	rows per page; injected as `{{page_size}}` — explicit, no default
`response_path`	yes	plain dotted path to the repeated element (e.g. `$.Body.Out.ListOf.Customer`)
`last_page_path`	no	path to a boolean "this is the last page"
`has_more_path`	no	path to a boolean "there are more pages"

One stop signal applies — chosen by configuration, not tried in sequence:

last_page_path set → it is authoritative: pagination continues only while the flag is explicitly false; a true, missing, or null flag stops (never loops forever).
else has_more_path set → continues only while explicitly true; missing/null stops.
else → stop on a page shorter than page_size.

(Flags arrive as XML strings — "true"/"false" are accepted as booleans.)

page_size is deliberately defaultless: a silent default that mismatched the service's real page size would skip or duplicate rows wherever the body template uses {{page_size}}. The validator requires the key; a non-positive or non-numeric value fails the run fast, before any request is sent.

response_path must be a plain dotted path

JsonPath operators ([*], ..) are rejected by the validator for SOAP pagination — with the XML-to-tree conversion they would silently match a single element and lose data.

Response handling¶

XXE-hardened: DTDs and external entities are disabled in the XML parser.
Namespace prefixes are stripped (ns1:Customer → Customer) before path matching.
The terminal segment of response_path is force-listed — one <Customer> or fifty, you always get a list. (v1 limit: intermediate repeated segments are not coerced.)
A SOAP Fault fails the run (no partial ingestion) — and is never retried: SOAP 1.1 faults arrive with HTTP 500, but the fault check runs before status-based retry, so a fault is treated as a business error, not a transient one.
Plain HTTP failures retry as transient on exactly 429, 500, 502, 503, 504, 408 (plus network errors) — the shared REST policy. Every other status (501, 505, any other 4xx, …) is permanent.

SOAP connector (kind: soap)¶