Skip to content

Architecture — Interactive Explorer

Eight views of the same system, no contradictions allowed. Each tab embeds the deterministic render of a canonical Mermaid source; the source files are the single source of truth (this page never restates them inline), and every panel links to its source and to the docs that explain the subsystem.

The tabs are plain radio buttons: use Tab to reach the tab row and ←/→ to switch views — no JavaScript required. Every panel also carries a text summary, so the content reads correctly with images off, with JavaScript off, and in dark mode (diagrams render on a light card by design; the palette is documented in the design system).

System architecture

Six layers: users and agents → four interfaces → application-services core → the eight-stage durable pipeline engine → infrastructure → three deployment profiles. Every interface drives the same core.

System architecture: users and agents, interfaces (CLI, MCP, REST and web console, SDK), application core, pipeline stages, infrastructure, deployment profiles
Open the image in a new tab for full-size reading; the render is deterministic from the Mermaid source.
  • Interfaces: knovaryn CLI, MCP server, knovaryn server REST + web console, Python SDK.
  • Pipeline stages are checkpointed — validate is the fail-closed gate.
  • Infrastructure: SQLite/PostgreSQL metadata, content-addressed artifacts, model gateway.

Canonical Mermaid source · Architecture overview

Pipeline flow

Two rows: prepare (intake → parse → split & chunk → plan) and produce (generate → validate → review → version & export). Validation failures quarantine with reason codes and are never exported.

Pipeline flow: prepare row from intake to plan, produce row from generation through validation, review, versioning and export
The plan stage is a dry run — cost is priced before any model call.
  • Intake preflights hash, size, license, and privacy class on untrusted documents.
  • Chunks carry source_span_ids and recorded location precision.
  • Exporters re-resolve every citation; publish is dry-run by default.

Canonical Mermaid source · Stage-by-stage notes

Durable jobs

Three lanes: normal execution (atomic claim, checkpoint, heartbeat), crash and recovery (lease expiry → re-lease → resume from checkpoint), and cancellation or budget stop (polled between stages, never mid-write, with a cost audit).

Durable job lifecycle: normal lane with claim, checkpoint and heartbeat; crash lane with lease expiry and resume; cancellation lane with safe points
Provider-call and cost-event dedup mean a replayed stage never bills twice.
  • Atomic claim via SKIP LOCKED — one winner among N workers.
  • Resume continues from the last checkpoint artifact; paid work is not repeated.
  • Budget caps pause the job before the limit is crossed.

Canonical Mermaid source · Job engine reference

Provenance chain

Every exported example chains back to source bytes: candidate → chunk → source span (page, section path, offsets, bounding box, quoted text with its own SHA-256) → parsed document → source document (file hash, license, privacy class). A provenance minimum is enforced.

Provenance lineage: exported example, generation candidate, chunk, source span, parsed document, source document, with hashes and location precision
knovaryn_lineage walks this chain for any example.
  • Spans quote their text and hash it — grounding is checkable, not asserted.
  • Parser configuration is hashed, so re-parses are comparable.
  • License and privacy classification happen at intake, before parsing.

Canonical Mermaid source · Provenance concepts

MCP session

Four phases — setup, estimate & generate, inspect & review, version & release — using the real tool names. The estimate step loops against the budget before start_pipeline; knovaryn_doctor is usable at any time.

MCP agent session: setup, estimate and generate with a budget check, inspect and review, version and release
Tool catalogue with schemas lives on the MCP tools reference page.
  • Dry-run cost estimate precedes every real generation run.
  • Review uses immutable revisions with evidence.
  • Publish is dry-run first, then confirmed.

Canonical Mermaid source · MCP tools reference

Security boundaries

Regions colored by trust: untrusted input boundary, local trust default, opt-in remote access, secrets handling, artifact integrity, publication authorization, and the release attestation chain.

Security and privacy boundaries: untrusted input, local trust, remote access, secrets, artifact integrity, publication authorization, release attestation
Red is untrusted input; green is the offline local default.
  • URL ingest is off by default; archives are verified before extraction.
  • Scopes are least-privilege per principal; tenants see only their projects.
  • Secrets come from the environment and are redacted at the display boundary.

Canonical Mermaid source · Security architecture

Deployment topology

One running system — access layer, core, worker pool, state — plus the external model providers (worker outbound calls only) and the scale decision: local, compose, or Kubernetes. Configuration, not a fork.

Deployment topology: clients, access layer, core, workers, state stores, external providers, and the three scale profiles
The same core runs at all three scales; only configuration differs.
  • Compose ships api, workers, PostgreSQL, MinIO, and a proxy.
  • Kubernetes base includes a migration Job, NetworkPolicies, and PDBs.
  • Providers are reached only by worker outbound calls.

Canonical Mermaid source · Deployment profiles

Release supply chain

Tag push → four gates on the exact release SHA → build → artifacts (detached checksums, CycloneDX SBOM, notes, prerelease enforcement) → OIDC publish to PyPI → clean-environment verification → your own knovaryn verify-release.

Release supply chain: gates on the release SHA, build, release artifacts, PyPI publish via OIDC, post-publish verification, local bundle verification
Verification ends on your machine, not in CI.
  • Deployment E2E runs the published tag through the compose stack.
  • Publishing uses OIDC trusted publishing — no long-lived tokens.
  • 0.x tags are enforced as pre-releases.

Canonical Mermaid source · Governance and CI

Where these renders come from

python scripts/render_diagrams.py   # sources in docs/assets/diagrams/ → PNG

The gallery page lists all renders with commentary; the diagram sources are plain Mermaid text you can diff and reuse.