Record validated suspended Kubernetes rollout and CI setup

PLAN.md

@@ -1,8 +1,8 @@
 # btcdata-archive implementation handoff
 
-Status: **IMPLEMENTED LOCALLY — restricted RustFS IAM identity, versioned archive bucket, PostgreSQL ledger and restricted runtime role provisioned on explicit requests; source index preflight, deployment and deletion have not been performed.**
+Status: **IMPLEMENTED AND DEPLOYED SUSPENDED — restricted identities, versioned bucket, ledger and indexes provisioned on explicit requests. The operator completed the initial catch-up. CI passed 65 tests; a verify-only Kubernetes Job completed on Citadel. Recurring pruning remains disabled pending separate approval.**
-Implementation artifacts and validation instructions are in `README.md`. The disposable
+Implementation artifacts, deployment state and validation results are in `README.md`.
-PostgreSQL/RustFS suite is prepared but requires a working Docker environment before rollout.
+The original requirements below remain the retention/archive contract.
 Prepared: 2026-10-03.
 Target repository: `/Users/pmagnus/pi/btcdata-archive`.
 

README.md

@@ -3,9 +3,9 @@
 Standalone Python archiver for the three fixed `btcprod` tables: `public.binance_price`,
 `public.strike_price`, and `public.kraken_minute`. **A restricted RustFS IAM identity
 and versioned archive bucket, progress ledger and restricted PostgreSQL role have
-been provisioned; source index preflight and deployment are still pending.**
+been provisioned. The Kubernetes CronJob is deployed through GitOps, suspended.**
-An operator run verified the first weekly archive, then hit a ledger transition bug
+The initial ledger transition bug is fixed; the operator completed 28 backlog weeks.
-before deletion. That bug is now fixed; deployment is being prepared suspended.
+Rollout validation ran verify-only and did not delete any source rows.
 The repository is hosted at https://git.pmagnus.cloud/magnus/btcdata-archive;
 Gitea Actions and project-scoped Harbor CI/pull credentials are configured.
 
@@ -30,10 +30,10 @@ set `TEST_PGHOST`, `TEST_PGPORT`, `TEST_PGUSER`, `TEST_PGPASSWORD`,
 at production infrastructure. Full S3 integration tests skip without both required
 TEST endpoints. PostgreSQL-only regressions require only the disposable PostgreSQL
 TEST variables and can run with `python -m pytest -q tests/integration/test_postgres.py`.
-Local validation of those regressions plus unit tests passed 43 tests on disposable
+Gitea CI passed all 65 unit/integration tests against disposable PostgreSQL 18.0 and
-PostgreSQL 18.6; the complete RustFS integration suite remains a rollout gate.
+RustFS 1.0.0, then published an immutable image. Boundary/crash tests inject interruptions
-Boundary/crash tests inject interruptions at workflow boundaries; actual process
+at workflow boundaries; actual process-kill/network/disk fault testing remains additional
-kill/network/disk fault testing and production connectivity remain rollout gates.
+operator validation. Production pod connectivity was verified on Citadel.
 
 `requirements.lock` pins runtime packages and hashes; `requirements-test.lock`
 adds pytest. Regenerate with Python 3.12, pip 25.2 and pip-tools 7.5.1:
@@ -235,7 +235,11 @@ Source INSERT/UPDATE/TRUNCATE/REFERENCES/TRIGGER and ledger DELETE are denied; t
 role has no permanent object creation privilege, elevated attributes or memberships.
 A real login/read-only ledger/source-access check succeeded. `.env` now uses this
 restricted role instead of the application's administrative credentials. No source
-rows or indexes were changed. Index/capacity preflight and deployment remain pending.
+rows were changed during that setup. During explicitly authorized rollout, valid
+`(recorded_at,id)` indexes were built concurrently for the price tables. PostgreSQL
+HBA access was added only for `btcprod`/`btcdata_archiver` from Citadel's
+`10.42.1.0/24` pod subnet, backed up and reloaded without a restart. No source rows
+were changed by these preparation steps.
 
 `scripts/rustfs-create-identity.py` is an explicit operator-only provisioning utility,
 run on RustFS with access to its administrative credential file. It refuses to overwrite
@@ -244,7 +248,32 @@ It is not executed by the runtime job. RustFS 1.0.0 rejects bare `Resource: "*"`
 mixed action families in a single statement; the policy uses separate admin/STS deny
 statements and S3 wildcard ARNs compatible with the deployed build.
 
-## Explicit operator setup and rollout (remaining steps not executed)
+## Kubernetes deployment state
+
+GitOps `magnus/deploy` main commit `c635511` registers `apps/btcdata-archive` under
+namespace `sejs`. Flux successfully reconciled it. CronJob `btcdata-archive` remains
+**suspended**, with Monday 00:00 UTC schedule, Citadel affinity, and the specified
+resource/storage limits. Existing feature-branch work in the deploy repository was
+preserved; rollout was committed from a separate main-based worktree.
+
+Deployed image (tested CI commit `d93735b`):
+`harbor.pmagnus.cloud/btcdata-archive/btcdata-archive@sha256:9d297326b4dc0d997a344b7353ee48257632be8020094a3a29f02b268e9f8a81`.
+Secrets `btcdata-archive-credentials` and `btcdata-archive-pull` were created in `sejs`
+outside Git, using the restricted PostgreSQL/S3 identities and a project-scoped
+Harbor pull-only identity. CI uses a separate project-scoped push/pull robot.
+
+Validation Job `btcdata-archive-verify-20261003193601` completed successfully on
+Citadel using this image, `run --mode verify-only --max-weeks 1`, non-root execution,
+read-only root filesystem and the actual work volume. There were no eligible weeks
+because the initial catch-up had completed, so it validated schema, lock, database
+access, S3 versioning and ledger discovery without uploading or pruning new data.
+The full export/upload/exact-version-download/prune/recovery path passed in CI.
+
+Enabling `suspend: false` is a separate explicit approval/GitOps step. No recurring
+archival/pruning schedule was enabled during deployment. Update image digests
+explicitly; this app has no floating-tag image automation.
+
+## Operator setup and rollout reference
 
 1. Run disposable integration tests. Inspect live schemas, constraints/DELETE triggers,
    RLS, existing indexes, source volumes/per-week sizes, source role privileges,
@@ -264,11 +293,11 @@ statements and S3 wildcard ARNs compatible with the deployed build.
    RustFS IAM compatibility. Runtime can inspect versioning but cannot enable it,
    administer the bucket, or delete objects/versions. No expiry/lifecycle rules.
    Provisioning and CI image pushes require separate credentials, never runtime keys.
-5. GitOps artifacts are prepared in `../deploy/apps/btcdata-archive/`, **not registered
+5. GitOps artifacts are in `magnus/deploy` main at `apps/btcdata-archive/`, registered
-   in the parent apps kustomization**. They remain suspended and have a deliberately
+   in the parent apps kustomization and currently suspended with a tested immutable
-   unpullable placeholder digest. Choose a remote, configure CI/Harbor, publish the
+   digest. For upgrades, publish a tested CI image and update the digest explicitly.
-   tested image and replace that digest before registration. Provision Secrets in
+   Provision Secrets in `sejs` through out-of-Git tooling; namespace references cannot
-   `sejs` through out-of-Git tooling; namespace references cannot cross namespaces.
+   cross namespaces. Never commit runtime or registry credentials.
 6. Confirm requests/limits/work capacity against the largest old week. Initial work
    limit is 8 GiB; disk estimates are conservative, not a capacity guarantee. Limit
    whole-week files require an increase for large weeks; streaming is not parallelism.

scripts/test-integration.sh

@@ -10,7 +10,7 @@ trap '"${compose[@]}" down -v' EXIT
 runner="$("${compose[@]}" ps -aq tests)"
 # CI's workspace is inside its job container, not on the Docker host. Copy only
 # test inputs; never bind-mount the workspace or copy .env, .git, or production secrets.
-COPYFILE_DISABLE=1 tar -cf - requirements-test.lock src tests sql \
+COPYFILE_DISABLE=1 tar -cf - pyproject.toml requirements-test.lock src tests sql \
   scripts/archive-old-weeks.sh scripts/list-archives.sh scripts/wait-test-infrastructure.py \
   | docker cp - "$runner:/app"
 docker start --attach "$runner"