Author SHA1 Message Date
magnus 0a27d01f12 Show 30-minute candles on the last-24-hours chart
pull-request / validate (pull_request) Successful in 2m34s
2026-11-03 10:32:23 +01:00
magnus 803df14813 Merge pull request 'Retry the Harbor manifest copy instead of a probe for it' (#3) from fix-harbor-tag-retry into main
build / build (push) Successful in 2m56s
Reviewed-on: #3
2026-10-11 07:58:50 +00:00
magnusandClaude Opus 5 7ea25ee4fd Retry the Harbor manifest copy instead of a probe for it
pull-request / validate (pull_request) Successful in 32s
The companion SHA tag step failed on the last build even though the image had
already been pushed and deployed. Harbor's vulnerability policy blocks manifest
reads while Trivy's scan is Running, and `imagetools create` copies the manifest,
so Harbor answered 412 Precondition Failed.

Two faults compounded. The loop gated on `imagetools inspect`, but a probe passing
is no guarantee the copy will be allowed. And `create` sat outside the condition,
so under the default `bash -e` its first failure aborted the step before `exit 0`
or any further attempt: the log shows one waiting line, then the error, with 58 of
60 retries unused.

Retry the real operation instead, inside the condition so a blocked copy is a
reason to wait rather than fatal. The trade-off is that a permanently failing copy
now takes the full ten minutes to report instead of failing fast; the previous
behaviour failed fast but wrongly, on a transient block.

Also records that only the primary tag matters for rollout, since the Flux image
policy matches the timestamp tag, so a failure in this step costs the audit tag
rather than the deployment.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-11 09:56:26 +02:00
magnus ac01388117 Merge pull request 'Recover from a btcdata restart without reloading the page' (#2) from ws-reconnect into main
build / build (push) Failing after 3m11s
Reviewed-on: #2
2026-10-11 07:26:42 +00:00
magnusandClaude Opus 5 4a2d8fabad Recover from a btcdata restart without reloading the page
pull-request / validate (pull_request) Successful in 24s
htmx's WebSocket extension reconnects only after close codes 1006/1011/1012/1013.
_ws_proxy closed the browser socket with a normal 1000 on every upstream exit, so
htmx read a btcdata restart as a deliberate shutdown and left both panels frozen
until the visitor reloaded. New browser connections always worked, which hid the
defect from non-browser checks.

The proxy now reconnects upstream behind the still-open browser socket on a bounded
schedule, resets that schedule after a connection that lasted long enough to count
as healthy, and closes with 1012 only once the schedule runs out, so recovery falls
back to htmx's own backoff. A concurrent reader detects a departed browser, which a
dead upstream would otherwise be indistinguishable from, and an unrenderable payload
is logged and skipped rather than ending the stream.

tests/test_ws_proxy.py covers this against a real loopback WebSocket server: same
socket recovery across an upstream restart for both streams, the retryable close
code when the upstream stays down, and skip-on-unrenderable.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-10-11 09:21:19 +02:00
magnus dd7058b6bd Add agent project guidance
build / build (push) Successful in 2m49s
2026-09-17 15:02:42 +02:00
magnus 161834a512 Merge pull request 'Show midnight price change in page title' (#1) from title-change into main
build / build (push) Successful in 3m7s
Reviewed-on: #1
2026-09-17 12:55:18 +00:00
magnus 0c86cf3b59 Show midnight change below heading
pull-request / validate (pull_request) Successful in 12s
2026-09-17 14:53:05 +02:00
magnus b5bbde8acb Show midnight price change in page title 2026-09-17 14:49:25 +02:00
magnus b30637ee4e Add pull request deployment workflow
build / build (push) Successful in 2m43s
2026-09-17 14:00:26 +02:00
magnus 6c21f172db Show live Bitcoin price in page title
build / build (push) Successful in 2m45s
2026-09-15 19:51:55 +02:00
magnus 61d6d21d88 Document live GitOps production deployment
build / build (push) Successful in 2m48s
2026-09-15 18:42:09 +02:00
magnus 645c00d8de Document btcprice GitOps deployment
build / build (push) Successful in 2m48s
2026-09-15 18:35:31 +02:00
magnus dd67eb7b23 Remove vulnerable build tooling from runtime image
build / build (push) Successful in 3m42s
2026-09-15 18:03:38 +02:00
magnus 62a02ba093 Upgrade vulnerable runtime Python packages
build / build (push) Failing after 2m56s
2026-09-15 17:59:09 +02:00
magnus 919d53cdf7 Use updated Alpine runtime image
build / build (push) Failing after 2m49s
2026-09-15 17:55:35 +02:00
magnus 9b42497e58 Avoid unavailable act runner cache service
build / build (push) Failing after 2m55s
2026-09-15 17:50:19 +02:00
magnus 87bd53ca65 Verify repaired Gitea repository hooks
build / build (push) Failing after 7m22s
2026-09-15 17:41:50 +02:00
magnus 29e5bd0548 Exercise btcprice build pipeline 2026-09-15 17:39:58 +02:00
magnus 69d0e4f3e7 Run registered btcprice workflow 2026-09-15 17:38:37 +02:00
magnus ea0d46c538 Trigger initial btcprice CI build 2026-09-15 17:22:53 +02:00
magnus 0470862cb4 Build and publish btcprice images in Gitea Actions 2026-09-15 17:16:35 +02:00
magnus 1b65330960 Record completion of deployment preparation 2026-09-15 17:12:55 +02:00
magnus 573eba2d86 Prepare btcprice for Kubernetes deployment 2026-09-15 17:12:25 +02:00
magnus dda00da77a Test for btc-data availbe 2026-09-14 16:20:28 +02:00
magnusandClaude Opus 5 864674bd09 Point citadel btcprice at the local btcdata over btc-network
omarchy is gone, but the citadel deployment still set
BTCDATA_URL=http://192.168.50.42:4102 and ran standalone on its own
compose network. Every request that proxies to btcdata failed with
httpx.ConnectError: All connection attempts failed.

btcdata now runs on citadel too, so attach btcprice to the external
btc-network there and reach it by service name at http://btcdata:4102.
deploy-citadel.sh creates the network on the remote if missing, as
btcdata's own deploy script already does.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TRkiLhK5qhkZjP2adxpA3e
2026-09-11 18:52:49 +02:00
magnusandClaude Opus 5 2af8b6f62d Add citadel deployment
Port of ../elprice/deploy-citadel.sh: build the image locally for the
remote's architecture, ship it over SSH with docker save | docker load,
and run docker compose up -d --no-build in /home/pmp/apps/btcprice.

The deployment compose is standalone rather than joining btc-network,
because citadel runs no btcdata — BTCDATA_URL points at the instance
already running on omarchy (http://192.168.50.42:4102).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BdMTk61JgUdNH86ttMi4ce
2026-09-10 14:24:38 +02:00
magnus e15f00e98c Update doc 2026-09-10 13:12:35 +02:00
magnus d5e5b78334 Fix 2026-09-10 13:01:11 +02:00
21 changed files with 2145 additions and 82 deletions
+9
View File
@@ -0,0 +1,9 @@
# Environment for the citadel deployment. deploy-citadel.sh copies this to
# $APP_DIR/.env on the target host. No secrets — btcprice has none.
# btcdata runs on citadel too, in the shared btc-network. Reached by service
# name; the port must match DOCKER_BTCDATA_PORT in ../btcdata/.env.
BTCDATA_URL=http://btcdata:4102
# Host port; container always listens on 4040.
BTCPRICE_PORT=4041
+16
View File
@@ -0,0 +1,16 @@
# Copy to .env and fill in. .env is gitignored.
# The Makefile does `include .env`, so this file must exist before `make run`.
# --- Local dev (make run) ---
# btcdata instance the server proxies WebSockets to. The code falls back to
# http://localhost:4100 when unset, but the Makefile needs the file to exist.
BTCDATA_URL=http://localhost:4100
# Used by `make run-local` to point at a btcdata on this machine while
# BTCDATA_URL above points elsewhere (e.g. a deployed instance).
BTCDATA_LOCAL_URL=http://localhost:4100
# --- Docker (docker compose up -d) ---
# Port btcdata listens on inside btc-network. Must match DOCKER_BTCDATA_PORT
# in ../btcdata/.env.
DOCKER_BTCDATA_PORT=4102
+95
View File
@@ -0,0 +1,95 @@
name: build
on:
push:
branches: [main]
tags: ['v*']
env:
REGISTRY: harbor.pmagnus.cloud
IMAGE: harbor.pmagnus.cloud/btcprice/btcprice
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.13'
# act_runner's Actions cache endpoint is unreachable from job
# containers and adds a five-minute timeout. Dependency install is
# fast enough here without setup-python's cache integration.
- name: Test
run: |
python -m pip install -r requirements-dev.txt
python -m pytest tests/
- name: Generate immutable tags
id: tags
shell: bash
run: |
set -euo pipefail
SHA=$(git rev-parse --short HEAD)
case "$GITHUB_REF" in
refs/tags/v*) VERSION="${GITHUB_REF_NAME#v}" ;;
*) VERSION="main-$(date -u +%Y%m%d%H%M%S)" ;;
esac
echo "sha=$SHA" >> "$GITHUB_OUTPUT"
echo "primary=$IMAGE:$VERSION" >> "$GITHUB_OUTPUT"
echo "qualified=$IMAGE:$VERSION-$SHA" >> "$GITHUB_OUTPUT"
# anvil is not a NetBird peer. Host networking makes BuildKit use the
# runner host's pinned Harbor address; keep this until anvil is enrolled.
- uses: docker/setup-buildx-action@v3
with:
driver-opts: network=host
- uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ secrets.HARBOR_ROBOT_USER }}
password: ${{ secrets.HARBOR_ROBOT_TOKEN }}
- uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64
push: true
# Harbor blocks manifest reads until Trivy has approved the image.
# Push one tag first, then add the companion tag after the scan.
tags: ${{ steps.tags.outputs.primary }}
build-args: |
BUILD_SHA=${{ steps.tags.outputs.sha }}
cache-from: type=registry,ref=${{ env.IMAGE }}:buildcache
cache-to: type=registry,ref=${{ env.IMAGE }}:buildcache,mode=max
# The primary tag is already pushed and deployable by the time this runs:
# the image policy selects `main-<timestamp>`, so a failure here costs the
# SHA audit tag, not the rollout.
- name: Wait for Harbor policy, then add SHA tag
shell: bash
env:
PRIMARY: ${{ steps.tags.outputs.primary }}
QUALIFIED: ${{ steps.tags.outputs.qualified }}
run: |
# Harbor's vulnerability policy blocks manifest reads while Trivy's scan
# is still Running, and `imagetools create` copies the manifest. Retry
# that copy itself rather than an `inspect` probe: a probe that passes is
# no guarantee the copy will, and under the default `bash -e` the first
# blocked copy aborted the step before its retries could run.
for attempt in $(seq 1 60); do
if docker buildx imagetools create --tag "$QUALIFIED" "$PRIMARY"; then
echo "tagged $QUALIFIED on attempt $attempt"
exit 0
fi
echo "waiting for Harbor scan/pull policy ($attempt/60)"
sleep 10
done
echo "Harbor did not approve $PRIMARY within 10 minutes" >&2
exit 1
- name: Summary
run: echo "pushed ${{ steps.tags.outputs.primary }} and ${{ steps.tags.outputs.qualified }}"
+20
View File
@@ -0,0 +1,20 @@
name: pull-request
on:
pull_request:
branches: [main]
jobs:
validate:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: '3.13'
- name: Test
run: |
python -m pip install -r requirements-dev.txt
python -m pytest tests/
+1
View File
@@ -7,3 +7,4 @@ __pycache__/
*.log *.log
/.env /.env
/token /token
/=0.27
+114
View File
@@ -0,0 +1,114 @@
# AGENTS.md
Bitcoin price tracker UI — Python FastAPI web application. Serves the HTML/Tailwind UI and
proxies live price data from btcdata over WebSocket.
## Build & Development Commands
- `make run` — Start dev server on port 4000; creates `.venv` and installs `requirements.txt`
on first run (the `venv` target, keyed off `requirements.txt`)
- `cp .env.example .env` — optional; `.env` is `-include`d, so a missing one is not fatal
- `make run-local` — Same, but with `BTCDATA_URL=$BTCDATA_LOCAL_URL`
- `make tailwind` — Watch Tailwind CSS for changes
- `make test` — Installs test-only dependencies from `requirements-dev.txt` and runs `pytest tests/`
- `make venv` — Bootstrap the virtualenv without starting the server
## Docker
Docker runs alongside local dev on different ports:
| Service | Local dev | Docker |
|----------|----------------|--------------------------------------|
| btcprice | localhost:4000 | localhost:4041 (container port 4040) |
| btcdata | localhost:4100 | localhost:4102 |
`docker compose up -d` here starts **btcprice only**. Both projects attach to the external
`btc-network` (`docker network create btc-network` once), and btcdata must be started from
`../btcdata`. Inside the network btcprice reaches it at `http://btcdata:${DOCKER_BTCDATA_PORT:-4102}`.
The image does not set `BTCDATA_URL`; deployments must provide it. `BUILD_SHA` is embedded as
`APP_VERSION` and exposed at `/api/version`. The port in `CMD` is hardcoded to 4040.
## CI/CD and production
`.gitea/workflows/build.yml` tests pushes to `main` and `v*` tags on the `anvil` Gitea Actions
runner, then publishes immutable linux/amd64 images to
`harbor.pmagnus.cloud/btcprice/btcprice`. It embeds the short Git SHA as `APP_VERSION`.
The build pushes two tags: `main-<timestamp>`, then a companion `main-<timestamp>-<sha>` for
audit. Harbor's vulnerability policy blocks manifest reads while Trivy is still scanning, so
the companion tag is retried until the scan clears. Only the first tag matters for deployment —
the Flux image policy matches `^main-<14 digits>$` — so a failure in that last step costs the
audit tag and nothing else.
Flux image automation updates the manifests in the separate `magnus/deploy` repository and Flux
reconciles k3s. CI has no Kubernetes credentials. Production runs two replicas—one on hostinger and
one on citadel—and is served at https://btcprice.k8s.pmagnus.cloud through ingress-nginx and
hostinger Traefik TLS. Deployment changes belong in `../deploy/apps/btcprice`; do not deploy with
Compose or manual `kubectl apply`.
## Legacy deployment
`./deploy-citadel.sh` — build locally for the remote's arch, `docker save | ssh docker load`,
`docker compose up -d --no-build` in `/home/pmp/apps/btcprice` on citadel (192.168.50.195).
Mirrors `../elprice/deploy-citadel.sh`; no registry.
- `docker-compose.citadel.yml` → remote `docker-compose.yml`; `.env.citadel` → remote `.env`.
- btcdata also runs on citadel, so both attach to the external `btc-network` there and
`BTCDATA_URL` is `http://btcdata:4102` (service name, not a LAN address). The deploy script
creates the network on the remote if it is missing.
- Temporary migration fallback at http://192.168.50.195:4041; it is not the production endpoint.
## Architecture
- **Framework:** FastAPI
- **Server:** Uvicorn
- **Templates:** Inline HTML strings + HTMX (WebSocket extension)
- **Styling:** Tailwind CSS v4
- **Data backend:** btcdata (separate service at `BTCDATA_URL`, default `http://localhost:4100`)
- **No database** — all data comes from btcdata
- **Probes:** `/api/health` is process-only liveness; `/api/ready` checks btcdata with a 3-second
timeout; `/api/version` exposes the build identifier
### btcdata connection
The browser connects only to btcprice: `ws-connect="/ws/price"` and `ws-connect="/ws/strike"`.
Each endpoint runs `_ws_proxy`, which opens an upstream WebSocket to btcdata's `/api/price/ws`
and `/api/strike/ws`, converts each JSON message to an HTML fragment (`price_json_to_html`,
`strike_json_to_html`) and forwards it. `BTCDATA_URL` is the single point of contact;
`_btcdata_ws_url` rewrites its scheme to `ws`/`wss`.
### Recovering from a btcdata restart
htmx's WebSocket extension reconnects only after an abnormal close
(1006/1011/1012/1013); a normal 1000 close means "the server is done" and the panel stays
frozen until the visitor reloads. So a btcdata restart must never reach the browser as a
normal close:
- `_ws_proxy` reconnects upstream behind the still-open browser socket, on the bounded
`UPSTREAM_RETRY_DELAYS_SECONDS` schedule. An ordinary restart is invisible to the page —
the last values stay on screen and updates resume.
- A connection that lasted `UPSTREAM_STABLE_SECONDS` resets that schedule, so a long-lived
page treats a later restart as a fresh outage rather than a continuing one.
- When the schedule runs out, the proxy closes with `BROWSER_RETRY_CLOSE_CODE` (1012) and
htmx's own backoff takes over. Every server-side exit uses that code; only a browser that
has already left can see a plain close.
- A payload that cannot be rendered is logged and skipped, not treated as a dead stream.
`tests/test_ws_proxy.py` covers this against a real loopback WebSocket server, including
same-socket recovery across an upstream restart and the retryable close code. The regression
to watch for is a 1000 close after upstream loss.
## Source Layout
btcprice acts as a WebSocket proxy between the browser and btcdata:
1. Browser connects via HTMX WebSocket to `/ws/price` or `/ws/strike`
2. btcprice proxies to btcdata's `/api/price/ws` or `/api/strike/ws`
3. JSON from btcdata is transformed into HTML fragments (in `price_json_to_html` / `strike_json_to_html`)
4. HTMX swaps the HTML into the DOM by matching element IDs (`price-display`, `strike-display`)
### Key Conventions
- Numbers use Danish locale formatting (dots for thousands, commas for decimals) via `da_fmt()`
- Price changes are color-coded: green (up), red (down), gray (unchanged)
- All HTML is built as string concatenation in Python — no template engine
+92 -14
View File
@@ -1,23 +1,63 @@
# btcprice # CLAUDE.md
Bitcoin price tracker UI — Python FastAPI web application. Serves the HTML/Tailwind UI. Connects to btcdata for live price data via WebSocket. Bitcoin price tracker UI — Python FastAPI web application. Serves the HTML/Tailwind UI and
proxies live price data from btcdata over WebSocket.
## Build & Development Commands ## Build & Development Commands
- `make run` — Start dev server on port 4000 - `make run` — Start dev server on port 4000; creates `.venv` and installs `requirements.txt`
on first run (the `venv` target, keyed off `requirements.txt`)
- `cp .env.example .env` — optional; `.env` is `-include`d, so a missing one is not fatal
- `make run-local` — Same, but with `BTCDATA_URL=$BTCDATA_LOCAL_URL`
- `make tailwind` — Watch Tailwind CSS for changes - `make tailwind` — Watch Tailwind CSS for changes
- `make test` — Run tests - `make test` — Installs test-only dependencies from `requirements-dev.txt` and runs `pytest tests/`
- `make venv` — Bootstrap the virtualenv without starting the server
## Docker ## Docker
Docker runs alongside local dev on different ports: Docker runs alongside local dev on different ports:
| Service | Local dev | Docker | | Service | Local dev | Docker |
|----------|----------------|----------------| |----------|----------------|--------------------------------------|
| btcprice | localhost:4000 | localhost:4041 | | btcprice | localhost:4000 | localhost:4041 (container port 4040) |
| btcdata | localhost:4100 | localhost:4101 | | btcdata | localhost:4100 | localhost:4102 |
btcprice's docker-compose includes btcdata via `include: ../btcdata/docker-compose.yml`. `docker compose up -d` here starts **btcprice only**. Both projects attach to the external
`btc-network` (`docker network create btc-network` once), and btcdata must be started from
`../btcdata`. Inside the network btcprice reaches it at `http://btcdata:${DOCKER_BTCDATA_PORT:-4102}`.
The image does not set `BTCDATA_URL`; deployments must provide it. `BUILD_SHA` is embedded as
`APP_VERSION` and exposed at `/api/version`. The port in `CMD` is hardcoded to 4040.
## CI/CD and production
`.gitea/workflows/build.yml` tests pushes to `main` and `v*` tags on the `anvil` Gitea Actions
runner, then publishes immutable linux/amd64 images to
`harbor.pmagnus.cloud/btcprice/btcprice`. It embeds the short Git SHA as `APP_VERSION`.
The build pushes two tags: `main-<timestamp>`, then a companion `main-<timestamp>-<sha>` for
audit. Harbor's vulnerability policy blocks manifest reads while Trivy is still scanning, so
the companion tag is retried until the scan clears. Only the first tag matters for deployment —
the Flux image policy matches `^main-<14 digits>$` — so a failure in that last step costs the
audit tag and nothing else.
Flux image automation updates the manifests in the separate `magnus/deploy` repository and Flux
reconciles k3s. CI has no Kubernetes credentials. Production runs two replicas—one on hostinger and
one on citadel—and is served at https://btcprice.k8s.pmagnus.cloud through ingress-nginx and
hostinger Traefik TLS. Deployment changes belong in `../deploy/apps/btcprice`; do not deploy with
Compose or manual `kubectl apply`.
## Legacy deployment
`./deploy-citadel.sh` — build locally for the remote's arch, `docker save | ssh docker load`,
`docker compose up -d --no-build` in `/home/pmp/apps/btcprice` on citadel (192.168.50.195).
Mirrors `../elprice/deploy-citadel.sh`; no registry.
- `docker-compose.citadel.yml` → remote `docker-compose.yml`; `.env.citadel` → remote `.env`.
- btcdata also runs on citadel, so both attach to the external `btc-network` there and
`BTCDATA_URL` is `http://btcdata:4102` (service name, not a LAN address). The deploy script
creates the network on the remote if it is missing.
- Temporary migration fallback at http://192.168.50.195:4041; it is not the production endpoint.
## Architecture ## Architecture
@@ -25,12 +65,50 @@ btcprice's docker-compose includes btcdata via `include: ../btcdata/docker-compo
- **Server:** Uvicorn - **Server:** Uvicorn
- **Templates:** Inline HTML strings + HTMX (WebSocket extension) - **Templates:** Inline HTML strings + HTMX (WebSocket extension)
- **Styling:** Tailwind CSS v4 - **Styling:** Tailwind CSS v4
- **Data backend:** btcdata (separate service at BTCDATA_URL, default http://localhost:4100) - **Data backend:** btcdata (separate service at `BTCDATA_URL`, default `http://localhost:4100`)
- **No database** — all data comes from btcdata - **No database** — all data comes from btcdata
- **Probes:** `/api/health` is process-only liveness; `/api/ready` checks btcdata with a 3-second
timeout; `/api/version` exposes the build identifier
### btcdata connection
The browser connects only to btcprice: `ws-connect="/ws/price"` and `ws-connect="/ws/strike"`.
Each endpoint runs `_ws_proxy`, which opens an upstream WebSocket to btcdata's `/api/price/ws`
and `/api/strike/ws`, converts each JSON message to an HTML fragment (`price_json_to_html`,
`strike_json_to_html`) and forwards it. `BTCDATA_URL` is the single point of contact;
`_btcdata_ws_url` rewrites its scheme to `ws`/`wss`.
### Recovering from a btcdata restart
htmx's WebSocket extension reconnects only after an abnormal close
(1006/1011/1012/1013); a normal 1000 close means "the server is done" and the panel stays
frozen until the visitor reloads. So a btcdata restart must never reach the browser as a
normal close:
- `_ws_proxy` reconnects upstream behind the still-open browser socket, on the bounded
`UPSTREAM_RETRY_DELAYS_SECONDS` schedule. An ordinary restart is invisible to the page —
the last values stay on screen and updates resume.
- A connection that lasted `UPSTREAM_STABLE_SECONDS` resets that schedule, so a long-lived
page treats a later restart as a fresh outage rather than a continuing one.
- When the schedule runs out, the proxy closes with `BROWSER_RETRY_CLOSE_CODE` (1012) and
htmx's own backoff takes over. Every server-side exit uses that code; only a browser that
has already left can see a plain close.
- A payload that cannot be rendered is logged and skipped, not treated as a dead stream.
`tests/test_ws_proxy.py` covers this against a real loopback WebSocket server, including
same-socket recovery across an upstream restart and the retryable close code. The regression
to watch for is a 1000 close after upstream loss.
## Source Layout ## Source Layout
``` btcprice acts as a WebSocket proxy between the browser and btcdata:
app/ 1. Browser connects via HTMX WebSocket to `/ws/price` or `/ws/strike`
└── main.py # FastAPI app: routes, WebSocket proxy, HTML rendering 2. btcprice proxies to btcdata's `/api/price/ws` or `/api/strike/ws`
``` 3. JSON from btcdata is transformed into HTML fragments (in `price_json_to_html` / `strike_json_to_html`)
4. HTMX swaps the HTML into the DOM by matching element IDs (`price-display`, `strike-display`)
### Key Conventions
- Numbers use Danish locale formatting (dots for thousands, commas for decimals) via `da_fmt()`
- Price changes are color-coded: green (up), red (down), gray (unchanged)
- All HTML is built as string concatenation in Python — no template engine
+9 -5
View File
@@ -1,4 +1,4 @@
FROM node:22-alpine AS css FROM harbor.pmagnus.cloud/dockerhub-cache/library/node:22-alpine AS css
WORKDIR /build WORKDIR /build
COPY package.json package-lock.json ./ COPY package.json package-lock.json ./
@@ -8,18 +8,22 @@ COPY app/ app/
COPY tailwind.config.js ./ COPY tailwind.config.js ./
RUN npm run css:build RUN npm run css:build
FROM python:3.13-slim FROM harbor.pmagnus.cloud/dockerhub-cache/library/python:3.13-alpine
WORKDIR /app WORKDIR /app
COPY requirements.txt . COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt RUN apk upgrade --no-cache \
&& pip install --no-cache-dir -r requirements.txt \
&& pip uninstall -y pip setuptools msgpack \
&& rm -rf /usr/local/lib/python3.13/ensurepip
COPY app/ app/ COPY app/ app/
COPY resources/public/ resources/public/ COPY resources/public/ resources/public/
COPY --from=css /build/resources/public/css/output.css resources/public/css/output.css COPY --from=css /build/resources/public/css/output.css resources/public/css/output.css
ARG BUILD_SHA=unknown
ENV APP_VERSION=${BUILD_SHA}
EXPOSE 4040 EXPOSE 4040
ENV PORT=4040
ENV BTCDATA_URL=http://btcdata:4101
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "4040"] CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "4040"]
+33 -8
View File
@@ -1,19 +1,44 @@
include .env -include .env
export export
.PHONY: clean run run-docker tailwind test VENV := .venv
UVICORN := $(VENV)/bin/uvicorn
TEST_DEPS := $(VENV)/.test-deps
K3S_BTCDATA_PORT := 4100
.PHONY: clean run run-local run-citadel run-k3s tailwind test venv
venv: $(UVICORN)
$(UVICORN): requirements.txt
python3 -m venv $(VENV)
$(VENV)/bin/pip install --upgrade pip
$(VENV)/bin/pip install -r requirements.txt
clean: clean:
rm -rf __pycache__ app/__pycache__ rm -rf __pycache__ app/__pycache__
run: run: venv
.venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 4000 --reload $(UVICORN) app.main:app --host 0.0.0.0 --port 4000 --reload
run-local: run-local: venv
BTCDATA_URL=$(BTCDATA_LOCAL_URL) .venv/bin/uvicorn app.main:app --host 0.0.0.0 --port 4000 --reload BTCDATA_URL=$(BTCDATA_LOCAL_URL) $(UVICORN) app.main:app --host 0.0.0.0 --port 4000 --reload
run-citadel: venv
BTCDATA_URL=$(BTCDATA_URL) $(UVICORN) app.main:app --host 0.0.0.0 --port 4000 --reload
run-k3s: venv
kubectl -n btcdata port-forward service/btcdata $(K3S_BTCDATA_PORT):4101 & \
port_forward_pid=$$!; \
trap 'kill $$port_forward_pid' EXIT INT TERM; \
BTCDATA_URL=http://127.0.0.1:$(K3S_BTCDATA_PORT) $(UVICORN) app.main:app --host 0.0.0.0 --port 4000 --reload
tailwind: tailwind:
npm run css:watch npm run css:watch
test: $(TEST_DEPS): requirements-dev.txt requirements.txt | venv
.venv/bin/python -m pytest tests/ $(VENV)/bin/pip install -r requirements-dev.txt
touch $(TEST_DEPS)
test: venv $(TEST_DEPS)
$(VENV)/bin/python -m pytest tests/
+219
View File
@@ -0,0 +1,219 @@
# Pull-request deployment workflow
## Intended result
Production deployment should happen only after a change has followed this path:
```text
feature branch
-> local test and local application run
-> pull request with automated checks
-> owner reviews the result and approves by merging
-> merge to main
-> Gitea Actions builds and publishes an immutable image
-> Flux updates magnus/deploy
-> Flux reconciles Kubernetes
-> production verification
```
For this single-user repository, clicking **Merge** after reviewing the diff and successful checks
is the approval. The merge creates the `push` event which starts the existing production build.
## Findings (2026-09-17)
- The default branch is `main`.
- Gitea reports no branch-protection rule for this repository. A contributor can therefore push
directly to `main`, bypassing a pull request and approval.
- `.gitea/workflows/pr.yml` now runs tests for pull requests targeting `main`; it has no publish
step and receives no Harbor credentials.
- `.gitea/workflows/build.yml` remains restricted to pushes to `main` and `v*` tags. A merged PR
therefore starts the existing image publication and automatic Kubernetes deployment path.
- Recent `main` workflow runs show that the existing post-merge build and publish workflow works.
- The workflow runs `pytest`, builds a linux/amd64 image, and publishes immutable
`main-<UTC timestamp>` and SHA-qualified tags to
`harbor.pmagnus.cloud/btcprice/btcprice`.
- Flux watches the plain `main-<UTC timestamp>` tags. Its image automation writes the selected tag
into `magnus/deploy`, after which Flux reconciles the Kubernetes cluster.
- Kubernetes runs two replicas with rolling updates, health/readiness probes, and one pod per node.
- Gitea currently has only one user (`magnus`). Mandatory reviewer approval is therefore not
appropriate: Gitea does not provide meaningful independent self-approval. The owner's explicit
merge action is the approval, while branch protection and CI remain the enforceable gates.
## Required one-time configuration
### 1. Pull-request validation workflow (implemented)
`.gitea/workflows/pr.yml` validates every pull request targeting `main`. It checks out the proposed
commit and runs the tests without publishing an image or receiving Harbor credentials. The
production Docker image is built locally before the PR and again by `build.yml` after merge.
```bash
python -m pip install -r requirements-dev.txt
python -m pytest tests/
```
The contributor performs the Docker build locally before opening the PR. Keeping it out of the PR
job avoids giving proposed code credentials for the private Harbor base images. After merge, the
trusted `build.yml` workflow builds the production image with Harbor credentials.
Keep image publication restricted to the existing `push` event for `main` and `v*` tags. PR code,
especially code from an untrusted fork, must not receive the Harbor credentials.
The workflows are intentionally separated:
- `pr.yml`: `pull_request` trigger; tests only; no secrets and no push.
- `build.yml`: existing `push` trigger; tests, image build, Harbor scan gate, and image push.
### 2. Protect `main` in Gitea (configuration still required)
In `magnus/btcprice`, open **Settings -> Branches -> Branch protection rules** and protect `main`.
Configure the rule to:
- disable direct pushes (and force pushes);
- require pull requests before merge;
- set required approvals to zero while the repository has only one user;
- block merge when a review requests changes;
- require the PR validation status check;
- require the branch to be up to date before merge;
- restrict bypass/whitelist permissions to emergency administrators only.
Create the PR workflow and let it run once before selecting its status-check context in the branch
rule; this avoids guessing Gitea's generated context name. With required approvals set to zero, the
owner reviews the diff and approves it by clicking **Merge** after the required check passes. If a
second trusted user is added later, change the rule to require one approval and dismiss stale
approvals when new commits are pushed.
## Per-change procedure
### 1. Create a feature branch
Start from the latest production branch:
```bash
git switch main
git pull --ff-only
git switch -c feature/<short-description>
```
Make and commit the change on this branch. Do not commit `.env`; it contains local configuration.
### 2. Test and run the branch locally before opening the PR
Run the automated tests:
```bash
make test
```
For a functional check, start `btcdata` in one terminal:
```bash
cd ../btcdata
make run
```
Then start this branch of `btcprice` in another terminal:
```bash
cd ../btcprice
make run
```
Open `http://localhost:4000` and confirm that both live price streams update. Also check the
application endpoints:
```bash
curl --fail http://localhost:4000/api/health
curl --fail http://localhost:4000/api/ready
curl --fail http://localhost:4000/api/version
```
`/api/health` checks the btcprice process; `/api/ready` also verifies that btcdata is reachable.
The local version normally reports `development`.
Finally, exercise the production build path:
```bash
docker build \
--build-arg BUILD_SHA="$(git rev-parse --short HEAD)" \
-t btcprice:pr-check .
```
If Docker validation needs the complete local stack, create `btc-network` once, start btcdata's
Compose project, and run `docker compose up --build` in this repository. This application then
listens at `http://localhost:4041`.
Record the tests and manual behavior checked in the PR description.
### 3. Push and open the pull request
```bash
git push -u origin feature/<short-description>
tea pulls create \
--repo magnus/btcprice \
--base main \
--head feature/<short-description> \
--title "<concise change title>"
```
The PR should explain the change, risk, local verification, and rollback considerations. Wait for
the PR validation job to pass, then review and merge it. Any new commit triggers validation again.
### 4. Approve and merge
As repository owner, check the diff, confirm the automated result, and verify the stated manual
test. Clicking **Merge** is the approval step in this single-user setup. Prefer **squash merge** for
small, single-purpose PRs or use another consistent merge policy.
Do not press merge if the check is failing or the PR branch is behind `main`. Do not deploy with
`kubectl apply` or `deploy-citadel.sh`; those bypass the GitOps audit trail.
### 5. Observe the deployment
After merge, observe this sequence:
1. Gitea Actions tests the merge commit.
2. It publishes `main-<UTC timestamp>` and `main-<UTC timestamp>-<short SHA>`.
3. Harbor scans the image and permits the companion tag after policy approval.
4. Flux image reflection selects the newest plain `main-<UTC timestamp>` tag.
5. Flux commits the updated image reference to `magnus/deploy`.
6. Flux applies that Git state and Kubernetes performs a rolling update.
The CI job intentionally has no Kubernetes credentials. A successful image build is not by itself
proof of a successful rollout, so verify production separately:
```bash
curl --fail https://btcprice.k8s.pmagnus.cloud/api/health
curl --fail https://btcprice.k8s.pmagnus.cloud/api/ready
curl --fail https://btcprice.k8s.pmagnus.cloud/api/version
```
The production version should equal the short SHA of the merged btcprice commit. Also confirm in
the deployment repository or cluster that both replicas became Ready. If rollout access is
available, useful checks are:
```bash
flux get kustomizations
kubectl -n btcprice rollout status deployment/btcprice
kubectl -n btcprice get pods -o wide
```
## Failure and rollback behavior
- If PR validation fails, fix the feature branch; nothing has been published or deployed.
- If the post-merge build fails, Kubernetes keeps running the previous image. Fix forward through a
new PR; do not manually push a replacement under an existing immutable tag.
- If Harbor rejects the image, Flux cannot select it and production stays on the previous image.
- If a new image deploys but is bad, revert the Flux image commit in `magnus/deploy`, or commit the
last known-good immutable tag there. Flux then restores that version. Follow up with a revert or
fix PR in this repository so future automation does not reintroduce the bad build.
## Security boundary
The separation is intentional:
- PR jobs validate untrusted proposed code without deployment or registry credentials.
- The protected `main` branch is the release authorization boundary.
- Gitea Actions may write images to Harbor only after merge.
- Flux alone has authority to update the GitOps repository and reconcile Kubernetes.
- Kubernetes changes remain reviewable and recoverable through Git history.
+110 -9
View File
@@ -1,6 +1,8 @@
# btcprice # btcprice
Bitcoin price tracker UI. Serves an HTML page with live price updates streamed from [btcdata](../btcdata) via WebSocket using HTMX. Bitcoin price tracker UI. Serves an HTML page with live price updates streamed from
[btcdata](../btcdata), which this server proxies over WebSocket and renders as HTML fragments
for HTMX.
## Quick Start ## Quick Start
@@ -8,25 +10,124 @@ Bitcoin price tracker UI. Serves an HTML page with live price updates streamed f
# Start btcdata first (provides price data) # Start btcdata first (provides price data)
cd ../btcdata && make run cd ../btcdata && make run
# Local dev # Local dev — creates .venv and installs requirements on first run
make run # starts on port 4000 make run # starts on port 4000
make tailwind # watch Tailwind CSS (separate terminal) make tailwind # watch Tailwind CSS (separate terminal)
# Docker (starts both btcprice + btcdata)
docker compose up -d # btcprice on 4041, btcdata on 4101
``` ```
Optionally `cp .env.example .env` to override `BTCDATA_URL`; without it the code default
(`http://localhost:4100`) applies. `make run-local` requires a `.env` defining
`BTCDATA_LOCAL_URL`.
Run the test suite with `make test`. Test-only packages are kept in `requirements-dev.txt` and are
not installed in the production image.
## Docker
`docker compose up -d` starts **btcprice only** — it does not bring up btcdata. Both compose
files attach to the external `btc-network`, so create it once and start btcdata separately:
```bash
docker network create btc-network # once
cd ../btcdata && docker compose up -d
cd ../btcprice && docker compose up -d
```
| Service | Local dev | Docker |
|----------|----------------|---------------------------------------|
| btcprice | localhost:4000 | localhost:4041 (container port 4040) |
| btcdata | localhost:4100 | localhost:4102 |
Inside `btc-network`, btcprice reaches btcdata as `http://btcdata:4102`.
## CI images
Pushes to the `main` branch and tags matching `v*` run
`.gitea/workflows/build.yml`. The workflow tests the application and publishes linux/amd64 images
to `harbor.pmagnus.cloud/btcprice/btcprice`. Branch builds receive an immutable
`main-<UTC timestamp>` tag (for Flux selection) and a SHA-qualified companion tag; release tags
receive the version without `v` and a SHA-qualified companion tag. The image embeds the short Git
SHA, available from `/api/version`.
The workflow requires the repository secrets `HARBOR_ROBOT_USER` and `HARBOR_ROBOT_TOKEN`. It only
pushes images and intentionally has no Kubernetes credentials.
## Production deployment
Production is GitOps-managed at **https://btcprice.k8s.pmagnus.cloud**:
```text
git push → Gitea Actions on anvil → Harbor → Flux → k3s → ingress-nginx → Traefik TLS
```
Flux image automation commits the newest approved `main-<UTC timestamp>` image to the separate
`magnus/deploy` repository. The manifests are in `../deploy/apps/btcprice`, and two replicas run with
one pod on each k3s node. CI never receives cluster credentials. Do not deploy production with
Docker Compose or manual `kubectl apply`; rollback by reverting the immutable image change in the
GitOps repository.
btcdata remains an external dependency running on citadel. Kubernetes represents it with a
selectorless Service and EndpointSlice, so pods use `http://btcdata:4102` without embedding its IP in
the Deployment.
## Legacy deployment (citadel)
```bash
./deploy-citadel.sh
```
Same method as `../elprice/deploy-citadel.sh`: builds the image locally for the target's
architecture (citadel is x86_64 → `linux/amd64`), copies `docker-compose.citadel.yml` and
`.env.citadel` to `/home/pmp/apps/btcprice` on the host as `docker-compose.yml` / `.env`,
pipes the image over SSH with `docker save | docker load`, and runs
`docker compose up -d --no-build`. No registry involved.
The temporary migration fallback listens on **http://192.168.50.195:4041** and proxies to the
btcdata running on citadel as well. It is retained only for the validation/rollback window; the
Kubernetes hostname is the production endpoint. Both attach to the external `btc-network` there, so `BTCDATA_URL` is
`http://btcdata:4102` (set in `.env.citadel`) — the service name inside the network, not a LAN
address. The deploy script creates `btc-network` on the host if it does not exist yet.
Overridable: `BTCPRICE_DEPLOY_HOST`, `BTCPRICE_DEPLOY_DIR`, `BTCPRICE_IMAGE_NAME`,
`BTCPRICE_PLATFORM`, `BTCPRICE_NETWORK`.
## Environment Variables ## Environment Variables
| Variable | Default | Description | | Variable | Default | Description |
|---|---|---| |---|---|---|
| `PORT` | `4000` | HTTP server port | | `BTCDATA_URL` | `http://localhost:4100` | btcdata service URL the server proxies to |
| `BTCDATA_URL` | `http://192.168.50.42:4100` | btcdata service URL (browser connects via WebSocket) | | `APP_VERSION` | `development` | Build identifier returned by `/api/version` |
| `BTCDATA_LOCAL_URL` | — | Used by `make run-local` instead of `BTCDATA_URL` |
| `DOCKER_BTCDATA_PORT` | `4102` | btcdata's port inside `btc-network` (local compose only) |
| `BTCPRICE_PORT` | `4041` | Host port in the citadel deployment (`.env.citadel`) |
The HTTP port is not configurable via environment: `make run` hardcodes 4000 and the Docker
image's `CMD` hardcodes 4040.
## Health endpoints
- `/api/health` is a process-only liveness check and remains healthy during a btcdata outage.
- `/api/ready` checks `${BTCDATA_URL}/api/health` with a three-second timeout and returns 503 when
btcdata is unavailable.
- `/api/version` returns the build SHA embedded by CI.
## How It Works ## How It Works
The server renders an HTML page with HTMX and the WebSocket extension (`hx-ext="ws"`). HTMX connects directly to btcdata's WebSocket endpoint (`BTCDATA_URL/api/price/ws`), which sends HTML fragments. HTMX swaps the received `<div id="price-display">` into the DOM automatically. Price changes are color-coded (green up, red down). Reconnection is handled by HTMX with exponential backoff. The browser loads a page with HTMX and the WebSocket extension, which connects back to
**btcprice** at `/ws/price` and `/ws/strike` — the browser never contacts btcdata directly.
Each of those endpoints opens an upstream WebSocket to btcdata (`/api/price/ws` and
`/api/strike/ws`), converts each JSON message to an HTML fragment (`price_json_to_html`,
`strike_json_to_html`), and forwards it to the browser. HTMX swaps the received
`<div id="price-display">` / `<div id="strike-display">` into the DOM. Price changes are
color-coded (green up, red down). Reconnection is handled by HTMX.
## Tech Stack ## Tech Stack
Clojure · Kit · Undertow · Reitit · Hiccup · HTMX · Tailwind CSS v4 Python · FastAPI · Uvicorn · websockets · HTMX · Tailwind CSS v4
The Clojure implementation this replaced is on the `clojure` branch.
The 24-hour chart uses 48 half-hour candles from rustbtcdata’s
`/api/kraken-half-hour` endpoint, aggregated from minute OHLC data. It refreshes
every five minutes and includes the current partial half-hour candle.
+270 -22
View File
@@ -1,17 +1,22 @@
import asyncio
import json import json
import logging import logging
import os import os
import time
from datetime import datetime from datetime import datetime
import httpx
import websockets import websockets
from fastapi import FastAPI, WebSocket, WebSocketDisconnect from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.responses import FileResponse, HTMLResponse from fastapi.responses import FileResponse, HTMLResponse, JSONResponse
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
logging.basicConfig(level=logging.INFO) logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("btcprice") logger = logging.getLogger("btcprice")
BTCDATA_URL = os.environ.get("BTCDATA_URL", "http://localhost:4100") BTCDATA_URL = os.environ.get("BTCDATA_URL", "http://localhost:4100").rstrip("/")
APP_VERSION = os.environ.get("APP_VERSION", "development")
READINESS_TIMEOUT_SECONDS = 3.0
app = FastAPI() app = FastAPI()
@@ -38,11 +43,13 @@ HOME_HTML = """\
<link rel="stylesheet" href="/css/output.css"> <link rel="stylesheet" href="/css/output.css">
<script src="https://unpkg.com/htmx.org@2/dist/htmx.min.js"></script> <script src="https://unpkg.com/htmx.org@2/dist/htmx.min.js"></script>
<script src="https://unpkg.com/htmx-ext-ws@2/ws.js"></script> <script src="https://unpkg.com/htmx-ext-ws@2/ws.js"></script>
<script src="https://unpkg.com/lightweight-charts@4/dist/lightweight-charts.standalone.production.js"></script>
</head> </head>
<body> <body>
<div class="mx-auto max-w-lg px-4 py-6"> <div class="mx-auto max-w-lg px-4 py-6">
<header class="text-center mb-8"> <header class="text-center mb-8">
<h1 class="text-3xl font-bold text-gray-900">BTC Price</h1> <h1 class="text-3xl font-bold text-gray-900">BTC Price</h1>
<p id="midnight-change" class="mt-1 text-sm text-gray-400">&nbsp;</p>
</header> </header>
<div id="price-panel" hx-ext="ws" ws-connect="/ws/price"> <div id="price-panel" hx-ext="ws" ws-connect="/ws/price">
<div id="price-display"> <div id="price-display">
@@ -58,8 +65,114 @@ HOME_HTML = """\
</div> </div>
</div> </div>
</div> </div>
<div class="mt-4">
<div class="bg-white rounded-2xl shadow-sm border border-gray-200 p-6">
<p class="text-sm text-gray-500 mb-3">1h Candles (1m)</p>
<div id="chart-minute" style="height:300px"></div>
</div>
</div>
<div class="mt-4">
<div class="bg-white rounded-2xl shadow-sm border border-gray-200 p-6">
<p class="text-sm text-gray-500 mb-3">24h Candles (30m)</p>
<div id="chart-hour" style="height:300px"></div>
</div>
</div>
<div class="hidden text-4xl text-2xl font-bold text-green-600 text-red-600 text-gray-900 text-xs text-gray-400 mt-2 flex justify-center justify-between items-center gap-4 mt-1 mt-3 text-sm text-gray-600 text-gray-500 mb-3 py-1 grid grid-cols-2 gap-x-6 gap-y-1" aria-hidden="true"></div> <div class="hidden text-4xl text-2xl font-bold text-green-600 text-red-600 text-gray-900 text-xs text-gray-400 mt-2 flex justify-center justify-between items-center gap-4 mt-1 mt-3 text-sm text-gray-600 text-gray-500 mb-3 py-1 grid grid-cols-2 gap-x-6 gap-y-1" aria-hidden="true"></div>
</div> </div>
<script>
(function() {
let currentPrice = null;
let midnightPrice = null;
function formatPriceChange(change) {
const sign = change > 0 ? '+' : change < 0 ? '-' : '±';
const amount = Math.abs(change).toLocaleString('da-DK', {
minimumFractionDigits: 2,
maximumFractionDigits: 2,
});
return sign + '$' + amount;
}
function updateTitleFromPrice() {
const priceDisplay = document.getElementById('price-display');
const price = priceDisplay && priceDisplay.dataset.titlePrice;
const priceValue = priceDisplay && Number(priceDisplay.dataset.priceValue);
if (price && Number.isFinite(priceValue)) {
currentPrice = priceValue;
}
if (price && currentPrice !== null && midnightPrice !== null) {
const change = currentPrice - midnightPrice;
const formattedChange = formatPriceChange(change);
document.title = price + ' | ' + formattedChange;
const midnightChange = document.getElementById('midnight-change');
midnightChange.textContent = formattedChange;
midnightChange.className = 'mt-1 text-sm font-bold ' + (
change > 0 ? 'text-green-600' : change < 0 ? 'text-red-600' : 'text-gray-500'
);
} else if (price) {
document.title = price + ' | BTC Price';
}
}
function updateMidnightPrice(candles) {
const midnight = new Date();
midnight.setHours(0, 0, 0, 0);
const midnightSeconds = Math.floor(midnight.getTime() / 1000);
const firstToday = candles.find(candle => candle.time >= midnightSeconds);
if (firstToday) {
midnightPrice = firstToday.open;
updateTitleFromPrice();
}
}
const pricePanel = document.getElementById('price-panel');
new MutationObserver(updateTitleFromPrice).observe(pricePanel, {
childList: true,
subtree: true,
});
updateTitleFromPrice();
function initChart(containerId, endpoint, intervalMs, onData) {
const container = document.getElementById(containerId);
const chart = LightweightCharts.createChart(container, {
layout: { background: { type: 'solid', color: '#ffffff' }, textColor: '#6b7280' },
grid: { vertLines: { color: '#f3f4f6' }, horzLines: { color: '#f3f4f6' } },
rightPriceScale: { borderColor: '#e5e7eb' },
timeScale: { borderColor: '#e5e7eb', timeVisible: true, secondsVisible: false },
});
const series = chart.addCandlestickSeries({
upColor: '#16a34a', downColor: '#dc2626',
borderUpColor: '#16a34a', borderDownColor: '#dc2626',
wickUpColor: '#16a34a', wickDownColor: '#dc2626',
});
function loadData() {
fetch(endpoint)
.then(r => r.json())
.then(candles => {
const data = candles.map(c => ({
time: c.ts,
open: parseFloat(c.open),
high: parseFloat(c.high),
low: parseFloat(c.low),
close: parseFloat(c.close),
})).sort((a, b) => a.time - b.time);
series.setData(data);
if (onData) onData(data);
chart.timeScale().fitContent();
})
.catch(e => console.error('Chart fetch error:', e));
}
loadData();
setInterval(loadData, intervalMs);
new ResizeObserver(() => {
chart.applyOptions({ width: container.clientWidth });
}).observe(container);
}
initChart('chart-minute', '/api/kraken-minute', 60 * 1000);
initChart('chart-hour', '/api/kraken-half-hour', 5 * 60 * 1000, updateMidnightPrice);
})();
</script>
</body> </body>
</html>""" </html>"""
@@ -77,6 +190,64 @@ async def health():
return {"status": "up"} return {"status": "up"}
@app.get("/api/ready")
async def ready():
"""Report readiness only while the btcdata dependency is healthy."""
url = f"{BTCDATA_URL}/api/health"
try:
async with httpx.AsyncClient() as client:
response = await client.get(url, timeout=READINESS_TIMEOUT_SECONDS)
response.raise_for_status()
except httpx.HTTPError:
logger.warning("btcdata readiness check failed for %s", url)
return JSONResponse(content={"status": "not ready"}, status_code=503)
return {"status": "ready"}
@app.get("/api/version")
async def version():
return {"version": APP_VERSION}
async def _proxy_btcdata_json(path: str) -> JSONResponse:
url = f"{BTCDATA_URL}{path}"
try:
async with httpx.AsyncClient() as client:
resp = await client.get(url, timeout=10)
return JSONResponse(content=resp.json(), status_code=resp.status_code)
except httpx.RequestError as exc:
logger.warning("Unable to connect to btcdata at %s: %s", url, exc)
return JSONResponse(
content={
"error": "btcdata unavailable",
"detail": f"Could not connect to {BTCDATA_URL}. Start btcdata first or set BTCDATA_URL.",
},
status_code=503,
)
except json.JSONDecodeError:
logger.warning("btcdata returned non-JSON response from %s", url)
return JSONResponse(
content={"error": "invalid btcdata response"},
status_code=502,
)
@app.get("/api/kraken-half-hour")
async def kraken_half_hour():
return await _proxy_btcdata_json("/api/kraken-half-hour")
@app.get("/api/kraken-hour")
async def kraken_hour():
return await _proxy_btcdata_json("/api/kraken-hour")
@app.get("/api/kraken-minute")
async def kraken_minute():
return await _proxy_btcdata_json("/api/kraken-minute")
# --- Formatting helpers (Danish locale) --- # --- Formatting helpers (Danish locale) ---
@@ -122,11 +293,12 @@ def price_json_to_html(text: str) -> str:
else: else:
ts = "" ts = ""
formatted_price = f'${da_fmt(",.2f", p)}'
html = ( html = (
'<div id="price-display">' f'<div id="price-display" data-title-price="{formatted_price}" data-price-value="{p}">'
'<div class="bg-white rounded-2xl shadow-sm border border-gray-200 p-6">' '<div class="bg-white rounded-2xl shadow-sm border border-gray-200 p-6">'
'<div class="text-center">' '<div class="text-center">'
f'<p class="text-4xl font-bold {color}">${da_fmt(",.2f", p)}</p>' f'<p class="text-4xl font-bold {color}">{formatted_price}</p>'
) )
if data.get("price_eur") is not None: if data.get("price_eur") is not None:
@@ -202,36 +374,112 @@ def strike_json_to_html(text: str) -> str:
# --- WebSocket proxy --- # --- WebSocket proxy ---
# htmx's WebSocket extension reconnects a browser socket only after an abnormal
# close (1006/1011/1012/1013). A normal 1000 close means "the server is done",
# so the extension never reconnects and the panel stays frozen until the visitor
# reloads. A btcdata restart must therefore never reach the browser as a normal
# close: the proxy reconnects upstream behind the still-open browser socket, and
# only if that keeps failing does it close with 1012 (Service Restart) and hand
# recovery to the page's own reconnect.
BROWSER_RETRY_CLOSE_CODE = 1012
UPSTREAM_OPEN_TIMEOUT_SECONDS = 5.0
UPSTREAM_RETRY_DELAYS_SECONDS = (0.5, 1.0, 2.0, 4.0, 8.0, 8.0)
UPSTREAM_STABLE_SECONDS = 30.0
def _btcdata_ws_url(path: str) -> str: def _btcdata_ws_url(path: str) -> str:
return BTCDATA_URL.replace("http", "ws", 1) + path return BTCDATA_URL.replace("http", "ws", 1) + path
async def _ws_proxy(browser_ws: WebSocket, upstream_path: str, transform): async def _await_browser_close(browser_ws: WebSocket) -> None:
"""Return as soon as the browser side goes away.
The browser sends nothing, but reading is the only way to notice that it
left while the proxy is waiting on a dead upstream rather than sending.
"""
try:
while True:
if (await browser_ws.receive())["type"] == "websocket.disconnect":
return
except (WebSocketDisconnect, RuntimeError):
return
async def _forward_upstream(browser_ws: WebSocket, upstream_url: str, transform) -> None:
"""Forward one upstream connection to the browser until that connection ends."""
async with websockets.connect(
upstream_url, ping_interval=None, open_timeout=UPSTREAM_OPEN_TIMEOUT_SECONDS
) as upstream:
unrenderable = 0
async for message in upstream:
if not message:
continue
try:
html = transform(message)
except Exception:
# One unexpected payload must not take down a working page. Log
# the first per connection, then stay quiet about the rest.
unrenderable += 1
logger.log(
logging.WARNING if unrenderable == 1 else logging.DEBUG,
"Discarding an upstream message that could not be rendered",
exc_info=True,
)
continue
await browser_ws.send_text(html)
async def _ws_proxy(browser_ws: WebSocket, upstream_path: str, transform) -> None:
await browser_ws.accept() await browser_ws.accept()
upstream_url = _btcdata_ws_url(upstream_path) upstream_url = _btcdata_ws_url(upstream_path)
logger.info("Browser connected, proxying to %s", upstream_url) logger.info("Browser connected, proxying to %s", upstream_url)
browser_closed = asyncio.create_task(_await_browser_close(browser_ws))
attempt = 0
try: try:
async with websockets.connect(upstream_url, ping_interval=None) as upstream: while True:
async for message in upstream: started = time.monotonic()
if not message: forwarding = asyncio.create_task(
continue _forward_upstream(browser_ws, upstream_url, transform)
try: )
html = transform(message) done, _ = await asyncio.wait(
await browser_ws.send_text(html) {forwarding, browser_closed}, return_when=asyncio.FIRST_COMPLETED
except WebSocketDisconnect: )
break if browser_closed in done:
except Exception: # The browser left; a healthy upstream would otherwise keep this
logger.debug("Error forwarding to browser", exc_info=True) # task forwarding into a socket nobody is reading.
break forwarding.cancel()
except WebSocketDisconnect: await asyncio.gather(forwarding, return_exceptions=True)
pass logger.info("Browser disconnected from %s", upstream_url)
except Exception: return
logger.debug("Upstream connection error", exc_info=True) try:
forwarding.result()
logger.info("Upstream %s closed; reconnecting", upstream_url)
except Exception:
logger.info("Upstream %s unavailable; reconnecting", upstream_url)
logger.debug("Upstream connection error", exc_info=True)
if time.monotonic() - started >= UPSTREAM_STABLE_SECONDS:
# A connection that worked for a while means a restart, not a
# persistently broken upstream: start the schedule over.
attempt = 0
if attempt >= len(UPSTREAM_RETRY_DELAYS_SECONDS):
logger.warning(
"Upstream %s still unavailable; asking the browser to reconnect",
upstream_url,
)
return
await asyncio.wait({browser_closed}, timeout=UPSTREAM_RETRY_DELAYS_SECONDS[attempt])
attempt += 1
if browser_closed.done():
logger.info("Browser disconnected from %s", upstream_url)
return
finally: finally:
browser_closed.cancel()
await asyncio.gather(browser_closed, return_exceptions=True)
try: try:
await browser_ws.close() # Every server-side exit, expected or not, closes with a code the
# page retries; a browser that already left ignores it.
await browser_ws.close(code=BROWSER_RETRY_CLOSE_CODE)
except Exception: except Exception:
pass pass
+336
View File
@@ -0,0 +1,336 @@
# btcprice GitOps migration plan
**Status (2026-09-15):** Phases 1–6 implemented and verified. The GitOps deployment is live with
one Ready replica on each k3s node, trusted TLS, working HTTP/WebSockets, and a successful unattended
source-to-production update. Compose remains online for the required 24-hour validation/rollback
window; Phase 7 is intentionally pending.
**Target path**
```text
git push (magnus/btcprice)
-> Gitea Actions on anvil
-> Harbor image + Trivy scan
-> Flux image policy commits magnus/deploy
-> Flux reconciles k3s
-> ingress-nginx
-> hostinger Traefik TLS
-> https://btcprice.k8s.pmagnus.cloud
```
Production deployment must not use `docker compose`, `kubectl apply` from CI, or a CI-held
kubeconfig. The existing Compose instance on citadel is retained only during validation and as a
short rollback path.
## Access and secret handling
Use `/Users/pmagnus/pi/system/.env` for Gitea, Harbor, host and Cloudflare credentials. The file is
mode `0600` and must remain outside Git.
```bash
set -a
. ../system/.env
set +a
```
Rules:
- never echo tokens/passwords or put them in command arguments;
- pass secrets via API headers, stdin, or temporary `0600` files;
- use the Harbor CI robot for image pushes, the existing cluster pull robot for pulls, and Harbor
admin only for project policy changes;
- use the Gitea token for API administration, not from application or workflow code;
- do not copy `../system/.env` into this repository, an image, a pod, or `magnus/deploy`.
## Audited starting point (2026-09-15)
Already working:
- `magnus/btcprice` uses `main`; `.gitea/workflows/build.yml` tests and publishes immutable amd64
images from `anvil`.
- The latest Gitea run for commit `dd67eb7` succeeded.
- Harbor project `btcprice` is private, scans on push, blocks High-or-worse vulnerable images, has a
retention policy and weekly garbage collection. The latest image is scanned successfully:
`main-20260915160353` / `main-20260915160353-dd67eb7`.
- The repository has `HARBOR_ROBOT_USER` and `HARBOR_ROBOT_TOKEN` Actions secrets.
- The image exposes `/api/health`, dependency-aware `/api/ready`, and `/api/version`.
- Flux and all image automation controllers are healthy. The `hello` canary proves unattended image
update, reconciliation, two-node scheduling, ingress, and TLS.
- k3s has two Ready amd64 nodes: `srv1893094` and `citadel`.
- Both nodes already have Harbor pull credentials.
- ingress-nginx is managed by Flux and is reached through hostinger Traefik on NodePort 30080.
- btcdata still runs under Compose on citadel and answers on `100.75.8.131:4102`.
- The old btcprice Compose container remains available on citadel port 4041.
Current gaps:
- `magnus/deploy` has no btcprice resources or image policy.
- `btcprice.k8s.pmagnus.cloud` currently reaches Traefik but receives its default certificate and an
nginx 404; the TLS domain and Kubernetes Ingress do not exist yet.
- Local `../deploy` can lag Flux's automated commits. Always run `git pull --ff-only` before editing.
- Harbor's release retention rule currently keeps `v*`, while the workflow strips the leading `v`
(`v1.2.3` becomes `1.2.3`). Correct that mismatch before relying on release retention.
- The Gitea btcprice repository is currently public. Confirm that this is intentional; it does not
expose the Actions secrets, but it does expose source and workflow definitions.
## Phase 1 — close pre-deployment policy gaps
1. In Harbor, change the release retention rule to match the tags the workflow actually emits
(plain semantic versions and their SHA-qualified companion tags), or change the workflow to retain
the `v` prefix. Prefer changing retention so existing image naming remains stable.
2. Confirm `buildcache` remains explicitly retained and excluded from application deployment policy.
3. Confirm the exact latest digest is pullable on both k3s nodes after its Trivy scan passes.
4. Run `make test` locally and confirm the published image reports `dd67eb7` from `/api/version`.
5. Decide whether `magnus/btcprice` should remain public.
**Gate:** no Kubernetes manifests are merged until tests pass, the latest immutable image is approved
by Harbor, and both nodes can pull it.
## Phase 2 — represent the external btcdata dependency in Kubernetes
Create the following in `../deploy/apps/btcprice/`:
- `namespace.yaml`
- `btcdata-service.yaml`
- `btcdata-endpointslice.yaml`
Use namespace `btcprice`. Define a selectorless ClusterIP Service named `btcdata` on port `4102` and
an EndpointSlice with:
- label `kubernetes.io/service-name: btcdata`;
- address type `IPv4`;
- endpoint `100.75.8.131`;
- TCP port `4102`.
The application then uses only:
```text
BTCDATA_URL=http://btcdata:4102
```
Do not embed citadel's IP in the Deployment. This abstraction lets btcdata move into Kubernetes later
without changing btcprice.
Test from disposable pods pinned once to `srv1893094` and once to `citadel`:
- `http://btcdata:4102/api/health` returns 200;
- `/api/kraken-minute` and `/api/kraken-hour` return JSON;
- WebSocket connections to `/api/price/ws` and `/api/strike/ws` receive data.
**Gate:** all HTTP and WebSocket checks pass from both nodes.
## Phase 3 — add the btcprice workload to `magnus/deploy`
First synchronize the GitOps checkout:
```bash
git -C ../deploy pull --ff-only
git -C ../deploy switch main
```
Add this structure:
```text
apps/btcprice/
namespace.yaml
btcdata-service.yaml
btcdata-endpointslice.yaml
deployment.yaml
service.yaml
ingress.yaml
pdb.yaml
kustomization.yaml
infrastructure/image-automation/
btcprice-image.yaml
```
Also update:
- `apps/kustomization.yaml` to include `btcprice`;
- `infrastructure/image-automation/kustomization.yaml` to include `btcprice-image.yaml`.
### Deployment specification
Use:
- image `harbor.pmagnus.cloud/btcprice/btcprice:<known-good-main-tag>`;
- Flux marker `# {"$imagepolicy": "flux-system:btcprice"}` on the image;
- two replicas and container port `4040`;
- `BTCDATA_URL=http://btcdata:4102`;
- `terminationGracePeriodSeconds: 30`;
- rolling update with `maxUnavailable: 1` and `maxSurge: 0`;
- initial resources of approximately `50m` CPU / `64Mi` memory requested and `250m` CPU / `256Mi`
memory limited, then tune from observed metrics.
Guarantee one replica on each node with:
- required node affinity allowing only hostnames `srv1893094` and `citadel`;
- required pod anti-affinity on `kubernetes.io/hostname`, matching
`app.kubernetes.io/name: btcprice`.
Hard anti-affinity plus two replicas and two eligible nodes gives one pod per node. `maxSurge: 0`
prevents rollout deadlock because a third pod cannot satisfy that anti-affinity.
Use probes:
- startup: `GET /api/health`;
- liveness: `GET /api/health`;
- readiness: `GET /api/ready`, with Kubernetes timeout longer than the application's three-second
upstream timeout.
Never use dependency-aware readiness as liveness; a btcdata outage must not restart healthy
btcprice processes.
### Service, disruption budget, and ingress
- ClusterIP Service: port `80` to named container port `4040`.
- PDB: `minAvailable: 1`.
- Ingress class: `nginx`.
- Host: `btcprice.k8s.pmagnus.cloud`.
- Path: `/`, `Prefix`, backend `btcprice:80`.
- No Kubernetes TLS secret: TLS terminates at hostinger Traefik.
- Add ingress-nginx proxy read/send timeouts suitable for long-lived WebSockets (for example 3600
seconds). WebSocket upgrade support itself is automatic.
## Phase 4 — add Flux image automation
Follow the existing `hello` convention in `../deploy/infrastructure/image-automation/`:
- `ImageRepository` named `btcprice` in `flux-system`;
- image `harbor.pmagnus.cloud/btcprice/btcprice`;
- existing pull-only secret `harbor-pull`;
- one-minute scan interval;
- `ImagePolicy` matching only plain branch tags:
```yaml
filterTags:
pattern: '^main-(?P<ts>[0-9]{14})$'
extract: '$ts'
policy:
numerical:
order: asc
```
This intentionally excludes SHA-qualified tags, releases, and `buildcache`. The existing
`ImageUpdateAutomation/apps` object will update files under `./apps`, commit to `magnus/deploy`, and
cause Flux to reconcile. CI receives no GitOps write key or kubeconfig.
Validate manifests before pushing:
```bash
kubectl kustomize ../deploy/apps/btcprice >/dev/null
kubectl kustomize ../deploy/apps >/dev/null
kubectl kustomize ../deploy/infrastructure/image-automation >/dev/null
```
Commit and push only to `magnus/deploy`; do not manually apply the resources. Watch Flux and verify
that all Kustomizations remain Ready.
## Phase 5 — enable ingress TLS
On hostinger, edit the file-provider configuration mounted at:
```text
/home/pmp/apps/netbird/traefik-dynamic.yaml
```
Add this entry under `http.routers.k8s-ingress.tls.domains`:
```yaml
- main: "btcprice.k8s.pmagnus.cloud"
```
Use `SUDO_HOSTINGER` from `../system/.env` only if privilege escalation is required. Back up the file
first and preserve permissions. Do **not** restart Traefik: its file provider hot-reloads, and a
restart can interrupt ACME and NetBird traffic.
Wait for issuance and verify externally:
```bash
curl --fail https://btcprice.k8s.pmagnus.cloud/api/health
curl --fail https://btcprice.k8s.pmagnus.cloud/api/ready
```
The certificate must be publicly trusted and named for `btcprice.k8s.pmagnus.cloud`; do not accept the
current self-signed `TRAEFIK DEFAULT CERT`.
## Phase 6 — acceptance and unattended deployment test
### Initial deployment
Confirm:
- both Flux application and infrastructure Kustomizations are Ready;
- `ImageRepository`, `ImagePolicy`, and `ImageUpdateAutomation` are Ready;
- exactly two btcprice pods are Ready, one on each node;
- Service endpoints contain both pods;
- `/`, CSS, favicon, chart APIs, health, readiness, and version endpoints work externally;
- `/ws/price` and `/ws/strike` remain connected and update through ingress;
- the running `/api/version` equals the expected image's embedded short SHA.
### Full pipeline test
Push a harmless visible change to `magnus/btcprice` and observe, without manual deployment actions:
1. Gitea queues the workflow on `anvil`.
2. Tests pass.
3. Harbor receives the timestamp and SHA-qualified tags.
4. Trivy approves the artifact.
5. Flux image reflection selects the new plain timestamp tag.
6. Flux commits the image change to `magnus/deploy`.
7. kustomize-controller rolls out the Deployment.
8. `/api/version` changes to the pushed source SHA.
### Failure tests
- Delete each pod in turn; one endpoint remains available and the missing node gets a replacement.
- During a rollout, verify browser WebSockets reconnect after old pods terminate.
- Temporarily make btcdata unreachable in a controlled window; readiness becomes false, liveness
remains true, and no restart loop occurs.
- Verify a Harbor-rejected image cannot become a healthy rollout.
- Revert a GitOps image commit and confirm Flux restores the prior immutable image.
**Acceptance criterion:** a normal source push changes production with no SSH, Compose, or manual
`kubectl` deployment, while Git records the deployed image.
## Phase 7 — cut over from Compose
Keep `http://192.168.50.195:4041` running in parallel for at least 24 hours after all acceptance tests
pass. It must not own the new Kubernetes hostname.
Then on citadel:
1. Record the legacy image ID and effective environment for rollback without recording secrets.
2. Stop only `/home/pmp/apps/btcprice`; do not stop btcdata, Frankfurter, PostgreSQL, or remove
`btc-network`.
3. Verify no clients still use port 4041.
4. Keep `deploy-citadel.sh`, `docker-compose.citadel.yml`, and `.env.citadel` for one defined rollback
window, clearly marked legacy/manual-only.
5. After that window, remove the legacy deployment files and update `README.md`, `CLAUDE.md`, and
`../system/infrastructure.md` so GitOps is the only production path.
## Rollback
Application rollback:
1. Suspend the btcprice `ImagePolicy` or image automation if it would immediately reselect the bad
tag.
2. Revert the automated image commit in `magnus/deploy`, or commit the previous known-good immutable
tag.
3. Let Flux reconcile and verify readiness/version.
4. Re-enable automation only after the bad tag is no longer selectable.
Migration rollback while Compose is retained:
1. Remove or suspend the btcprice Ingress through Git and let Flux reconcile.
2. Restart only the recorded btcprice Compose service on citadel.
3. Restore clients temporarily to port 4041.
4. Diagnose without deleting Harbor artifacts or rewriting Git history.
## Known limitation and follow-up
Two frontend replicas do not make the full system highly available: btcdata and its database remain
single-node dependencies on citadel. If citadel fails, the hostinger btcprice pod may serve static
content but readiness and live data fail. Migrate btcdata and decide PostgreSQL storage/availability
in a separate change; then replace the selectorless Service and EndpointSlice with a normal
Kubernetes Service.
+2
View File
@@ -0,0 +1,2 @@
phase2 trigger 2026-09-15T15:39:58Z
hooks fixed 2026-09-15T15:41:50Z
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${BTCPRICE_DEPLOY_HOST:-citadel}"
APP_DIR="${BTCPRICE_DEPLOY_DIR:-/home/pmp/apps/btcprice}"
IMAGE_NAME="${BTCPRICE_IMAGE_NAME:-btcprice:latest}"
PLATFORM="${BTCPRICE_PLATFORM:-}"
COMPOSE_FILE="docker-compose.citadel.yml"
ENV_FILE=".env.citadel"
NETWORK="${BTCPRICE_NETWORK:-btc-network}"
FILES=("$COMPOSE_FILE" "$ENV_FILE")
if [[ -z "$PLATFORM" ]]; then
remote_arch="$(ssh "$HOST" uname -m)"
case "$remote_arch" in
x86_64 | amd64) PLATFORM="linux/amd64" ;;
aarch64 | arm64) PLATFORM="linux/arm64" ;;
*)
echo "Unsupported target architecture: $remote_arch" >&2
echo "Set BTCPRICE_PLATFORM manually, e.g. BTCPRICE_PLATFORM=linux/amd64" >&2
exit 1
;;
esac
fi
for file in "${FILES[@]}"; do
if [[ ! -f "$file" ]]; then
echo "Missing required file: $file" >&2
exit 1
fi
done
echo "Building $IMAGE_NAME locally for $PLATFORM..."
docker build --platform "$PLATFORM" -t "$IMAGE_NAME" .
echo "Preparing $APP_DIR on $HOST..."
ssh "$HOST" "mkdir -p '$APP_DIR'"
echo "Ensuring external network $NETWORK exists on $HOST..."
ssh "$HOST" "docker network inspect '$NETWORK' >/dev/null 2>&1 || docker network create '$NETWORK'"
echo "Copying $COMPOSE_FILE and $ENV_FILE to $HOST:$APP_DIR..."
scp "$COMPOSE_FILE" "$HOST:$APP_DIR/docker-compose.yml"
scp "$ENV_FILE" "$HOST:$APP_DIR/.env"
echo "Copying Docker image $IMAGE_NAME to $HOST..."
docker save "$IMAGE_NAME" | ssh "$HOST" docker load
echo "Starting docker compose on $HOST in $APP_DIR..."
ssh "$HOST" "cd '$APP_DIR' && docker compose up -d --no-build"
+19
View File
@@ -0,0 +1,19 @@
# Deployment compose for citadel. Copied to $APP_DIR/docker-compose.yml by
# deploy-citadel.sh, which also copies .env.citadel to $APP_DIR/.env.
# btcdata also runs on citadel, so both attach to the external btc-network and
# btcprice reaches it by service name.
networks:
btc-network:
name: btc-network
external: true
services:
btcprice:
image: btcprice:latest
restart: unless-stopped
networks:
- btc-network
ports:
- "${BTCPRICE_PORT:-4041}:4040"
environment:
BTCDATA_URL: "${BTCDATA_URL:?set BTCDATA_URL in .env}"
+2
View File
@@ -0,0 +1,2 @@
-r requirements.txt
pytest>=8.3
+1
View File
@@ -1,3 +1,4 @@
fastapi>=0.115 fastapi>=0.115
uvicorn[standard]>=0.34 uvicorn[standard]>=0.34
websockets>=14.0 websockets>=14.0
httpx>=0.27
File diff suppressed because one or more lines are too long
+128
View File
@@ -0,0 +1,128 @@
import httpx
from fastapi.testclient import TestClient
from app import main
class StubAsyncClient:
def __init__(self, response=None, error=None):
self.response = response
self.error = error
self.requested_url = None
self.timeout = None
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, traceback):
return None
async def get(self, url, timeout):
self.requested_url = url
self.timeout = timeout
if self.error:
raise self.error
return self.response
def test_health_is_independent_of_btcdata():
response = TestClient(main.app).get("/api/health")
assert response.status_code == 200
assert response.json() == {"status": "up"}
def test_home_updates_title_from_latest_price():
response = TestClient(main.app).get("/")
assert response.status_code == 200
assert "new MutationObserver(updateTitleFromPrice)" in response.text
assert "currentPrice - midnightPrice" in response.text
assert "document.title = price + ' | ' + formattedChange" in response.text
assert '<h1 class="text-3xl font-bold text-gray-900">BTC Price</h1>' in response.text
assert "page-title" not in response.text
assert 'id="midnight-change"' in response.text
assert "midnightChange.textContent = formattedChange" in response.text
assert "since 00:00" not in response.text
assert "change > 0 ? 'text-green-600' : change < 0 ? 'text-red-600'" in response.text
assert "formatPriceChange(change)" in response.text
assert "midnight.setHours(0, 0, 0, 0)" in response.text
def test_price_fragment_exposes_formatted_price_for_title():
html = main.price_json_to_html(
'{"price": 67432.1, "prev_price": 67400, "recorded_at": ""}'
)
assert 'data-title-price="$67.432,10"' in html
assert 'data-price-value="67432.1"' in html
assert ">$67.432,10</p>" in html
def test_ready_when_btcdata_is_healthy(monkeypatch):
request = httpx.Request("GET", "http://btcdata.test/api/health")
upstream = httpx.Response(200, request=request)
client = StubAsyncClient(response=upstream)
monkeypatch.setattr(main, "BTCDATA_URL", "http://btcdata.test")
monkeypatch.setattr(main.httpx, "AsyncClient", lambda: client)
response = TestClient(main.app).get("/api/ready")
assert response.status_code == 200
assert response.json() == {"status": "ready"}
assert client.requested_url == "http://btcdata.test/api/health"
assert client.timeout == main.READINESS_TIMEOUT_SECONDS
def test_not_ready_when_btcdata_is_unavailable(monkeypatch):
request = httpx.Request("GET", "http://btcdata.test/api/health")
error = httpx.ConnectError("connection refused", request=request)
client = StubAsyncClient(error=error)
monkeypatch.setattr(main, "BTCDATA_URL", "http://btcdata.test")
monkeypatch.setattr(main.httpx, "AsyncClient", lambda: client)
response = TestClient(main.app).get("/api/ready")
assert response.status_code == 503
assert response.json() == {"status": "not ready"}
assert "btcdata.test" not in response.text
def test_not_ready_when_btcdata_returns_an_error(monkeypatch):
request = httpx.Request("GET", "http://btcdata.test/api/health")
upstream = httpx.Response(503, request=request)
client = StubAsyncClient(response=upstream)
monkeypatch.setattr(main.httpx, "AsyncClient", lambda: client)
response = TestClient(main.app).get("/api/ready")
assert response.status_code == 503
assert response.json() == {"status": "not ready"}
def test_version_endpoint(monkeypatch):
monkeypatch.setattr(main, "APP_VERSION", "abc1234")
response = TestClient(main.app).get("/api/version")
assert response.status_code == 200
assert response.json() == {"version": "abc1234"}
def test_24_hour_chart_uses_half_hour_candles():
response = TestClient(main.app).get("/")
assert "24h Candles (30m)" in response.text
assert "initChart('chart-hour', '/api/kraken-half-hour', 5 * 60 * 1000, updateMidnightPrice)" in response.text
assert "initChart('chart-minute', '/api/kraken-minute', 60 * 1000)" in response.text
def test_half_hour_proxy_preserves_candles_and_upstream_errors(monkeypatch):
for status, payload in [(200, [{"ts": 1800, "open": "10", "high": "12", "low": "9", "close": "11"}]), (503, {"error": "database unavailable"})]:
request = httpx.Request("GET", "http://btcdata.test/api/kraken-half-hour")
client = StubAsyncClient(response=httpx.Response(status, json=payload, request=request))
monkeypatch.setattr(main, "BTCDATA_URL", "http://btcdata.test")
monkeypatch.setattr(main.httpx, "AsyncClient", lambda: client)
response = TestClient(main.app).get("/api/kraken-half-hour")
assert response.status_code == status
assert response.json() == payload
assert client.requested_url == "http://btcdata.test/api/kraken-half-hour"
+142
View File
@@ -0,0 +1,142 @@
"""Proxy recovery tests.
htmx reconnects a browser WebSocket only after an abnormal close, so a btcdata
restart must either be invisible to the browser or arrive as a retryable close
code. These tests run a real loopback WebSocket server as the stand-in btcdata:
the recovery being tested is connection lifecycle behaviour, which a stubbed
client would not exercise.
"""
import asyncio
import json
import socket
import threading
import pytest
import websockets
from fastapi.testclient import TestClient
from app import main
PRICE = json.dumps({"price": 67432.1, "prev_price": 67400, "recorded_at": ""})
LATER_PRICE = json.dumps({"price": 67500.0, "prev_price": 67432.1, "recorded_at": ""})
class FakeBtcdata:
"""Serves one list of messages per incoming connection.
Every session but the last closes as soon as its messages are sent, which is
what a btcdata restart looks like from the proxy's side. The last session
stays open so the test can assert on a live stream.
"""
def __init__(self, sessions):
self._sessions = [list(session) for session in sessions]
self._ready = threading.Event()
self.connections = 0
self.port = None
async def _handler(self, connection):
self.connections += 1
session = self._sessions.pop(0) if self._sessions else []
for message in session:
await connection.send(message)
if not self._sessions:
await connection.wait_closed()
async def _start(self):
# websockets.serve needs a running loop, so it cannot be called from
# run_until_complete's argument list.
self._server = await websockets.serve(self._handler, "127.0.0.1", 0)
self.port = self._server.sockets[0].getsockname()[1]
async def _stop(self):
self._server.close()
await self._server.wait_closed()
def _run(self):
self._loop = asyncio.new_event_loop()
asyncio.set_event_loop(self._loop)
self._loop.run_until_complete(self._start())
self._ready.set()
self._loop.run_forever()
self._loop.run_until_complete(self._stop())
self._loop.close()
def __enter__(self):
self._thread = threading.Thread(target=self._run, daemon=True)
self._thread.start()
assert self._ready.wait(10), "fake btcdata did not start"
return self
def stop(self):
"""Stop listening. Idempotent, so a test may stop the upstream early."""
if self._thread.is_alive():
self._loop.call_soon_threadsafe(self._loop.stop)
self._thread.join(10)
def __exit__(self, *exception):
self.stop()
return False
@pytest.fixture(autouse=True)
def quick_retries(monkeypatch):
"""Keep the real schedule's shape without waiting out its real delays."""
monkeypatch.setattr(main, "UPSTREAM_RETRY_DELAYS_SECONDS", (0.01, 0.01, 0.01))
def closed_port():
with socket.socket() as probe:
probe.bind(("127.0.0.1", 0))
return probe.getsockname()[1]
def test_upstream_restart_recovers_on_the_same_browser_socket(monkeypatch):
with FakeBtcdata([[PRICE], [LATER_PRICE]]) as upstream:
monkeypatch.setattr(main, "BTCDATA_URL", f"http://127.0.0.1:{upstream.port}")
with TestClient(main.app).websocket_connect("/ws/price") as browser:
assert "$67.432,10" in browser.receive_text()
# Same browser socket, no reload: the proxy reconnected upstream.
assert "$67.500,00" in browser.receive_text()
assert upstream.connections == 2
def test_browser_is_asked_to_reconnect_when_upstream_stays_down(monkeypatch):
monkeypatch.setattr(main, "BTCDATA_URL", f"http://127.0.0.1:{closed_port()}")
with TestClient(main.app).websocket_connect("/ws/price") as browser:
with pytest.raises(main.WebSocketDisconnect) as disconnect:
browser.receive_text()
# 1000 is the regression: htmx treats it as "done" and never reconnects.
assert disconnect.value.code == main.BROWSER_RETRY_CLOSE_CODE
assert disconnect.value.code in (1006, 1011, 1012, 1013)
def test_browser_is_asked_to_reconnect_after_a_lost_upstream(monkeypatch):
with FakeBtcdata([[PRICE]]) as upstream:
port = upstream.port
monkeypatch.setattr(main, "BTCDATA_URL", f"http://127.0.0.1:{port}")
with TestClient(main.app).websocket_connect("/ws/price") as browser:
assert "$67.432,10" in browser.receive_text()
# The fake stops listening entirely, so in-proxy retries cannot win.
upstream.stop()
with pytest.raises(main.WebSocketDisconnect) as disconnect:
browser.receive_text()
assert disconnect.value.code == main.BROWSER_RETRY_CLOSE_CODE
def test_unrenderable_message_does_not_end_the_stream(monkeypatch):
with FakeBtcdata([["not json", "", PRICE]]) as upstream:
monkeypatch.setattr(main, "BTCDATA_URL", f"http://127.0.0.1:{upstream.port}")
with TestClient(main.app).websocket_connect("/ws/price") as browser:
assert "$67.432,10" in browser.receive_text()
assert upstream.connections == 1
def test_strike_stream_recovers_on_the_same_browser_socket(monkeypatch):
rates = json.dumps({"BTC/USDT": 67432.1, "BTC/EUR": 61000.5, "quote-sats": 1476})
with FakeBtcdata([[rates], [rates]]) as upstream:
monkeypatch.setattr(main, "BTCDATA_URL", f"http://127.0.0.1:{upstream.port}")
with TestClient(main.app).websocket_connect("/ws/strike") as browser:
assert "1.476 <span" in browser.receive_text()
assert 'id="strike-display"' in browser.receive_text()
assert upstream.connections == 2