No more healthcheck
This commit is contained in:
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
||||
|
||||
## What this is
|
||||
|
||||
Backup + disaster-recovery tooling for a Linux server: Borg (encrypted, offsite-synced via rclone) backing up `/home/srv/files/content`, including MariaDB running in Docker. Plain bash, no build system, no CI. `RUNBOOK.md` is the operational doc (setup, deploy, cron, day-2 ops, recovery). `old/` holds the legacy scripts this replaced — reference only, not maintained.
|
||||
Backup + disaster-recovery tooling for a Linux server: Borg (encrypted, offsite-synced via rclone) backing up `/home/srv/files/content`, including MariaDB running in Docker. Plain bash, no build system, no CI, no test suite. `RUNBOOK.md` is the operational doc (setup, deploy, cron, day-2 ops, recovery).
|
||||
|
||||
- `borg-backup.sh` — daily backup orchestrator (cron). Never stops MariaDB: `dump_db.sh`'s `--single-transaction` dump is consistent on its own, and the raw data directory is excluded from the archive via a `.nobackup` marker file.
|
||||
- `dump_db.sh` — per-database `mysqldump`/`mariadb-dump` with atomic staging/swap. Deployed to `/home/srv/files/content/mariadb/dump_db.sh` (borg-backup.sh invokes it at that exact path).
|
||||
@@ -21,22 +21,12 @@ Backup + disaster-recovery tooling for a Linux server: Borg (encrypted, offsite-
|
||||
- `resolve_archive()` in `restore.sh` deliberately does *not* filter archives by hostname (unlike backup's prune/list, which does) — disaster recovery may run from a different host than made the backup.
|
||||
- Logs: `/var/log/borg/{backup,restore}-*.log` (override with `BORG_BACKUP_LOGFILE` / `RESTORE_LOGDIR` for local testing).
|
||||
|
||||
## Testing
|
||||
## Verifying changes
|
||||
|
||||
```
|
||||
RESTORE_LOGDIR=/tmp/restore-test-logs bash tests/test_restore.sh
|
||||
```
|
||||
|
||||
Plain bash, no framework (`assert_eq`/`assert_contains` + a `$FAILURES` counter in `tests/test_restore.sh`). `tests/lib/setup_mocks.sh` builds fake `borg`/`docker`/`mysql`/`mariadb`/`flock` executables into a temp dir.
|
||||
|
||||
Mock isolation requires all three of:
|
||||
- `PATH="$mockdir:$PATH"`
|
||||
- `BASH_ENV="$(mock_bash_env "$mockdir")"` — both scripts prepend a hardened `PATH` with system dirs (`/usr/local/bin` etc.) *ahead* of `$PATH`, so a real binary there would win over a PATH-only mock. `BASH_ENV` shell functions take priority regardless of PATH order.
|
||||
- Env overrides for `LOCKFILE` / `BORG_PASSPHRASE_FILE` / `ROOT_PASSWORD_FILE` pointed at throwaway paths, so `acquire_lock()`/`preflight()` never touch real `/var/lock` or `/root`.
|
||||
|
||||
`flock(1)` (util-linux) doesn't exist on macOS — that's why it's mocked too, not just for isolation.
|
||||
No test suite. Verify script changes with `bash -n <script>.sh` (syntax) and `shellcheck <script>.sh`, and by tracing the change against `RUNBOOK.md`'s documented behavior. Both `borg-backup.sh` and `restore.sh` prepend a hardened `PATH` (`/usr/local/sbin:/usr/local/bin:...:$PATH`) and require `flock`, `borg`, `docker`, and a `mysql`/`mariadb` client to actually exercise end-to-end — this only really runs on the target Linux server, not locally on macOS (`flock(1)` doesn't even exist there).
|
||||
|
||||
## Conventions
|
||||
|
||||
- Conventional commits (`feat:`, `fix:`, `docs:`, `chore:`). Direct commits to `master`, no PR workflow.
|
||||
- New restore/backup logic should mirror the existing style in `borg-backup.sh`/`restore.sh`: `log()`/`step()`/`die()`/`run_cmd()` helpers, `set -euo pipefail`, config constants up top.
|
||||
- Run `shellcheck` on any script you touch before considering it done.
|
||||
|
||||
+1
-6
@@ -47,8 +47,6 @@ Run once, by hand, on the server:
|
||||
# create a remote named "scaleway", type S3, matching your Scaleway
|
||||
# Object Storage credentials and region
|
||||
```
|
||||
7. Set a real healthcheck URL in `borg-backup.sh` (`HEALTHCHECK_URL=`), for
|
||||
example from https://healthchecks.io.
|
||||
|
||||
## 2. Deploying the Scripts
|
||||
|
||||
@@ -78,9 +76,6 @@ ls -lt /var/log/borg/backup-*.log | head -1 # latest log file
|
||||
tail -50 /var/log/borg/backup-*.log # inspect it
|
||||
```
|
||||
|
||||
Or watch the healthcheck dashboard configured in step 1.7 — a missed or
|
||||
failed run pages/alerts there.
|
||||
|
||||
## 4. Day-2 Operations
|
||||
|
||||
List archives:
|
||||
@@ -205,4 +200,4 @@ borg extract --lock-wait 600 "::$LATEST"
|
||||
Confirm the dump files under `mariadb/dump/` are present, non-empty, and
|
||||
importable (`mysql -u root -p < mariadb/dump/somedb.sql` against a
|
||||
throwaway MariaDB container). Log the drill date and outcome somewhere
|
||||
durable (e.g. the healthcheck dashboard's notes, or a team wiki page).
|
||||
durable (e.g. a team wiki page).
|
||||
|
||||
+1
-21
@@ -30,8 +30,6 @@ REPO_MOUNT=""
|
||||
LOGDIR="/var/log/borg"
|
||||
LOG_RETENTION_DAYS=90
|
||||
|
||||
HEALTHCHECK_URL="https://hc-ping.com/your-uuid-here"
|
||||
|
||||
RCLONE_REMOTE="scaleway"
|
||||
RCLONE_PATH="par-backup-1/$NAME"
|
||||
RCLONE_MAX_DELETE=200
|
||||
@@ -50,7 +48,7 @@ CHECK_TIMEOUT="4h"
|
||||
# Create it with: echo 'your-strong-passphrase' > /root/.borg-passphrase
|
||||
BORG_PASSPHRASE_FILE="${BORG_PASSPHRASE_FILE:-/root/.borg-passphrase}"
|
||||
|
||||
REQUIRED_CMDS=(borg rclone docker curl timeout flock date find)
|
||||
REQUIRED_CMDS=(borg rclone docker timeout flock date find)
|
||||
|
||||
# =================================================================
|
||||
|
||||
@@ -85,21 +83,6 @@ run_cmd() {
|
||||
"$@"
|
||||
}
|
||||
|
||||
send_healthcheck() {
|
||||
local status="${1:-0}"
|
||||
[[ -n "${HEALTHCHECK_URL:-}" ]] || return 0
|
||||
[[ "$HEALTHCHECK_URL" != *"your-uuid-here"* ]] || return 0
|
||||
command -v curl >/dev/null 2>&1 || return 0
|
||||
|
||||
local curl_opts=(-fsS -m 15 --retry 3 --retry-connrefused)
|
||||
case "$status" in
|
||||
start) curl "${curl_opts[@]}" "${HEALTHCHECK_URL}/start" >/dev/null || true ;;
|
||||
0) curl "${curl_opts[@]}" "$HEALTHCHECK_URL" >/dev/null || true ;;
|
||||
*) curl "${curl_opts[@]}" "${HEALTHCHECK_URL}/${status}" \
|
||||
--data-raw "exit=$status archive=$ARCHIVE log=$LOGFILE" >/dev/null || true ;;
|
||||
esac
|
||||
}
|
||||
|
||||
container_running() {
|
||||
[[ "$(docker inspect -f '{{.State.Running}}' "$1" 2>/dev/null || echo false)" == "true" ]]
|
||||
}
|
||||
@@ -154,7 +137,6 @@ cleanup() {
|
||||
log "=== Backup FAILED (exit $exit_code) ==="
|
||||
fi
|
||||
log "Full log: $LOGFILE"
|
||||
send_healthcheck "$exit_code"
|
||||
}
|
||||
|
||||
# Returns 0 if the repo is encrypted, 1 if not.
|
||||
@@ -230,8 +212,6 @@ echo "Target: $TARGET"
|
||||
echo "Archive: $ARCHIVE"
|
||||
echo "Log: $LOGFILE"
|
||||
|
||||
send_healthcheck start
|
||||
|
||||
step "Step 0: Preflight"
|
||||
preflight
|
||||
|
||||
|
||||
Reference in New Issue
Block a user