Ignore files, remove old and add CLAUDE.md

This commit is contained in:
kbe
2026-07-25 21:16:30 +02:00
parent 592ef45bab
commit 86c7731d0c
7 changed files with 44 additions and 509 deletions
+2
View File
@@ -0,0 +1,2 @@
CLAUDE.local.md
codedb.snapshot
+42
View File
@@ -0,0 +1,42 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## What this is
Backup + disaster-recovery tooling for a Linux server: Borg (encrypted, offsite-synced via rclone) backing up `/home/srv/files/content`, including MariaDB running in Docker. Plain bash, no build system, no CI. `RUNBOOK.md` is the operational doc (setup, deploy, cron, day-2 ops, recovery). `old/` holds the legacy scripts this replaced — reference only, not maintained.
- `borg-backup.sh` — daily backup orchestrator (cron). Never stops MariaDB: `dump_db.sh`'s `--single-transaction` dump is consistent on its own, and the raw data directory is excluded from the archive via a `.nobackup` marker file.
- `dump_db.sh` — per-database `mysqldump`/`mariadb-dump` with atomic staging/swap. Deployed to `/home/srv/files/content/mariadb/dump_db.sh` (borg-backup.sh invokes it at that exact path).
- `restore.sh` — recovery CLI: `full` (disaster recovery), `db <name>`, `file <path>`, `--list-archives`, all supporting `--dry-run`.
## Global conventions (don't re-derive these)
- `TARGET=/home/srv/files/content`, `REPO=/home/srv/files/backups/borg-2025`, `DB_CONTAINER=mariadb`
- `BORG_PASSPHRASE_FILE=/root/.borg-passphrase` (chmod 600, read via `BORG_PASSCOMMAND`)
- Backup DB user `backup` (password file `/root/.mariadb-backup.pw`, limited SELECT/LOCK grants) is separate from restore's `root` creds (`MYSQL_ROOT_PASSWORD` env or `/root/.mariadb-root.pw`) — restore needs CREATE/DROP privileges the backup user doesn't have.
- Dumps live at `mariadb/dump/*.sql` plus `00-users-and-grants.sql`, inside `$TARGET`.
- `restore.sh` and `borg-backup.sh` share one lockfile (`/var/lock/borg-backup.lock` via `flock -n 9`) so a restore and the nightly backup can never run concurrently.
- Destructive-op safety: `full` refuses a non-empty `$TARGET` without `--force`; `db` requires typed dbname confirmation unless `--yes`; every mode supports `--dry-run`.
- `resolve_archive()` in `restore.sh` deliberately does *not* filter archives by hostname (unlike backup's prune/list, which does) — disaster recovery may run from a different host than made the backup.
- Logs: `/var/log/borg/{backup,restore}-*.log` (override with `BORG_BACKUP_LOGFILE` / `RESTORE_LOGDIR` for local testing).
## Testing
```
RESTORE_LOGDIR=/tmp/restore-test-logs bash tests/test_restore.sh
```
Plain bash, no framework (`assert_eq`/`assert_contains` + a `$FAILURES` counter in `tests/test_restore.sh`). `tests/lib/setup_mocks.sh` builds fake `borg`/`docker`/`mysql`/`mariadb`/`flock` executables into a temp dir.
Mock isolation requires all three of:
- `PATH="$mockdir:$PATH"`
- `BASH_ENV="$(mock_bash_env "$mockdir")"` — both scripts prepend a hardened `PATH` with system dirs (`/usr/local/bin` etc.) *ahead* of `$PATH`, so a real binary there would win over a PATH-only mock. `BASH_ENV` shell functions take priority regardless of PATH order.
- Env overrides for `LOCKFILE` / `BORG_PASSPHRASE_FILE` / `ROOT_PASSWORD_FILE` pointed at throwaway paths, so `acquire_lock()`/`preflight()` never touch real `/var/lock` or `/root`.
`flock(1)` (util-linux) doesn't exist on macOS — that's why it's mocked too, not just for isolation.
## Conventions
- Conventional commits (`feat:`, `fix:`, `docs:`, `chore:`). Direct commits to `master`, no PR workflow.
- New restore/backup logic should mirror the existing style in `borg-backup.sh`/`restore.sh`: `log()`/`step()`/`die()`/`run_cmd()` helpers, `set -euo pipefail`, config constants up top.
BIN
View File
Binary file not shown.
-145
View File
@@ -1,145 +0,0 @@
#!/bin/bash
# =============================================================================
# Borg Backup Script - MariaDB Restart Guaranteed
# =============================================================================
set -euo pipefail
# ========================= CONFIGURATION =========================
NAME="borg-2025"
REPO="/home/srv/files/backups/$NAME"
TARGET="/home/srv/files/content"
LOGDIR="/var/log/borg"
LOGFILE="${LOGDIR}/backup-$(date +%Y-%m-%d-%H%M%S).log"
# Optional but strongly recommended
HEALTHCHECK_URL="https://hc-ping.com/your-uuid-here"
RCLONE_REMOTE="scaleway"
# =================================================================
mkdir -p "$LOGDIR"
exec > >(tee -a "$LOGFILE")
exec 2>&1
echo "=== Backup started at $(date '+%Y-%m-%d %H:%M:%S') ==="
echo "Repository: $REPO"
echo "Target: $TARGET"
echo "Log: $LOGFILE"
echo "-----------------------------------------------------------"
# ----------------------- Lockfile -----------------------
LOCKFILE="/var/lock/borg-backup.lock"
MARIADB_STOPPED=false
cleanup() {
local exit_code=${1:-$?}
if [ "$MARIADB_STOPPED" = true ]; then
echo "=== Cleanup: Starting MariaDB container ==="
docker start mariadb || echo "WARNING: Failed to start mariadb container"
sleep 3
echo "MariaDB restart completed."
fi
rm -f "$LOCKFILE"
if [ "$exit_code" -eq 0 ]; then
echo "=== Backup completed SUCCESSFULLY at $(date '+%Y-%m-%d %H:%M:%S') ==="
send_healthcheck
else
echo "=== Backup FAILED at $(date '+%Y-%m-%d %H:%M:%S') (exit code $exit_code) ==="
send_healthcheck "fail"
fi
echo "Full log: $LOGFILE"
}
if [ -e "$LOCKFILE" ]; then
echo "ERROR: Another backup is already running (lockfile exists)"
echo "Remove it manually if stale: $LOCKFILE"
exit 1
fi
touch "$LOCKFILE"
trap 'cleanup $?' EXIT
# ----------------------- Borg & Functions -----------------------
export BORG_REPO="$REPO"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes
send_healthcheck() {
local status="${1:-}"
if [[ -n "${HEALTHCHECK_URL:-}" ]] && [[ "$HEALTHCHECK_URL" != *"your-uuid-here"* ]]; then
case "$status" in
start) curl -s -m 10 --retry 3 "${HEALTHCHECK_URL}/start" >/dev/null || true ;;
fail) curl -s -m 10 --retry 3 "${HEALTHCHECK_URL}/fail" -d "Backup failed - check $LOGFILE" >/dev/null || true ;;
*) curl -s -m 10 --retry 3 "$HEALTHCHECK_URL" >/dev/null || true ;;
esac
fi
}
run_cmd() {
echo "[RUN] $*"
"$@"
}
# =================================================================
send_healthcheck "start"
# 1. MariaDB dump (while running)
echo "=== Step 1: MariaDB dump ==="
DUMP_SCRIPT="${TARGET}/mariadb/dump_db.sh"
if [[ -x "$DUMP_SCRIPT" ]]; then
run_cmd "$DUMP_SCRIPT"
else
echo "WARNING: Dump script not found or not executable: $DUMP_SCRIPT"
fi
# 2. Stop MariaDB for consistent backup
echo "=== Step 2: Stopping MariaDB container ==="
run_cmd docker stop mariadb
MARIADB_STOPPED=true
# 3. Create Borg archive
echo "=== Step 3: Creating Borg archive ==="
run_cmd borg create \
--stats \
--progress \
--list \
--filter=AME \
--compression zstd,8 \
--exclude-caches \
--exclude-if-present .nobackup \
"::${now:%Y-%m-%dT%H-%M-%S}" \
"$TARGET"
# 4. Restart MariaDB (this line is now also in the cleanup trap)
echo "=== Step 4: Starting MariaDB container ==="
run_cmd docker start mariadb
MARIADB_STOPPED=false
# 5. Offsite sync
echo "=== Step 5: Syncing to Scaleway S3 ==="
run_cmd rclone sync -v \
--fast-list \
--transfers=8 \
--checkers=16 \
"$REPO" "${RCLONE_REMOTE}:/par-backup-1/$NAME"
# 6. Prune & Compact
echo "=== Step 6: Pruning and compacting ==="
run_cmd borg prune \
--list \
--keep-daily=7 \
--keep-weekly=4 \
--keep-monthly=6 \
--keep-within=7d \
"$REPO"
run_cmd borg compact --progress "$REPO"
# The cleanup trap will run automatically and mark success
-36
View File
@@ -1,36 +0,0 @@
root@chaudron:/home/srv/files/content/mariadb# cat dump_db.sh
#!/bin/bash
# Load configuration from environment variables
MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD:-MySuperDatabase}
DUMP_DIR=${DUMP_DIR:-dump}
DOCKER_CONTAINER_NAME=${DOCKER_CONTAINER_NAME:-mariadb}
# Check dependencies
if ! command -v docker &>/dev/null; then
echo "Docker is not installed or not in the PATH"
exit 1
fi
if ! docker ps -q -f name=$DOCKER_CONTAINER_NAME; then
echo "Docker container '$DOCKER_CONTAINER_NAME' is not running"
exit 1
fi
# Create dump directory if it doesn't exist
mkdir -p "$(dirname "$0")/$DUMP_DIR"
# Delete old dump files
rm -f "$(dirname "$0")/$DUMP_DIR"/*.sql
# Get list of databases and exclude system databases
databases=$(docker exec -it $DOCKER_CONTAINER_NAME mysql -u root -p"$MYSQL_ROOT_PASSWORD" -e "SHOW DATABASES;" --skip-column-names -s | grep -Ev "(information_schema|mysql|performance_schema|sys)" | tr -d '\r')
# Iterate through databases and dump them individually
for db in $databases; do
echo "Dumping database: $db"
docker exec -i $DOCKER_CONTAINER_NAME mysqldump -u root -p"$MYSQL_ROOT_PASSWORD" "$db" >"$(dirname "$0")/$DUMP_DIR/$db.sql"
echo "Dumped database: $db"
done
echo "All non-system databases dumped to individual files in '$DUMP_DIR' directory."
-110
View File
@@ -1,110 +0,0 @@
#!/bin/bash
# Builds fake borg/docker/mysql/mariadb executables into $1 so restore.sh
# can be exercised without real infrastructure. Each mock logs its
# invocation to $1/mock.log.
setup_mock_bin() {
local dir="$1"
mkdir -p "$dir"
local mocklog="$dir/mock.log"
: > "$mocklog"
cat > "$dir/borg" <<EOF
#!/bin/bash
echo "borg \$*" >> "$mocklog"
case "\$1" in
list)
echo "host-2026-01-01T00-00-00"
echo "host-2026-06-01T00-00-00"
;;
extract)
shift
paths=()
for a in "\$@"; do
case "\$a" in
-*) ;;
*::*) ;;
*) paths+=("\$a") ;;
esac
done
for p in "\${paths[@]}"; do
if [[ "\$p" == *.* ]]; then
mkdir -p "\$(dirname "\$p")"
printf 'mock-content\n' > "\$p"
else
mkdir -p "\$p"
touch "\$p/RESTORED_MARKER"
fi
done
;;
*) ;;
esac
exit 0
EOF
cat > "$dir/docker" <<EOF
#!/bin/bash
echo "docker \$*" >> "$mocklog"
case "\$1" in
inspect)
if [[ "\$*" == *"State.Running"* ]]; then
echo true
else
echo healthy
fi
;;
start|stop)
exit 0
;;
exec)
case "\$*" in
*"command -v mariadb"*) exit 0 ;;
*) cat >/dev/null 2>&1 || true; exit 0 ;;
esac
;;
*) ;;
esac
exit 0
EOF
cat > "$dir/mysql" <<EOF
#!/bin/bash
echo "mysql \$*" >> "$mocklog"
cat >/dev/null 2>&1 || true
exit 0
EOF
cp "$dir/mysql" "$dir/mariadb"
# flock(1) is Linux-only (util-linux) and absent on macOS dev machines;
# restore.sh only ever uses the "flock -n FD" form (never the
# command-wrapping form), so a no-op success is a faithful stand-in for
# exclusion testing purposes - no test here exercises actual contention.
cat > "$dir/flock" <<EOF
#!/bin/bash
echo "flock \$*" >> "$mocklog"
exit 0
EOF
chmod +x "$dir/borg" "$dir/docker" "$dir/mysql" "$dir/mariadb" "$dir/flock"
}
# restore.sh hardens PATH with system dirs (/usr/local/bin etc.) placed
# ahead of $PATH, so `PATH="$mockdir:$PATH" ...` alone does NOT guarantee the
# mock is what actually runs: a real borg/docker/mysql/mariadb installed in
# one of those system dirs would shadow it. BASH_ENV is sourced by bash
# before running a script and shell functions win over PATH lookup for
# simple commands regardless of PATH ordering, so this is what actually
# guarantees isolation on any machine, not just ones that happen to lack
# those binaries in the hardened prefix.
mock_bash_env() {
local dir="$1"
local bashenv="$dir/bash_env.sh"
cat > "$bashenv" <<EOF
borg() { "$dir/borg" "\$@"; }
docker() { "$dir/docker" "\$@"; }
mysql() { "$dir/mysql" "\$@"; }
mariadb() { "$dir/mariadb" "\$@"; }
flock() { "$dir/flock" "\$@"; }
EOF
echo "$bashenv"
}
-218
View File
@@ -1,218 +0,0 @@
#!/bin/bash
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
RESTORE="$HERE/../restore.sh"
FAILURES=0
assert_eq() {
local expected="$1" actual="$2" msg="$3"
if [[ "$expected" != "$actual" ]]; then
echo "FAIL: $msg (expected '$expected', got '$actual')"
FAILURES=$((FAILURES + 1))
else
echo "PASS: $msg"
fi
}
assert_contains() {
local haystack="$1" needle="$2" msg="$3"
if [[ "$haystack" != *"$needle"* ]]; then
echo "FAIL: $msg (expected to contain '$needle', got '$haystack')"
FAILURES=$((FAILURES + 1))
else
echo "PASS: $msg"
fi
}
test_help_exits_zero() {
local out rc
out="$(bash "$RESTORE" -h)"
rc=$?
assert_eq "0" "$rc" "help exits 0"
assert_contains "$out" "Usage:" "help prints usage"
}
test_no_args_exits_one() {
local rc
bash "$RESTORE" >/dev/null 2>&1
rc=$?
assert_eq "1" "$rc" "no args exits 1"
}
test_unknown_command_exits_one() {
local rc
bash "$RESTORE" bogus >/dev/null 2>&1
rc=$?
assert_eq "1" "$rc" "unknown command exits 1"
}
source "$HERE/lib/setup_mocks.sh"
# Every test below isolates two things beyond PATH: BASH_ENV forces borg/
# docker/mysql/mariadb to the mock regardless of what's really installed on
# this machine's PATH (restore.sh's own hardened PATH would otherwise shadow
# the mock with any real binary in a system dir), and LOCKFILE/
# BORG_PASSPHRASE_FILE point at throwaway files so acquire_lock()/preflight()
# never touch real system paths like /var/lock or /root.
test_list_archives() {
local mockdir out
mockdir="$(mktemp -d)"
setup_mock_bin "$mockdir"
out="$(PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" bash "$RESTORE" --list-archives)"
assert_contains "$out" "host-2026-01-01T00-00-00" "list-archives shows first archive"
assert_contains "$out" "host-2026-06-01T00-00-00" "list-archives shows latest archive"
rm -rf "$mockdir"
}
test_file_restore_extracts_to_dest() {
local mockdir dest out final passfile lockfile
mockdir="$(mktemp -d)"
dest="$(mktemp -d)"
setup_mock_bin "$mockdir"
passfile="$mockdir/passphrase"
lockfile="$mockdir/lock"
echo "s3cr3t" > "$passfile"
out="$(PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" \
BORG_PASSPHRASE_FILE="$passfile" LOCKFILE="$lockfile" \
bash "$RESTORE" file photos/img.jpg --dest "$dest")"
final="$dest/home/srv/files/content/photos/img.jpg"
assert_contains "$out" "Restored file available at: $final" "file mode reports final path"
if [[ -f "$final" ]]; then
echo "PASS: extracted file exists on disk"
else
echo "FAIL: extracted file missing at $final"
FAILURES=$((FAILURES + 1))
fi
rm -rf "$mockdir" "$dest"
}
test_file_restore_dry_run_makes_no_borg_call() {
local mockdir dest
mockdir="$(mktemp -d)"
dest="$(mktemp -d)"
setup_mock_bin "$mockdir"
PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" LOCKFILE="$mockdir/lock" \
bash "$RESTORE" file photos/img.jpg --dest "$dest" --dry-run >/dev/null
if [[ -s "$mockdir/mock.log" ]] && grep -q "^borg extract" "$mockdir/mock.log"; then
echo "FAIL: dry-run invoked borg extract"
FAILURES=$((FAILURES + 1))
else
echo "PASS: dry-run made no borg extract call"
fi
rm -rf "$mockdir" "$dest"
}
test_db_restore_with_yes_runs_full_sequence() {
local mockdir out passfile
mockdir="$(mktemp -d)"
setup_mock_bin "$mockdir"
echo "rootpass" > "$mockdir/rootpw"
passfile="$mockdir/passphrase"
echo "s3cr3t" > "$passfile"
out="$(PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" \
ROOT_PASSWORD_FILE="$mockdir/rootpw" BORG_PASSPHRASE_FILE="$passfile" \
LOCKFILE="$mockdir/lock" \
bash "$RESTORE" db shopdb --yes </dev/null)"
assert_contains "$out" "Database 'shopdb' restored from" "db mode reports success"
if grep -q "CREATE DATABASE IF NOT EXISTS" "$mockdir/mock.log"; then
echo "PASS: db mode issued CREATE DATABASE"
else
echo "FAIL: db mode did not issue CREATE DATABASE"
FAILURES=$((FAILURES + 1))
fi
rm -rf "$mockdir"
}
test_db_restore_dry_run_skips_confirmation_and_calls() {
local mockdir out
mockdir="$(mktemp -d)"
setup_mock_bin "$mockdir"
out="$(PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" LOCKFILE="$mockdir/lock" \
bash "$RESTORE" db shopdb --dry-run </dev/null)"
assert_contains "$out" "DRY-RUN" "db dry-run prints DRY-RUN plan"
# cmd_db calls resolve_archive() (a borg list call) before checking DRY_RUN,
# same as cmd_file does, so mock.log legitimately gets a "borg list" entry.
# What must NOT happen in dry-run is any docker/mysql/mariadb interaction.
if grep -qE "^(docker|mysql|mariadb)" "$mockdir/mock.log"; then
echo "FAIL: dry-run invoked a docker/db-client mock binary"
FAILURES=$((FAILURES + 1))
else
echo "PASS: dry-run made no docker/db-client calls"
fi
rm -rf "$mockdir"
}
test_db_restore_aborts_on_wrong_confirmation() {
local mockdir rc
mockdir="$(mktemp -d)"
setup_mock_bin "$mockdir"
set +e
echo "wrongname" | PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" LOCKFILE="$mockdir/lock" \
bash "$RESTORE" db shopdb >/dev/null 2>&1
rc=$?
set -e
assert_eq "1" "$rc" "db mode aborts on mismatched confirmation"
rm -rf "$mockdir"
}
test_full_restore_refuses_nonempty_target_without_force() {
local mockdir target out rc
mockdir="$(mktemp -d)"
target="$(mktemp -d)"
touch "$target/existing-file"
setup_mock_bin "$mockdir"
set +e
out="$(PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" LOCKFILE="$mockdir/lock" bash -c '
sed "s#^TARGET=\"/home/srv/files/content\"#TARGET=\"'"$target"'\"#; s@^ARCHIVE_TARGET_PATH=.*@ARCHIVE_TARGET_PATH=\"\${TARGET#/}\"@" "'"$RESTORE"'" > "'"$mockdir"'/restore_patched.sh"
bash "'"$mockdir"'/restore_patched.sh" full
' 2>&1)"
rc=$?
set -e
assert_eq "1" "$rc" "full mode refuses non-empty target without --force"
assert_contains "$out" "is not empty - pass --force" "refusal message names the reason"
rm -rf "$mockdir" "$target"
}
test_full_restore_dry_run_makes_no_calls() {
local mockdir target out
mockdir="$(mktemp -d)"
target="$(mktemp -d)"
setup_mock_bin "$mockdir"
out="$(PATH="$mockdir:$PATH" BASH_ENV="$(mock_bash_env "$mockdir")" LOCKFILE="$mockdir/lock" bash -c '
sed "s#^TARGET=\"/home/srv/files/content\"#TARGET=\"'"$target"'\"#; s@^ARCHIVE_TARGET_PATH=.*@ARCHIVE_TARGET_PATH=\"\${TARGET#/}\"@" "'"$RESTORE"'" > "'"$mockdir"'/restore_patched.sh"
bash "'"$mockdir"'/restore_patched.sh" full --dry-run
')"
assert_contains "$out" "DRY-RUN" "full dry-run prints DRY-RUN plan"
# cmd_full calls resolve_archive() (a borg list call) before checking
# DRY_RUN, same as cmd_db/cmd_file, so mock.log legitimately gets a
# "borg list" entry. What must NOT happen in dry-run is a borg extract,
# or any docker/db-client interaction.
if grep -qE "^(borg extract|docker|mysql|mariadb)" "$mockdir/mock.log" 2>/dev/null; then
echo "FAIL: full dry-run invoked borg extract or a docker/db-client mock binary"
FAILURES=$((FAILURES + 1))
else
echo "PASS: full dry-run made no borg extract or docker/db-client calls"
fi
rm -rf "$mockdir" "$target"
}
test_help_exits_zero
test_no_args_exits_one
test_unknown_command_exits_one
test_list_archives
test_file_restore_extracts_to_dest
test_file_restore_dry_run_makes_no_borg_call
test_db_restore_with_yes_runs_full_sequence
test_db_restore_dry_run_skips_confirmation_and_calls
test_db_restore_aborts_on_wrong_confirmation
test_full_restore_refuses_nonempty_target_without_force
test_full_restore_dry_run_makes_no_calls
echo "-----"
if [[ "$FAILURES" -gt 0 ]]; then
echo "$FAILURES failure(s)"
exit 1
fi
echo "All tests passed"