fix: make rclone sync more resilient to Scaleway throttling
Offsite sync hit 17 fatal IO errors deleting stale segment files after a large one-off prune batch (from the recent log/-exclude change), and rclone's default behavior is to bail immediately on delete errors rather than retry. Lower concurrency (--transfers 8->4, --checkers 16->8) to reduce rate-limit pressure, and make retries explicit (--retries 5, --retries-sleep 10s, --low-level-retries 20) instead of relying on defaults. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
+5
-2
@@ -298,8 +298,11 @@ step "Step 6: Syncing to ${RCLONE_REMOTE}:${RCLONE_PATH}"
|
||||
run_cmd timeout "$SYNC_TIMEOUT" \
|
||||
rclone sync -v \
|
||||
--fast-list \
|
||||
--transfers=8 \
|
||||
--checkers=16 \
|
||||
--transfers=4 \
|
||||
--checkers=8 \
|
||||
--retries 5 \
|
||||
--retries-sleep 10s \
|
||||
--low-level-retries 20 \
|
||||
--s3-no-check-bucket \
|
||||
--max-delete "$RCLONE_MAX_DELETE" \
|
||||
"$REPO" "${RCLONE_REMOTE}:${RCLONE_PATH}"
|
||||
|
||||
Reference in New Issue
Block a user