rsync, scp, and sftp: Incremental Sync, Delete Flags, and Bandwidth Limits
Why the trailing slash decides whether rsync merges or mirrors, how --delete behaves with filters, safe resume with --partial, and where scp and sftp still beat a delta transfer.

Quick Summary
rsync, scp, and sftp all move files over SSH, and they are not interchangeable. rsync computes a delta: it compares the destination with the source using size and modification time by default (or a full checksum with -c), then transfers only what differs, with optional compression, deletion of stale destination files, and a bandwidth cap. scp copies whole files with almost no options and no resume. sftp is an interactive or batch file-transfer protocol with get, put, reget, and chmod, useful when you need a few files and no synchronization logic.
The single most consequential detail in rsync is the trailing slash: rsync -a src/ dest/ copies the contents of src into dest, while rsync -a src dest/ creates dest/src. That one character decides whether a --delete run merges a tree or mirrors it — and --delete is the flag that has erased more production directories than any other option in this list.
Bandwidth limits are --bwlimit=RATE in kilobytes per second (rsync 3 accepts K, M, G suffixes), applied per connection, so a mirror over a shared link stays polite. This guide covers the option semantics that matter (-a versus -aHAX, --delete variants, --partial, --link-dest), the scp/scp-versus-sftp differences in modern OpenSSH, safe dry-run practice, and the exit codes that tell you whether a transfer actually converged.
Basic Syntax & Core Signatures
# Local copy: archive mode, verbose, human-readable
rsync -av src/ dst/
# Remote pull / push (SSH transport is the default when a host is given)
rsync -av user@host:/srv/app/ ./app/
rsync -av --delete ./app/ user@host:/srv/app/
# Custom port and a non-standard remote binary
rsync -avz -e 'ssh -p 2222 -i ~/.ssh/deploy' ./dist/ user@host:/var/www/dist/
rsync -av --rsync-path='sudo rsync' ./etc/ user@host:/etc/
# Progress and resumability
rsync -avh --progress --partial src/ dst/
rsync -a --info=progress2 --stats src/ dst/
# Bandwidth limit: 5000 KB/s (~5 MB/s), per connection
rsync -av --bwlimit=5000 big/ user@host:/backup/big/
rsync -av --bwlimit=2M big/ user@host:/backup/big/
# Dry run first: prints what would change, transfers nothing
rsync -avn --delete --itemize-changes ./app/ user@host:/srv/app/
# Excludes and includes (evaluated in order)
rsync -av --exclude='node_modules/' --exclude='*.log' \
--include='logs/' --include='logs/*.json' --exclude='logs/*' \
./app/ /backup/app/
rsync -av --exclude-from=.rsyncignore --files-from=filelist.txt ./ /backup/# scp: whole-file copy, port is -P, -p preserves modes and times
scp file.txt user@host:/tmp/
scp -r ./dir user@host:/srv/
scp -P 2222 -p user@host:/var/log/app.log ./
scp -C user@host:/data/dump.sql ./dump.sql # on-the-fly compression
# sftp: batch mode, scripted
sftp -b - user@host <<'EOF'
put -r ./release /var/www/release
reget /var/log/app.log ./app.log
chmod 640 /var/www/release/config.yml
EOF
sftp -P 2222 user@host # interactive: ls, cd, get, put, reget, df, llsExit codes from rsync that change how a script must react:
0 success
1 syntax or usage error
2 protocol incompatibility
3 file selection error
5 client error starting the protocol
10 socket I/O error
11 file I/O error (disk full, permissions)
12 protocol data stream error
13 errors with diagnostics
20 SIGUSR1 or SIGINT received
23 partial transfer (some files were not copied)
24 partial transfer due to vanished source files
30 timeout in data send/receive
35 timeout waiting for the daemon connectionKey Comparison Table
| Aspect | rsync | scp | sftp |
|---|---|---|---|
| Transfer model | Delta (size+mtime, or -c checksum) | Whole file | Whole file, resumable with reget |
| Resume a broken transfer | --partial --append-verify | No | reget / reput |
| Delete destination extras | --delete | No | rm manually |
| Bandwidth limit | --bwlimit | trickle externally | -l limits throughput |
| Compression | -z (skip for pre-compressed data) | -C | -C |
| Preserve owner/group | -o -g as root, --numeric-ids | -p (modes/times only) | chmod after |
| ACLs / xattrs | -A -X (-aHAX) | No | No |
| Excludes / filters | Rich filter rules | No | No |
| Symlink handling | -l (as link), -L | Follows by default | Follows |
| Incremental snapshots | --link-dest | No | No |
| Scriptable progress | --info=progress2, --stats | None | Progress in batch output |
| Port flag | -e 'ssh -p 2222' | -P 2222 | -P 2222 |
| Protocol in OpenSSH 9+ | SSH | SFTP by default (-O restores legacy) | SFTP |
| Option | What it actually does |
|---|---|
-a | -r -l -p -t -g -o -D; archives but does not include ACLs or xattrs |
-z | Compresses the wire; wastes CPU on local copies and on .gz/.zst sources |
--delete | Removes destination files absent from the source, during the transfer by default |
--delete-delay | Decides deletions at the end; lower peak deletion risk on full disks |
--delete-excluded | Also deletes destination files matching --exclude — rarely what you want |
-c | Compares by checksum instead of size+mtime; slow, catches same-size edits |
-u | Skips files that are newer on the receiver |
--inplace | Writes into the destination file instead of a temp file; no atomic replace |
--partial-dir=.rsync-partial | Keeps partial files next to the target for later resume |
--numeric-ids | Transfers UID/GID numbers, not names; correct for cross-host restores |
Practical Edge Cases & Deep Dives
1. The trailing slash decides whether --delete merges or mirrors
# Mirrors: dst becomes an exact copy of the contents of src, extra files deleted
rsync -av --delete /srv/app/ /backup/app/
# Copies the directory itself: /backup/app/app/... is created
rsync -av --delete /srv/app /backup/app/The second form with --delete deletes nothing extra inside /backup/app — it adds a nested directory and leaves stale files alone, which is how "the backup is not actually pruning" incidents happen. Write both paths in the same style (both with, or both without a trailing slash) and dry run with -n --itemize-changes before trusting a new command line:
rsync -an --delete --itemize-changes /srv/app/ /backup/app/
# *deleting old/file.txt
# >f+++++++++ new/file.txt--delete never runs while the transfer is failing for another reason unless you pass --ignore-errors, and it is refused entirely if the source path has a wildcard that matched nothing. Note also that --delete interacts with filters: an excludes rule protects nothing on the destination unless --delete-excluded is absent (it is absent by default) — so excluded files survive, which is usually what protects caches and .env files.
2. Why an unchanged file transfers again: mtime, size, and checksums
# Default quick check: same size AND same mtime -> skipped
rsync -av src/ dst/
# Same size, different content, same mtime: skipped. Use checksums to catch it.
rsync -avc src/ dst/
# Restores that must not trust timestamps at all
rsync -av --size-only src/ dst/Plain rsync compares size and modification time with one-second granularity. Files generated by a build process often keep the same size and a copied timestamp, so a corrupt or stale file survives every sync. -c fixes correctness at the cost of reading every file on both sides. Windows/SMB destinations, filesystems with coarse timestamps (FAT: two seconds), and containers that copy files with --no-preserve=timestamps are the common sources of a mismatch in the other direction: everything transfers every time. --modify-window=2 absorbs coarse-granularity differences.
3. Resuming interrupted transfers safely
# Option A: keep partial files, resume by re-running the same command
rsync -av --partial --partial-dir=.rsync-partial big.iso user@host:/data/
# Option B: append-only for a growing file (log tail, append-only dump)
rsync -av --append-verify bigfile user@host:/data/
# Never combine --append with --inplace on a file that can be rewritten--partial keeps the partially transferred file at the destination, and the next run reuses it. --append tells rsync the destination is a strict prefix of the source — true for an append-only log, false for a database dump that was rewritten. --append-verify (rsync 3.0+) re-checks the existing bytes with a checksum before appending, which is the safer choice; plain --append trusts them. For scp there is no resume at all: a 4 GB copy that dies at 90% starts over, which is the practical reason to reach for rsync on anything large or flaky.
4. Bandwidth limits and link sharing
# 2 MB/s ceiling, including for the SSH transport
rsync -av --bwlimit=2M --timeout=300 /data/ user@host:/backup/data/
# scp is throttled externally
trickle -s -d 2048 -u 2048 scp -r ./data user@host:/backup/
# sftp has a per-command throughput limit in Kbit/s
sftp -l 16000 user@host--bwlimit applies per rsync process; -P/parallel rsync runs multiply the ceiling. --timeout is an I/O inactivity timeout, not a total runtime limit, so a slow-but-alive link does not trip it. On long-haul copies, -z usually helps text and hurts already-compressed archives; measure once with --stats rather than assuming.
5. Snapshots with --link-dest, and the backup that is not a backup
BASE=/backup/app
DATE=$(date +%F)
rsync -aHAX --delete --numeric-ids \
--link-dest="$BASE/current" \
/srv/app/ "$BASE/$DATE/"
rm -f "$BASE/current" && ln -s "$DATE" "$BASE/current"--link-dest hard-links unchanged files from the reference directory, so each daily snapshot costs only the changed bytes while looking like a full tree. Three conditions make it work: same filesystem for the snapshots (hard links cannot cross one), --delete so removed files disappear, and never modifying a completed snapshot in place. Adding -H (hard links) for the source's own hard-linked files and -A -X for ACLs and xattrs avoids losing the capabilities some binaries need.
# Verify a mirror actually matches before declaring success
rsync -an --delete --itemize-changes /srv/app/ /backup/app/ | grep -v '^\.d' || echo "in sync"Common Runtime & Configuration Mistakes
| Symptom | Cause | Fix |
|---|---|---|
Destination has src/src/... | Trailing slash missing on the source | src/ to copy contents |
--delete removed more than expected | Source was a wrong/empty path or a mounted filesystem | Always -n first, check the source exists |
| Files re-transfer every run | mtime granularity or a filesystem that does not keep times | --modify-window=2, --size-only, -c |
| Changed file not transferred | Same size and mtime | -c |
rsync: connection unexpectedly closed | Remote binary missing or wrong --rsync-path | Install rsync on both ends |
| Permission denied on read | Running as the wrong user, or -o -g without root | Match the file owners; sudo via --rsync-path |
Owner became nobody | Name lookup differs across hosts | --numeric-ids and root on the receiver |
| Interrupted copy restarts from zero | --partial missing | Add --partial --partial-dir=... |
| Exit code 24 in a script treated as failure | Source changed during the run | Treat 24 as success, fail on 23+ |
| Transfer saturates the link | No --bwlimit, or many parallel runs | --bwlimit, serialise, --timeout |
| scp fails with "subsystem request failed" | Server has no sftp subsystem, OpenSSH 9+ uses SFTP | scp -O for the legacy protocol |
| Restored archive loses xattrs/ACLs | -a does not include them | -aHAX |
The pattern behind nearly all of these is that rsync's defaults are fast, not strict: quick size-and-time checks, filters that keep excluded files on the receiver, and deletion that trusts your source path completely. A transfer is trustworthy when the source path is spelled exactly as intended, the first run was a dry run with --itemize-changes, and the exit code was read rather than ignored.
Try matching browser tools
All toolsSmall utilities that fit Linux work — they run entirely in this browser tab, so nothing you paste leaves your device.
Related Linux guides
All Linux guidesBash Indexed Arrays, Associative Maps, and Parameter Expansion
Quoting decides whether an array stays a list, and the expansion operators replace most external text tools. Includes set -u, mapfile, namerefs, and the bash 3.2 differences that break scripts on macOS.
Parsing JSON and YAML in Shell Pipelines with jq and yq
jq is a filter language, not a pretty-printer. The operators that do the work, --arg versus string interpolation, in-place YAML edits, and how to tell which yq is installed before writing a command.
Linux Memory, Swap, the OOM Killer, vmstat, and sysctl Tuning
MemAvailable versus used, the vmstat columns that prove thrashing, how the OOM killer picks its victim, and the sysctl keys that belong in /etc/sysctl.d instead of a one-off command.
Linux Firewalls: ufw, iptables, and nftables Rules and Persistence
First-match ordering, stateful filtering, and persistence: the three ideas behind every Linux firewall. Plus why Docker-published ports ignore ufw and how DOCKER-USER closes them.
More in Linux
All Linux guidesThe guides filed next to this one in Linux, in library order.