Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Linux Archiving and Compression: tar, gzip, xz, and zstd Syntax and Trade-offs

tar archives and the compressors only shrink the stream. Argument order, -C path handling, ownership and xattr preservation, streaming over SSH, and which algorithm to pick for size versus speed.

~5 min readLinux
#Linux#CLI#Backup#Compression
3D illustration of a tar archive box feeding gzip, xz and zstd compression cylinders in a streaming pipeline

Quick Summary

Two separate jobs are usually typed as one command. Archiving packs a directory tree into a single stream with its metadata; compression shrinks a stream. tar does the first, gzip, xz, bzip2, and zstd do the second, and tar merely runs one of them for you through -z, -J, -j, or --zstd. That distinction explains the flags that otherwise look arbitrary: -f archive.tar.gz names a file, -c creates, -x extracts, -t lists, and compression is a filter applied to the stream, so tar -cf - dir | zstd > dir.tar.zst and tar --zstd -cf dir.tar.zst dir produce equivalent results.

Modern distributions are moving toward zstd because it decompresses several times faster than xz at a comparable ratio, and gzip remains the universal lowest common denominator that every system can read. xz still wins on size when the data is text-like and the archive is written once and read rarely.

Two operational details matter more than the algorithm. First, -C (change directory) combined with relative paths decides whether the archive extracts into a clean tree or scatters absolute paths across the filesystem. Second, metadata is preserved only when asked: ownership needs root, and --xattrs/--acls are separate flags. This guide covers the argument order, each compressor's trade-offs, streaming and incremental archives, extract-into-place safety, and the failures that leave a half-written archive behind.

Basic Syntax & Signature

bash
# Create: tar -c[compression] -f ARCHIVE [paths...] tar -czf backup.tar.gz /srv/app # gzip tar -cJf backup.tar.xz /srv/app # xz tar -cjf backup.tar.bz2 /srv/app # bzip2 tar --zstd -cf backup.tar.zst /srv/app # zstd (tar >= 1.31) tar -cf backup.tar /srv/app # uncompressed # List / extract / compare tar -tzf backup.tar.gz # -t list contents tar -tzvf backup.tar.gz | head # verbose: mode, owner, size, date tar -xzf backup.tar.gz # -x extract tar -xzf backup.tar.gz -C /restore # extract into another directory tar -df backup.tar /srv/app # compare archive against the filesystem # Verbose creation, and stripping the leading directory tar -czvf backup.tar.gz -C /srv app # stores "app/..." not "srv/app/..." # Include / exclude tar -czf site.tar.gz --exclude='*/node_modules' --exclude='*.log' -C /srv site tar -czf files.tar.gz -T list.txt # paths from a file, one per line tar -czf files.tar.gz --null -T list.txt # NUL-separated (from find -print0)
bash
# Standalone compression of a single stream (reads stdin, writes stdout) gzip -9 < notes.txt > notes.txt.gz xz -9e -T0 < dump.sql > dump.sql.xz # -T0: all cores, -e: extra effort zstd -19 -T0 dump.sql -o dump.sql.zst zstd -3 --long=27 big.log -o big.log.zst # Decompress to stdout gunzip -c notes.txt.gz zstd -dc dump.sql.zst | psql db xz -dc dump.sql.xz | head -5 # Test integrity without writing anything gzip -t backup.tar.gz xz -t backup.tar.xz zstd -t backup.tar.zst # Parallel compression of independent files find /var/log -name '*.log' -print0 | xargs -0 -P4 -n1 gzip -9

Key long options worth knowing: --strip-components=N (drop N leading path elements on extract), --one-file-system (do not cross mount points), --same-owner/--no-same-owner, --numeric-owner (store UIDs/GIDs, not names), --xattrs, --acls, --sparse, --format=pax|gnu|posix, and --listed-incremental=FILE for incremental snapshots.

Key Comparison Table

Aspectgzipbzip2xz (LZMA2)zstd
Typical ratio on textBaseline~10-15% smaller~20-25% smaller~5-15% smaller
Decompression speedFastSlowSlowestFastest (multi-core)
Compression speedFastSlowVery slowVery fast at low levels
Multi-threadedNo (pigz exists)No-T0 for blocks-T0
Memory to decompressVery low~4 MB/blockCan be high, depends on dictConfigurable, low
Streaming friendlyYesYesNeeds large blocksYes (--long)
Universal availabilityEverywhereCommonCommonGrowing, not in older base images
tar flag-z-j-J--zstd
Best forPortability, speedLegacyLong-term archivesBackups, CI artifacts, logs
TaskCommand
List without extractingtar -tf a.tar.zst --zstd
Extract one filetar -xf a.tar.gz path/inside/archive
Extract with directory flatteningtar -xf a.tar.gz --strip-components=2 -C /dst
Extract as a different ownersudo tar -xf a.tar.gz --same-owner -C /dst
Re-pack with no recompressiontar -xf a.tar.gz && tar --zstd -cf b.tar.zst <tree>
Append to an uncompressed archivetar -rf a.tar newfile (not possible with compression)
Verify against sourcetar -df a.tar -C /

Practical Edge Cases & Deep Dives

1. -f position and the "Cowardly refusing to create an empty archive" error

bash
$ tar -czf backup.tar.gz /srv/app # correct: f directly before the archive name $ tar -cfz backup.tar.gz /srv/app # wrong: -f takes "z" as the filename tar: z: Cannot open: No such file or directory

GNU tar tolerates the older bundling of single-letter options without a dash (tar czf archive file), and in that style f must be the last letter because the next argument becomes the archive name. Mixing GNU's long options with the historical form is where the confusion starts. If you ever see Cowardly refusing to create an empty archive, the paths were consumed as options or never reached tar — check that the source path exists and is not being eaten by a shell glob that matched nothing.

2. Absolute paths, -C, and why archives unpack into odd places

bash
# Stores /srv/app/... — extraction tries to write absolute paths tar -czf app.tar.gz /srv/app # Stores app/... — extraction is relative to where you run it tar -czf app.tar.gz -C /srv app # GNU tar strips a leading "/" on extract and warns, but the structure remains tar -tzf app.tar.gz | head -2 # srv/app/bin/server

Extracting such an archive in /tmp creates /tmp/srv/app/..., which people then mistake for a restore that "did not work". Store relative paths with -C, and normalize archives that were built badly:

bash
# Restore with the ancestor stripped sudo tar -xzf app.tar.gz --strip-components=2 -C /

-C is also how to include two distant trees in one archive, since it applies to the paths that follow it:

bash
tar -czf bundle.tar.gz -C /srv app -C /etc/nginx nginx.conf

3. Preserving ownership, permissions, and extended attributes

bash
# local: store numeric IDs and restore them sudo tar --numeric-owner -cpzf etc.tar.gz /etc sudo tar --numeric-owner --same-owner -xpzf etc.tar.gz -C / # preserve ACLs and SELinux labels as well sudo tar --acls --xattrs --selinux -cpzf etc.tar.gz /etc

Without -p on extract as root, GNU tar defaults to restoring permissions for the extracting user, which can silently change modes on system files. Without --acls/--xattrs, file capabilities (security.capability — needed by ping, traceroute, and containers' setcap binaries) and user xattrs are lost, which is why a "restored" system sometimes fails in surprising ways. tar -tvf showing root/root is not proof of the numeric IDs; check with --numeric-owner -tvf when the archive crosses hosts whose user databases differ.

4. Streaming directly between hosts and compressors

bash
# Pipe over SSH without a temporary file tar -czf - -C /srv app | ssh backup@host 'tar -xzf - -C /restore' # Same, with zstd and a progress readout outside the pipe tar --zstd -cf - -C /srv app \ | pv -bep \ | ssh backup@host 'zstd -dc | tar -xf - -C /restore'

The failure mode of this pattern is a broken pipe that tar reports as write error: Broken pipe while the receiving side reports a short read — the actual error (disk full on the receiver, permission denied) is on the far side and gets lost. Tee the tar stream so a copy exists locally when the transfer is one-way:

bash
tar -czf - -C /srv app | tee /backup/app.tar.gz | ssh backup@host 'cat > /backup/app.tar.gz'

Never append to a compressed archive. tar -rf refuses on compressed files, and gzip -c of two tars concatenated produces a stream most tools read only up to the first member (gzip itself accepts concatenated members, but tar stops at the first end-of-archive marker unless -i is given).

5. Extracting one path, and listing without unpacking huge trees

bash
# List paths matching a pattern, then extract only those tar -tf release.tar.gz | grep '^release/config/' tar -xf release.tar.gz release/config/app.yaml # Extract everything except a directory tar -xf release.tar.gz --exclude='release/node_modules/*' # List a remote archive member's content without full extraction tar -xOf release.tar.gz release/VERSION # Print size totals per top-level directory tar -tvf release.tar.gz | awk '{n=$6; split(n,p,"/"); s[p[1]]+=$3} END {for (k in s) print k, s[k]}'

--exclude and member names are matched with shell-style patterns against the full stored path, so --exclude='node_modules' may not match anything if the archive stores release/node_modules. Anchor patterns with a leading */ or list the archive first.

6. Incremental archives and what they are not

bash
# Level 0 snapshot, then a level 1 of what changed since tar --listed-incremental=/var/backup/app.snar -czf app-0.tar.gz -C /srv app tar --listed-incremental=/var/backup/app.snar -czf app-1.tar.gz -C /srv app # Always restore incrementals in order, into an empty directory tar --listed-incremental=/dev/null -xzf app-0.tar.gz -C /restore tar --listed-incremental=/dev/null -xzf app-1.tar.gz -C /restore

The snapshot file (*.snar) is state; losing or reusing the wrong one produces archives that silently omit files. --listed-incremental tracks mtime-based changes and handles deletions by storing the full directory listing — it is not a deduplicating backup, and it does not detect corruption. For anything that must be provably restorable, add a verification step:

bash
gzip -t app-0.tar.gz && tar --compare -f app-0.tar.gz -C /srv app && echo verified

Common Runtime & Configuration Mistakes

SymptomCauseFix
Cowardly refusing to create an empty archiveNo paths, or a glob that matched nothingCheck the source path, quote globs
tar: z: Cannot open-f not last in bundled flagstar -czf name file
Archive unpacks to ./srv/appAbsolute path storedBuild with -C /srv app
Restored files owned by the wrong userNo --numeric-owner --same-owner, or not rootAdd both, extract as root
File capabilities lost--xattrs missing--acls --xattrs --selinux
--exclude has no effectPattern does not match the stored pathList first, anchor the pattern
Compressed archive cannot be appendedCompression is a single streamUse uncompressed tar, or re-create
xz: Memory usage limit reached on decompressDictionary larger than available RAMxz -d --memlimit=..., or use zstd
Sparse files balloon to full sizeNo --sparseAdd -S/--sparse
Broken pipe during a pipe transferReceiver failedCheck the far side, keep a tee'd copy
Truncated archive after a full diskCompression wrote partial streamgzip -t/zstd -t before trusting it

tar is stable enough that almost every incident is about paths, ownership, or a stream that never finished rather than about the format. Building archives from a known directory with -C, storing numeric owners, and verifying the archive with -t or --compare before deleting the source turns a fragile habit into a restorable backup.

Try matching browser tools

All tools

Small utilities that fit Linux work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Linux guides

More in Linux

All Linux guides

The guides filed next to this one in Linux, in library order.