Moving a home directory to a new machine, I sized the job with du -sh ~/*/. It
reported roughly 149G. The real total was 521G. The destination had 86G
free. The transfer started, and the only reason it did not fill a root filesystem is that somebody happened
to be watching it.
~/*/ is expanded by the shell, and the shell does not match hidden entries. Every dotdir is
silently absent from the output. Not shown as zero. Not shown at all.
350G ~/.local <-- invisible to du -sh ~/*/
120G ~/UnrealEngine-src
13G ~/Downloads
9.6G ~/.var <-- also invisible
8.1G ~/Overlays
---
521G TOTAL.local alone was more than twice the entire estimate. On a desktop that is where Steam and
flatpak payloads live, so it is often the largest thing in a home directory and reliably the thing
nobody counts.
# the whole thing, one number, no glob involved
du -sh ~
# top 20 entries including hidden ones, largest first
du -sh ~/.[!.]* ~/* 2>/dev/null | sort -rh | head -20
# or let find do the walking, no shell globbing at all
du -h --max-depth=1 ~ | sort -rh | head -20du -h --max-depth=1 is the one to default to. It takes a path rather than a glob, so the shell
never gets the chance to hide anything from it.
cp ~/* dest/, rsync ~/*, tar cf a.tar ~/* and
any exclude list built by reading that output. If your backup was scoped from a glob, it may not contain
.ssh, .gnupg or .config.
The tempting conclusion is "add .local to the excludes". That fixes this transfer and none of
the next ones, because the actual failure was starting a copy without knowing whether it could finish.
Measure both ends and refuse to run:
SRC_GB=$(du -sBG --exclude-from="$EXCLUDES" "$SRC" | awk '{print int($1)}')
DST_GB=$(ssh "$DEST_HOST" "df -PBG '$DEST_PATH' | tail -1 | awk '{print int(\$4)}'")
HEADROOM=20
if (( DST_GB < SRC_GB + HEADROOM )); then
echo "ABORT: need $((SRC_GB + HEADROOM))G at the destination, have ${DST_GB}G." >&2
exit 1
fiThree details make that guard worth having. It measures the source after exclusions, so it checks what you are really sending. It asks the destination rather than assuming, because the destination is the one that runs out. And it keeps headroom, because filling a filesystem to exactly 100% is its own outage.
Point df at the actual target directory, not at the host generally. A machine with a small
root disk and a large mounted volume gives completely different answers depending on which path you ask
about, and the failure mode is copying onto the root disk because your target directory was not where you
thought.
ssh dest 'df -h /mnt/bigdisk; findmnt -no SOURCE,TARGET /mnt/bigdisk'Worth knowing before you need it. When a bulk transfer has to stop, killing the sender can leave the receiver writing whatever is already in flight. Stop the process actually creating files:
ssh dest "pkill -f 'rsync --server'"rsync is restartable, so an interrupted run costs you nothing but time. That is exactly why the preflight is worth more than the exclude list: the expensive failure is not a stopped copy, it is a filled disk on a machine doing other work.
Whenever a size estimate informs a decision, spend one command finding the biggest thing you did not expect:
du -h --max-depth=2 ~ 2>/dev/null | sort -rh | head -30Two levels deep is usually where the answer lives. It is what separates "my home directory is 149G of documents" from "my home directory is 350G of game data and some documents", and those two facts lead to completely different plans.
du -sh ~. No glob, no omission, nothing to fix.rsync ~/ dest/ with a trailing slash sends
the whole directory including dotfiles. It is the shell glob that loses things, not the transfer tool.