Backup (rsync) -> [E] Error: rsync: [sender] write error: Broken pipe (32)

JZL240I-U · 09-19-2021, 11:14 AM

I do automatic backups via "backintime" after every boot up. Today I noticed a briefly shown popup with that message "[E] Error: rsync: [sender] write error: Broken pipe (32)". So I checked and found that this happened since october 2019 i.e. for all of my backups since then. *grrrr*

The disk is probably full:

Code:

me@PC:~> df -h /backup
Dateisystem    Größe Benutzt Verf. Verw% Eingehängt auf
/dev/sdc1       960G    860G   52G   95% /backup

First question: Are these backups still usable or rather, how can I find out which files are not "kosher" anymore?

These backups also have a history. The disk that held them originally was dying, so I formatted a new one with ext4, same size partition, and "rsync -aXHAv"ed my backups over. The new disk is what throws the error now. I could enlarge this partition with gparted.

Now, I have more backup media. I checked the next:

Code:

Failed to get take_snapshot log from None:
[Errno 2] No such file or directory: '/home/me/.local/share/backintime/takesnapshot_3.log'

Ahmm, what? This is from the backintime utility which worked fine until now. What does that mean for my older backups? I just created a new one with no errors but now for all three users in the local home but with only one in the backup profile. What the...

Anyone with experience in such cases? Thanks in advance for sharing

.

business_kid · 09-20-2021, 05:41 AM

Pipes are the list of components: rsync reads disk1 | network |internet or wherever......disk2. If it's sending away, and some link goes AWOL, there is a broken pipe error thrown, & things exit. It requires a restart or retry of the connection.

For the integrity of the backup, there is no solution, as presumably you have nothing to compare them to. But I don't think it's a real worry. They may, however be incomplete, if the backup stopped prematurely.

If it's a scripted backup, check the exit status of rsync. If it exits 0, backup is complete. If it's not 0, you can take it as incomplete.

EDIT: I'd try size for finding usable backups. Otherwise do them in a terminal, and make sure they are not finishing on an error.

JZL240I-U · 09-20-2021, 10:26 AM

This is the kind of error I hate. All errors are gone! Even the logs back to 2019 list no more errors. What the ...

What I did is delete some snapshots of my children from times where I could be sure they didn't change anything being out of house. Then I created new snapshots on three different media to be on the secure side. Now all is well, pffff.

I'll just enlarge my /backup partition. Thanks for your input

. Oh, and as to your "EDIT" backintime is a very handy tool. It even shows the rsync command it uses when you start it by hand, I'd hate to type that long a command

.

business_kid · 09-20-2021, 11:56 AM

I always, always use rsync the same way

cd to dest dir, renaming dir if necessary
ls source dir, making sure I have the path correct.
Use rsync <options> source (which I can copy & paste) dest(which is always '.') && sync

That way I can do it in my sleep. Otherwise, I spend more time on the man page than I do on the backup

computersavvy · 09-20-2021, 01:57 PM

Running rsync to a remote host from a script is difficult to do if you have not set up keys and configured both systems for ssh connection using keys. That may be your entire issue since it cannot connect without either a password or using keys.

Do you connect instantly with ssh and no password or do you have to enter a password to connect?

You do not say if the backups are on the same system or a different one so my comment is about a remote backup system.

JZL240I-U · 09-21-2021, 09:48 AM

Ups, I answered yesterday but that post is gone. So again:

@business_kid This is a darn long command. Backintime shows is, it is at least three lines. Also, it is on automatic, it always does a backup after booting (or when I tell it to, e.g. using external disk(s)). I don't want to do it by hand every time (too lazy and forgetful

).

@computersavvy It is an other disk in the same machine mounted to /backup at every boot. So no ssh shenanigans.

business_kid · 09-21-2021, 10:21 AM

We still have not had a look at the command. does it use the internet? FQDNs and directory paths can be long or short, depending on how you go about it. You can probably simplify it down somewhat

JZL240I-U · 09-21-2021, 10:54 AM

Yes, but I want it automated. Here is the command:

rsync --recursive --times --devices --specials --hard-links --human-readable --links --acls --xattrs --perms --executability --group --owner --info=progress2 --no-inc-recursive --delete --delete-excluded -v -i --out-format=BACKINTIME: %i %n%L --link-dest=../../20210920-170233-374/backup --chmod=Du+wx --exclude=/backup --exclude=/home/me/.local/share/backintime --exclude=.local/share/backintime/mnt --include=/home/me/ --include=/home/ --include=/home/me statisch/ --exclude=.gvfs --exclude=.cache* --exclude=[Cc]ache* --exclude=.thumbnails* --exclude=[Tt]rash* --exclude=*.backup* --exclude=*~ --exclude=.dropbox* --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/tmp/backintime --exclude=/home/me statisch/Virtual Machines --exclude=.cache/* --exclude=.local/share/[Tt]rash* --exclude=/proc/* --exclude=/sys/* --exclude=/dev/* --exclude=/run/* --exclude=/etc/mtab --exclude=/var/cache/apt/archives/*.deb --exclude=lost+found/* --exclude=/tmp/* --exclude=/var/tmp/* --exclude=/var/backups/* --exclude=.Private --exclude=/home/me statisch/Android-SD-Card-backup/TWRP/BACKUPS --exclude=/home/me/backintime-mint --include=/home/me/** --include=/home/me statisch/** --exclude=* / /backup/backintime/PC/me/1/new_snapshot/backup

Oh, and by the way, the error messages are back also in the old logs...

JZL240I-U · 09-21-2021, 11:08 AM

Hmm. I just about doubled the size of the backup partition and started a new run of backintime. It hangs with this message:

Code:

[I] Schnappschuss erstellen (rsync: BACKINTIME: >f+++++++++ home/me/.local/share/akonadi/db_data/mysql.err.old)

I don't need that file (mysql.err.old) anymore (KMail is broken anyway), should I delete it and try again?

And look ant this:

Code:

me@PC:~> ps ax | grep backintime
 1399 ?        S      0:00 /usr/bin/python3 -Es /usr/share/backintime/qt/serviceHelper.py
 4782 ?        S      0:00 /bin/sh /usr/bin/backintime-qt
 4787 ?        Sl     0:05 python3 -Es /usr/share/backintime/qt/app.py
 4793 ?        Z      0:00 [backintime] <defunct>
 4818 ?        S      0:00 /bin/sh /usr/bin/backintime backup
 4822 ?        S      0:04 python3 -Es /usr/share/backintime/common/backintime.py backup
 4825 ?        Z      0:00 [backintime] <defunct>
 4826 ?        Sl     0:01 /usr/bin/python3 /usr/share/backintime/qt/qtsystrayicon.py 1
 4832 ?        D      1:22 rsync --recursive --times --devices --specials --hard-links --human-readable --links --acls --xattrs --perms --executability --group --owner --info=progress2 --no-inc-recursive --delete --delete-excluded -v -i --out-format=BACKINTIME: %i %n%L --link-dest=../../20210921-163354-999/backup --chmod=Du+wx --exclude=/backup --exclude=/home/matthias/.local/share/backintime --exclude=.local/share/backintime/mnt --include=/home/matthias/ --include=/home/ --include=/home/matthias statisch/ --exclude=.gvfs --exclude=.cache* --exclude=[Cc]ache* --exclude=.thumbnails* --exclude=[Tt]rash* --exclude=*.backup* --exclude=*~ --exclude=.dropbox* --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/tmp/backintime --exclude=/home/matthias statisch/Virtual Machines --exclude=.cache/* --exclude=.local/share/[Tt]rash* --exclude=/proc/* --exclude=/sys/* --exclude=/dev/* --exclude=/run/* --exclude=/etc/mtab --exclude=/var/cache/apt/archives/*.deb --exclude=lost+found/* --exclude=/tmp/* --exclude=/var/tmp/* --exclude=/var/backups/* --exclude=.Private --exclude=/home/matthias statisch/Android-SD-Card-backup/TWRP/BACKUPS --exclude=/home/matthias/backintime-mint --include=/home/matthias/** --include=/home/matthias statisch/** --exclude=* / /backup/backintime/PC/matthias/1/new_snapshot/backup
 4833 ?        S      0:18 rsync --recursive --times --devices --specials --hard-links --human-readable --links --acls --xattrs --perms --executability --group --owner --info=progress2 --no-inc-recursive --delete --delete-excluded -v -i --out-format=BACKINTIME: %i %n%L --link-dest=../../20210921-163354-999/backup --chmod=Du+wx --exclude=/backup --exclude=/home/matthias/.local/share/backintime --exclude=.local/share/backintime/mnt --include=/home/matthias/ --include=/home/ --include=/home/matthias statisch/ --exclude=.gvfs --exclude=.cache* --exclude=[Cc]ache* --exclude=.thumbnails* --exclude=[Tt]rash* --exclude=*.backup* --exclude=*~ --exclude=.dropbox* --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/tmp/backintime --exclude=/home/matthias statisch/Virtual Machines --exclude=.cache/* --exclude=.local/share/[Tt]rash* --exclude=/proc/* --exclude=/sys/* --exclude=/dev/* --exclude=/run/* --exclude=/etc/mtab --exclude=/var/cache/apt/archives/*.deb --exclude=lost+found/* --exclude=/tmp/* --exclude=/var/tmp/* --exclude=/var/backups/* --exclude=.Private --exclude=/home/matthias statisch/Android-SD-Card-backup/TWRP/BACKUPS --exclude=/home/matthias/backintime-mint --include=/home/matthias/** --include=/home/matthias statisch/** --exclude=* / /backup/backintime/PC/matthias/1/new_snapshot/backup
 4840 ?        S      2:01 rsync --recursive --times --devices --specials --hard-links --human-readable --links --acls --xattrs --perms --executability --group --owner --info=progress2 --no-inc-recursive --delete --delete-excluded -v -i --out-format=BACKINTIME: %i %n%L --link-dest=../../20210921-163354-999/backup --chmod=Du+wx --exclude=/backup --exclude=/home/matthias/.local/share/backintime --exclude=.local/share/backintime/mnt --include=/home/matthias/ --include=/home/ --include=/home/matthias statisch/ --exclude=.gvfs --exclude=.cache* --exclude=[Cc]ache* --exclude=.thumbnails* --exclude=[Tt]rash* --exclude=*.backup* --exclude=*~ --exclude=.dropbox* --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/tmp/backintime --exclude=/home/matthias statisch/Virtual Machines --exclude=.cache/* --exclude=.local/share/[Tt]rash* --exclude=/proc/* --exclude=/sys/* --exclude=/dev/* --exclude=/run/* --exclude=/etc/mtab --exclude=/var/cache/apt/archives/*.deb --exclude=lost+found/* --exclude=/tmp/* --exclude=/var/tmp/* --exclude=/var/backups/* --exclude=.Private --exclude=/home/matthias statisch/Android-SD-Card-backup/TWRP/BACKUPS --exclude=/home/matthias/backintime-mint --include=/home/matthias/** --include=/home/matthias statisch/** --exclude=* / /backup/backintime/PC/matthias/1/new_snapshot/backup
 4927 pts/1    S+     0:00 grep --color=auto backintime
me@PC:~>

Why three times rsync?

business_kid · 09-21-2021, 11:37 AM

So it's not an rsync command, and you have been leading us up the garden path.

Backintime is at least three python scripts, one of which probably has an rsync command.

Mark this solved, and file a bug, or raise an issue on the github a/c. I'm getting off this thread.

/Unsubscribes

JZL240I-U · 09-21-2021, 11:49 AM

No. You didn't pay attention. This is my very first sentence in this thread:

Quote:

I do automatic backups via "backintime" after every boot up.

Backintime's log showed the rsync errors. That's what I posted. Sorry that you are p*ssed, but I don't really see why.