borg compact makes my repo wrong #6781
Comments
|
duplicate of #6687? if you could run a full borg check (including the archives check), we could see if there is any serious issue. |
|
And please keep your copy of the original repo that causes this. It sounds like you are able to reproduce the issue, which will be helpful in debugging it. Maybe the compact command can be modified to print its actions, so we can figure out what it is doing wrong? |
|
Sorry for my lateness, I was not at home a few days. Now I made run the whole process again and the result is below. It was very sloooow on my 10+ years old PC. Demonstrate borg repo damage after borg compact command echo "Demonstrate borg repo damage after borg compact command" date +%T export BORG_PASSCOMMAND='cat /root/work/borg-backup/pass' echo "delete repo" echo "restore saved config and cache files" echo "restore saved repo" echo "rename saved repo to borg" echo "check repo before compact" echo "check archives before compact" echo "run borg compact" echo "check repo after compact" echo "check archives after compact" echo "try to repair" echo "check archives after repair" echo "******************** end ********************" Type 'YES' if you understand this and want to continue: Starting repository check |
|
That makes it really look like there is a bug in @ThomasWaldmann |
|
@jdchristensen : I use borg since 2018. A backup runs in every hours via cron, if the machine is turned on. After the first run on every day runs a "borg prune" and a "borg check" via anacron. Borg check never found any errors. The repo is on a partition of the hard disk, which is mounted to /backup. I copy the repo (via rsync) to an external USB disk, to a micro SD card and to a remote PC via SSH in every month, sometimes - if I work a lot - more frequently. I had never disk error as I can remember. Sometimes I had a disk full situation on the root partition (not on the backup) which could affect the /root directory. (And so the borg cache on it, because borg runs as root.) I never needed to restore the repo from any external source. If I need to restore some files from backup, I usually use borg mount. There was the first run of the "borg compact" now because my repo didn't fit to my 128G SD card. (Which I don't understand exactly because repo dir was about 70G only. Maybe it is due the encryption.) Did I forget something? Ask me. I'm glad to help make borg better. |
|
May I ask some help? What is the safest method if I would like to use my old repo for testing? Current and old repo have same id, so it is not so simple. I don't want to damage my current repo again. |
|
You can change the repo id in the config file in the borg repo you saved for testing. (Note that in general there are some risks if you do this with encrypted repos and plan to continue using both. But in this situation it should be fine.) |
|
I played with my old backup in last days. I made run copy_original_repo-purge-compact-check cycles to check which archives are affected these errors. (This was a time consuming play because one cycle runtime was about 50 minutes.) purge keep-last count / error count / comment, if any 99 / 4 / not purged at all, there is four errors as listed earlier There were borg updates during this four years and unfortunately I can't say which borg version was used to backup these archives. (Except if the archive doesn't contain it and you tell me how can I to see it.) I hope I could help. (To make chaos;) |
|
@DSpammy if you simplify your reproducer script by removing |
|
@ThomasWaldmann The result is same as with --cleanup-commits. There is no difference. |
|
Perhaps I can, but I need some help. My current borg version is 1.2.1, which is the latest version on Manjaro now. I downloaded the borg-compaction-debugging-1.2.zip file. May I simply replace (temporarily) repository.py, archive.py, archiver.py and base.py with downloaded files? |
|
You can either take the repository.py file and copy it over the original one or apply the patch. |
|
I copied repository.py first but when I run there was an exception related to archiver.py if I can remember. Then I copied each of the four files. After that I got the following message at borg check (before prune and compact): Platform: Linux cornflower 5.10.133-1-MANJARO #1 SMP PREEMPT Mon Jul 25 20:12:34 UTC 2022 x86_64 Maybe I made something wrong. I will try again later but now I have no time for that. |
|
Hmm, yeah, there were changes in other parts of the code also, so just applying the patch would work, but taking the whole repository.py does not. |
|
Patch: |
|
Here is the result of the "borg compact --cleanup-commits --debug ..." command: And the result of "borg check": Before compacting there were no errors. |
|
The three segment files that gave errors were 48506 (twice), 47096, 26597. All three were not compacted during the |
|
it dropped 1st DEL for it dropped 2nd DEL for after all the DELs were gone, if i don't know yet why the shadow index is wrong. |
|
Yes, "borg check --repair" could repair the repo. I use that repo since repair without problems. |
|
Did it complain about orphans (in the archives checking part) as I suspected above? |
|
Yes, It did. I checked again. |
|
I would just report that this problem still exists in borg version 1.2.2. |
|
This fix was not included in 1.2.2, you have to wait for 1.2.3 I guess |
|
@infectormp Is there a known fix? As far as I know, this hasn't been debugged yet. I've been running with --debug on one machine for a while, and haven't seen the problem since I switched to the version with --debug. But on other machines, I still get the problem very frequently. |
|
I get the issue as well unfortunately. Is more info needed to help debugging this? |
|
@szaimen guess I would need a repo copy where the issue can be reproduced. I tried to track this down by logs (see above) and reading source code, but did not really find a problem yet. And maybe I even need the state before any issue is introduced and the action that introduces the issue. |
Okay, actually I indeed have a testrepo and will try to reproduce the issue with that. I'll let you know how it goes and will let you know the exact steps if I should suceed. |
|
Unfortunately I wasn't able to reproduce the issue with my test-archive. :/ |
|
Likely related to #5661. |
|
Likely also a duplicate of #6687, so closing this one here. I just merged a potential fix to 1.2-maint branch, testing of that would be very welcome. |
Have you checked borgbackup docs, FAQ, and open Github issues?
Yes
Is this a BUG / ISSUE report or a QUESTION?
BUG
System information. For client/server mode post info for both machines.
Your borg version (borg -V).
1.2.0
Operating system (distribution) and version.
Manjaro linux
Hardware / network configuration, and filesystems used.
How much data is handled by borg?
repo directory size is 77 Gb
Full borg commandline that lead to the problem (leave away excludes and passwords)
borg compact /backup/borg
Describe the problem you're observing.
I used borg since 2018 without any problems. It runs via cron and makes backups in every hours. There is an anacron job starting every day which runs borg prune and borg check on the repository. Borg check never found any problem before now. Today I started a borg compact command to save some disk space. After that I started borg check again and it found some index problems.
Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.
Yes, I can. I copied back an earlier backup from an external drive. Borg check runs without errors on it. After borg compact, borg check find index errors again.
Include any warning/errors/backtraces from the system logs
The text was updated successfully, but these errors were encountered: