Yeah, I've been searching for a great general archiver for a long time. I tried borgbackup a couple of years ago and immediately hit a handful of issues that put me off, though I can't really recall the specific technical details clearly (I'm vaguely recollecting something about incompatible block sizes).
Whatever the details were, I came away with the distinct impression that the borgbackup developers didn't really respect the gravity of archived records and long-term storage. While that's probably not a totally fair conclusion from a one-shot use, in my defense, I'll offer up the latest version of their changelog [0], where:
* the first thing listed is an apparently serious data corruption bug that lived through several stable releases
* the second thing listed is an apparently-very-serious security vulnerability ("a flaw in the cryptographic authentication scheme used in Borg allowed an attacker to spoof the manifest ...")
* the third thing listed is another data corruption bug titled "Pre-1.0.9 data loss"
* the fourth thing listed is another data corruption bug titled "Pre-1.0.4 potential repo corruption"
Note these are all post-1.0 versions. To be frank, I dare not scroll further.
Users are depending on this software to safeguard important files with the assumption that it will be able to reproduce them bit-for-bit-intact some years down the road. Any long-term storage software requires developers with a fanatical devotion to compatibility, longevity, and integrity if it's ever going to be more than a toy.
I hate to pile on open-source devs who're trying their hand at making software that a lot of people appreciate, but overall, it's hard for me to say I regret the choice to pass it up and stick to combinations of ZFS, rsync, and the trusty old tarball.
I still don't know how to really judge software from changelogs, but I think it's really hard to do and this post is not a great example either.
This is a changelog backup software which does data storage and encryption mainly. If it ever has bugs worth talking about they're going to be in... data storage and encryption. If we look at rsync for a change which is much older, simpler and mature: https://download.samba.org/pub/rsync/src/rsync-3.1.1-NEWS
- fixing .. traversal in V3 (how did that survive so long?)
- issues with copying attributes
(There won't be much data corruption since rsync does 1:1 copies)
Time-to-fix could give us a better idea maybe? Either way, software doing X has bugs in X is "normal".
I mean, the issue isn't so much that bugs exist -- like you say, that's par for the course, even among the best developers. It's about a large incidence of severe eat-your-data bugs in such a short span of time (roughly a year) in something that's quote-unquote stable and whose entire purpose is long-term data storage and retrieval.
It's even worse because unlike rsync, which synchronizes filetrees from point A to B and can be immediately confirmed to have either worked correctly or not, borg uses a bespoke storage format that's not easily verified or operated upon by standard utilities. You just have to trust it to pull the correct data out when you need it. That's a heavy burden to put on a tool, and the borg devs seem to be struggling under its weight.
Consider the standards expected of filesystem or database maintainers. borg repos are not really that different -- they're a big, opaque chunks of bytes that you expect to be able to produce specific data on demand with perfect reliability.
These types of systems don't get marked stable when they're still in primordial, eat-your-data development phases, and on the small handful of unfortunate occasions when data loss bugs sneak in, they're a) usually limited to some bizarre corner case; and b) taken extremely seriously, almost to the point of solemnity, and often result in major overhauls to a project's validation and QA routine.
If any stable filesystem or database had 4 widely-applicable database corruption bugs within a year, it'd be lights out for that project. All trust lost, reputation irreparably ruined, angry letters to a variety of mailing lists, permanently-increased scrutiny on any new projects or maintainers among the same class of projects, etc.
All I'm saying is that based on my admittedly-murky prior experience, the project's observed track record, and the high standard of care that must be met to qualify a project for storing authoritative copies of data, I'm personally not comfortable trusting borg with anything important any time soon. No one is obliged to share that evaluation, of course.
If compression is purely a function of the price, things are not so obvious.
I think Backblaze B2 is the cheapest, but it's not supported by Borg. So a backup system without compression support, but with B2 support, may ultimately be cheaper than a system with opposite features.
It's also important to consider the use case. If a user's bulk of data is photos/videos/audio files, there's virtually no use of compression - but of course, the proportion will vary per user.
I’m using restic too for part of my backups, having moved from duplicity. My experience has been (on a dataset which is relatively compressible ie not audio/video files) that deduplication has saved me quite a lot more space vs duplicity than I lost through not having compression.
Whatever the details were, I came away with the distinct impression that the borgbackup developers didn't really respect the gravity of archived records and long-term storage. While that's probably not a totally fair conclusion from a one-shot use, in my defense, I'll offer up the latest version of their changelog [0], where:
* the first thing listed is an apparently serious data corruption bug that lived through several stable releases
* the second thing listed is an apparently-very-serious security vulnerability ("a flaw in the cryptographic authentication scheme used in Borg allowed an attacker to spoof the manifest ...")
* the third thing listed is another data corruption bug titled "Pre-1.0.9 data loss"
* the fourth thing listed is another data corruption bug titled "Pre-1.0.4 potential repo corruption"
Note these are all post-1.0 versions. To be frank, I dare not scroll further.
Users are depending on this software to safeguard important files with the assumption that it will be able to reproduce them bit-for-bit-intact some years down the road. Any long-term storage software requires developers with a fanatical devotion to compatibility, longevity, and integrity if it's ever going to be more than a toy.
I hate to pile on open-source devs who're trying their hand at making software that a lot of people appreciate, but overall, it's hard for me to say I regret the choice to pass it up and stick to combinations of ZFS, rsync, and the trusty old tarball.
[0] https://github.com/borgbackup/borg/commit/75dcf9356334188276...