points by doctorpangloss 2 years ago

> Veeam

As my 2 year old would say, "uh oh."

Are you saying your backup solution is:

- there's a folder somewhere whose contents are "synchronized", crying laugh emojis galore, using a "Dogshit" protocol, as it appears in the programs Microsoft OneShit or DropShit or Shit.net or Google Shit.

- you have a machine that has a "copy" of a dogshit-synchronized file system, perhaps using a vendor's implementation of Dogshit, such as Synology Dogshit Manager, a piece of software with which I am intimately acquainted

- that machine "regularly" "backs up" its "copy" via a procedure known confusingly as "dogShit"

- wait a minute, the files aren't right!

I hear you. This is very surprising, but it would occur no matter which file sharing system you use, so long as it's based on Dogshit and/or dogShit.

What is the specific flaw with dogShit? The simplest explanation for everything you observe, like the constant moving by multiple users, is that while using ordinary file system enumerations of the form "walk," a folder may be moved in a way such that it is never visited by "walk," even though the destination of the moved folder is still within the root directory of the walk. You can easily verify this in Python.

Okay, so now that I solved your bug in dogShit, what should you use instead? Snapshotters have already solved this problem, you have to create a dedicated filesystem for the shared directory on your Synology, then snapshot the whole filesystem, which will only have issues with "missing" since the moment it started but never traversing the filesystem incorrectly. Then you can backup the file system snapshots to S3. In Synology you can achieve this with btrfs and some elbow grease.

But the right answer is to not use dogshit. At the end of the day the people who are authoring Dogshit file sharing products, they should be giving you a backup approach, not demanding you do it ad-hoc.