Hello.
I am using cloudsync to backup my nas to a s3 bucket (with some small modification on config file and sqlite3 database, I can change from Amazon to Ceph s3 endpoint).
I have two problems: cloudsyncd uploads all the files with a temporary filename (.cloudsyncd_downloading_ORIGINALFILENAME), after completed renames it with the original filename.
But with S3 there are 2 big problems:
- as I know (but I can be wrong) S3 protocol doesn't have a "rename" function, but it copies the old file with a new file (using the new filename) and then deletes the old one. But, if the file is very big (I have 30/40Gbyte files), this is a long operation, and cloudysncd returned an error (due to timeout).
- as I know (but, again, i can be wrong), if I enable bucket versioning (and I NEED to do it to be safe against ransomware), every temporary files will be not deleted, and the used space will be double (no problem if I have enought space because with lifecycle will be deleted after some days). I can't find any setting to exclude some pattern (ex: .cloudsyncd_*) from versioning.
So, the best choice could be to not use a temporary filename for s3 sync (better if we can choose from sync settings).
Anyone with my same problem ?
I am using cloudsync to backup my nas to a s3 bucket (with some small modification on config file and sqlite3 database, I can change from Amazon to Ceph s3 endpoint).
I have two problems: cloudsyncd uploads all the files with a temporary filename (.cloudsyncd_downloading_ORIGINALFILENAME), after completed renames it with the original filename.
But with S3 there are 2 big problems:
- as I know (but I can be wrong) S3 protocol doesn't have a "rename" function, but it copies the old file with a new file (using the new filename) and then deletes the old one. But, if the file is very big (I have 30/40Gbyte files), this is a long operation, and cloudysncd returned an error (due to timeout).
- as I know (but, again, i can be wrong), if I enable bucket versioning (and I NEED to do it to be safe against ransomware), every temporary files will be not deleted, and the used space will be double (no problem if I have enought space because with lifecycle will be deleted after some days). I can't find any setting to exclude some pattern (ex: .cloudsyncd_*) from versioning.
So, the best choice could be to not use a temporary filename for s3 sync (better if we can choose from sync settings).
Anyone with my same problem ?
Statistics: Posted by fabry68 — Today, 16:30