Answer the question
In order to leave comments, you need to log in
How to organize an incremental S3 backup?
The customer asked for a daily backup of objects from S3.
In a couple of minutes I wrote
aws s3 sync s3://origin-bucket/folder s3://backup-bucket/folder- CURRENT_TIMESTAMP
But then I thought about it. The customer dreams
- about something incremental
- and even a zipped archive would be nice (for simplicity, let's say that documents are stored, and I want the ability to roll back to the desired version, "like in Git").
As far as I understand, these wishes are not easy to fulfill - the first is due to the lack of a simple ability to filter objects by date, and the second - because this will require downloading files to a local machine.
I'm thinking of enabling versioning in S3 for backup. Or is there a more elegant solution?
UPDATE: Thanks a lot to everyone who replied! You made me think, and I realized that I should use S3 versioning and replication, instead of reinventing the wheel with scripts and cronjobs.
Answer the question
In order to leave comments, you need to log in
As a purely "knee" option, use:
EC2 instance with running software:
1. Mount S3 (source, target) as a folder: goofys or s3fs .
2. As backup software: Bup for deduplication. Able like Git...
At the input is the folder of the source bucket, and at the output - the target.
Haven't tried it myself. Just like an idea.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question