Migrating S3 Data to ObjectiveFS
This article describes the steps to migrate data such as images, videos, documents, etc, from an existing S3 bucket to an ObjectiveFS filesystem.
Recommendations
We recommend using an ObjectiveFS filesystem and an EC2 instance in the same region as your existing S3 bucket for best performance and to save on data transfer cost.
Steps
Start an EC2 instance in the same region as your S3 bucket.
Configure AWS CLI.
- Run
aws configureand provide your existing AWS S3 bucket's credentials. - For details, see AWS CLI Configuration Guide
Install ObjectiveFS on the EC2 instance.
- See installation guide.
- See get started guide if you need to create a new ObjectiveFS filesystem.
Mount your ObjectiveFS filesystem.
# mount.objectivefs <filesystem> /ofs
Copy your existing S3 bucket's content to ObjectiveFS.
# mkdir /ofs/dest
# aws s3 cp s3://<existing S3 bucket> /ofs/dest --recursive
Note: For large migrations, aws s3 sync instead of cp lets you re-run the same command to resume an interrupted copy or pick up files added since the last run.
References
Last updated ObjectiveFS staff
ObjectiveFS is a shared file system for Linux and macOS that automatically scales and gives you scalable cloud storage. If you have questions or article idea suggestions, please email us at support@objectivefs.com