Compaction is a background process that optimizes the storage layout of your filesystem. An optimized storage layout provides faster performance and can reduce memory usage.
This article describes how to speed up the compaction process to quickly optimize the layout, for example after an initial import to the filesystem.
Install ObjectiveFS 6.8 or newer on an EC2 instance in the same region as your S3 bucket. Check that the instance type does not have limited network bandwidth for best performance.
Note: High compaction can use a lot of bandwidth and data transfer cost is free for EC2 in the same region as the S3 bucket.
Mount your filesystem with the
compact=5,noratelimit,freebw mount options.
# mount.objectivefs -ocompact=5,noratelimit,freebw <filesystem> <dir>
/etc/fstabentry for mounting on boot is:
<filesystem> <dir> objectivefs auto,_netdev,compact=5,noratelimit,freebw 0 0
noratelimitis needed for compaction levels 4 and 5 to take effect.
Depending on the size of your filesystem, it could take a few days to achieve the optimal layout. You can monitor the compaction progress by checking the rate of object deletion (
DELETE) in the log file or by checking the IUsed value from
df -i (which shows the number of objects in the bucket).
Last updated by ObjectiveFS staff, May 4, 2021
ObjectiveFS is a shared file system for macOS and Linux that automatically scales and gives you scalable cloud storage. If you have questions or article idea suggestions, please email us at firstname.lastname@example.org