Storage Cleaner Guide

The storage cleaner works together with compaction to reclaim unused storage from deleted snapshots. This document describes how to use the cleaner+ feature (available in the 6.0 release and newer) to reclaim storage from your filesystem.

Background Information

The storage cleaner reclaims unused storage from deleted snapshots. It works together with compaction and the speed of cleaning is determined by the compaction level. Higher compaction levels will result in faster storage reclamation.

The storage cleaner has two modes: clean=1 (standard cleaner) and clean=2 (cleaner+). The cleaner+ is better at identifying unused storage but requires all mounts of a filesystem to use the 6.0 release or newer since it uses an extended storage format that is only available in these releases.

To enable cleaner+, use the clean=2 mount option with compaction enabled. Verify that all other mounts of this filesystem are running the 6.0 release or newer.


Using Cleaner+ For Storage Reclamation

This section outlines the steps you can use to speed up the storage reclamation process using a dedicated cleaner+ mount.

Before you begin: We recommend having a current backup of your filesystem before proceeding with the steps below.

  1. Start an EC2 spot instance in the same region as your S3 bucket.

    • Running in the same region will avoid AWS data transfer charges.
    • For best performance, use a server with sufficient memory and network performance for your filesystem size and verify that the memory cache size (CACHESIZE) (default: 20% of server memory) is at least 320MB for smaller filesystems and at least 1GB for very large filesystems.
    • For best result, this spot instance should be dedicated for the storage reclamation mount only.

  2. Mount your filesystem using the 6.0 release with cleaner+ enabled (clean=2), highest compaction level (compact=5,noratelimit), noocache, nosnapshots, noatime on the spot instance. The noratelimit option is needed for compaction levels 4 and 5.
    You can check the starting line of the objectivefs log to verify the mount options have taken effect. The starting line should contain clean+ and compact 5.

    $ sudo mount.objectivefs -o clean=2,compact=5,noratelimit,noocache,nosnapshots,noatime <filesystem> <directory>

  3. Upgrade all other mounts of this filesystem to the 6.0 release for the cleaner+ to activate.

Monitoring Progress

  1. The CLEAN field in the objectivefs log shows the cumulative storage reclaimed by this mount since it was mounted.

  2. The first time the cleaner+ is used on an existing filesystem, it needs to gather information about the state of the objects. It may take some time before the storage reclamation starts especially on large filesystems.

  3. During active storage reclamation, df -h output of the mount may report slightly higher storage than the actual amount used. To get the current storage size, you can run df -h on a fresh read-only mount with noocache.

    $ sudo mount.objectivefs -oro,noocache <filesystem> /ofs_readonly
    $ df -h

  4. The rate of storage reclamation depends on the filesystem size, amount of reclaimable storage, EC2 instance bandwidth and other factors. We recommend running the spot instance with cleaner+ and maximum compaction for several days and monitor the progress. Larger filesystems may require longer to reclaim the storage.

Note

  1. For filesystems that are 10TB or smaller, we suggest running only one mount with compact=5 and cleaner+. The other mounts can run with default compaction level (or at most level 3).

  2. For fastest cleaning, we suggest keeping the mount with cleaner+ running continuously and avoid restarting this mount.

  3. ObjectiveFS is backwards and forward compatible. You can run mixed versions of ObjectiveFS and do a rolling upgrade. The cleaner+ feature will only activate when all mounts of the filesystem are running the 6.0 release due to the extended storage format.

Reference

by ObjectiveFS staff, January 25, 2019
ObjectiveFS is a shared file system for OS X and Linux that automatically scales and gives you scalable cloud storage. If you have questions or article idea suggestions, please email us at support@objectivefs.com

Get updates about ObjectiveFS