This article provides an example of how to create a transparently rotating filesystem with a two-week retention schedule. It utilizes the ObjectiveFS filesystem pool feature together with an overlay filesystem.
In this example, we use two filesystems in a filesystem pool to store our weekly filesystem data: one for the current week and one for the previous week, since our retention schedule is two weeks. We then mount these two filesystems on an overlay filesystem: one as the primary filesystem for reading and writing this week’s data and the other as a read-only filesystem for last week’s data. At the end of the week, we recycle the older weekly filesystem that we don’t need anymore per our retention schedule. We then mount the freshly recycled filesystem for reading and writing this week’s data and move last week’s filesystem into read-only mode on our overlay filesystem.
# apt-get update
# apt-get install linux-image-extra-virtual
# apt-get install linux-image-extra-4.4.0-53-generic
# mount.objectivefs create <pool name>/weekly-a
# mount.objectivefs create <pool name>/weekly-b
# mkdir -p /ofs/weekly-{a,b}
# mkdir -p /ofs/overlay
# mount.objectivefs <pool name>/weekly-a /ofs/weekly-a
# mount.objectivefs <pool name>/weekly-b /ofs/weekly-b
# mount -t aufs -o br=/ofs/weekly-b:/ofs/weekly-a none /ofs/overlay
# ln -s /ofs/overlay /ofs-target
Your applications can read and write to /ofs-target/* as usual.
All data written to /ofs-target/* will be stored on the weekly-b filesystem.
Reads from /ofs-target/* will contain data from both weekly-a and weekly-b.
At the end of each week, use the following weekly rotation schedule:
Note: Make sure that the filesystem is unmounted on all machines before running destroy.
a. Week N
# ln -s /ofs/weekly-b /ofs.new && mv -Tf /ofs.new /ofs-target
# umount /ofs/overlay
# umount /ofs/weekly-a
// Recycle the weekly-a filesystem
// Verify weekly-a is unmounted on all machines before running destroy
# mount.objectivefs destroy <pool name>/weekly-a
# mount.objectivefs create <pool name>/weekly-a
# mount.objectivefs <pool name>/weekly-a /ofs/weekly-a
// Create an overlay filesystem with weekly-a on top
# mount -t aufs -o br=/ofs/weekly-a:/ofs/weekly-b none /ofs/overlay
# ln -s /ofs/overlay /ofs.new && mv -Tf /ofs.new /ofs-target
Your applications can continue to read and write to /ofs-target/* as usual.
All data written to /ofs-target/* will be stored on the weekly-a filesystem.
Reads from /ofs-target/* will contain data from both weekly-a and weekly-b.
b. Week N+1
# ln -s /ofs/weekly-a /ofs.new && mv -Tf /ofs.new /ofs-target
# umount /ofs/overlay
# umount /ofs/weekly-b
// Recycle the weekly-b filesystem
// Verify weekly-b is unmounted on all machines before running destroy
# mount.objectivefs destroy <pool name>/weekly-b
# mount.objectivefs create <pool name>/weekly-b
# mount.objectivefs <pool name>/weekly-b /ofs/weekly-b
// Create an overlay filesystem with weekly-b on top
# mount -t aufs -o br=/ofs/weekly-b:/ofs/weekly-a none /ofs/overlay
# ln -s /ofs/overlay /ofs.new && mv -Tf /ofs.new /ofs-target
Your applications can continue to read and write to /ofs-target/* as usual.
All data written to /ofs-target/* will be stored on the weekly-b filesystem.
Reads from /ofs-target/* will contain data from both weekly-a and weekly-b.
by ObjectiveFS staff, January 25, 2017
ObjectiveFS is a shared file system for Linux and OS X that automatically scales and gives you scalable cloud storage.
If you have questions or article idea suggestions, please email us at support@objectivefs.com