Large File Performance: Amazon EFS vs ObjectiveFS

For processing large files such as videos, images and life science data, sequential read and write performance is a key factor when selecting a filesystem. We compared the performance of two shared filesystems that are commonly used to scale these workloads on AWS EC2 instances: Amazon Elastic File System (EFS) and ObjectiveFS. For details on how Amazon EFS differs from ObjectiveFS, see here.

Setup

We selected an EC2 instance type with 10Gbit network so that the performance test is not limited by network bandwidth. We used the us-west-1 region (Northern California) for our testing with a c4.8xlarge instance and an S3 bucket in the same region. For ObjectiveFS filesystem, we used the default memory setting and enabled multithreading. Since EFS is not available in many regions, including us-west-1, we created a similar setup with a c4.8xlarge instance and EFS in the us-west-2 region. We used the default general purpose performance mode for EFS since the test was done from a single machine.

Note: For EFS, we kept the test within the burst time limit, to get the best possible EFS performance. For a 100GiB filesystem, the EFS burst limit is 72 minutes per day. After that, Amazon EFS performance is limited to 5MiB/s. For more details, see the Throughput Scaling section of Amazon EFS Performance. Note that ObjectiveFS does not have any burst limits and can sustain high throughput consistently 24x7.

Setup Summary

Component Settings
EC2 instance c4.8xlarge EC2 instance
Amazon EFS general purpose performance mode
ObjectiveFS multithreading, default memory cache size

Performance

We performed sequential read and write of a large non-compressible 10G file, with the file content pregenerated from /dev/urandom.

Read Throughput

For read throughput, we compared EFS with ObjectiveFS. For ObjectiveFS, we measured the throughput for the initial read from S3 (cold cache) and subsequent read when the data is cached in memory. Kernel caches are dropped using echo 3 > /proc/sys/vm/drop_caches before each run for all cases to avoid any effects from kernel caching.

Throughput – large sequential reads

Filesystem Read Throughput Improvement
EFS 105 MB/s1 1x
ObjectiveFS (cold cache) 355 MB/s  3x
ObjectiveFS (warm memory cache) 1.9 GB/s  18x

1: The EFS performance is limited to 72 minutes per day for a 100GiB filesystem. Subsequent throughput is limited to 5.25MB/s (see “Throughput Scaling” in Amazon EFS Documentation).

Write Throughput

For write, the data was cached in memory before being written to the filesystem to ensure that reading the file was not the bottleneck.

Throughput – large sequential writes

Filesystem Write Throughput Improvement
EFS 103 MB/s1 1x
ObjectiveFS                                  382 MB/s  3.7x

1: The EFS performance is limited to 72 minutes per day for a 100GiB filesystem. Subsequent throughput is limited to 5.25MB/s (see “Throughput Scaling” in Amazon EFS Documentation).

Conclusion

For large files workloads, ObjectiveFS can provide 3X or higher throughput for both reads and writes compared to Amazon EFS. To see how small file workloads compare, see how ObjectiveFS compared to EFS on small file workloads.

To see how ObjectiveFS performs for your workload, you can download a copy and try it with our free trial.

by ObjectiveFS staff, March 27, 2017
ObjectiveFS is a shared file system for OS X and Linux that automatically scales and gives you unlimited storage. If you have questions or article idea suggestions, please email us at support@objectivefs.com

Get updates about ObjectiveFS