Performance: Amazon EFS vs ObjectiveFS

When scaling web servers and content management systems (CMS), a shared filesystem is useful for storing and sharing file assets. Performance for small files is an important factor to consider when choosing a shared filesystem for web server and CMS workloads. We compared the performance of two shared filesystems that are commonly used to scale these workloads on AWS EC2 instances: Amazon Elastic File System (EFS) and ObjectiveFS. For details on how Amazon EFS differs from ObjectiveFS, see here.

Setup

We set up an r3.xlarge EC2 instance in the us-east-1 region. For EFS, we used a filesystem in the us-east-1 region with general purpose performance mode. The general purpose performance mode was selected because Amazon states it is better for latency-sensitive use cases like web servers and CMS than the max I/O performance mode. For ObjectiveFS, we used an S3 bucket in the us-east-1 region as the object store with the default memory cache setting and multithreading enabled. For the disk cache test, we used the local instance store on the EC2 instance.

Setup Summary

Component Settings
EC2 instance r3.xlarge EC2 instance in us-east-1
Amazon EFS general purpose performance mode in us-east-1
ObjectiveFS multithreading, s3 bucket in us-east-1 (disk cache)

Read Performance

We compared the read performance for random read access of small files, similar to a web server workload. The test performs 16 parallel reads to 10K small files on the filesystem with an 80/20 access pattern, and the latency and throughput are measured. Each filesystem contains the linux 4.7.2 kernel source tree with ~54K files in 3657 directories; the total size is 726MB and all files are smaller than 2MB.

The tables below show the latency and throughput results. For ObjectiveFS, it shows the results when the caches are not warmed (e.g. on initial boot), when the disk cache is warmed (e.g. after system reboots), or when the memory cache is warmed (during use). Kernel caches are dropped using echo 3 > /proc/sys/vm/drop_caches before each run for both filesystems.

Latency – small random reads

Filesystem Latency (95th-percentile) Improvement
EFS 81 ms 1x
ObjectiveFS (cold cache) 80 ms 1x
ObjectiveFS (warm disk cache) 6 ms 13x
ObjectiveFS (warm memory cache) <1 ms  81x

Throughput – small random reads

Filesystem               Read Throughput Improvement
EFS 3 MB/s 1x
ObjectiveFS (cold cache) 10 MB/s 3x
ObjectiveFS (warm disk cache) 54 MB/s 18x
ObjectiveFS (warm memory cache) 387 MB/s 129x

Write Performance

We also compared the write performance with small files. This test writes ~54K small files in the filesystem by unpacking the linux 4.7.2 kernel source tree. We measured the time it takes to untar these files, totaling 762MB on each filesystem and compared the throughput. For writes, ObjectiveFS performance is independent of the cache settings.

Performance – untar

Filesystem Time Write Throughput Improvement
EFS 2283 s 0.3 MB/s 1x
ObjectiveFS 33 s 22 MB/s 69x

Conclusion

For web server workloads, ObjectiveFS has higher performance and lower latency compared to Amazon EFS. ObjectiveFS read latency is up to 81x better than Amazon EFS. The read and write throughputs are also significantly better (up to 129x for reads and 69x for writes).

To see how large file workloads compare, see Large File Performance: Amazon EFS vs ObjectiveFS.

To see how ObjectiveFS performs for your workload, you can download a copy and try it with our free trial.

by ObjectiveFS staff, September 20, 2016
ObjectiveFS is a shared file system for OS X and Linux that automatically scales and gives you unlimited storage. If you have questions or article idea suggestions, please email us at support@objectivefs.com

Reference: Linux kernel source tree: https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.7.2.tar.xz

Get updates about ObjectiveFS