Features

Unlimited Storage that Scales Automatically

ObjectiveFS automatically allocates more storage as your data grows, without any cluster reconfiguration or storage pre-provision. You get unlimited cloud storage that doesn’t take up any local disk space by default. Files are downloaded on demand, so you get instant access to far larger volumes of data than what your local storage can support.

Scales Out with High Performance

ObjectiveFS is designed to scale with high performance when you add more servers. We support as many concurrent connections as the object store backend can handle, so you will maintain good throughput as the number of servers increase.

Performance is similar to a local hard drive. We use workload adaptive heuristics, advanced in-memory caching and write bundling to achieve high performance.

Easy and Fast to Set Up

It is easy and fast to get up and running with ObjectiveFS. You only need ObjectiveFS running on your servers and an object store such as Amazon S3. No extra metadata server, coordination service or storage cluster is needed, since ObjectiveFS handles all the coordination among the clients through the object store.

No Storage Cluster Management

We offload your storage cluster management to the cloud provider, so you save time and effort. You don’t need to maintain a storage cluster or set up multiple file servers to get redundant storage. You also don’t have to rebalance the storage cluster or restripe data when adding more capacity, since it is taken care of by the object store. You can avoid split-brain and long healing process seen in other filesystems.

Live File System

ObjectiveFS supports multiple concurrent readers and writers. Updates are automatically synced and visible to all your machines, so there is no need to maintain scripts or use rsync to copy data anymore.

POSIX Compliant

Your existing software and tools run directly and without modification on your cloud file system. We support standard file system semantics with our POSIX-compliant FUSE-based file system, so all existing Linux and macOS programs just work. No extra software is required to access your cloud data.

Access from Everywhere

You can access your data from anywhere, regardless of where your object store is located. You can now share data between your EC2 instances in all regions, Linux servers in your office and macOS laptops at home by mounting ObjectiveFS.

Reliable and Redundant

Your data is stored in an object store such as Amazon S3. Amazon S3 has 99.999999999% durability and redundantly stores your data across multiple facilities. Your data remains persistent and is safely stored in the object store, even when no server is running.

Our end-to-end strong data integrity checks also verify that your file system data is transmitted, stored and received without corruption, protecting against network errors.

Choose your own cloud

ObjectiveFS supports S3-compatible object stores. You can choose to store your data on Amazon S3, Google Cloud Storage or your own on-premise S3-compatible object store.

Encrypted Data at Rest and in Motion

ObjectiveFS uses strong end-to-end encryption. Your data is encrypted locally on your machine, with a key known only to you, before leaving for the cloud. It stays encrypted at rest and in motion, and is only decrypted when it returns to your machine. ObjectiveFS uses cloud secure end-to-end client side crypto that is designed and implemented to be secure, even in multi-tenant VM environments.

Advanced Features

Performance

Management

Security and Data Integrity


Performance

Memory and Disk Cache

To improve performance and reduce the number of S3 operations, ObjectiveFS caches data and metadata in memory. The memory cache size is user-configurable and cache statistics are shown in the log file. learn more

ObjectiveFS can also use local disks for caching. Using a disk cache can lower file access latency and increase data throughput. The disk cache’s content persists when your filesystem is unmounted, so it is fast to restart and access when you remount your filesystem. ObjectiveFS handles the disk cache operations automatically without additional maintenance from users. learn more

Multithreading

Multithreading can lower latency and improve throughput for your workload. ObjectiveFS will spawn dedicated CPU and IO threads to handle operations such as data decompression, data integrity check, disk cache accesses and updates. learn more

Compaction

Compaction improves performance and reduces the number of object store operations needed for filesystem access. It combines multiple small objects in the object store into a larger object and brings related data close together. Compaction is a background process and adjusts dynamically depending on your workload and your filesystem’s characteristics. learn more

Compression

ObjectiveFS uses fast compression algorithm to compress your data before sending them to the object store. It dynamically detects compressible data and skips over already compressed data.

S3 Transfer Acceleration

ObjectiveFS supports AWS S3 Transfer Acceleration that enables fast transfers of files over long distances between your server and S3 bucket. learn more

Management

Snapshots

ObjectiveFS supports both automatic snapshots and checkpoint/manual point-in-time snapshots. Snapshots can be mounted as a read-only filesystem to access your filesystem data as it was at that point in time. There is no limit to the number of filesystem snapshots that you can create and use. learn more

Local License Check

While our regular license check is very robust and can handle multi-day outages, some companies prefer to minimize external dependencies. For these cases, we offer a local license check feature that lets you run your infrastructure independent of any license server. learn more

IAM Roles for EC2

You can use IAM roles for Amazon EC2 to automatically manage your S3 keys. ObjectiveFS is aware of the expiration time of your keys and will automatically renew the keys before it expires, so that your filesystem will continue to run without interruptions. learn more

Live Rekeying

ObjectiveFS supports live rekeying which lets you update your AWS keys while keeping your filesystem mounted. ObjectiveFS supports both automatic rekeying using IAM roles and manual rekeying. learn more

Filesystem Pool

Filesystem pool lets you have multiple file systems per S3 bucket. A filesystem pool simplifies the management of lots of filesystems. You can also use pools to organize your company’s file systems by teams or departments. learn more

Admin Mode

The admin mode provides an easy way to manage filesystems in a programmatic way. You can use the admin mode to easily script the creation and deletion of filesystems. learn more

User/Group ID Mapping

The user and group id mapping feature lets you map local user ids and group ids to different ids in the remote filesystem. This feature is useful when managing users on systems with colliding ids and lets you map each user to a unique id. learn more

HTTP Proxy Support

ObjectiveFS can use an http proxy to connect to your object store. This makes it easy to comply with your corporate network policies or to connect to the object store via a caching proxy. learn more

Security and Data Integrity

End-to-End Encryption

Your data is encrypted locally on your server, with a key known only to you. Your data remains encrypted at rest and in transit. ObjectiveFS uses the standard NaCl crypto library using Salsa20 and Poly1305 algorithms and protects your data against cache timing attacks. There is also no data-dependent branches or data dependent array indices, so your data is securely protected even in multi-tenant VM environments.

AWS KMS and Server Side Encryption

Your data is always encrypted by client-side encryption that protects your data in transit and at rest. If you need additional server side encryption for compliance, ObjectiveFS also supports AWS server-side encryption using AWS KMS-managed keys and Amazon S3-managed keys. learn more

Strong Data Integrity Checks

ObjectiveFS uses cryptographically strong checksums to verify your data hasn’t been accidentally or maliciously modified. It will always retry operations if the checksum doesn’t match.

Technical Specification

ObjectiveFS is a log-structured filesystem using an object store for persistent storage. It is a POSIX-compliant FUSE-based file system for Linux and macOS and works natively with all Linux and macOS tools and software.

Supported Backends

Performance

Scalability

POSIX features

Security

Management

Other Features

Amazon S3 Shared File System Throughput Performance

Throughput number is for large compressed files (~1GB each) running on m3.2xlarge instances

Get updates about ObjectiveFS