ObjectiveFS automatically allocates more storage as your data grows, without any cluster reconfiguration or storage pre-provision. You get unlimited cloud storage that doesn’t take up any local disk space by default. Files are downloaded on demand, so you get instant access to far larger volumes of data than what your local storage can support.
ObjectiveFS is designed to scale with high performance when you add more servers. We support as many concurrent connections as the object store backend can handle, so you will maintain good throughput as the number of servers increases.
Performance is similar to a local hard drive. We use workload adaptive heuristics, advanced in-memory caching and write bundling to achieve high performance.
It is easy and fast to get up and running with ObjectiveFS. You only need ObjectiveFS running on your servers and an object store such as Amazon S3. No extra metadata server, coordination service or storage cluster is needed, since ObjectiveFS handles all the coordination among the clients through the object store.
We offload your storage cluster management to the cloud provider, so you save time and effort. You don’t need to maintain a storage cluster or set up multiple file servers to get redundant storage. You also don’t have to rebalance the storage cluster or restripe data when adding more capacity, since it is taken care of by the object store. You can avoid split-brain and long healing process seen in other filesystems.
ObjectiveFS supports multiple concurrent readers and writers. Updates are automatically synced and visible to all your machines, so there is no need to maintain scripts or use rsync to copy data anymore.
Your existing software and tools run directly and without modification on your cloud file system. We support standard file system semantics with our POSIX-compliant FUSE-based file system, so all existing Linux and macOS programs just work. No extra software is required to access your cloud data.
You can access your data from anywhere, regardless of where your object store is located. You can now share data between your EC2 instances in all regions, Linux servers in your office and macOS laptops at home by mounting ObjectiveFS.
Your data is stored in an object store such as Amazon S3. Amazon S3 has 99.999999999% durability and redundantly stores your data across multiple facilities. Your data remains persistent and is safely stored in the object store, even when no server is running.
Our end-to-end strong data integrity checks also verify that your file system data is transmitted, stored and received without corruption, protecting against network errors.
ObjectiveFS supports S3-compatible object stores. You can choose to store your data on Amazon S3, Google Cloud Storage or your own on-premise S3-compatible object store.
ObjectiveFS uses strong end-to-end encryption. Your data is encrypted locally on your machine, with a key known only to you, before leaving for the cloud. It stays encrypted at rest and in motion, and is only decrypted when it returns to your machine. ObjectiveFS uses cloud secure end-to-end client side crypto that is designed and implemented to be secure, even in multi-tenant VM environments.
To improve performance and reduce the number of S3 operations, ObjectiveFS caches data and metadata in memory. The memory cache size is user-configurable and cache statistics are shown in the log file. learn more
ObjectiveFS can also use local disks for caching. Using a disk cache can lower file access latency and increase data throughput. The disk cache’s content persists when your filesystem is unmounted, so it is fast to restart and access when you remount your filesystem. ObjectiveFS handles the disk cache operations automatically without additional maintenance from users. learn more
ObjectiveFS can leverage the kernel cache to speed up re-read performance and is particularly useful for web server and simulation workloads. ObjectiveFS also has an object store cache to speed up mount time for large filesystems.
Multithreading can lower latency and improve throughput for your workload. ObjectiveFS will spawn dedicated CPU and IO threads to handle operations such as data decompression, data integrity check, disk cache accesses and updates. learn more
Compaction improves performance and reduces the number of object store operations needed for filesystem access. It combines multiple small objects in the object store into a larger object and brings related data close together. Compaction is a background process and adjusts dynamically depending on your workload and your filesystem’s characteristics. learn more
ObjectiveFS uses fast compression algorithm to compress your data before sending them to the object store. It dynamically detects compressible data and skips over already compressed data.
ObjectiveFS supports AWS S3 Transfer Acceleration that enables fast transfers of files over long distances between your server and S3 bucket. learn more
ObjectiveFS supports tuning options for different workload types, including high performance computing mode that trades off throughput over latency for high throughput workloads and bulk data mode for large file workloads.
ObjectiveFS has native support for ARM64 (NEON), x86-64-v4 (AVX-512), x86-64-v3 (AVX2), x86-64-v2, x86-64-v1. SHA-1, SHA-256 and AES native instructions are also supported. The smart binary uses the fastest vector instructions available on the platform to achieve the best performance.
ObjectiveFS supports both automatic snapshots and checkpoint/manual point-in-time snapshots. Snapshots can be mounted as a read-only filesystem to access your filesystem data as it was at that point in time. This is useful to recover accidentally deleted or modified files or as the source for creating a consistent point-in-time backup. learn more
While our regular license check is very robust and can handle multi-day outages, some companies prefer to minimize external dependencies. For these cases, we offer a local license check feature that lets you run your infrastructure independent of any license server. learn more
You can use IAM roles for Amazon EC2 to automatically manage your S3 keys. ObjectiveFS is aware of the expiration time of your keys and will automatically renew the keys before it expires, so that your filesystem will continue to run without interruptions. learn more
ObjectiveFS supports live rekeying which lets you update your AWS keys while keeping your filesystem mounted. ObjectiveFS supports both automatic rekeying using IAM roles and manual rekeying. learn more
Filesystem pool lets you have multiple file systems per S3 bucket. A filesystem pool simplifies the management of lots of filesystems. You can also use pools to organize your company’s file systems by teams or departments. learn more
The admin mode provides an easy way to manage filesystems in a programmatic way. You can use the admin mode to easily script the creation and deletion of filesystems. learn more
The user and group id mapping feature lets you map local user ids and group ids to different ids in the remote filesystem. This feature is useful when managing users on systems with colliding ids and lets you map each user to a unique id. learn more
ObjectiveFS can use an http proxy to connect to your object store. This makes it easy to comply with your corporate network policies or to connect to the object store via a caching proxy. learn more
Your data is encrypted locally on your server, with a key known only to you. Your data remains encrypted at rest and in transit. ObjectiveFS uses the standard NaCl crypto library using Salsa20 and Poly1305 algorithms and protects your data against cache timing attacks. There is also no data-dependent branches or data dependent array indices, so your data is securely protected even in multi-tenant VM environments.
Your data is always encrypted by client-side encryption that protects your data in transit and at rest. If you need additional server side encryption for compliance, ObjectiveFS also supports AWS server-side encryption using AWS KMS-managed keys and Amazon S3-managed keys. learn more
ObjectiveFS uses cryptographically strong checksums to verify your data hasn’t been accidentally or maliciously modified. It will always retry operations if the checksum doesn’t match.
ObjectiveFS is a log-structured filesystem using an object store for persistent storage. It is a POSIX-compliant FUSE-based file system for Linux and macOS and works natively with all Linux and macOS tools and software.
hpc that trades off throughput over latency for high throughput workloads, bulkdata for large file workloads
Throughput number is for large compressed files (~1GB each) running on m3.2xlarge instances