Sign Up Free Log In
Why OFS? Features Pricing Docs FAQ

ObjectiveFS Release Notes

6.2

Release date: April 23, 2019

  • Support AWS S3 transfer acceleration through http proxy
  • Fixed cleaner+ low memory issue (only for clean=2, not enabled by default)
  • Added /opt/bin to search path for fusermount

6.1

Release date: March 23, 2019

  • Amazon S3 only: Use Signature Version 4 (SigV4) in all regions to support the upcoming Amazon S3 deprecation of Signature Version 2 (SigV2) on June 24, 2019 (see AWS announcement). S3 regions that need this upgraded version are:

    • us-east-1 (N. Virginia)
    • us-west-1 (N. California)
    • us-west-2 (Oregon)
    • ap-southeast-1 (Singapore)
    • ap-southeast-2 (Sydney)
    • ap-northeast-1 (Tokyo)
    • eu-west-1 (Ireland)
    • sa-east-1 (Sao Paulo)

  • Added fix to better handle quick unmounts during initial filesystem startup.

5.5.3

Release date: March 23, 2019

  • Backported: Amazon S3 only: Use Signature Version 4 (SigV4) in all regions to support the upcoming Amazon S3 deprecation of Signature Version 2 (SigV2) on June 24, 2019 (see AWS announcement). S3 regions that need this upgraded version are:

    • us-east-1 (N. Virginia)
    • us-west-1 (N. California)
    • us-west-2 (Oregon)
    • ap-southeast-1 (Singapore)
    • ap-southeast-2 (Sydney)
    • ap-northeast-1 (Tokyo)
    • eu-west-1 (Ireland)
    • sa-east-1 (Sao Paulo)

6.0

Release date: January 26, 2019

A. New Features

  • New kernel cache kcache to improve re-read performance (learn more)
  • New cleaner+ storage cleaner (learn more)
  • New rdirplus that reduces stat call overhead for many common workloads (learn more)
  • New fuse_conn to set the max background FUSE connections (learn more)

B. Performance

  • Faster directory listing which can speed up metadata-heavy operations by up to 50%
  • Read-ahead performance improvements, including bigger step size for hpc mode
  • New early-issue for certain read requests for faster read performance
  • Compaction heuristics improvements with focus on metadata and boot time performance
  • Object size optimizations for filesystems in the TB to PB+ ranges
  • Lower latency for parallel directory operations
  • Lower memory usage for active directories and extended attributes
  • Lower memory usage for OS portion of cache for some workloads
  • Faster repeated metadata lookup to lower cpu usage for active workloads

C. General Improvements

  • Support for newer FUSE versions up to 7.26
  • Additional export flag tuning for NFS and Samba exports
  • Read and write operations cpu usage efficiency improvements
  • Increase compaction rate for max compaction level (compact=5)
  • Support newer EC2 instances with Nitro hypervisor for compaction level setting
  • Refresh indexes more often for more up-to-date filesystem size
  • Log availability zone and instance type on the starting line
  • Error message improvements

D. Fixes

  • Fixed link count for directories with block devices
  • Handled directly setting atime when using relatime or noatime mount options
  • Improved cleaning for filesystems which never enabled snapshots when used with the nosnapshots flag

E. Compatibility

  • Preferred block size is now reported as 128KB instead of 4KB for efficient filesystem I/O
  • ObjectiveFS 6.0 uses an extended storage format
    • Version 6.0 can mount filesystems created by all previous versions
    • Version 3.0 and newer can mount filesystems with the 6.0 extended storage format
    • Version 2.1.1 (2014) and older are not compatible with the 6.0 extended storage format, but a filesystem with the 6.0 extended storage format can be downgraded to be compatible to work with these versions if needed.

5.5.2

Release date: January 14, 2019

  • Fixed issue where nodes that create large number of files have a small chance of inode reuse
  • MacOS only: updated FUSE for macOS to version 3.8.3

5.4.2

Release date: January 14, 2019

  • Backported: Fixed issue where nodes that create large number of files have a small chance of inode reuse
  • Backported: MacOS only: updated FUSE for macOS to version 3.8.3

5.5.1

Release date: October 28, 2018

  • MacOS only: updated FUSE for macOS to version 3.8.2

5.5

Release date: June 21, 2018

  • Write performance improvements including improved buffer and link management
  • Improved write speed for remote object stores and high network latency links
  • Improved write speed for geo-distributed object stores
  • Improved write speed for on-premise object stores
  • Improved link bandwidth predictor
  • Improved compressible data write performance
  • New fsavail mount option to set the available filesystem space
  • bulkdata and ocache are now enabled by default
  • Added support for SI/IEC when setting CACHESIZE and DISKCACHE_SIZE
  • No changes in file system format. All versions of ObjectiveFS are compatible with 5.5

5.4.1

Release date: April 10, 2018

  • Fixed issue when using new bulkdata mode with slow links and certain write patterns
  • Improved support for very large memory cache size (500GB+ CACHESIZE)

5.4

Release date: March 26, 2018

  • New cache in object store (ocache) to improve mount time
  • New bulk data mode (bulkdata) to improve performance for filesystems with high write activity
  • New memory reduction option (mboost) to balance performance/memory for larger filesystems
  • Use default ACL handling for removexattr for better AUFS compatibility
  • Improved log messages
  • No changes in file system format. All versions of ObjectiveFS are compatible with 5.4

5.3.1

Release date: December 3, 2017

  • Fixed a resource leak in handling certain region-dependent contention cases, issue introduced in 5.3

5.3

Release date: November 22, 2017

  • Improved change detection heuristics for lower latency updates from other nodes
  • Reduced average write latency to S3 with new buffer management heuristics
  • Added support for getting filesystem passphrase from secrets management tools (learn more)
  • Allow larger writes when hpc option is set for better throughput on high latency links
  • Storage cleaner is enabled by default
  • Improved write throughput for very compressible data
  • Metadata host is automatically tried when no keys are provided
  • New config command option for IAM roles
  • Linux only: Updated network settings for faster detection of low-level connection errors
  • Set disk cache directory to be excluded from backup programs (learn more)
  • Updated config command to trim leading and trailing whitespace from user inputs
  • Added verbose flags -v and -vv for more verbose messages
  • Added diagnostic checks for config file permissions when using -v
  • Fixed request back-off handling when object store returns large number of errors
  • Fixed a case where the default compact level was used if multiple -o options were given
  • Fixes and improvements for error and informational messages
  • No changes in file system format. All versions of ObjectiveFS are compatible with 5.3

5.2

Release date: July 28, 2017

  • Optimized storage layout to improve mount time, list time and access time
  • New compaction algorithm to support the optimized storage layout
  • Added more compaction levels to enable fast storage layout optimization
  • Added storage cleaner to reclaim storage from snapshots
  • Tuned compaction on write to improve filesystem performance
  • Tuned fair queue algorithm to work better with large requests
  • Added new export option to better support remount for NFS/Samba exports
  • Added improvement to return free memory to the Linux kernel faster
  • Increased the number of reported inodes
  • MacOS only: updated FUSE for macOS to version 3.6.3
  • No changes in file system format. All versions of ObjectiveFS are compatible with 5.2

5.1.1

Release date: April 25, 2017

  • Tunneling proxy support for on-premise object store
  • Support for filesystem pool creation using an existing bucket

5.1

Release date: March 26, 2017

  • Performance improvements. Metadata up to 30% faster, large file writes up to 2.5X and large file reads up to 1.4X compared to v.5.0
  • Multithreaded write and compaction
  • Dynamically adjustable object sizes and parallelism on writes
  • Improved compaction heuristics for large filesystems
  • Improved multithreading performance
  • Improved DNS performance: faster retries and reduce number of total queries
  • Improved read and write performance for high-latency cross-region buckets
  • Optimized index layout for large and mixed file size workloads
  • Improved read ahead heuristics
  • Improved responsiveness during large file reads
  • Increased max reported filesystem size
  • Monitor disk cache performance and bypass to S3 if disk cache is too slow
  • Faster identification of DNS configuration changes
  • Google Cloud Storage: use AWS_DEFAULT_REGION on filesystem creation
  • No changes in file system format. All versions of ObjectiveFS are compatible with 5.1

5.0

Release date: January 12, 2017

  • Snapshots: automatic and checkpoint (learn more)
  • S3 Transfer Acceleration support (learn more)
  • Added protection against OOM killer (Linux)
  • Multithreading support on macOS
  • MacOS Sierra v.10.12 support
  • Simplified setup of on-premise object stores by connecting directly to endpoints
  • Simplified use of on-premise object stores with http AWS_DEFAULT_REGION support
  • Improved performance for large sequential reads
  • Improved Samba support for 16GB+ files
  • Better list command handling of large number of filesystems
  • Better compatibility for empty extended attributes
  • Extended config to configure AWS_DEFAULT_REGION
  • Added AWS S3 new regions: Ohio (us-east-2), Canada (ca-central-1), London (eu-west-2)
  • No changes in file system format. All versions of ObjectiveFS are compatible with 5.0

4.3.4

Release date: October 3, 2016

  • Expanded AWS_DEFAULT_REGION support for all commands
  • Added AWS Asia Pacific (Mumbai) region: ap-south-1
  • Added Google Cloud Storage regional locations

4.3.3

Release date: August 15, 2016

  • Fixed a scenario where update propagation between nodes could sometimes be delayed
  • Improved user interface message when creating a filesystem using the default region

4.3.2

Release date: July 26, 2016

  • Linux only: Fixed Windows directory listing interoperability when exporting using Samba/NFS

4.3.1

Release date: June 28, 2016

  • Improved handling of DNS queries for certain network errors

4.2.2

Release date: June 28, 2016

  • Backported: improved handling of DNS queries for certain network errors

4.3

Release date: May 30, 2016

  • Local license check support (learn more)
  • Memory improvements to significantly reduce memory usage
  • Cache changes to increase memory efficiency
  • Disk cache cleaner with faster response and less memory and cpu usage
  • Disk cache cleaner updated to be inode-aware and to work better with EBS disks
  • Improved cache efficiency for reads of large files
  • Compaction rate improvements
  • New flag to enable mount on a non-empty directory
  • Improved help text
  • No changes in file system format. All versions of ObjectiveFS are compatible with 4.3

4.2.1

Release date: May 30, 2016

  • Fix for disk cache cleaner start-up with background mount

4.2

Release date: April 26, 2016

  • Multithreading support for Linux (learn more)
  • I/O thread pools
  • Automatically increases parallel fuse connections for parallel workloads
  • Up to 20% speed up in cached responses
  • Various caching performance improvements
  • Wait for mount directory to be ready before daemonizing
  • Caching statistics logging (learn more)
  • OSX only: support new OSX fuse version 3.2
  • Improved some error messages
  • No changes in file system format. All versions of ObjectiveFS are compatible with 4.2

4.1.6

Release date: April 26, 2016

  • Improved handling of slow S3 list responses

4.1.5

Release date: April 3, 2016

  • Faster retries on truncated S3 list responses

4.1.4

Release date: March 20, 2016

  • Added noratelimit flag to disable S3 requests rate limits

4.1.3

Release date: February 7, 2016

  • Http proxy only: support for getting settings in directory

4.1.2

Release date: January 28, 2016

  • IBM Cleversafe object store change back off timing for retries

4.1.1

Release date: January 25, 2016

  • Fix for fair queue reads, issue introduced in 4.1

4.1

Release date: January 11, 2016

This major release has many new features and improvements including a large reduction in memory usage, HPC support for large sequential reads and writes, http proxy, user id mapping, Amazon server-side encryption (AWS KMS) support and ap-northeast-2 support.

A. Major Changes and Features

  • High-performance computing (HPC) support for faster (100+MB/s) read/write of large sequential files
  • Amazon server side encryption support including AWS Key Management Service (KMS) (learn more)
  • HTTP proxy support to access object store (learn more)
  • User ID and Group ID mapping support (learn more)
  • High-bandwidth mode support when hpc flag is set
  • AWS Asia Pacific (Seoul) region: ap-northeast-2
  • More robust clock skew handling and detection

B. Performance

  • Reduced memory usage for directories and extended attributes
  • Reduced memory usage for the memory cache
  • Updated the main index to a more memory-efficient data structure
  • Updated compaction heuristics to be more aggressive in reducing memory usage
  • Faster compression for small blocks
  • Added fair queue policy to let small requests bypass big/slow requests
  • Lowered the minimum required free space for disk cache for dedicated cache partitions
  • Optimized caching heuristics for web server workload to reduce latency bumps under high loads
  • Improved memory cache performance

C. Others

D. Compatibility

  • No changes in file system format. All versions of ObjectiveFS are compatible with 4.1

4.0.3

Release date: October 31, 2015

  • Reliability fix for compaction (for v.4.0+)

4.0.2

Release date: October 15, 2015

  • OS X only: Added support for El Capitan System Integrity Protection (SIP) and changed kernel interface

4.0.1

Release date: September 28, 2015

  • Added workaround for Amazon S3 us-east-1 issue where incorrect bucket creation status code is returned
  • Removed overly-strict bucket name check in list command to handle non-ObjectiveFS buckets better
  • Improved reporting and error messaging

4.0

Release date: September 8, 2015

This major release has many new features such as disk cache, compaction on write, connection pooling, us-east-1 support and significant performance improvements. These improvements reduce latency, lower memory usage and reduce the number of S3 operations.

A. Major Changes and Features

  • Disk cache support learn more
    • Content is compressed, encrypted and has strong integrity checks
    • Can be shared between multiple filesystems
    • Faster start up time and fewer S3 operations
  • Connection pooling to reduce request latency
  • Compaction on write to reduce the number of S3 objects stored
  • US Standard (us-east-1) region support
  • Faster detection of compaction done on other nodes to lower memory usage
  • Index generation heuristics improvements to lower memory usage and to reduce S3 operations
  • Request queue improvement to reduce latency for parallel read operations

B. Performance

  • Caching algorithm update with more detailed tracking to improve latency for large directories with 100k+ files
  • Cache operations optimizations to improve speed
  • Prioritize metadata in cache
  • Prefetch algorithm improvements for small files
  • Read ahead improvements for very large files
  • Read ahead optimizations to improve performance for multiple streams of videos
  • Lower cpu usage
  • General responsiveness and latency improvements

C. Improvements

  • Reduce default cache size for low memory machines
  • Make use of Linux o_noatime hint
  • Count kernel-referenced data as part of cache usage
  • Retry list operations for additional S3 error conditions
  • Improve user interface error messages
  • Reduce inode memory usage

D. Fixes

  • Fix for potential stale read corner case with very low cache size (only in 3.2)

E. Compatibility

  • No changes in file system format. All versions of ObjectiveFS are compatible with 4.0

3.2

Release date: July 21, 2015

New user interface with easy-to-use commands

  • New config command to easily set up the required variables
  • New list command to display user’s file systems with region and type info
  • Updated create command with simpler usage and ability to specify regions directly
  • Updated mount command with improved messaging and ability to run in background
  • Updated destroy command with simpler usage
  • Simpler installation with OSX package, and Linux rpm and deb packages
  • More detailed log messages for error responses from AWS S3/GCS

Features

  • Multiple file systems per bucket support
  • AWS v4 signature support, including Frankfurt data center support
  • Smart key-based selection of AWS S3 vs GCS storage backend
  • Preliminary support for auto mount on OSX
  • Support for GCS regions
  • Mount on boot support for additional Linux distributions: e.g. Ubuntu, Debian, SUSE
  • OS shutdown signal response handling

Performance Improvements

  • Improved responsiveness for directory listing
  • Improved caching responsiveness to updates from other nodes
  • Significant reduction in list operations for common usage patterns
  • Improved start up performance
  • Improved compaction heuristics
  • Reduced memory usage

Fixes

  • Various GCS compatibility fixes, including adapting to variable response time from GCS

Compatibility

  • No changes in file system format. All versions of ObjectiveFS are compatible with 3.2

3.1

Release date: April 20, 2015

  • Significant reduction of memory footprint
  • Reduced inode cache usage
  • Fixed initial rsync slowdown for certain directory structures
  • Better support for mount on boot
  • Small latency improvement for first uncached directory access
  • Enable compaction by default
  • No changes in file system format. All versions of ObjectiveFS are compatible with 3.1

3.0.1

Release date: March 3, 2015

  • Beta support for Google Cloud Storage
  • Fix for small cache for handling certain workloads
  • Output cache size and S3 region at file system startup
  • No changes in file system format. All versions of ObjectiveFS are compatible with 3.0.1

3.0

Release date: February 25, 2015

A. Major Changes and Features

  • Easy mount on boot
    • Support for mounting file system from /etc/fstab on boot
  • Faster synchronization speed between machines
  • Re-key live file system
    • User can change AWS keys without restarting the file system
  • User settable cache size support
    • User can set the read cache size as a percentage of memory or absolute value
  • Read-only mode mount option
  • Additional access time settings
    • Added reltime, noatime and nodiratime, in addition to strictatime
  • Parallel writes to S3
    • For write performance speed up
  • Compaction support
  • Dropping of root after file system start to run as a different user

B. Performance and Improvements

  • Reduced memory usage for large file systems
  • Improved decompression speed
  • Improved synchronization speed between machines
  • Added environment variables in files support
  • Added additional error handling support

C. Fixes

  • Fixed file hole creation for uncached files
  • Added compatibility with Linux for non-empty directory move target

D. Compatibility

  • No changes in file system format. All versions of ObjectiveFS are compatible with 3.0

2.1.1

Release date: October 26, 2014

  • Better handling of sleeping on OS X
  • Fix for undocumented OS X limitation
  • No changes in file system format. All versions of ObjectiveFS are compatible with 2.1.1

2.1

Release date: July 28, 2014

  • Handle rekeying from Amazon
  • Minor improvements for DNS
  • No changes in file system format. All versions of ObjectiveFS are compatible with 2.1

2.0.1

Release date: July 16, 2014

  • Added support for session security token when using AWS STS
  • No changes in file system format. All versions of ObjectiveFS are compatible with 2.0.1

2.0

Release date: June 3, 2014

A. Major Changes and Features

  • Added unified metadata and data cache
  • Improved handling of Linux ACL
  • Added support for compressed data detection
  • Support environment variables for configuration
  • Support for user’s non-DevPay S3 buckets
  • Improved sync speed to object store

B. Performance and Improvements

  • Improved read ahead algorithm
  • Reduce cache memory use
  • Various performance optimization
  • Faster synchronization between nodes
  • Speed improvement for compressed data

C. Fixes

  • Improved ctime handling
  • Better symlink compatibility

Compatibility

  • No changes in file system format. All versions of ObjectiveFS are compatible with 2.0

Pre 2.0

1.0 (Release date: July 30, 2013)

1.0 Release Candidate (Release date: May 23, 2013)

First public beta (Release date: April 3, 2013)

Get updates about ObjectiveFS