ObjectiveFS Release Notes
6.8
Release date: March 5, 2021
A. New Features
- New storage cleaner architecture with more efficient compaction and improved cleaning heuristic (learn more)
- New Size Tiered and Time Based dynamic compaction heuristic
- New multithreaded storage cleaner and compactor
- New object store tier aware compaction algorithm
- New
freebw, autofreebw mount options
- New
mtplus mount option (learn more)
- New compaction progress monitoring and logging (learn more)
- A free cleaner thread for all users and additional cleaner threads with multithreading
B. Performance
- Improved worker threads scaling algorithm
- Improved storage cleaner heuristics when used with snapshots
- Increased concurrency for compaction operations at higher compaction levels
- Improved disk cache performance for larger disk caches
- Performance optimizations for Petabyte-size filesystems
- Optimized object sizes for improved object store performance
- Increased ocache size for very large filesystems
- Improved storage layout heuristics
- Faster performance for creation of lots of small files
- Faster response time in the event of object store errors
- Increased I/O threads for regular multithreading from 8 to 16
C. General Improvements
- Improved EC2 detection for autofreebw and compaction level setting
- Used newer AWS metadata server API version
- Tuned concurrency to match updated AWS recommendations
- Tuned compaction to utilize file system idle periods
- Support filesystem creation in an existing bucket with restrictive permission
- Adjusted disk cache queue response time
- Tuned compaction on write for metadata and data
- Tuned quick sync heuristic
- Unified endpoint support for all AWS regions
- Adjusted compaction rate limits
- Updated various logging messages and exit codes
- Increased retry handling during initial startup
- MacOS only: Increased backwards compatibility to macOS 10.8
D. Fixes
6.7.2
Release date: June 16, 2020
- Fix compability when using http proxy and sigv2 with non-AWS object stores
6.7.1
Release date: April 25, 2020
- Linux only: Make EC2 Instance Metadata Service v2 support work better with Docker
6.7
Release date: April 9, 2020
- Dynamic scaling of threads to improve efficiency and resource usage
- Reduced memory usage when running with multithreading
- Speed up performance when running with disk cache
- Support for EC2 Instance Metadata Service v2 (IMDSv2)
- Improved disk cache performance resiliency when dealing with slow or faulty local disks
- Improved performance for small file operations
- Improved cleaner+ efficiency for active directories and files with many hard links
- Improved write queue management to better handle contention during slow responses from the object store
- Reduced max CPU usage for some workloads with improved heuristics
- Reduced the number of GET requests when running with multiple nodes
- Tuned S3 connection settings
- Tuned destroy filesystem operation
- Added beta support for workflows that communicate new filenames through a separate channel
- Report final statistics to log file upon shutdown
- Error message improvements
- Compatibility: All releases after 2.1.1 are compatible with this release.
6.6
Release date: January 6, 2020
- New memory caching algorithm for faster cache operations and better memory usage
- New heuristic for memory cache to better adapt to workload changes over time
- Improved parallelism for backend operations to speed up performance
- Reduced memory cache usage for active directories
- Added several optimizations to reduce OS memory usage
- Reduced number of FUSE interface operations in certain cases
- Added fix for
rdirplus memory usage
- Added fix to address a case that could cause high cpu usage
- Amazon S3 only: use ISO 8601 date format for x-amz-date header for http proxy connections
- MacOS only: added support for inode creation time queries
- Various minor improvements for write workload performance and checkpoint snapshots
- Compatibility: All releases after 2.1.1 are compatible with this release
6.5
Release date: November 28, 2019
A. New Features
- Extended ACL support for Linux with new
acl mount option learn more
- Support for immutable and append-only file attributes for Linux
- File capabilities support learn more
- File attributes support for Linux learn more
- File flags support for macOS learn more
- Improved xattr support including more xattr namespace compatibility
B. Improvements
- Faster cross-node locking with new directory locking algorithm
- Improved synchronization speed between nodes
- Improved speed and efficiency of cleaner+
- Improved 99th percentile latency for filesystem operations
- Improved write queue management to object store
- Match atime, ctime and mtime updates closer with ext4
C. General
- Additional error message logging for http retries
- Support for macOS Catalina
- Support for newer FUSE versions up to 7.31
- Moved oldest supported FUSE version to 7.12
- Compatibility: All releases after 2.1.1 are compatible with this release
6.4
Release date: August 18, 2019
- Important fix for cleaner+ in the presence of clock skew between nodes (only for
clean=2, not enabled by default)
- Fixed an issue which can cause a directory to have slow file creation
- Improved
ocache handling of updates from slow writer nodes
- Improved
rdirplus inode memory usage
- EC2 only: Nitro instance type and region are now reported in syslog
- Compatibility: All releases after 2.1.1 are compatible with this release
6.3
Release date: June 24, 2019
- Updated memory allocator to be more robust in low memory situations
- New
nomem mount option to select mount behavior when out of memory
- New
retry mount option for retrying connection to object store upon start up
- Cleaner+ now skips ocache on initial mount
- MacOS only: updated FUSE for macOS to version 3.9.2
- Minor error message updates
- Compatibility: All releases after 2.1.1 are compatible with this release
6.2
Release date: April 23, 2019
- Support AWS S3 transfer acceleration through http proxy
- Fixed cleaner+ low memory issue (only for
clean=2, not enabled by default)
- Added
/opt/bin to search path for fusermount
- Compatibility: All releases after 2.1.1 are compatible with this release
6.1
Release date: March 23, 2019
-
Amazon S3 only: Use Signature Version 4 (SigV4) in all regions to support the upcoming Amazon S3 deprecation of Signature Version 2 (SigV2) on June 24, 2019 (see AWS announcement). S3 regions that need this upgraded version are:
- us-east-1 (N. Virginia)
- us-west-1 (N. California)
- us-west-2 (Oregon)
- ap-southeast-1 (Singapore)
- ap-southeast-2 (Sydney)
- ap-northeast-1 (Tokyo)
- eu-west-1 (Ireland)
- sa-east-1 (Sao Paulo)
-
Added fix to better handle quick unmounts during initial filesystem startup
- Compatibility: All releases after 2.1.1 are compatible with this release
5.5.3
Release date: March 23, 2019
6.0
Release date: January 26, 2019
A. New Features
- New kernel cache
kcache to improve re-read performance (learn more)
- New cleaner+ storage cleaner
- New
rdirplus that reduces stat call overhead for many common workloads (learn more)
- New
fuse_conn to set the max background FUSE connections (learn more)
B. Performance
- Faster directory listing which can speed up metadata-heavy operations by up to 50%
- Read-ahead performance improvements, including bigger step size for hpc mode
- New early-issue for certain read requests for faster read performance
- Compaction heuristics improvements with focus on metadata and boot time performance
- Object size optimizations for filesystems in the TB to PB+ ranges
- Lower latency for parallel directory operations
- Lower memory usage for active directories and extended attributes
- Lower memory usage for OS portion of cache for some workloads
- Faster repeated metadata lookup to lower cpu usage for active workloads
C. General Improvements
- Support for newer FUSE versions up to 7.26
- Additional
export flag tuning for NFS and Samba exports
- Read and write operations cpu usage efficiency improvements
- Increase compaction rate for max compaction level (
compact=5)
- Support newer EC2 instances with Nitro hypervisor for compaction level setting
- Refresh indexes more often for more up-to-date filesystem size
- Log availability zone and instance type on the starting line
- Error message improvements
D. Fixes
- Fixed link count for directories with block devices
- Handled directly setting atime when using relatime or noatime mount options
- Improved cleaning for filesystems which never enabled snapshots when used with the
nosnapshots flag
E. Compatibility
- Preferred block size is now reported as 128KB instead of 4KB for efficient filesystem I/O
- ObjectiveFS 6.0 uses an extended storage format
- Version 6.0 can mount filesystems created by all previous versions
- Version 3.0 and newer can mount filesystems with the 6.0 extended storage format
- Version 2.1.1 (2014) and older are not compatible with the 6.0 extended storage format, but a filesystem with the 6.0 extended storage format can be downgraded to be compatible to work with these versions if needed.
5.5.2
Release date: January 14, 2019
- Fixed issue where nodes that create large number of files have a small chance of inode reuse
- MacOS only: updated FUSE for macOS to version 3.8.3
5.4.2
Release date: January 14, 2019
- Backported: Fixed issue where nodes that create large number of files have a small chance of inode reuse
- Backported: MacOS only: updated FUSE for macOS to version 3.8.3
5.5.1
Release date: October 28, 2018
- MacOS only: updated FUSE for macOS to version 3.8.2
5.5
Release date: June 21, 2018
- Write performance improvements including improved buffer and link management
- Improved write speed for remote object stores and high network latency links
- Improved write speed for geo-distributed object stores
- Improved write speed for on-premise object stores
- Improved link bandwidth predictor
- Improved compressible data write performance
- New
fsavail mount option to set the available filesystem space
bulkdata and ocache are now enabled by default- Added support for SI/IEC when setting
CACHESIZE and DISKCACHE_SIZE
- No changes in file system format. All versions of ObjectiveFS are compatible with 5.5
5.4.1
Release date: April 10, 2018
- Fixed issue when using new bulkdata mode with slow links and certain write patterns
- Improved support for very large memory cache size (500GB+ CACHESIZE)
5.4
Release date: March 26, 2018
- New cache in object store (ocache) to improve mount time
- New bulk data mode (bulkdata) to improve performance for filesystems with high write activity
- New memory reduction option (mboost) to balance performance/memory for larger filesystems
- Use default ACL handling for removexattr for better AUFS compatibility
- Improved log messages
- No changes in file system format. All versions of ObjectiveFS are compatible with 5.4
5.3.1
Release date: December 3, 2017
- Fixed a resource leak in handling certain region-dependent contention cases, issue introduced in 5.3
5.3
Release date: November 22, 2017
- Improved change detection heuristics for lower latency updates from other nodes
- Reduced average write latency to S3 with new buffer management heuristics
- Added support for getting filesystem passphrase from secrets management tools (learn more)
- Allow larger writes when hpc option is set for better throughput on high latency links
- Storage cleaner is enabled by default
- Improved write throughput for very compressible data
- Metadata host is automatically tried when no keys are provided
- New config command option for IAM roles
- Linux only: Updated network settings for faster detection of low-level connection errors
- Set disk cache directory to be excluded from backup programs (learn more)
- Updated config command to trim leading and trailing whitespace from user inputs
- Added verbose flags
-v and -vv for more verbose messages
- Added diagnostic checks for config file permissions when using
-v
- Fixed request back-off handling when object store returns large number of errors
- Fixed a case where the default compact level was used if multiple
-o options were given
- Fixes and improvements for error and informational messages
- No changes in file system format. All versions of ObjectiveFS are compatible with 5.3
5.2
Release date: July 28, 2017
- Optimized storage layout to improve mount time, list time and access time
- New compaction algorithm to support the optimized storage layout
- Added more compaction levels to enable fast storage layout optimization
- Added storage cleaner to reclaim storage from snapshots
- Tuned compaction on write to improve filesystem performance
- Tuned fair queue algorithm to work better with large requests
- Added new
export option to better support remount for NFS/Samba exports
- Added improvement to return free memory to the Linux kernel faster
- Increased the number of reported inodes
- MacOS only: updated FUSE for macOS to version 3.6.3
- No changes in file system format. All versions of ObjectiveFS are compatible with 5.2
5.1.1
Release date: April 25, 2017
- Tunneling proxy support for on-premise object store
- Support for filesystem pool creation using an existing bucket
5.1
Release date: March 26, 2017
- Performance improvements. Metadata up to 30% faster, large file writes up to 2.5X and large file reads up to 1.4X compared to v.5.0
- Multithreaded write and compaction
- Dynamically adjustable object sizes and parallelism on writes
- Improved compaction heuristics for large filesystems
- Improved multithreading performance
- Improved DNS performance: faster retries and reduce number of total queries
- Improved read and write performance for high-latency cross-region buckets
- Optimized index layout for large and mixed file size workloads
- Improved read ahead heuristics
- Improved responsiveness during large file reads
- Increased max reported filesystem size
- Monitor disk cache performance and bypass to S3 if disk cache is too slow
- Faster identification of DNS configuration changes
- Google Cloud Storage: use AWS_DEFAULT_REGION on filesystem creation
- No changes in file system format. All versions of ObjectiveFS are compatible with 5.1
5.0
Release date: January 12, 2017
- Snapshots: automatic and checkpoint (learn more)
- S3 Transfer Acceleration support (learn more)
- Added protection against OOM killer (Linux)
- Multithreading support on macOS
- MacOS Sierra v.10.12 support
- Simplified setup of on-premise object stores by connecting directly to endpoints
- Simplified use of on-premise object stores with http AWS_DEFAULT_REGION support
- Improved performance for large sequential reads
- Improved Samba support for 16GB+ files
- Better list command handling of large number of filesystems
- Better compatibility for empty extended attributes
- Extended config to configure AWS_DEFAULT_REGION
- Added AWS S3 new regions: Ohio (us-east-2), Canada (ca-central-1), London (eu-west-2)
- No changes in file system format. All versions of ObjectiveFS are compatible with 5.0
4.3.4
Release date: October 3, 2016
- Expanded AWS_DEFAULT_REGION support for all commands
- Added AWS Asia Pacific (Mumbai) region: ap-south-1
- Added Google Cloud Storage regional locations
4.3.3
Release date: August 15, 2016
- Fixed a scenario where update propagation between nodes could sometimes be delayed
- Improved user interface message when creating a filesystem using the default region
4.3.2
Release date: July 26, 2016
- Linux only: Fixed Windows directory listing interoperability when exporting using Samba/NFS
4.3.1
Release date: June 28, 2016
- Improved handling of DNS queries for certain network errors
4.2.2
Release date: June 28, 2016
- Backported: improved handling of DNS queries for certain network errors
4.3
Release date: May 30, 2016
- Local license check support (learn more)
- Memory improvements to significantly reduce memory usage
- Cache changes to increase memory efficiency
- Disk cache cleaner with faster response and less memory and cpu usage
- Disk cache cleaner updated to be inode-aware and to work better with EBS disks
- Improved cache efficiency for reads of large files
- Compaction rate improvements
- New flag to enable mount on a non-empty directory
- Improved help text
- No changes in file system format. All versions of ObjectiveFS are compatible with 4.3
4.2.1
Release date: May 30, 2016
- Fix for disk cache cleaner start-up with background mount
4.2
Release date: April 26, 2016
- Multithreading support for Linux (learn more)
- I/O thread pools
- Automatically increases parallel fuse connections for parallel workloads
- Up to 20% speed up in cached responses
- Various caching performance improvements
- Wait for mount directory to be ready before daemonizing
- Caching statistics logging (learn more)
- OSX only: support new OSX fuse version 3.2
- Improved some error messages
- No changes in file system format. All versions of ObjectiveFS are compatible with 4.2
4.1.6
Release date: April 26, 2016
- Improved handling of slow S3 list responses
4.1.5
Release date: April 3, 2016
- Faster retries on truncated S3 list responses
4.1.4
Release date: March 20, 2016
- Added noratelimit flag to disable S3 requests rate limits
4.1.3
Release date: February 7, 2016
- Http proxy only: support for getting settings in directory
4.1.2
Release date: January 28, 2016
- IBM Cleversafe object store change back off timing for retries
4.1.1
Release date: January 25, 2016
- Fix for fair queue reads, issue introduced in 4.1
4.1
Release date: January 11, 2016
This major release has many new features and improvements including a large reduction in memory usage, HPC support for large sequential reads and writes, http proxy, user id mapping, Amazon server-side encryption (AWS KMS) support and ap-northeast-2 support.
A. Major Changes and Features
- High-performance computing (HPC) support for faster (100+MB/s) read/write of large sequential files
- Amazon server side encryption support including AWS Key Management Service (KMS) (learn more)
- HTTP proxy support to access object store (learn more)
- User ID and Group ID mapping support (learn more)
- High-bandwidth mode support when hpc flag is set
- AWS Asia Pacific (Seoul) region: ap-northeast-2
- More robust clock skew handling and detection
B. Performance
- Reduced memory usage for directories and extended attributes
- Reduced memory usage for the memory cache
- Updated the main index to a more memory-efficient data structure
- Updated compaction heuristics to be more aggressive in reducing memory usage
- Faster compression for small blocks
- Added fair queue policy to let small requests bypass big/slow requests
- Lowered the minimum required free space for disk cache for dedicated cache partitions
- Optimized caching heuristics for web server workload to reduce latency bumps under high loads
- Improved memory cache performance
C. Others
D. Compatibility
- No changes in file system format. All versions of ObjectiveFS are compatible with 4.1
4.0.3
Release date: October 31, 2015
- Reliability fix for compaction (for v.4.0+)
4.0.2
Release date: October 15, 2015
- OS X only: Added support for El Capitan System Integrity Protection (SIP) and changed kernel interface
4.0.1
Release date: September 28, 2015
- Added workaround for Amazon S3 us-east-1 issue where incorrect bucket creation status code is returned
- Removed overly-strict bucket name check in list command to handle non-ObjectiveFS buckets better
- Improved reporting and error messaging
4.0
Release date: September 8, 2015
This major release has many new features such as disk cache, compaction on write, connection pooling, us-east-1 support and significant performance improvements. These improvements reduce latency, lower memory usage and reduce the number of S3 operations.
A. Major Changes and Features
- Disk cache support learn more
- Content is compressed, encrypted and has strong integrity checks
- Can be shared between multiple filesystems
- Faster start up time and fewer S3 operations
- Connection pooling to reduce request latency
- Compaction on write to reduce the number of S3 objects stored
- US Standard (us-east-1) region support
- Faster detection of compaction done on other nodes to lower memory usage
- Index generation heuristics improvements to lower memory usage and to reduce S3 operations
- Request queue improvement to reduce latency for parallel read operations
B. Performance
- Caching algorithm update with more detailed tracking to improve latency for large directories with 100k+ files
- Cache operations optimizations to improve speed
- Prioritize metadata in cache
- Prefetch algorithm improvements for small files
- Read ahead improvements for very large files
- Read ahead optimizations to improve performance for multiple streams of videos
- Lower cpu usage
- General responsiveness and latency improvements
C. Improvements
- Reduce default cache size for low memory machines
- Make use of Linux o_noatime hint
- Count kernel-referenced data as part of cache usage
- Retry list operations for additional S3 error conditions
- Improve user interface error messages
- Reduce inode memory usage
D. Fixes
- Fix for potential stale read corner case with very low cache size (only in 3.2)
E. Compatibility
- No changes in file system format. All versions of ObjectiveFS are compatible with 4.0
3.2
Release date: July 21, 2015
New user interface with easy-to-use commands
- New
config command to easily set up the required variables
- New
list command to display user’s file systems with region and type info
- Updated
create command with simpler usage and ability to specify regions directly
- Updated
mount command with improved messaging and ability to run in background
- Updated
destroy command with simpler usage
- Simpler installation with OSX package, and Linux rpm and deb packages
- More detailed log messages for error responses from AWS S3/GCS
Features
- Multiple file systems per bucket support
- AWS v4 signature support, including Frankfurt data center support
- Smart key-based selection of AWS S3 vs GCS storage backend
- Preliminary support for auto mount on OSX
- Support for GCS regions
- Mount on boot support for additional Linux distributions: e.g. Ubuntu, Debian, SUSE
- OS shutdown signal response handling
Performance Improvements
- Improved responsiveness for directory listing
- Improved caching responsiveness to updates from other nodes
- Significant reduction in list operations for common usage patterns
- Improved start up performance
- Improved compaction heuristics
- Reduced memory usage
Fixes
- Various GCS compatibility fixes, including adapting to variable response time from GCS
Compatibility
- No changes in file system format. All versions of ObjectiveFS are compatible with 3.2
3.1
Release date: April 20, 2015
- Significant reduction of memory footprint
- Reduced inode cache usage
- Fixed initial rsync slowdown for certain directory structures
- Better support for mount on boot
- Small latency improvement for first uncached directory access
- Enable compaction by default
- No changes in file system format. All versions of ObjectiveFS are compatible with 3.1
3.0.1
Release date: March 3, 2015
- Beta support for Google Cloud Storage
- Fix for small cache for handling certain workloads
- Output cache size and S3 region at file system startup
- No changes in file system format. All versions of ObjectiveFS are compatible with 3.0.1
3.0
Release date: February 25, 2015
A. Major Changes and Features
- Easy mount on boot
- Support for mounting file system from /etc/fstab on boot
- Faster synchronization speed between machines
- Re-key live file system
- User can change AWS keys without restarting the file system
- User settable cache size support
- User can set the read cache size as a percentage of memory or absolute value
- Read-only mode mount option
- Additional access time settings
- Added reltime, noatime and nodiratime, in addition to strictatime
- Parallel writes to S3
- For write performance speed up
- Compaction support
- Dropping of root after file system start to run as a different user
B. Performance and Improvements
- Reduced memory usage for large file systems
- Improved decompression speed
- Improved synchronization speed between machines
- Added environment variables in files support
- Added additional error handling support
C. Fixes
- Fixed file hole creation for uncached files
- Added compatibility with Linux for non-empty directory move target
D. Compatibility
- No changes in file system format. All versions of ObjectiveFS are compatible with 3.0
2.1.1
Release date: October 26, 2014
- Better handling of sleeping on OS X
- Fix for undocumented OS X limitation
- No changes in file system format. All versions of ObjectiveFS are compatible with 2.1.1
2.1
Release date: July 28, 2014
- Handle rekeying from Amazon
- Minor improvements for DNS
- No changes in file system format. All versions of ObjectiveFS are compatible with 2.1
2.0.1
Release date: July 16, 2014
- Added support for session security token when using AWS STS
- No changes in file system format. All versions of ObjectiveFS are compatible with 2.0.1
2.0
Release date: June 3, 2014
A. Major Changes and Features
- Added unified metadata and data cache
- Improved handling of Linux ACL
- Added support for compressed data detection
- Support environment variables for configuration
- Support for user’s non-DevPay S3 buckets
- Improved sync speed to object store
B. Performance and Improvements
- Improved read ahead algorithm
- Reduce cache memory use
- Various performance optimization
- Faster synchronization between nodes
- Speed improvement for compressed data
C. Fixes
- Improved ctime handling
- Better symlink compatibility
Compatibility
- No changes in file system format. All versions of ObjectiveFS are compatible with 2.0
Pre 2.0
1.0 (Release date: July 30, 2013)
1.0 Release Candidate (Release date: May 23, 2013)
First public beta (Release date: April 3, 2013)
