The ObjectiveFS CSI driver can be used to let Kubernetes provision and manage the full lifetime of an ObjectiveFS filesystem. Kubernetes will then automatically create and destroy your filesystems based on the Kubernetes configuration. The ObjectiveFS CSI driver is integrated in the democratic-csi project.
Important: When Kubernetes fully manages the storage, your Kubernetes-managed filesystems will be destroyed and your data will be permanently deleted when Kubernetes determines that the storage is no longer needed.
For the common use case of using the CSI driver to mount your existing filesystems or for persistent storage, see this document.
I. Create the following three YAML files:
a. helm.yaml
b. pvc.yaml
c. pod.yaml
This helm values file configures the Objectivefs CSI driver during installation. The driver section uses the admin key to create and destroy the filesystems. The storageClasses section specificies the mount options and environment variables used to mount the filesystems.
csiDriver:
name: "org.democratic-csi.node-manual"
fsGroupPolicy: File
storageClasses:
- name: objectivefs
defaultClass: false
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: false
parameters:
fsType: objectivefs
mountOptions:
# Specify ObjectiveFS mount options here (note 1)
- mt # optional
secrets:
provisioner-secret:
controller-publish-secret:
node-stage-secret:
# Specify ObjectiveFS environment variables here (note 2)
"env.OBJECTSTORE": "<OBJECT STORE PREFIX>"
"env.OBJECTIVEFS_PASSPHRASE": "<YOUR FILESYSTEM PASSPHRASE>"
"env.OBJECTIVEFS_LICENSE": "<YOUR LICENSE KEY>"
"env.ACCESS_KEY": "<YOUR OBJECT STORE ACCESS KEY>"
"env.SECRET_KEY": "<YOUR OBJECT STORE SECRET KEY>"
node-publish-secret:
controller-expand-secret:
controller:
externalSnapshotter:
enabled: false
externalResizer:
enabled: false
# Forwards ObjectiveFS logs to the host
driver:
extraVolumeMounts:
- mountPath: /dev/log
name: dev-log
extraVolumes:
- hostPath:
path: /dev/log
type: ""
name: dev-log
node:
# Forwards ObjectiveFS logs to the host
driver:
extraVolumeMounts:
- mountPath: /dev/log
name: dev-log
extraVolumes:
- hostPath:
path: /dev/log
type: ""
name: dev-log
driver:
config:
driver: objectivefs
objectivefs:
pool: <YOUR EMPTY BUCKET NAME> # (note 3)
cli:
sudoEnabled: false
env:
# Specify ObjectiveFS environment variables here (note 4)
"OBJECTSTORE": "<OBJECT STORE PREFIX>"
"OBJECTIVEFS_PASSPHRASE": "<YOUR FILESYSTEM PASSPHRASE>"
"OBJECTIVEFS_LICENSE": "<YOUR LICENSE KEY + ADMIN KEY>"
"REGION": "<YOUR BUCKET REGION>"
"ACCESS_KEY": "<YOUR OBJECT STORE ACCESS KEY>"
"SECRET_KEY": "<YOUR OBJECT STORE SECRET KEY>"
_private:
csi:
volume:
# volume name is hashed to fit filesystem name length
idHash:
strategy: md5
Note:
ObjectiveFS mount options can be specified in storageClasses:mountOptions.
The ObjectiveFS environment variables specified in storageClasses:secrets:node-stage-secret are used to mount your filesystems. Required environment variables:
env.OBJECTSTORE : your object store prefix, e.g. s3://env.OBJECTIVEFS_PASSPHRASE : should match driver:config:objectivefs:env:OBJECTIVEFS_PASSPHRASEenv.OBJECTIVEFS_LICENSE : your regular license keyenv.ACCESS_KEY (optional if running on EC2 with IAM role attached)env.SECRET_KEY (optional if running on EC2 with IAM role attached)
A new empty bucket name should be specified in driver:config:objectivefs:pool. Kubernetes will automatically provision new filesystems using the filesystem pool feature in this bucket. This bucket should be considered as fully managed by the CSI driver and a dedicated bucket per cluster or deployment would be best practice.
The following ObjectiveFS environment variables are required in driver:config:objectivefs:env for the controller to create and destroy your filesystems:
env.OBJECTSTORE : your object store prefix, e.g. s3://env.OBJECTIVEFS_PASSPHRASE : this passphrase is used to create the managed filesystems. Should match storageClasses:secrets:node-stage-secret:OBJECTIVEFS_PASSPHRASE.env.OBJECTIVEFS_LICENSE : your admin license key, see this doc for details.env.REGION : the region for your bucket, e.g. us-west-2env.ACCESS_KEY (optional if running on EC2 with IAM role attached)env.SECRET_KEY (optional if running on EC2 with IAM role attached)
The environment variables and mount options in helm.yaml apply to all managed mounts in this Kubernetes cluster.
This file contains the PersistentVolumeClaim specifications.
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: objectivefs-provisioned-pvc # Unique within a namespace
spec:
storageClassName: "objectivefs"
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Pi
This file contains an example pod mounting the filesystem.
---
apiVersion: v1
kind: Pod
metadata:
name: objectivefs-provisioned-pod
spec:
volumes:
- name: ofs-volume # (note 1)
persistentVolumeClaim:
claimName: objectivefs-provisioned-pvc # (note 2)
containers:
- name: objectivefs-container
image: ubuntu
command: [ "/bin/bash", "-c", "--" ]
args: [ "while true; do echo 'sleeping'; sleep 5; done;" ]
volumeMounts:
- mountPath: "<MOUNT DIRECTORY>" # (note 3)
name: ofs-volume # (note 1)
Note:
Pod:spec:volumes:name must match Pod:spec:containers:volumeMounts:name. This name should be unique within a pod.Pod:spec:volumes:persistentVolumeClaim:claimName in pod.yaml must match PersistentVolumeClaim:metadata:name in pvc.yaml.Pod:spec:containers:volumeMounts:mountPath.These commands will install K3s and the ObjectiveFS CSI driver, then start a pod with a newly created ObjectiveFS filesystem mounted. These commands are for Ubuntu. You can use equivalent commands for other Linux distributions.
Update helm.yaml, pvc.yaml and pod.yaml with your ObjectiveFS configuration.
Install K3s
# apt update
# curl -sfL https://get.k3s.io | sh -
# export KUBECONFIG=/etc/rancher/k3s/k3s.yaml
3. Install ObjectiveFS CSI driver
# snap install helm --classic
# helm repo add objectivefs-csi https://democratic-csi.github.io/charts
# helm repo update
# helm upgrade --install --create-namespace --values helm.yaml --namespace objectivefs-csi objectivefs objectivefs-csi/democratic-csi
4. Configure the Persistent Volume Claim
# kubectl apply -f pvc.yaml
5. Start the Pod
# kubectl apply -f pod.yaml Note: To view the state of the K3s cluster, you can use K9s.
snap install k9s
/snap/k9s/155/bin/k9s
Last updated by ObjectiveFS staff, September 4, 2024
ObjectiveFS is a shared file system for Linux and macOS that automatically scales and gives you scalable cloud storage.
If you have questions or suggestions, please email us at support@objectivefs.com