ObjectiveFS CSI Driver - Kubernetes Managed

The ObjectiveFS CSI driver can be used to let Kubernetes provision and manage the full lifetime of an ObjectiveFS filesystem. Kubernetes will then automatically create and destroy your filesystems based on the Kubernetes configuration. The ObjectiveFS CSI driver is integrated in the democratic-csi project.

Important: When Kubernetes fully manages the storage, your Kubernetes-managed filesystems will be destroyed and your data will be permanently deleted when Kubernetes determines that the storage is no longer needed.

For the common use case of using the CSI driver to mount your existing filesystems or for persistent storage, see this document.

Prerequisites

Kubernetes YAML files

I. Create the following three YAML files:
a. helm.yaml
b. pvc.yaml
c. pod.yaml

a. helm.yaml

This helm values file configures the Objectivefs CSI driver during installation. The driver section uses the admin key to create and destroy the filesystems. The storageClasses section specificies the mount options and environment variables used to mount the filesystems.

csiDriver:
  name: "org.democratic-csi.node-manual"
  fsGroupPolicy: File

storageClasses:
  - name: objectivefs
    defaultClass: false
    reclaimPolicy: Delete
    volumeBindingMode: Immediate
    allowVolumeExpansion: false
    parameters:
      fsType: objectivefs

    mountOptions:
      # Specify ObjectiveFS mount options here (note 1)
      - mt                                             # optional
    secrets:
      provisioner-secret:
      controller-publish-secret:
      node-stage-secret:
        # Specify ObjectiveFS environment variables here (note 2)
        "env.OBJECTSTORE": "<OBJECT STORE PREFIX>"
        "env.OBJECTIVEFS_PASSPHRASE": "<YOUR FILESYSTEM PASSPHRASE>"
        "env.OBJECTIVEFS_LICENSE": "<YOUR LICENSE KEY>"
        "env.ACCESS_KEY": "<YOUR OBJECT STORE ACCESS KEY>"
        "env.SECRET_KEY": "<YOUR OBJECT STORE SECRET KEY>"
      node-publish-secret:
      controller-expand-secret:

controller:
  externalSnapshotter:
    enabled: false
  externalResizer:
    enabled: false

  # Forwards ObjectiveFS logs to the host
  driver:
    extraVolumeMounts:
    - mountPath: /dev/log
      name: dev-log

  extraVolumes:
  - hostPath:
      path: /dev/log
      type: ""
    name: dev-log

node:
  # Forwards ObjectiveFS logs to the host
  driver:
    extraVolumeMounts:
    - mountPath: /dev/log
      name: dev-log

  extraVolumes:
  - hostPath:
      path: /dev/log
      type: ""
    name: dev-log

driver:
  config:
    driver: objectivefs
    objectivefs:
      pool: <YOUR EMPTY BUCKET NAME>               # (note 3)
      cli:
        sudoEnabled: false
      env:
        # Specify ObjectiveFS environment variables here (note 4)
        "OBJECTSTORE": "<OBJECT STORE PREFIX>"
        "OBJECTIVEFS_PASSPHRASE": "<YOUR FILESYSTEM PASSPHRASE>"
        "OBJECTIVEFS_LICENSE": "<YOUR LICENSE KEY + ADMIN KEY>"
        "REGION": "<YOUR BUCKET REGION>"
        "ACCESS_KEY": "<YOUR OBJECT STORE ACCESS KEY>"
        "SECRET_KEY": "<YOUR OBJECT STORE SECRET KEY>"
    _private:
      csi:
        volume:
          # volume name is hashed to fit filesystem name length
          idHash:
            strategy: md5

Note:

  1. ObjectiveFS mount options can be specified in storageClasses:mountOptions.

  2. The ObjectiveFS environment variables specified in storageClasses:secrets:node-stage-secret are used to mount your filesystems. Required environment variables:

    • env.OBJECTSTORE : your object store prefix, e.g. s3://
    • env.OBJECTIVEFS_PASSPHRASE : should match driver:config:objectivefs:env:OBJECTIVEFS_PASSPHRASE
    • env.OBJECTIVEFS_LICENSE : your regular license key
    • env.ACCESS_KEY (optional if running on EC2 with IAM role attached)
    • env.SECRET_KEY (optional if running on EC2 with IAM role attached)

  3. A new empty bucket name should be specified in driver:config:objectivefs:pool. Kubernetes will automatically provision new filesystems using the filesystem pool feature in this bucket. This bucket should be considered as fully managed by the CSI driver and a dedicated bucket per cluster or deployment would be best practice.

  4. The following ObjectiveFS environment variables are required in driver:config:objectivefs:env for the controller to create and destroy your filesystems:

    • env.OBJECTSTORE : your object store prefix, e.g. s3://
    • env.OBJECTIVEFS_PASSPHRASE : this passphrase is used to create the managed filesystems. Should match storageClasses:secrets:node-stage-secret:OBJECTIVEFS_PASSPHRASE.
    • env.OBJECTIVEFS_LICENSE : your admin license key, see this doc for details.
    • env.REGION : the region for your bucket, e.g. us-west-2
    • env.ACCESS_KEY (optional if running on EC2 with IAM role attached)
    • env.SECRET_KEY (optional if running on EC2 with IAM role attached)

  5. The environment variables and mount options in helm.yaml apply to all managed mounts in this Kubernetes cluster.

b. pvc.yaml

This file contains the PersistentVolumeClaim specifications.

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: objectivefs-provisioned-pvc      # Unique within a namespace
spec:
  storageClassName: "objectivefs"
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Pi

c. pod.yaml

This file contains an example pod mounting the filesystem.

---
apiVersion: v1
kind: Pod
metadata:
  name: objectivefs-provisioned-pod
spec:
  volumes:
    - name: ofs-volume                          # (note 1)
      persistentVolumeClaim:
        claimName: objectivefs-provisioned-pvc  # (note 2)
  containers:
    - name: objectivefs-container
      image: ubuntu
      command: [ "/bin/bash", "-c", "--" ]
      args: [ "while true; do echo 'sleeping'; sleep 5; done;" ]
      volumeMounts:
        - mountPath: "<MOUNT DIRECTORY>"        # (note 3)
          name: ofs-volume                      # (note 1)

Note:

  1. Pod:spec:volumes:name must match Pod:spec:containers:volumeMounts:name. This name should be unique within a pod.
  2. Pod:spec:volumes:persistentVolumeClaim:claimName in pod.yaml must match PersistentVolumeClaim:metadata:name in pvc.yaml.
  3. Your filesystem’s mount directory on the pod is specified in Pod:spec:containers:volumeMounts:mountPath.

Steps

These commands will install K3s and the ObjectiveFS CSI driver, then start a pod with a newly created ObjectiveFS filesystem mounted. These commands are for Ubuntu. You can use equivalent commands for other Linux distributions.

  1. Update helm.yaml, pvc.yaml and pod.yaml with your ObjectiveFS configuration.

  2. Install K3s

 # apt update
 # curl -sfL https://get.k3s.io | sh -
 # export KUBECONFIG=/etc/rancher/k3s/k3s.yaml


3. Install ObjectiveFS CSI driver

 # snap install helm --classic
 # helm repo add objectivefs-csi https://democratic-csi.github.io/charts
 # helm repo update
 # helm upgrade --install --create-namespace --values helm.yaml --namespace objectivefs-csi objectivefs objectivefs-csi/democratic-csi


4. Configure the Persistent Volume Claim

 # kubectl apply -f pvc.yaml 


5. Start the Pod

 # kubectl apply -f pod.yaml 

Note: To view the state of the K3s cluster, you can use K9s.

snap install k9s
/snap/k9s/155/bin/k9s

Last updated by ObjectiveFS staff, September 4, 2024
ObjectiveFS is a shared file system for Linux and macOS that automatically scales and gives you scalable cloud storage. If you have questions or suggestions, please email us at support@objectivefs.com