ObjectiveFS CSI Driver

The ObjectiveFS CSI driver can mount an ObjectiveFS filesystem in a Kubernetes cluster. The ObjectiveFS CSI driver is integrated in the democratic-csi project.

This document describes the steps to use the CSI driver to mount your existing ObjectiveFS filesystem. This is the most common use case for ObjectiveFS users with existing filesystems.

It is also possible to let Kubernetes manage the lifetime of an ObjectiveFS filesystem. In this case, Kubernetes will automatic create and destroy filesystems. See here for details.

Prerequisites

Your object store credentials
Your ObjectiveFS license key
An existing ObjectiveFS filesystem and its passphrase

Kubernetes YAML files

Create the following three YAML files:
a. helm.yaml
b. objectivefs.yaml
c. pod.yaml

a. helm.yaml

This helm values file configures the Objectivefs CSI driver during installation.

csiDriver:
  name: "org.democratic-csi.node-manual"
  fsGroupPolicy: File

controller:
  externalAttacher:
    enabled: true
  externalProvisioner:
    enabled: false
  externalResizer:
    enabled: false
  externalSnapshotter:
    enabled: false

node:                      # Forward ObjectiveFS logs to the host
  driver:
    extraVolumeMounts:
    - mountPath: /dev/log
      name: dev-log
  extraVolumes:
  - hostPath:
      path: /dev/log
      type: ""
    name: dev-log

driver:
  config:
    driver: node-manual

b. objectivefs.yaml

This file contains the ObjectiveFS Secret and PersistentVolume specifications.

---
apiVersion: v1
kind: Secret
metadata:
  name: objectivefs-secret
  namespace: kube-system
stringData:
  # Specify ObjectiveFS environment variables here (note 1)
  "env.OBJECTIVEFS_PASSPHRASE": "<YOUR FILESYSTEM PASSPHRASE>"
  "env.OBJECTIVEFS_LICENSE": "<YOUR LICENSE KEY>"
  "env.ACCESS_KEY": "<YOUR OBJECT STORE ACCESS KEY>"
  "env.SECRET_KEY": "<YOUR OBJECT STORE SECRET KEY>"
  "env.OBJECTSTORE": "<OBJECT STORE PREFIX>"         # optional, e.g. s3://
  "env.DISKCACHE_SIZE": "10G:0G"                     # optional
  "env.CACHESIZE": "25%"                             # optional
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: objectivefs-volume
spec:
  capacity:
    storage: 1Pi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  mountOptions:
    # Specify ObjectiveFS mount options here (note 2)
    - mt                                             # optional
    - noatime                                        # optional
  csi:
    driver: org.democratic-csi.node-manual
    readOnly: false
    fsType: objectivefs
    volumeHandle: objectivefs-volumeid      # must be unique per cluster
    nodeStageSecretRef:
      name: objectivefs-secret
      namespace: kube-system
    volumeAttributes:
      node_attach_driver: objectivefs
      provisioner_driver: node-manual
      filesystem: "<YOUR FILESYSTEM NAME>"           # (note 3)
      # Specify ObjectiveFS environment variables here (note 1)
      env.TLS: "1"                                   # optional

Note:

ObjectiveFS environment variables can be specified in Secret or PersistentVolume. The parameters specified in PersistentVolume have precedence over Secret since they are processed later. Required environment variables:
- env.OBJECTIVEFS_PASSPHRASE
- env.OBJECTIVEFS_LICENSE
- env.ACCESS_KEY (optional if running on EC2 with IAM role attached)
- env.SECRET_KEY (optional if running on EC2 with IAM role attached)
ObjectiveFS mount options can be specified in PersistentVolume:spec:mountOptions.
Your filesystem name is required and can be specified either as <bucket name> (e.g. my-ofs-bucket) or as <object store>:<bucket name> (e.g. s3://my-ofs-bucket) in PersistentVolume:spec:csi:volumeAttributes. If the object store prefix is not specified, env.OBJECTSTORE should be specified in either Secret or PersistentVolume.
Commonly used environment variables [optional]
- env.CACHESIZE : Specify the memory cache size
- env.DISKCACHE_SIZE : Specify the disk cache size

c. pod.yaml

This file contains the PersistentVolumeClaim specification and an example pod mounting the filesystem.

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: objectivefs-pvc           # Unique within a name space.
spec:
  volumeName: objectivefs-volume  # Must match PV. Unique within a cluster.
  storageClassName: ""
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Pi
---
apiVersion: v1
kind: Pod
metadata:
  name: objectivefs-pod
spec:
  volumes:
    - name: ofs-volume              # Unique within pod.
      persistentVolumeClaim:
        claimName: objectivefs-pvc  # Must match PVC metadata:name
  containers:
    - name: objectivefs-container
      image: ubuntu
      command: [ "/bin/bash", "-c", "--" ]
      args: [ "while true; do echo 'sleeping'; sleep 5; done;" ]
      volumeMounts:
        - mountPath: "<MOUNT DIRECTORY>"
          name: ofs-volume          # Must match volumes:name

Note:

Your filesystem’s mount directory on the pod is specified in Pod:spec:containers:volumeMounts:mountPath.
PersistentVolumeClaim:spec:volumeName in pod.yaml must match PersistentVolume:metadata:name in objectivefs.yaml.
Pod:spec:volumes:PersistentVolumeClaim:claimName must match PersistentVolume:metadata:name (both are in pod.yaml).

Steps

These commands will install K3s, the ObjectiveFS CSI driver and start a pod with your ObjectiveFS filesystem mounted. These commands are for Ubuntu. You can use equivalent commands for other Linux distributions.

Update helm.yaml, objectivefs.yaml and pod.yaml with your ObjectiveFS configuration.
Install K3s

 # apt update
 # curl -sfL https://get.k3s.io | sh -
 # export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

3. Install ObjectiveFS CSI driver

 # snap install helm --classic
 # helm repo add objectivefs-csi https://democratic-csi.github.io/charts
 # helm repo update
 # helm upgrade --install --create-namespace --values helm.yaml --namespace objectivefs-csi objectivefs objectivefs-csi/democratic-csi

4. Configure the Persistent Volume

 # kubectl apply -f objectivefs.yaml

5. Start the Pod

 # kubectl apply -f pod.yaml

Note: To view the state of the K3s cluster, you can use K9s.

 # snap install k9s
 # KUBECONFIG=/etc/rancher/k3s/k3s.yaml /snap/k9s/155/bin/k9s

Last updated by ObjectiveFS staff, September 4, 2024
ObjectiveFS is a shared file system for Linux and macOS that automatically scales and gives you scalable cloud storage. If you have questions or suggestions, please email us at support@objectivefs.com