Volume Snapshots (Clones) Broken on 2.7.0

[nick-oconnor]

What steps did you take and what happened:

I use volsync to backup PVCs. After upgrading to 2.7.0, volume snapshot (clone) creation fails. The only error I saw was logged by the node daemonset:

E0211 06:20:45.622639       1 zfs_util.go:470] zfs: could not clone volume pool-2/vol-0/k8s/pvc-51e7634f-a9b9-440f-bf2f-1d8d05e87b29 cmd [clone -o =1073741824 -o mountpoint=legacy pool-2/vol-0/k8s/pvc-55f9cb2f-2a92-4b91-a251-074e4b35ce27@pvc-51e7634f-a9b9-440f-bf2f-1d8d05e87b29 pool-2/vol-0/k8s/pvc-51e7634f-a9b9-440f-bf2f-1d8d05e87b29] error: cannot create 'pool-2/vol-0/k8s/pvc-51e7634f-a9b9-440f-bf2f-1d8d05e87b29': invalid property ''

Based on the message, it looks like the quota option label is missing from the command (i.e. -o =1073741824). I'm not setting quota type explicitly. Rolling back to 2.6.2 resolves the issue.

What did you expect to happen:

I expected the snapshot to be successfully created.

The output of the following commands will help us better understand what's going on:

I didn't get a chance to grab full logs before rolling back to 2.6.2.

Anything else you would like to add:

Nope.

Environment:

• LocalPV-ZFS version: 2.7.0
• Kubernetes version (use kubectl version): 1.31.4
• Kubernetes installer & version: kubeadm, n/a
• Cloud provider or hardware configuration: Baremetal + Containerd + Calico + zfs-localpv (via helm)
• OS (e.g. from /etc/os-release): Ubuntu 24.04.1

[Abhinandan-Purkait]

Oops, seems like GitHub auto-closed this. @nick-oconnor Thanks for reporting this, we have tried to add fix for the same. Would it be possible for you to test this with a prerelease version before we make a patch release. That would be really helpful as in to understand if this really fixes the issue.

[nick-oconnor]

Hi @Abhinandan-Purkait! Thanks for the fix. Yep, I can test it. Lmk when there's a prerelease ready.

[Abhinandan-Purkait]

Thanks @nick-oconnor, you can try with the helm chart as below

➜  ~ helm search repo openebs-zfs --devel -l
NAME                   	CHART VERSION   	APP VERSION     	DESCRIPTION
openebs-zfs/zfs-localpv	2.7.1-prerelease	2.7.1-prerelease	Helm chart for CSI Driver for dynamic provision...

[nick-oconnor]

Looks good 👍

I0212 05:43:00.135200       1 zfs_util.go:452] creating snapshot pvc-55f9cb2f-2a92-4b91-a251-074e4b35ce27@pvc-eb78159b-80a1-4f41-a4be-9c425855a608 for the clone pool-2/vol-0/k8s/pvc-eb78159b-80a1-4f41-a4be-9c425855a608
I0212 05:43:00.181225       1 zfs_util.go:701] created snapshot pvc-55f9cb2f-2a92-4b91-a251-074e4b35ce27@pvc-eb78159b-80a1-4f41-a4be-9c425855a608
I0212 05:43:00.254287       1 zfs_util.go:475] created clone pool-2/vol-0/k8s/pvc-eb78159b-80a1-4f41-a4be-9c425855a608

[Abhinandan-Purkait]

Thanks @nick-oconnor, We will do the patch release. Also tracking the automated upgrade testing here #635

[Abhinandan-Purkait]

@nick-oconnor 2.7.1 has been released. Please check and close this issue if it is resolved. Thanks

[nick-oconnor]

Thanks for getting this fixed so quickly! LGTM.