Armbian: move root to ZVOLs over iSCSI

I have 4 nodes involved in this setup, a NAS and few Armbian Orange Pi Zero3s board. The main reason for this tutorial is being able to deploy k3s cluster on the Armbian boards, without having the SDCards degrade quickly. Here is an alternative setup to pivot the root over iSCSI devices, we are keeping /boot mounted on the SDCard still, although there are alternatives out there to fully pxe boot. Also I have not seen successful attempts to use overlayfs over NFS, this solution seems to works fine for me, and is one of the fastest I have seen to implement.

The hosts are as follow, there is a dhcp server on my network handing over the IP addresses

nas.lan 192.168.0.5          # NAS
k3s-node-01.lan 192.168.0.51 # k3s node 01
k3s-node-02.lan 192.168.0.52 # k3s node 02
k3s-node-03.lan 192.168.0.53 # k3s node 03

On the NAS I have a tank ZFS pool, I have created for each node a zvol and mapped it in with tgt. My SDCards on each node are 32GB, but you can do different sizes as you wish, zvols just need to be the same size or bigger.

root@nas:~# zfs list
NAME                    USED  AVAIL  REFER  MOUNTPOINT
tank                   6.86T  11.2T    96K  /tank
tank/k3s-node-01.root  65.0G  11.2T    56K  -
tank/k3s-node-02.root  65.0G  11.2T    56K  -
tank/k3s-node-03.root  65.0G  11.2T    56K  -


<target iqn.2026-09.lan.nas:k3s-node-01.root>
backing-store /dev/zvol/tank/k3s-node-01.root
initiator-address 192.168.0.51
</target>
<target iqn.2026-09.lan.nas:k3s-node-02.root>
backing-store /dev/zvol/tank/k3s-node-02.root
initiator-address 192.168.0.52
</target>
<target iqn.2026-09.lan.nas:k3s-node-03.root>
backing-store /dev/zvol/tank/k3s-node-03.root
initiator-address 192.168.0.53
</target>

Moving root to iSCSI device

This is just for node-01, repeat on each node as necessary. On the node install open-iscsi, list the target and get the initiator name, each initiator name must be unique per node

root@k3s-node-01# apt -y install open-iscsi

root@k3s-node-01# iscsiadm --mode discovery --type sendtargets --portal 192.168.0.5
192.168.0.5:3260,1 iqn.2026-09.lan.nas:k3s-node-02.root

root@k3s-node-01:~# cat /etc/iscsi/initiatorname.iscsi
## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.1993-08.org.debian:01:eed87f37e093

Regen initiramfs to include iscsi modules


root@k3s-node-01# touch /etc/iscsi/iscsi.initramfs

root@k3s-node-01# update-initramfs -v -k `uname -r` -c

Configure the kernel and add the iscsi target at boot time on /boot/armbianEnv.txt, I am disabling ipv6 and setting ip=dhcp

extraargs=ipv6.disable=1 ip=dhcp ISCSI_INITIATOR=iqn.1993-08.org.debian:01:eed87f37e093 ISCSI_TARGET_NAME=iqn.2026-09.lan.nas:k3s-node-01.root ISCSI_TARGET_IP=192.168.0.5 ISCSI_TARGET_PORT=3260 rw

Reboot the node and verify that the iscsi the device attaches

root@k3s-node-01:~# dmesg | grep sd
[   19.148231] sd 0:0:0:1: Power-on or device reset occurred
[   19.149427] sd 0:0:0:1: [sda] 134217728 512-byte logical blocks: (68.7 GB/64.0 GiB)
[   19.149775] sd 0:0:0:1: [sda] Write Protect is off
[   19.149790] sd 0:0:0:1: [sda] Mode Sense: 69 00 10 08
[   19.150550] sd 0:0:0:1: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   19.209258] sd 0:0:0:1: [sda] Attached SCSI disk

Shutdown the node and get the disk imaged over the the zvol device, I use a card reader for this. Reset the UUID for the FS, and fsck and resize it as zvol is larger than sdcard:

root@nas:~# dd if=/dev/sdf1 of=/dev/zvol/tank/k3s-node-01.root bs=1M

root@nas:~# tune2fs -U random /dev/zvol/tank/k3s-node-01.root

root@nas:~# e2fsck -f /dev/zvol/tank/k3s-node-01.root
e2fsck 1.47.2 (1-Jan-2025)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
armbi_root: 114168/1850240 files (0.6% non-contiguous), 2024528/7711744 blocks

root@nas:~# resize2fs /dev/zvol/tank/k3s-node-02.root
resize2fs 1.47.2 (1-Jan-2025)
Resizing the filesystem on /dev/zvol/tank/k3s-node-02.root to 16777216 (4k) blocks.
The filesystem on /dev/zvol/tank/k3s-node-02.root is now 16777216 (4k) blocks long

root@nas:~# lsblk --fs /dev/zvol/tank/k3s-node-02.root
NAME FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
zd16 ext4   1.0   armbi_root d6d2f966-b7a8-4a17-9e8e-02746204049e

Reinsert the card on the node, reboot, mount the iscsi device, create a sdcard mount directory and alter its /etc/fstab to mount the sdcard root into /sdcard/ (reuse the UUID, just change the mount point), also add a bind mount for boot from the sd card, place at the top the new the new root mount for the iscsi device with the new UUID.

root@k3s-node-01:~# mount /dev/sda /mnt/

root@k3s-node-01:~# mkdir /mnt/sdcard

root@k3s-node-01:~# vim /mnt/etc/fstab

root@k3s-node-01:~# cat /mnt/etc/fstab
UUID=d6d2f966-b7a8-4a17-9e8e-02746204049e / ext4 defaults,commit=120,errors=remount-ro 0 1
UUID=f46a0f5d-7d49-4f24-b71e-1d804780f431 /sdcard ext4 defaults,commit=120,errors=remount-ro 0 1
/sdcard/boot	                          /boot   none bind,nofail
tmpfs /tmp tmpfs defaults,nosuid 0 0

root@k3s-node-01:~# umount /mnt

Alter the with the new rootdev with UUID on the iscsi device in /boot/armbianEnv.txt:

rootdev=UUID=d6d2f966-b7a8-4a17-9e8e-02746204049e

Reboot the node and enjoy your new armbian running from iscsi, cleaning the sdcard mount for everything but /boot is optional

root@k3s-node-01:~# df -h / /boot/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda         63G  7.3G   56G  12% /
/dev/mmcblk0p1   29G  7.3G   22G  26% /boot