I have 4 nodes involved in this setup, a NAS and few Armbian Orange Pi Zero3. The main reason for is being able to deploy k3s cluster on the armbian boards, and since SDCards degrade quickly here is an alternative setup to pivot the root over iSCSI devices, we are keeping /boot mounted on the SDCard, but there are alternatives out there to fully pxe boot. Also I have not seen successful attempts to use overlayfs over NFS, and this solution seems to works fine.

The hosts are are follow, there is a dhcp server on my network handing over IP addresses

nas.lan 192.168.0.5       # NAS
k3s-node-01.lan 192.168.0.51 # k3s node 01
k3s-node-02.lan 192.168.0.52 # k3s node 02
k3s-node-03.lan 192.168.0.53 # k3s node 03

On the NAS I have a tank ZFS pool, I have created for each node zvol and mapped it in with tgt for each node. My sdcards on the node are 32GB, but you can do your sizes as you wish

root@nas:~# zfs list
NAME                    USED  AVAIL  REFER  MOUNTPOINT
tank                   6.86T  11.2T    96K  /tank
tank/k3s-node-01.root  65.0G  11.2T    56K  -
tank/k3s-node-02.root  65.0G  11.2T    56K  -
tank/k3s-node-03.root  65.0G  11.2T    56K  -

<target iqn.2026-09.lan.nas:k3s-node-01.root>
backing-store /dev/zvol/tank/k3s-node-01.root
initiator-address 192.168.0.51
</target>
<target iqn.2026-09.lan.nas:k3s-node-02.root>
backing-store /dev/zvol/tank/k3s-node-02.root
initiator-address 192.168.0.52
</target>
<target iqn.2026-09.lan.nas:k3s-node-03.root>
backing-store /dev/zvol/tank/k3s-node-03.root
initiator-address 192.168.0.53
</target>

Moving root to iSCSI device

This is just for node-01, repeat on each node as necessary. On the node install open-iscsi, list the target and get the initiator name, each initiator name must be unique per node

root@k3s-node-01# apt -y install open-iscsi

root@k3s-node-01# iscsiadm --mode discovery --type sendtargets --portal 192.168.0.5
192.168.0.5:3260,1 iqn.2026-09.lan.nas:k3s-node-02.root

root@k3s-node-01:~# cat /etc/iscsi/initiatorname.iscsi
## DO NOT EDIT OR REMOVE THIS FILE!
## If you remove this file, the iSCSI daemon will not start.
## If you change the InitiatorName, existing access control lists
## may reject this initiator.  The InitiatorName must be unique
## for each iSCSI initiator.  Do NOT duplicate iSCSI InitiatorNames.
InitiatorName=iqn.1993-08.org.debian:01:eed87f37e093

Regen kernel to include iscsi modules

root@k3s-node-01# touch /etc/iscsi/iscsi.initramfs

root@k3s-node-01# update-initramfs -v -k `uname -r` -c

Configure the kernel to add the add the iscsi target at boot time on /boot/armbianEnv.txt, I am disabling ipv6 and setting ip=dhcp

extraargs=ipv6.disable=1 ip=dhcp ISCSI_INITIATOR=iqn.1993-08.org.debian:01:eed87f37e093 ISCSI_TARGET_NAME=iqn.2026-09.lan.nas:k3s-node-01.root ISCSI_TARGET_IP=192.168.0.5 ISCSI_TARGET_PORT=3260 rw

Reboot the node and verify the iscsi the device attaches

root@k3s-node-01:~# dmesg | grep sd
[   19.148231] sd 0:0:0:1: Power-on or device reset occurred
[   19.149427] sd 0:0:0:1: [sda] 134217728 512-byte logical blocks: (68.7 GB/64.0 GiB)
[   19.149775] sd 0:0:0:1: [sda] Write Protect is off
[   19.149790] sd 0:0:0:1: [sda] Mode Sense: 69 00 10 08
[   19.150550] sd 0:0:0:1: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   19.209258] sd 0:0:0:1: [sda] Attached SCSI disk

Shutdown the node and get the disk imaged over the the zvol device, I use a card reader for this. I reset the UUID for the FS, and fsck and resize it as zvol is larger than sdcard:

root@nas:~# dd if=/dev/sdf1 of=/dev/zvol/tank/k3s-node-01.root bs=1M

root@nas:~# tune2fs -U random /dev/zvol/tank/k3s-node-01.root

root@nas:~# e2fsck -f /dev/zvol/tank/k3s-node-01.root
e2fsck 1.47.2 (1-Jan-2025)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
armbi_root: 114168/1850240 files (0.6% non-contiguous), 2024528/7711744 blocks
root@nas:~# resize2fs /dev/zvol/tank/k3s-node-02.root
resize2fs 1.47.2 (1-Jan-2025)
Resizing the filesystem on /dev/zvol/tank/k3s-node-02.root to 16777216 (4k) blocks.
The filesystem on /dev/zvol/tank/k3s-node-02.root is now 16777216 (4k) blocks long

root@nas:~# lsblk --fs /dev/zvol/tank/k3s-node-02.root
NAME FSTYPE FSVER LABEL      UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
zd16 ext4   1.0   armbi_root d6d2f966-b7a8-4a17-9e8e-02746204049e

Reinsert the card on the node, reboot, mount the iscsi device, create a sdcard mount directory and alter its /etc/fstab to mount the sdcard root into /sdcard/ (reuse the UUID), also add a bind mount for boot from the sd card, etc

root@k3s-node-01:~# mount /dev/sda /mnt/

root@k3s-node-01:~# mkdir /mnt/sdcard

root@k3s-node-01:~# vim /mnt/etc/fstab

root@k3s-node-01:~# cat /mnt/etc/fstab
UUID=d6d2f966-b7a8-4a17-9e8e-02746204049e / ext4 defaults,commit=120,errors=remount-ro 0 1
UUID=f46a0f5d-7d49-4f24-b71e-1d804780f431 /sdcard ext4 defaults,commit=120,errors=remount-ro 0 1
/sdcard/boot	                          /boot   none bind,nofail
tmpfs /tmp tmpfs defaults,nosuid 0 0

root@k3s-node-01:~# umount /mnt

Alter the with the new rootdev UUID on the iscsi device in /boot/armbianEnv.txt:

rootdev=UUID=d6d2f966-b7a8-4a17-9e8e-02746204049e

Reboot the node and enjoy your new armbian running from iscsi, cleaning the sdcard mount for everything but /boot is optional

root@k3s-node-01:~# df -h / /boot/
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda         63G  7.3G   56G  12% /
/dev/mmcblk0p1   29G  7.3G   22G  26% /boot

References

Also You can take it the extra mile and also use pxe boot per the following post

Great references to that were great help: