Hi,
I've tried few things for my raid 5 array in my workstation. Windows storage spaces (including powershell commands to make it faster) horribly slow. Btrfs for windows worked great until I learned the hard way that btrfs + raid 5 is no no.
Long story short, I end up with setup:
Host: windows 10 enterprise iot ltsc 2021
VMWare Workstation VM running Alpine Linux, VM flavor, 8G ram, 8c setup with host-only network
Passthrough of my disks into VM
ZFS setup in RAIDZ1 exposing volume /dev/zd0
tgtd with following config:
tgtadm --lld iscsi --op new --mode target --tid 1 --targetname iqn.2025-01.local.raidvm:r5store
tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /dev/zd0
I also tried replacing last line with:
tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 --backing-store /dev/zd0 --bsoflags direct
And windows iSCSI connector. It connected and let me format the partition no issue.
Crystal disk mark shows numbers that look +- legitmite, and align with performence tests I done inside the VM.
When I try to copy files into the partition with windows explorer, the speeds skyrocket to +-4GB/s for few seconds, and after that it gets stuck, VM freezes, SSH sessions to it stop responding. After waiting some time, VM un-freezes, ssh are back responsive. Smaller transfers (like 10 gb), don't freeze the VM (VM has 8 gigs of ram, and 2 gigs of persistent storage for system), and running iftop I can see that long after windows says it finished the transfer, its still pushing the data (!). For transfers over 30 gigs, during the transfer windows times out during VM freeze, the operation gets canceled, it drops iSCSI connection and shortly after reconnects.
When I try to use WSL for copying, it also shows stupidly high speeds, but it doesnt even produce network traffic on VM, and the files never appear on the targeted file system.
As sanity check, if maybe thats some ZFS or maybe something I didnt consider, Ive setup temporary Samba. Samba, like I expected underperforms (still overperforming storagespaces drastically), but works. Nothing to write home about, but I noticed that while writing with samba, the speed drops from 220MB/s to 50MB/s and recovers, back to orginal 220MB/s speed and than drops once again. But it does indeed finalize transfers correctly.
Other things:
-
I disabled write cache in windows, one of first troubleshooting steps (disk manager -> disk properties -> policy)
-
I played around zfs ARC, but its not the cause of the issue
-
I tried to figure out other server than tgt, but scst is deprecated, and the LIO setup requires KMod that isn't available in this flavor of linux. And I think that since Crystal Disk Mark can get true results, the blame squarely falls on windows iSCSI client.
-
dmesg from lockup: https://pastebin.com/mGUSEKzv https://pastebin.com/xbxq0pQ1
-
after few stalls and reconnects, it seems like windows figures out that its pushing it too hard, and the transfer speed gets normal for specific file transfer operation.
Im at loss here, I would like to use the iSCSI, but this setup is unusable (or bearly usable).
I don't know how to proceed with this. Please advise.