I recently replaced my old Synology NAS with a newer model. The old NAS is still in great shape, has good disks, and more than a terabyte of storage, so I figured I’d try setting up a Storj storage node with it.

However, there’s a limitation: the older NAS can’t run Docker containers, and so cannot host the storage node software directly. Instead, I need to run it on another local system that uses the NAS as storage. Storj is pretty clear that iSCSI is the only proper option for accessing storage hardware over the network:

A network-attached storage location may work, but this is neither supported nor recommended! Please consider running the node locally on your file server/NAS instead. If that is not possible, then the only working network protocol for network storage is iSCSI.

Fair enough. Let’s get things started. This post documents my process for future reference, and I’m sharing it publicly in the hopes that it will be useful for others.

Here’s my setup:

• A Proxmox cluster running a variety of VMs and LXC containers for my homelab.
• The old Synology NAS.
• A switched gigabit ethernet network with several VLANs (one for internal services, one for internet-exposed services, etc.).

For security, I want to run an internet-facing service like a Storj node on a VM that runs its own kernel (rather than an LXC, which shares the host kernel), in the internet-service VLAN (40, in my case), and use Alpine Linux because it’s extremely lightweight.

Basic Setup & Networking

1. After creating a VM, assigning its interface to vlan40, and installing and updating Alpine, I have a basic VM running on 10.200.40.151.
2. My NAS is running on a separate VLAN as 192.168.3.65.
3. I have a dynamic DNS address that automatically updates to my public IPv4 address.
4. Following the instructions from Storj, I set up port forwarding on my firewall (running opnsense, for what it’s worth) so that TCP and UDP port 28967 is forwarded to that VM.
5. I also increase the UDP buffer size on the VM using the steps here, which work as expected on Alpine.
6. Since the NAS and VM are on separate VLANs that are isolated by a firewall, configure a firewall rule to allow the VM to access TCP port 3260 on the NAS.

Install & Configure iSCSI

iSCSI lets you connect to a remote network device and mount a logical storage device as a block device, that is, as a local block-based hard disk. This differs from, for example, NFS, which is a file-based system, and is apparently needed by the Storj storage node software.

The terminology can be a bit overwhelming, but in brief:

• An iSCSI “initiator” is the client. In my case, the Alpine Linux system.
• An iSCSI “target” is the “server” running on the NAS.
• An iSCSI “LUN” is the virtual disk I want to share.
• An iSCSI “IQN” is a unique name used to identify initiators and targets on the network.

Setup Synology NAS as an iSCSI Target

1. Log into the NAS, open the menu, and select iSCSI Manager.
2. Select “Target”.
3. Select “Create”
4. In the “Create a new iSCSI Target” menu, enter a descriptive name in the Name field. I chose “storj-target”. Leave the IQN as-is, but make a note of it for later use (mine is iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219 but yours will be different). I also leave CHAP authentication disabled since the NAS will only have this single iSCSI target and there’s a firewall rule that allows it to only be accessed by the VM. Your mileage may vary and you may need to enable it. Click Next.
5. Select “Create a new iSCSI LUN” and click Next.
6. Pick a descriptive name for the LUN. Mine is “storj-lun”. I wanted to share 1.5 TB, so I specified the size of the LUN as (1.5 TB * 1024 GB/TB) + 1 GB = 1537 GB to leave a little buffer. I selected Thin Provisoning, as I read that performance is essentially identical to Thick while not having to preallocate all the storage required by the LUN. Click “Next” and “Apply”.
7. If all goes well, the target and LUN are created and the target has the LUN mapped. By default, the target is set to allow anyone to access the target. I use firewall rules to restrict access.

Setup Alpine Linux as an iSCSI Initiator

Alpine Linux is extremely lightweight and comes with minimal software. We need to install and configure the iSCSI package. All commands in this section are run as root on the VM.

1. Read the iSCSI instructions on the Alpine Linux wiki, then:
   • Run apk add open-iscsi
   • Start the iSCSI service by running rc-service iscsid start
   • Set the iSCSI service to start at boot: rc-update add iscsid
   • Query the NAS to verify connectivity and list the LUNs by running iscsiadm --mode discovery --type sendtargets --portal IP_OF_TARGET
     • Specific example: iscsiadm --mode discovery --type sendtargets --portal 192.168.3.65
     • It should return something like this, 192.168.3.65:3260,1 iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219, which indicates it was able to connect to the NAS and find the target.
   • Next, let’s mount the target as a local disk: iscsiadm --mode node --targetname NAME_OF_TARGET --portal IP_OF_TARGET --login
     • Specific example: iscsiadm --mode node --targetname iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219 --portal 192.168.3.65 --login
     • It should return something like this:
       Logging in to [iface: default, target: iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219, portal: 192.168.3.65,3260]
Login to [iface: default, target: iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219, portal: 192.168.3.65,3260] successful.
2. Next, we need to partition the new disk.
   • Let’s confirm the device name of the mounted iSCSI target. Running iscsiadm -m session -P 3 will show a lot of detail about the current iSCSI session, including device names. On the last line you should see something like Attached scsi disk sdb State: running — this tells us that the iSCSI disk is mounted to /dev/sdb. It may be different on your system.
   • Alpine doesn’t come with lsblk, which is a handy utility, so let’s install it: apk add lsblk
   • Now, run lsblk to show all block devices connected to the computer. I see /dev/sda as my VM’s main disk (along with its several partitions) and /dev/sdb as an unpartitioned disk.
   • Again, on my system the iSCSI disk is mounted to /dev/sdb — it may be different on your system. Please verify that you’re referring to the correct device in the following setps, as partitioning it will destroy any existing data on the device.
   • Let’s partition the disk: fdisk /dev/sdb
     • Type “n” for a new partition, then follow the prompts and select “p” for primary partition, then “1” for the partition number, and accept the default size.
     • Type “p” to print the partition table. Confirm that everything is in order.
     • Note: If you wish to quit without saving your changes, type “q” and press Enter.
     • (WARNING: THIS IS DESTRUCTIVE) Save and write the changes to disk by typing “w” and press Enter.
3. Now that we have the iSCSI disk partitioned, let’s format it. ext4 is fine.
   • Run mkfs.ext4 /dev/sdb1 and follow the prompts.
4. In some cases the device name or partition name (e.g. /dev/sdb and /dev/sdb1, respectively) might change, so it’s useful to unambiguously identify the partition by in configuration files using a UUID. The partition’s UUID can be found by running blkid /dev/sdb1. In my case, it’s f88fc19f-f4ab-4a33-9383-ce9a72b3d6ca. Yours will be different. Make a note of it for now.
5. Disconnect and reconnect the iSCSI connection by running iscsiadm --mode node --targetname NAME_OF_TARGET --portal IP_OF_TARGET -u and iscsiadm --mode node --targetname NAME_OF_TARGET --portal IP_OF_TARGET --login so the system will recognize it as a partitioned disk.
6. Make a mount point where you’ll mount the partition. In my case, I want it to be in /mnt/storj-iscsi, so I run mkdir -p /mnt/storj-iscsi (the “-p” ensures it creates any needed intermediate directories like /mnt if they don’t already exist).
7. Let’s configure the /etc/fstab file so the system can mount the partition. Open it in a text editor and add the following line: /dev/disk/by-uuid/YOUR_DISK_UUID /path/to/mount/point ext4 defaults,_netdev 0 0
   • Specific example: /dev/disk/by-uuid/f88fc19f-f4ab-4a33-9383-ce9a72b3d6ca /mnt/storj-iscsi ext4 defaults,_netdev 0 0
   • Note: The _netdev option tells the system that it’s a network drive and should wait until after the networking stack is up before trying to mount it. However, it can take a few seconds for the iSCSI system to connect to the target after networking is up, and if it’s not connected by the time the system tries to mount the partition then it will remain unmounted. We’ll set up a more robust mounting method later, so stay tuned.
8. Test that the partition mounts by running mount -a -v. Assuming it works, let’s unmount it to finish up a few more things by running umount /mnt/storj-iscsi/.
9. Configure the iSCSI client to make the connection persistent (that is, so it’ll start at boot) by running iscsiadm -m node -T NAME_OF_TARGET -p IP_OF_TARGET --op update -n node.conn[0].startup -v automatic
   • Specific example: iscsiadm -m node -T iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219 -p 192.168.3.65 --op update -n node.conn[0].startup -v automatic
   • Note: “node.conn[0].startup” is the term used in the Alpine Linux wiki. The “conn[0]” refers to the first TCP connection in that session. The wiki calls it out to be explicit since it’s possible that iSCSI sessions have multiple connections, but in practice this is rarely used so you can also simply use “node.startup” instead if that’s simpler to understand.
   • Note: To remove the connection’s persistence, run the previous command with -n node.conn[0].startup -v manual instead.
   • Note: To remove the whole connection entirely, run iscsiadm -m node -T iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219 --op delete
   • Node: On Alpine Linux, iSCSI node information and configurations are available in /var/lib/iscsi/nodes and it’s subdirectories.
10. Configure the iSCSI system to automatically start connections marked as “automatic” by opening the /etc/iscsi/iscsid.conf file and changing node.startup = manual to node.startup = automatic.

I noticed that that localmount (which mounts fstab entries) was running before the iSCSI target was connected, so the system would read the fstab and try to mount /dev/sdb1 to /mnt/storj-iscsi/ but the mount would fail since /dev/sdb1 wasn’t available yet. I didn’t want to change the default init scripts for the system, so I just added a new init script called iscsi-wait that verifies that the iSCSI target is connected and the /dev/sdb1 partition (identified by UUID) is available and then tries to mount entries in fstab again.

Create the file /etc/init.d/iscsi-wait, make it executable with chmod +x /etc/init.d/iscsi-wait, and giving it the following contents:

#!/sbin/openrc-run

description="Check iSCSI session and mount fstab entries"

# Change the ISCSI_TARGET and DISK_UUID to values specific for your system.
ISCSI_TARGET="iqn.2000-01.com.synology:DiskStation.storj-target.9bdcdcf219"
DISK_UUID="f88fc19f-f4ab-4a33-9383-ce9a72b3d6ca"
ISCSI_WAIT_TIMEOUT="10"
UUID_WAIT_TIMEOUT="10"

# Load this script after the network is up and iscsid has started. Be sure to run this before Docker.
depend() {
    need net iscsid
    before docker
}

start() {
    ebegin "Waiting for iSCSI session for ${ISCSI_TARGET}"
    local retries="${ISCSI_WAIT_TIMEOUT}"
    while [ "${retries}" -gt 0 ]; do
        iscsiadm -m session 2>/dev/null | grep -q "${ISCSI_TARGET}" && break
        sleep 1
        retries=$((retries - 1))
    done
    if ! iscsiadm -m session 2>/dev/null | grep -q "${ISCSI_TARGET}"; then
        eend 1 "No active iSCSI session found for ${ISCSI_TARGET}"
        return 1
    fi
    eend 0

    ebegin "Waiting for /dev/disk/by-uuid/${DISK_UUID}"
    retries="${UUID_WAIT_TIMEOUT}"
    while [ "${retries}" -gt 0 ]; do
        [ -e "/dev/disk/by-uuid/${DISK_UUID}" ] && break
        sleep 1
        retries=$((retries - 1))
    done
    if [ ! -e "/dev/disk/by-uuid/${DISK_UUID}" ]; then
        eend 1 "Timed out waiting for /dev/disk/by-uuid/${DISK_UUID}"
        return 1
    fi
    eend 0

    ebegin "Mounting iSCSI fstab entries"
    local mount_output mount_status
    mount_output=$(mount -a 2>&1)
    mount_status=$?
    [ -n "${mount_output}" ] && einfo "${mount_output}"
    eend ${mount_status} "Failed to mount fstab entries"
}

Set the script to

rc-update add iscsi-wait

It’d probably be a good idea to reboot your system and make sure that the iSCSI disk connects as expected and the partition is mounted automatically. Repeating this several times for good measure to make sure it works probably wouldn’t hurt.

Here’s what my startup process looks like with that script on a test system (that’s talking to Target-2 rather than storj-target):

As you can see, localmount tries mounting the filesystem early on (lines 4-5) but fails. However, later, iscsid loads and the script above immediately loads, checks the iSCSI target is connected, then mounts the filesystem. Docker starts immediately after, and so the Storj storage node container will see the filesystem it needs as expected.

Install and Configure the Storj Storage Node

1. The Storj storage node requires docker, so install it with apk add docker, start the service with service docker start, and have it load when the system starts up with rc-update add docker default.
2. Create an Storj identity file following the instructions here. This can be time-consuming and may be better done on a more powerful computer than the Alpine VM and then transferred over to the NAS. Be sure to back up the file.
3. With the iSCSI disk connected and the partition mounted, create a subdirectory on the iSCSI directory: mkdir -p /mnt/storj-iscsi/storagenode/ — this can be named whatever you want, but I call it “storagenode”. Copy your identity directory into the storagenode directory.
4. Install the Storj storage node in a Docker container in accordance with their directions. In the docker commands, be sure to correctly configure your email, dynamic DNS address, port, payout wallet address to receive your earnings (I entered my Storj account’s deposit address since that’s all I use STORJ tokens for.), and the amount of storage you’re offering (I selected 1.5TB, again, enter what you’re making available. Make sure it’s less than the iSCSI LUN size, including the space lost due to partitioning and formatting.).
   • In both of the docker run commands during the installation process, I set <identity-dir> to /mnt/storj-iscsi/identity/storagenode/ and the <storage-dir> to /mnt/storj-iscsi/config. I also changed -p 127.0.0.1:14002:14002 to -p 14002:14002 so the dashboard interface would be available from outside the docker container on my LAN (I set up a firewall rule so I could access the dashboard interface from my other VLAN by going to http://10.200.40.151:14002). With the 127.0.0.1 present you can only access it from within the docker container itself, which isn’t very useful.
     • Don’t expose the dashboard to the internet. Keep it local to your LAN.
     • You might also prefer to keep the identity file on the VM itself, which you can do. In my case, I keep it on the NAS so it’s easily accessible if I need to create a new VM or migrate it somewhere else.
5. Install Watchtower to make sure the Storj storage node image is updated periodically.
6. Check the dashboard to make sure everything’s running properly, you’re connected to the network, the vetting process has started, and you’re getting test data.

Conclusion

I hope you found this post to be useful. I welcome any feedback, suggestions, corrections, etc. Any bugs or errors are my own. Your mileage may vary. Please take care to understand the consequences of any commands rather than just blindly copy-pasting them.