How to Setup Linux iSCSI Storage Target and Remote Client Access

iSCSI stands for Internet Small Computer System Interface.

This works on TCP/IP layer and allows SCSI commands to be sent over local area networks (LAN).

It is used to access the storage on your network over TCP/IP using block-level data transfers.

There is often a comparison between NFS vs iSCSI. The key difference is NFS is file-level implementation whereas iSCSI is a block-level implementation.

In block-level implementations such as iSCSI and Fiber Channel, the storage array offers up a collection of blocks to the client, which then formats that raw storage with whatever file system it decides to use.

In order to access the storage over iscsi, we need to have the following components:

1. iSCSI initiator (Hardware/Software) – This is iscsi client that can send iscsi commands over TCP/IP network. This is usually your NIC adapter on your server and can be physical or virtual with an IP configured on the same subnet with the storage array’s iscsi interface.
2. iSCSI Target (Your storage node) – This is your storage node that you can configure on your own or can be physical array from a vendor that supports iscsi configuration
3. iSCSI switch (optional) – The iscsi switch is optional here but inorder to separate the host traffic from the iscsi traffic in enterprise deployments, it is recommended to use a separate switch for iscsi traffic.

In this tutorial, we are going to see how to setup our own storage node that supports the iscsi configuration and access the storage from the iscsi client on a Centos 6 machine.

The same procedures should work for RHEL 6 (CentOS 6) version. For CentOS 7, the procedure is slightly different using targetcli, which we will cover in a different article.

1. Install SCSI Target Utils and Start tgtd

First, install the scsi-target-utils using yum command on the storage node, which is required to configure the iSCSI.

yum install scsi-target-utils

If you don’t have Perl installed on your system already, the above command will install perl and several other dependencies (for example: sg3_utils) automatically.

This will install the tgtadm command as shown below.

# which tgtadm
/usr/sbin/tgtadm

Star the SCSI target daemon as shown below.

# service tgtd start
Starting SCSI target daemon: [ OK ]

Using chkconfig command, make sure it starts automatically during system startup.

chkconfig tgtd on

2. Add New iSCSI Target using tgtadm

Use tgtadm command to add a new iSCSI target on the storage node using the following syntax:

tgtadm --lld --mode target --op new --tid --targetname

The following example creates a target with target id 1 with the below iqn,

tgtadm --lld iscsi --op new --mode target --tid 1 \
       -T iqn.2016-03.com.thegeekstuff:storage.lun01.thegeekstuff.com

In the above command:

• tgtadm command is the Linux SCSI Target Administration Utility
• –lld iscsi specifies that we are using iscsi driver (You can also use -L instead of –lld)
• –op new specifies the operation, in this case, we are creating a new target (You can also use -o instead of –op)
• –mode target indicates that we are creating a target. Other possible modes are: logicalunit, account, portal, conn, sys. (You can also use -m instead of –mode)
• –tid 1 specifies the target id, which is 1 in this example (You can also use -t instead of –tid)
• -T specifies the target name (You can also use –targetname instead of -T)

3. View the iSCSI Target configuration

Once the target is created, you can view the details using the “show” operation as shown below. This will provide the system information, and LUN information of the all the targets available on your system.

As you see from the following output, we have only one target (Target 1), which is iSCSI driver, with one LUN (LUN 0).

# tgtadm --lld iscsi --op show --mode target
Target 1: iqn.2016-03.com.thegeekstuff:storage.lun01.thegeekstuff.com
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
Account information:
ACL information:

Note: In the above output, the “Backing store path” is currently None, as we have not yet associated any physical storage path to this iSCSI yet.

4. Add the LUN to the iSCSI target

In the following example, we will be adding an existing lun, which is /storage/lun01 that is mounted and available on our storage node to the iSCSI target that we created.

For the existing LUN (/storage/lun01), you should also set the SELinux context appropriately as shown below.

semanage fcontext -a -t tgtd_var_lib_t /storage/lun01

restorecon -Rv /storage/lun01

If you are getting “Invalid Request”, for temporary testing purpose, you may want to disable the SELinux by setting SELinux to permissive mode to see whether it solves the problem, and narrow down the issue to SELinux permission.

Now, using the tgtadm command, add the /storage/lun01 to the iSCSI target that we created earlier.

tgtadm --lld iscsi --op new --mode logicalunit --tid 1 \
       --lun 1 --backing-store /storage/lun01

In the above command:

• most of the options are same as we explained in our previous tgtadm command above.
• –mode logicalunit. Please note that earlier we used “target” for mode, but here we are using logicalunit for mode.
• –lun indicates the the LUN id, which is 1 in this example (You can also use -l instead of –lun)
• –backing-store indicates the physical storage path on the storage server. In our example, it is /storage/lun01 (You can also use -b instead of –backing-store option)
• –tid 1 indicates that connect this given LUN to the existing target id 1, which we created earlier.

Now, view the details of the iSCSI on your system again.

But, this time, In the LUN 1, the “Backing store path” in the following output will point to the storage path that we associated with this iSCSI. In our example, it is /storage/lun01.

# tgtadm --lld iscsi --op show --mode target
Target 1: iqn.2016-03.com.thegeekstuff:storage.lun01.thegeekstuff.com
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 5369 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /storage/lun01
Backing store flags:
Account information:
ACL information:

For some reason, if you’ve made a mistake in the above command, and like to delete the LUN1 association that we created above, use the following command.

tgtadm --lld iscsi --mode logicalunit --op delete --tid 1 --lun 1

In the above command:

• Everything is same as the previous command
• But, here the -op (which is operation) is delete.
• This will delete the lun1 from the iSCSI tid 1 (target 1)

5. Setup ACL Restriction for the iSCSI

Next, you need to setup ACL to allow the initiators that can access this scsi target.

tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL

In the above command:

• Most of the options are pretty much similar to our previous commands
• -I ALL option indicates that we are allowing the target to accept ALL initiators (You can also use –initiator-address instead of -I)

When you view the iSCSI details using “show” operation, you’ll notice the ACL information towards the end of the output as shown below.

# tgtadm --lld iscsi --op show --mode target
..
..
Backing store path: /storage/lun01
Backing store flags:
Account information:
ACL information:
ALL

If you want to restrict only the client that needs to access the storage, you can use the following command and specify a specific ip-address.

tgtadm --lld iscsi --mode target --op bind --tid 1 192.168.2.109

If you want to restrict access to a particular sub-net, you can specific the sub-net instead of the ip-address as shown below.

tgtadm --lld iscsi --mode target --op bind --tid 1 192.168.2.0/24

6. Allow Access to Port 3260

If you are running ip-tables, add a ip-tables rules and allow port 3260 as shown below.

iptables -I INPUT -p tcp -m tcp --dport 3260 -j ACCEPT

Verify that the port 3260 is open using netstat command as shown below.

# netstat -tunlp | grep 3260
tcp 0 0 0.0.0.0:3260 0.0.0.0:* LISTEN 1248/tgtd
tcp 0 0 :::3260 :::* LISTEN 1248/tgtd

Note: Up until this point, all of the above commands were executed on the storage server node. From now-on-awards, all the commands in the following section will be executed on the client machine, which needs to connect to the storage server.

7. On Client: Discover iSCSI

On the client, before you discover the iSCSI, you’ll not see the new iSCSI disk. For example, the following output will not display anything.

fdisk -l /dev/sdb

Now all the target side configuration is complete and the lun can be accessed over iscsi from the client. The below procedures will show you how to configure the client side to access the storage.

On the client server, install iscsi-initiator-utils package as shown below.

yum install iscsi-initiator-utils

Next, discover the iSCSI target using iscsiadm command as shown below.

[root@client ~]# iscsiadm --mode discovery --type sendtargets \
                --portal 192.168.2.166

The above command will display the following output:

192.168.2.166:3260,1 iqn.2016-03.com.thegeekstuff:storage.lun01.thegeekstuff.com

In the above command:

• iscsiadm command is the open-iscsi administration utility
• –mode discover indicates that we are discovering the iSCSI server here. Other possible modes are: discoverydb, node, fw, host iface or session. (You can also use -m instead of –mode)
• –type specificy sendtargets here. Other posible values are: sendtargets, fw, and iSNS. You can also use st abbreviation instead of sendtargets. (You can also use -t instead of –type)
• –portal specify the ip-address of the iSCSI storage node that we setup earlier (You can also use -p instead of –portal).
• For the ip-address, if you are running iSCSI on non-default port (anything other than 3260), specify it after ip-address. For example, 192.168.2.166:3261

Note: If you get “No route to host” error, then the iptables on the target or the client is not configured properly.

8. On Client: Login to iSCSI Target Session

Once the discovery process is completed successfully, you need to login to the iscsi target session by using the following command:

[root@client ~]# iscsiadm --mode node --targetname iqn.2016-03.com.thegeekstuff:storage.lun01.thegeekstuff.com \
                          --portal 192.168.2.166 --login

The above command will display the following output:

Logging in to [iface: default, target: iqn.2016-03.com.thegeekstuff:storage.lun01.thegeekstuff.com, portal: 192.168.2.166,3260] (multiple)
Login to [iface: default, target: iqn.2016-03.com.thegeekstuff:storage.lun01.thegeekstuff.com, portal: 192.168.2.166,3260] successful.

In the above command:

• Most of the options are similar to the previous command.
• For mode, here we are using node (instead of discovery)
• -targetname specifies the target name. This is the tragetname that we specified on the storage node earlier when we created the iSCSI (You can also use -T instead of –targetname)
• –login option will login to all the discovered targets. In our case, it will login to our storage iSCSI server specified in the –portal option.

Now you should be seeing your new iscsi device on your fdisk output. You can format it, add it to lvm and use it as needed.

The fdisk command output will now show the device.

[root@client ~]# fdisk -l /dev/sdb
Disk /dev/sdb: 5368 MB, 5368709120 bytes
166 heads, 62 sectors/track, 1018 cylinders
Units = cylinders of 10292 * 512 = 5269504 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Note: The above disk is not coming from the client machine, this is coming from the storage server as iSCSI.