ISCSI网络存储服务（my version）
Table 26.2 iSCSI Terminology
| Item | Description |
|---|---|
| IQN | The iSCSI qualified name. A unique name that is used for identifying targets as well as initiators. |
| Backend storage | The storage devices on the iSCSI target that the iSCSI target component is providing access to. |
| Target | The service on an iSCSI server that gives access to backend storage devices. |
| Initiator | The iSCSI client that connects to a target and is identified by an IQN. |
| ACL | The access control list that is based on the iSCSI initiator IQNs and used to provide access to a specific target. While setting up the iSCSI target on RHEL7, creating an ACL is mandatory. This ACL is based on the IQN of the iSCSI initiator that should be granted access. |
| LUN | A logical unit number. The backend storage devices that are shared through the target. This can be any device that supports read/write operations, such as disks, partitions, logical volumes, files, or tape drives. |
| Portal | The IP address and port that a target or initiator uses to establish connections.Also referred to as node. |
| TPG | The target portal group. This is the collection of IP address and TCP ports to which a specific iSCSI target will listen. |
| Discovery | The process whereby an initiator finds the targets that are configured on a portal and stores this information locally for future reference. Discovery is done by using the iscsiadm command. |
| Login | Authentication that gives an initiator access to LUNs on the target. After successful login, the login information is stored on the initiator automatically.Login is performed using the iscsiadm command. |
iscsi实现网络存储，提供存储端叫 target，使用存储端叫 initiator。target 上可以提供存储空间，initiator负责连接ISCSI设备，在ISCSI设备中创建文件系统，以及存取数据，在 initiator 上看上去是多了一块硬盘。
服务器端配置target，准备做为LUN发布的ISCSI的存储设备（可以是一整块磁盘、一个分区、逻辑卷或RAID阵列）。 本人事先准备好两个逻辑卷作为ISCSI的存储设备。
一、前期准备
硬盘
/dev/sdb/
首先安装target
[root@localhost yum.repos.d]# yum -y install targetd targetcli
启动服务
[root@localhost ~]# systemctl enable target
Created symlink from /etc/systemd/system/multi-user.target.wants/target.service to /usr/lib/systemd/system/target.service.
[root@localhost ~]# systemctl start target
设置防火墙
[root@localhost ~]# firewall-cmd --permanent --add-port=3260/tcp
success
[root@localhost ~]# firewall-cmd --reload
success
执行targetcli工具
[root@localhost ~]# targetcli
Warning: Could not load preferences file /root/.targetcli/prefs.bin.
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/> exit
Global pref auto_save_on_exit=true
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
[root@localhost ~]#
[root@localhost ~]# firewall-cmd --reload
success
[root@localhost ~]# targetcli
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
ls查看
/> ls
o- / ............................................................................................. [...]
o- backstores .................................................................................. [...]
| o- block ...................................................................... [Storage Objects: 0]
| o- fileio ..................................................................... [Storage Objects: 0]
| o- pscsi ...................................................................... [Storage Objects: 0]
| o- ramdisk .................................................................... [Storage Objects: 0]
o- iscsi ................................................................................ [Targets: 0]
o- loopback ............................................................................. [Targets: 0]
注：输入help可以查看targetcli的帮助
/> help
GENERALITIES
============
This is a shell in which you can create, delete and configure
configuration objects.
The available commands depend on the current path or target
path you want to run a command in: different path have
different sets of available commands, i.e. a path pointing at
an iscsi target will not have the same availaible commands as,
say, a path pointing at a storage object.
The prompt that starts each command line indicates your
current path. Alternatively (useful if the prompt displays
an abbreviated path to save space), you can run the
pwd command to display the complete current path.
Navigating the tree is done using the cd command. Without
any argument, cd will present you wil the full objects
tree. Just use arrows to select the destination path, and
enter will get you there. Please try help cd for navigation
tips.
COMMAND SYNTAX
==============
Commands are built using the following syntax:
[TARGET_PATH] COMMAND_NAME [OPTIONS]
The TARGET_PATH indicates the path to run the command from.
If ommited, the command will be run from your current path.
The OPTIONS depend on the command. Please use help
COMMAND to get more information.
AVAILABLE COMMANDS
==================
The following commands are available in the
current path:
- bookmarks action [bookmark]
- cd [path]
- clearconfig [confirm]
- exit
- get [group] [parameter...]
- help [topic]
- ls [path] [depth]
- pwd
- refresh
- restoreconfig [savefile] [clear_existing]
- saveconfig [savefile]
- sessions [action] [sid]
- set [group] [parameter=value...]
- status
- version
二、target端（服务器端）
基本思路：
先把准备共享的块做出来，创建一个target，在target上创建lun，一个lun连接一个块
st=>start: 准备块
e=>end
op=>operation: 创建一个target
op1=>operation: target上创建lun
op2=>operation: lun连接块
st->op->op1->op2
st=>start: Start:>http://www.google.com[blank]
e=>end:>http://www.google.com
op1=>operation: My Operation
sub1=>subroutine: My Subroutine
cond=>condition: Yes
or No?:>http://www.google.com
io=>inputoutput: catch something...
st->op1->cond
cond(yes)->io->e
cond(no)->sub1(right)->op1
1、创建块即给要发布的sdb起个名字
/> backstores/block create disk1 /dev/sdb
Created block storage object disk1 using /dev/sdb.
给/dev/sdb设置的名字为disk1
查看块如下图所示：
/> ls
o- / ............................................................................................. [...]
o- backstores .................................................................................. [...]
| o- block ...................................................................... [Storage Objects: 1]
| | o- disk1 ............................................ [/dev/sdb (100.0GiB) write-thru deactivated]
2、创建 iqn 名字即创建ISCSI对象
/> iscsi/ create iqn.2016-07.com.perma.disk1
Created target iqn.2016-07.com.perma.disk1.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
查看ISCSI对象如下图所示：
/> ls
o- / ............................................................................................. [...]
o- backstores .................................................................................. [...]
| o- block ...................................................................... [Storage Objects: 0]
| o- fileio ..................................................................... [Storage Objects: 0]
| o- pscsi ...................................................................... [Storage Objects: 0]
| o- ramdisk .................................................................... [Storage Objects: 0]
o- iscsi ................................................................................ [Targets: 1]
| o- iqn.2016-07.com.perma.disk1 ........................................................... [TPGs: 1]
这里要注意命名的规则是
- iqn.是固定的
- 后面是YY-MM
3、设置ACL即将ISCSI对象与客户端IP或主机名绑定
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/acls create iqn.2016.-07.com.jason:client
WWN not valid as: iqn, naa, eui
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/acls create iqn.2016-07.com.jason:client
Created Node ACL for iqn.2016-07.com.jason:client
查看设置的ACL
/> ls
o- / ............................................................................................. [...]
o- backstores .................................................................................. [...]
| o- block ...................................................................... [Storage Objects: 1]
| | o- disk1 ............................................ [/dev/sdb (100.0GiB) write-thru deactivated]
| o- fileio ..................................................................... [Storage Objects: 0]
| o- pscsi ...................................................................... [Storage Objects: 0]
| o- ramdisk .................................................................... [Storage Objects: 0]
o- iscsi ................................................................................ [Targets: 1]
| o- iqn.2016-07.com.perma.disk1 ........................................................... [TPGs: 1]
| o- tpg1 ................................................................... [no-gen-acls, no-auth]
| o- acls .............................................................................. [ACLs: 1]
| | o- iqn.2016-07.com.jason:client ............................................. [Mapped LUNs: 0]
iqn.2016-07.com.jason:client是 initiator 的名字，需要在客户端中设置的。
4、创建LUN并绑定块
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/luns create /backstores/block/disk1
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2016-07.com.jason:client
一个ISCSI对象可以创建多个LUN（LUN0、LUN1……）。
执行ls查看绑定后的结果
/> ls
o- / ............................................................................................. [...]
o- backstores .................................................................................. [...]
| o- block ...................................................................... [Storage Objects: 1]
| | o- disk1 .............................................. [/dev/sdb (100.0GiB) write-thru activated]
| o- fileio ..................................................................... [Storage Objects: 0]
| o- pscsi ...................................................................... [Storage Objects: 0]
| o- ramdisk .................................................................... [Storage Objects: 0]
o- iscsi ................................................................................ [Targets: 1]
| o- iqn.2016-07.com.perma.disk1 ........................................................... [TPGs: 1]
| o- tpg1 ................................................................... [no-gen-acls, no-auth]
| o- acls .............................................................................. [ACLs: 1]
| | o- iqn.2016-07.com.jason:client ............................................. [Mapped LUNs: 1]
| | o- mapped_lun0 ..................................................... [lun0 block/disk1 (rw)]
| o- luns .............................................................................. [LUNs: 1]
| | o- lun0 ............................................................. [block/disk1 (/dev/sdb)]
启动监听程序
删除0.0.0.0 3260
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/portals/ create 173.16.16.20
Using default IP port 3260
Could not create NetworkPortal in configFS
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/portals/0.0.0.0:3260
last bookmarks cd enable_iser exit get help info
ls pwd refresh set status
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/portals/
@last bookmarks cd create delete exit get help ls
pwd refresh set status
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/portals/ delete
0.0.0.0 ip_address= ip_port=
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/portals/ delete 0.0.0.0
Missing required parameter ip_port
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/portals/ delete ip_address=0.0.0.0 ip_port=3260
Deleted network portal 0.0.0.0:3260
/> ls
o- / ............................................................................................. [...]
o- backstores .................................................................................. [...]
| o- block ...................................................................... [Storage Objects: 1]
| | o- disk1 .............................................. [/dev/sdb (100.0GiB) write-thru activated]
| o- fileio ..................................................................... [Storage Objects: 0]
| o- pscsi ...................................................................... [Storage Objects: 0]
| o- ramdisk .................................................................... [Storage Objects: 0]
o- iscsi ................................................................................ [Targets: 1]
| o- iqn.2016-07.com.perma.disk1 ........................................................... [TPGs: 1]
| o- tpg1 ................................................................... [no-gen-acls, no-auth]
| o- acls .............................................................................. [ACLs: 1]
| | o- iqn.2016-07.com.jason:client ............................................. [Mapped LUNs: 1]
| | o- mapped_lun0 ..................................................... [lun0 block/disk1 (rw)]
| o- luns .............................................................................. [LUNs: 1]
| | o- lun0 ............................................................. [block/disk1 (/dev/sdb)]
| o- portals ........................................................................ [Portals: 0]
o- loopback ............................................................................. [Targets: 0]
删除后创建
/> /iscsi/iqn.2016-07.com.perma.disk1/tpg1/portals create 173.16.16.20
Using default IP port 3260
Created network portal 173.16.16.20:3260.
173.16.16.20是ISCSI服务端网卡IP
/> ls
o- / ............................................................................................. [...]
o- backstores .................................................................................. [...]
| o- block ...................................................................... [Storage Objects: 1]
| | o- disk1 .............................................. [/dev/sdb (100.0GiB) write-thru activated]
| o- fileio ..................................................................... [Storage Objects: 0]
| o- pscsi ...................................................................... [Storage Objects: 0]
| o- ramdisk .................................................................... [Storage Objects: 0]
o- iscsi ................................................................................ [Targets: 1]
| o- iqn.2016-07.com.perma.disk1 ........................................................... [TPGs: 1]
| o- tpg1 ................................................................... [no-gen-acls, no-auth]
| o- acls .............................................................................. [ACLs: 1]
| | o- iqn.2016-07.com.jason:client ............................................. [Mapped LUNs: 1]
| | o- mapped_lun0 ..................................................... [lun0 block/disk1 (rw)]
| o- luns .............................................................................. [LUNs: 1]
| | o- lun0 ............................................................. [block/disk1 (/dev/sdb)]
| o- portals ........................................................................ [Portals: 1]
| o- 173.16.16.20:3260 .................................................................... [OK]
可以查看/etc/target/saveconfig.json配置文件，该配置文件保存着ISCSI的配置。
三、在initiator端的配置：
1、安装软件
[root@localhost ~]# rpm -qa | grep iscsi
iscsi-initiator-utils-6.2.0.873-32.el7.x86_64
iscsi-initiator-utils-iscsiuio-6.2.0.873-32.el7.x86_64
libiscsi-1.9.0-6.el7.x86_64
2、给 initiator 起个名字
[root@localhost ~]# vim /etc/iscsi/initiatorname.iscsi
内容如下：
[root@localhost ~]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.2016-07.com.jason:client
3、启动服务
[root@localhost ~]# systemctl enable iscsi
[root@localhost ~]# systemctl start iscsi
4、发现存储
| Item | Description |
|---|---|
| --type | sendtargets This tells the discovery mode how to find the iSCSI targets. In some configurations, an iSNS service can be configured to make discovery easier, but setting up iSNS is not an RHCE requirement. Therefore, you’ll find the targets you need using --type sendtargets. |
| --portal | This argument tells the iscsiadm command which IP address and port to address to perform the discovery. You can use an IP address or node name as the argument, and optionally, you can specify a port as well. If no port is specified, the default port 3260 is used. |
| --discover | This argument tells the iscsid service to perform a discovery. |
[root@localhost ~]# iscsiadm -m discovery -t st -p 173.16.16.20
173.16.16.20:3260,1 iqn.2016-07.com.perma.disk1
5、登录存储
| Item | Description |
|---|---|
| --mode node | This specifies iscsiadm to enter “node” mode. This is the mode in which the actual connection with the target can be established. |
| --targetname | This specifies the name of the target as discovered when using the iSCSI discovery process. |
| --portal | This is the IP address and port on which the target is listening. |
| --login | This authenticates to the target and will store credentials as well to ensure that on reboot the connection can be reestablished again. |
# iscsiadm -m node -T iqn.2015-06.com.benet:disk1 -p 172.24.3.5 -l
登录失败过程
[root@localhost ~]# iscsiadm -m node -T iqn.2016-07.com.perma:disk1 -p 173.16.16.20 -l
iscsiadm: No records found
[root@localhost ~]# iscsiadm -m node -T iqn.2016-07.com.perma:disk1 -p 173.16.16.20 -l
iscsiadm: No records found
[root@localhost ~]# iscsiadm -m discovery -t st -p 173.16.16.20
173.16.16.20:3260,1 iqn.2016-07.com.perma.disk1
[root@localhost ~]# iscsiadm -m node -T iqn.2016-07.com.per:disk1 -p 173.16.16.20 -l
iscsiadm: No records found
[root@localhost ~]# ping 173.16.16.20
PING 173.16.16.20 (173.16.16.20) 56(84) bytes of data.
64 bytes from 173.16.16.20: icmp_seq=1 ttl=64 time=0.531 ms
64 bytes from 173.16.16.20: icmp_seq=2 ttl=64 time=0.482 ms
^C
--- 173.16.16.20 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.482/0.506/0.531/0.033 ms
[root@localhost ~]# iscsiadm -m node -T iqn.2016-07.com.per.disk1 -p 173.16.16.20 -l
iscsiadm: No records found
[root@localhost ~]#
[root@localhost ~]# iscsiadm -m discovery -t st -p 173.16.16.20
173.16.16.20:3260,1 iqn.2016-07.com.perma.disk1
[root@localhost ~]# iscsiadm -m node -T iqn.2016-07.com.per.disk1 -p 173.16.16.20 -l
iscsiadm: No records found
- 第一次是因为将"."错误打成了":"
- 第二次是将"perma"错误打成了"per"
成功登录
[root@localhost ~]# iscsiadm -m node -T iqn.2016-07.com.perma.disk1 -p 173.16.16.20 -l
Logging in to [iface: default, target: iqn.2016-07.com.perma.disk1, portal: 173.16.16.20,3260] (multiple)
Login to [iface: default, target: iqn.2016-07.com.perma.disk1, portal: 173.16.16.20,3260] successful.
[root@localhost ~]#
注：-l表示连接ISCSI目标；-u表示断开和ISCSI目标的连接 验证客户端ISCSI连接
剩下来的操作就和管理本地磁盘一样了。
The second issue when making persistent iSCSI mounts is that normally the /etc/fstab file is processed before the network is available. To make sure the iSCSI disk can be mounted, you need to use the _netdev mount option in /etc/fstab. So to ensure that an iSCSI mount is configured persistently, put an entry in /etc/ fstab that looks like this:
UUID-XXXXXXXX-XXXX-XXXX-XXXXXXXX /iscsi xfs _netdev 0 2