收集了一些還不錯的連結
http://wiki.alpinelinux.org/w/index.php?title=High_performance_SCST_iSCSI_Target_on_Linux_software_Raid&oldid=4999
介紹用 RAID 和 SCST 的搭配使用，不過後半段很清楚的描述從無到有把 iSCSI Target export 出去的流程
Add Target | -add_target |
Open Device | -open_dev |
Add LUN | -add_lun |
Enable Target | -enable_target |
Enable Driver | -set_drv_attr iscsi -attributes enabled=1 |
Disable Driver | -set_drv_attr iscsi -attributes enabled=0 |
Disable Target | -disable_target |
Remove LUN | -rem_lun |
Close Device | -close_dev |
Remove Target | -rem_target |
整個流程 Sample
# 加一個 Target
pjack@ubuntu3371:~$ sudo scstadmin -add_target iqn.2011-12.org.pjack:foo1 -driver iscsi
Collecting current configuration: done.
-> Making requested changes.
-> Creating target 'iqn.2011-12.org.pjack:foo1' for driver 'iscsi': done.
-> Done.
All done.
# 列出所有的 target 或是列出屬於 iscsi driver 的 Target, 看有沒有出現剛剛新增的
pjack@ubuntu3371:~$ sudo scstadmin -list_target
Collecting current configuration: done.
Driver Target
--------------------------------
iscsi iqn.2011-12.org.pjack:foo1
All done.
pjack@ubuntu3371:~$ sudo scstadmin -list_driver iscsi
Collecting current configuration: done.
Driver Target
--------------------------------
iscsi iqn.2011-12.org.pjack:foo1
All done.
# 看一下這個 Target 的一些屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_tgt_attr iqn.2011-12.org.pjack:foo1 -driver iscsi -nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
--------------------------------------------------------------------------------------------------------------------------
HeaderDigest None Yes No
RspTimeout 90 Yes No
InitialR2T No Yes No
MaxBurstLength 1048576 Yes No
QueuedCommands 32 Yes No
MaxRecvDataSegmentLength 1048576 Yes No
tid 2 No No
redirect < not set=""> Yes No
NopInTimeout 30 Yes No
rel_tgt_id 0 Yes No
NopInInterval 30 Yes No
ImmediateData Yes Yes No
MaxXmitDataSegmentLength 1048576 Yes No
cpu_mask ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff Yes No
enabled 0 Yes No
per_portal_acl 0 Yes No
addr_method PERIPHERAL Yes No
FirstBurstLength 1048576 Yes No
MaxSessions 0 Yes No
MaxOutstandingR2T 32 Yes No
DataDigest None Yes No
comment < n a=""> Yes No
io_grouping_type auto Yes No
Dynamic attributes available
----------------------------
allowed_portal
OutgoingUser
IncomingUser
LUN CREATE attributes available
-------------------------------
read_only
# 新增一個 Device
pjack@ubuntu3371:~$ sudo scstadmin -open_dev disk01 -handler vdisk_blockio -attributes filename=/dev/rbd0
Collecting current configuration: done.
-> Making requested changes.
-> Opening device 'disk01' using handler 'vdisk_blockio': done.
-> Done.
All done.
# 列出所有的 Device
pjack@ubuntu3371:~$ sudo scstadmin -list_device
Collecting current configuration: done.
Handler Device
------------------------
vdisk_nullio -
vdisk_fileio -
vdisk_blockio disk01
vcdrom -
All done.
# 列出使用 vdisk_blockio handler 的 Device
pjack@ubuntu3371:~$ sudo scstadmin -list_handler vdisk_blockio
Collecting current configuration: done.
Handler Device
------------------------
vdisk_blockio disk01
All done.
# 列出這個 device 的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_dev_attr disk01 -nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
--------------------------------------------------------------------------------------
usn 27cddc71 Yes No
blocksize 512 No No
t10_dev_id 27cddc71-disk01 Yes No
rotational 1 No No
nv_cache 0 No No
type_string Direct-access device (e.g., magnetic disk) No No
removable 0 No No
read_only 0 No No
resync_size < n a=""> Yes No
threads_num 1 Yes No
size_mb 1024 No No
write_through 0 No No
thin_provisioned 0 No No
handler vdisk_blockio No No
filename /dev/rbd0 Yes Yes
dump_prs < n a=""> Yes No
exported < n a=""> No No
threads_pool_type per_initiator Yes No
type 0 No No
# 新增一個 LUN
pjack@ubuntu3371:~$ sudo scstadmin -add_lun 0 -driver iscsi -target iqn.2011-12.org.pjack:foo1 -device disk01
Collecting current configuration: done.
-> Making requested changes.
-> Adding device 'disk01' at LUN 0 to driver/target 'iscsi/iqn.2011-12.org.pjack:foo1': done.
-> Driver/target is not a fibre channel target, ignoring.
-> Done.
All done.
# 透過 list_target 再多加 driver 參數, 可以看到這個 target 的 LUN
pjack@ubuntu3371:~$ sudo scstadmin -list_target iqn.2011-12.org.pjack:foo1 -driver iscsi
Collecting current configuration: done.
Driver: iscsi
Target: iqn.2011-12.org.pjack:foo1
Assigned LUNs:
LUN Device
-----------
0 disk01
All done.
# 看 LUN 的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_lun_attr 0 -driver iscsi -target iqn.2011-12.org.pjack:foo1 -nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
------------------------------------------
read_only 0 No No
device disk01 No No
All done.
# Enable Target
pjack@ubuntu3371:~$ sudo scstadmin -enable_target iqn.2011-12.org.pjack:foo1 -driver iscsi
Collecting current configuration: done.
-> Making requested changes.
-> Enabling driver/target 'iscsi/iqn.2011-12.org.pjack:foo1': done.
-> Done.
All done.
# 多了一個 rel_tgt_id 的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_tgt_attr iqn.2011-12.org.pjack:foo1 -driver iscsi
Collecting current configuration: done.
Attribute Value Writable KEY
------------------------------------------
rel_tgt_id 1 Yes Yes
Dynamic attributes available
----------------------------
allowed_portal
OutgoingUser
IncomingUser
LUN CREATE attributes available
-------------------------------
read_only
All done.
# 把 iscsi enable
pjack@ubuntu3371:~$ sudo scstadmin -set_drv_attr iscsi -attributes enabled=1 -noprompt
Collecting current configuration: done.
-> Making requested changes.
-> Enabling driver 'iscsi': done.
-> Done, 0 change(s) made.
All done.
# 從另一台看是否能找到
pjack@ubuntu64-33-7:~$ sudo iscsiadm -m discovery -t st -p 172.16.33.71
172.16.33.71:3260,1 iqn.2011-12.org.pjack:foo1
# Login
pjack@ubuntu64-33-7:~/Git/WiStor/Node/src/wistor/ext/itri/iscsi$ sudo iscsiadm -m node -T iqn.2011-12.org.pjack:foo1 -p 172.16.33.71 -l
Logging in to [iface: default, target: iqn.2011-12.org.pjack:foo1, portal: 172.16.33.71,3260]
Login to [iface: default, target: iqn.2011-12.org.pjack:foo1, portal: 172.16.33.71,3260]: successful
# 回到本機看一下連線 session
pjack@ubuntu3371:~$ sudo scstadmin -list_sessions
Collecting current configuration: done.
Driver/Target: iscsi/iqn.2011-12.org.pjack:foo1
Session: iqn.1993-08.org.debian:01:4e712cd57f99
Attribute Value Writable KEY
-----------------------------------------------------------------------------------------
write_cmd_count 0 Yes No
HeaderDigest None Yes No
sid 10000023d0200 Yes No
read_cmd_count 118 Yes No
bidi_io_count_kb 0 Yes No
commands 0 Yes No
InitialR2T No Yes No
MaxBurstLength 1048576 Yes No
MaxRecvDataSegmentLength 1048576 Yes No
active_commands 0 Yes No
none_cmd_count 7 Yes No
unknown_cmd_count 0 Yes No
ImmediateData Yes Yes No
MaxXmitDataSegmentLength 262144 Yes No
reinstating 0 Yes No
FirstBurstLength 262144 Yes No
read_io_count_kb 354 Yes No
MaxOutstandingR2T 1 Yes No
bidi_cmd_count 0 Yes No
write_io_count_kb 0 Yes No
DataDigest None Yes No
initiator_name iqn.1993-08.org.debian:01:4e712cd57f99 Yes No
force_close Yes No
# Logout
pjack@ubuntu64-33-7:~/Git/WiStor/Node/src/wistor/ext/itri/iscsi$ sudo iscsiadm -m node -T iqn.2011-12.org.pjack:foo1 -p 172.16.33.71 -u
Logging out of session [sid: 2, target: iqn.2011-12.org.pjack:foo1, portal: 172.16.33.71,3260]
Logout of [sid: 2, target: iqn.2011-12.org.pjack:foo1, portal: 172.16.33.71,3260]: successful
# disable target
pjack@ubuntu3371:~$ sudo scstadmin -disable_target iqn.2011-12.org.pjack:foo1 -driver iscsi -noprompt
Collecting current configuration: done.
-> Making requested changes.
-> Disabling driver/target 'iscsi/iqn.2011-12.org.pjack:foo1': done.
-> Done.
All done.
# remove lun
pjack@ubuntu3371:~$ sudo scstadmin -rem_lun 0 -driver iscsi -target iqn.2011-12.org.pjack:foo1 -device disk01 -noprompt
Collecting current configuration: done.
-> Making requested changes.
-> Removing LUN 0 from driver/target 'iscsi/iqn.2011-12.org.pjack:foo1': done.
-> Driver/target is not a fibre channel target, ignoring.
-> Done.
All done.
# close the device
pjack@ubuntu3371:~$ sudo scstadmin -close_dev disk01 -handler vdisk_blockio -noprompt
Collecting current configuration: done.
-> Making requested changes.
-> Closing device 'disk01' using handler 'vdisk_blockio': done.
-> Driver/target is not a fibre channel target, ignoring.
-> Driver/target is not a fibre channel target, ignoring.
-> Done.
All done.
# remove the target
pjack@ubuntu3371:~$ sudo scstadmin -rem_target iqn.2011-12.org.pjack:foo1 -driver iscsi -noprompt
Collecting current configuration: done.
-> Making requested changes.
-> Removing virtual target 'iqn.2011-12.org.pjack:foo1' from driver 'iscsi': done.
-> Driver/target is not a fibre channel target, ignoring.
-> Done.
All done.
http://iscsi-scst.sourceforge.net/SCST_Gentoo_HOWTO.txt
這篇裡面有不少設定 scstadmin 的範例
這篇大部份不透過 scstadmin 設定
這篇是滿早期的文章，有些已經不適用，包括安裝的 ppa 不能用，其它的部份則是可參考
這應該是把 scst 寫的最詳細的資料，而且有很多專有名詞的解釋及用法，改天應該要好好研究一下。
裡面有提到 /sys/kernel/scst_tgt/ 裡面的目錄結構，如果要查看每一個 target 有那一些連線，就要查看這個目錄，
/sys/kernel/scst_tgt/targets/iscsi//sessions
裡面有提到 /sys/kernel/scst_tgt/ 裡面的目錄結構，如果要查看每一個 target 有那一些連線，就要查看這個目錄，
/sys/kernel/scst_tgt/targets/iscsi/
這是 scstadmin 官方的 README
https://git.ipxe.org/mirror/scst/.git/blob/master:/scstadmin/scstadmin.sysfs/scstadmin
這份有整理好的 scstadmin 指令
General Operations | |
-write_config < file> | Writes the current configuration to < file>. |
-clear_config | Clear all SCST configuration. |
Query Operations | |
-list_handler | List all available handlers |
-list_handler [< hndlr>] | List all available device under specific handler |
-list_driver | List all available drivers |
-list_driver [< driver>] | List all available target under specific driver |
-list_target | List all available targets |
-list_target [< target>][-driver < driver>] | List all available LUN under specific target |
List Attribute Operations | |
-list_scst_attr | List all attributes for SCST |
-list_hnd_attr < hndlr> | List all attributes for a given handler. |
-list_dev_attr < device> | List all attributes for a given device. |
-list_drv_attr < driver> | List all attributes for a given driver. |
-list_lun_attr < lun> -driver < driver> -target < target> [-group < group>] | List all attributes for a driver/target/lun |
-list_sessions | List all current initiator sessions |
# 看一下有支援那些 handler
pjack@ubuntu3371:~$ sudo scstadmin -list_handler
Collecting current configuration: done.
Handler
-------------
vdisk_fileio
vdisk_blockio
vdisk_nullio
vcdrom
All done.
# 看一下有支援那些 driver
pjack@ubuntu3371:~$ sudo scstadmin -list_driver
Collecting current configuration: done.
Driver
-----
iscsi
All done.
# 看一下有 blockio 支援的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_hnd_attr vdisk_blockio -nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
--------------------------------------------------------------------------------
type_string Direct-access device (e.g., magnetic disk) No No
disk01 < n a=""> Yes No
trace_level out_of_mem Yes No
minor Yes No
pid Yes No
line Yes No
function Yes No
special Yes No
mgmt Yes No
mgmt_dbg Yes No
type 0 No No
Possible trace levels:
(use trace_level="add < level>", none, all or default to set):
debug, function, line, pid, entryexit,
buff, mem, sg, out_of_mem, special,
scsi, mgmt, minor, mgmt_dbg, scsi_serializing,
retry, recv_bot, send_bot, recv_top, pr,
send_top, order,
Device CREATE attributes available
----------------------------------
blocksize
write_through
rotational
nv_cache
thin_provisioned
filename
removable
read_only
# 看一下有 fileio 支援的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_hnd_attr vdisk_fileio -nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
--------------------------------------------------------------------------------
type_string Direct-access device (e.g., magnetic disk) No No
trace_level out_of_mem Yes No
minor Yes No
pid Yes No
line Yes No
function Yes No
special Yes No
mgmt Yes No
mgmt_dbg Yes No
type 0 No No
Possible trace levels:
(use trace_level="add < level>", none, all or default to set):
debug, function, line, pid, entryexit,
buff, mem, sg, out_of_mem, special,
scsi, mgmt, minor, mgmt_dbg, scsi_serializing,
retry, recv_bot, send_bot, recv_top, pr,
send_top, order,
Device CREATE attributes available
----------------------------------
o_direct
blocksize
write_through
rotational
nv_cache
thin_provisioned
filename
removable
read_only
All done.
# 看一下有 nullio 支援的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_hnd_attr vdisk_nullio -nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
--------------------------------------------------------------------------------
type_string Direct-access device (e.g., magnetic disk) No No
trace_level out_of_mem Yes No
minor Yes No
pid Yes No
line Yes No
function Yes No
special Yes No
mgmt Yes No
mgmt_dbg Yes No
type 0 No No
Possible trace levels:
(use trace_level="add < level>", none, all or default to set):
debug, function, line, pid, entryexit,
buff, mem, sg, out_of_mem, special,
scsi, mgmt, minor, mgmt_dbg, scsi_serializing,
retry, recv_bot, send_bot, recv_top, pr,
send_top, order,
Device CREATE attributes available
----------------------------------
removable
blocksize
read_only
rotational
# 看一下有 cdrom 支援的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_hnd_attr vcdrom -nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
------------------------------------------------------
type_string CD-ROM device No No
trace_level out_of_mem Yes No
minor Yes No
pid Yes No
line Yes No
function Yes No
special Yes No
mgmt Yes No
mgmt_dbg Yes No
type 5 No No
Possible trace levels:
(use trace_level="add < level>", none, all or default to set):
debug, function, line, pid, entryexit,
buff, mem, sg, out_of_mem, special,
scsi, mgmt, minor, mgmt_dbg, scsi_serializing,
retry, recv_bot, send_bot, recv_top, pr,
send_top, order,
All done.
# 看一下有 iscsi driver 支援的屬性
pjack@ubuntu3371:~$ sudo scstadmin -list_drv_attr iscsi --nonkey
Collecting current configuration: done.
Attribute Value Writable KEY
-----------------------------------------------------
open_state open No No
isns_entity_name < not set=""> Yes No
version 3.0.0-pre1 No No
trace_level out_of_mem Yes No
minor Yes No
pid Yes No
line Yes No
function Yes No
special Yes No
mgmt Yes No
mgmt_dbg Yes No
conn Yes No
iSNSServer < not set=""> Yes No
enabled 0 Yes No
Possible trace levels:
(use trace_level="add < level>", none, all or default to set):
debug, function, line, pid, entryexit,
buff, mem, sg, out_of_mem, special,
scsi, mgmt, minor, mgmt_dbg, scsi_serializing,
retry, recv_bot, send_bot, recv_top, pr,
send_top, d_write, conn, conn_dbg, iov,
pdu, net_page,
Dynamic attributes available
----------------------------
OutgoingUser
IncomingUser
scstadmin script 內容
把原本的 IET 改接到 SCST，另外也有提到一些 performance fine tune 的建議，另外也有 /sys/kernel/scst_tgt/ 的目錄結構
http://linux-iscsi.org/index.php/ALUA
對於 ALUA 的解釋
沒有留言:
張貼留言