Jump to content
Welcome to our new Citrix community!
  • 0

Multipathing on IBM v5030 from XenServer 6.5 SP1 does not seem to work automatically


Erik Schellekens

Question

Hi, 

 

I have been struggling with this issue now for a couple of days. I seem to have made some steps, though multipathing does not seem to work automatically.

What I see is that when I issue the command iscsiadm -m discovery -t st -p <IP> it only returns 1 path, it does not return the 2 paths. Same with the other node, only returns 1 path.

If I first run discovery manually with that command for both of the nodes and issue iscsiadm -m node --loginall=all, then the server reports 2 of 2 paths active (1 ISCSI session).

If I restart the service with service multipathd restart, it will still be ok as mpathutil status will return:
show topology
3600507XXXX818408f00000000000000X dm-0 IBM,2145
size=200G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=enabled
| `- 3:0:0:0 sdb 8:16 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 4:0:0:0 sdc 8:32 active ready running
 

I could of course put these commands in a script and run it at start-up though if I put those commands after each other coupled by && it will produce mixed results, some of the servers will work, but some of them it will not. When that happens it's also not possible to re-run the commands to get it to work. 

 

Below the output of the manual workaround.

[root@nldxensrv1 ~]# multipath -ll
3600507XXXX818408f00000000000000X dm-1 IBM,2145
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=50 status=active
  `- 3:0:0:0 sdb 8:16 active ready running
[root@nldxensrv1 ~]# iscsiadm -m discovery --type sendtargets --portal 10.x.x.1
10.x.x.1:3260,1 iqn.1986-03.com.ibm:2145.v5030-iscsi.node1
[root@nldxensrv1 ~]# iscsiadm -m discovery --type sendtargets --portal 10.x.x.2
10.x.x.2:3260,1 iqn.1986-03.com.ibm:2145.v5030-iscsi.node2
[root@nldxensrv1 ~]# iscsiadm -m node --loginall=all
Logging in to [iface: default, target: iqn.1986-03.com.ibm:2145.v5030-iscsi.node2, portal: 10.x.x.2,3260] (multiple)
Login to [iface: default, target: iqn.1986-03.com.ibm:2145.v5030-iscsi.node2, portal: 10.x.x.2,3260] successful.
[root@nldxensrv1 ~]# multipath -ll
3600507XXXX818408f00000000000000X dm-1 IBM,2145
size=200G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=enabled
| `- 3:0:0:0 sdb 8:16 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 4:0:0:0 sdc 8:32 active ready running

 

(Probably the discovery of the already connected path is not needed)

 

My multipath.conf

[root@nldxensrv1 ~]# cat /etc/multipath.conf
# This configuration file is used to overwrite the built-in configuration of
# multipathd.
# For information on the syntax refer to `man multipath.conf` and the examples
# in `/usr/share/doc/device-mapper-multipath-*/`.
# To check the currently running multipath configuration see the output of
# `multipathd -k"show conf"`.
defaults {
        user_friendly_names     no
        replace_wwid_whitespace yes
        dev_loss_tmo            30
}
devices {
        device {
                vendor                  "DataCore"
                product                 "SAN*"
                path_checker            "tur"
                path_grouping_policy    failover
                failback                30
        }
        device {
                vendor                  "DELL"
                product                 "MD36xx(i|f)"
                features                "2 pg_init_retries 50"
                hardware_handler        "1 rdac"
                path_selector           "round-robin 0"
                path_grouping_policy    group_by_prio
                failback                immediate
                rr_min_io               100
                path_checker            rdac
                prio                    rdac
                no_path_retry           30
        }
        device {
                vendor                  "DGC"
                product                 ".*"
                detect_prio             yes
                retain_attached_hw_handler yes
        }
        device {
                vendor                  "EQLOGIC"
                product                 "100E-00"
                path_grouping_policy    multibus
                path_checker            tur
                failback                immediate
                path_selector           "round-robin 0"
                rr_min_io               3
                rr_weight               priorities
        }
        device {
                vendor                  "NETAPP"
                product                 "LUN.*"
                dev_loss_tmo            30
        }
        device {
                vendor "IBM"
                product "2145"
                path_grouping_policy "group_by_prio"
                path_selector "round-robin 0" # Used by Red Hat 6.2
                prio "alua"
                path_checker "tur"
                failback "immediate"
                no_path_retry 5
                rr_weight uniform
                rr_min_io_rq "1"
                dev_loss_tmo 120
        }
#       device {
#                vendor                  "IBM"
#                product                 "1723*"
#                hardware_handler        "1 rdac"
#                path_selector           "round-robin 0"
#                path_grouping_policy    group_by_prio
#                failback                immediate
#                path_checker            rdac
#                prio                    rdac
#        }

}
 

During boot I see the following events:

Aug  8 08:08:27 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|PBD.plug R:dce1431e8a42|storage_access] SR 0d1fca2e-8fa5-a468-3840-27b91915bab4 will be implemented by /services/SM/lvmoiscsi in VM OpaqueRef:c369177e-4b1e-3bfa-8f3c-e230b3a76dba
Aug  8 08:08:27 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|PBD.plug R:dce1431e8a42|storage_impl] SR.attach dbg:OpaqueRef:dce1431e-8a42-52f1-957c-8329cff9ba37 sr:0d1fca2e-8fa5-a468-3840-27b91915bab4 device_config:[targetIQN:iqn.1986-03.com.ibm:2145.v5030-hb.node1; SCSIid:360050XXX0818408f00000000000000X; target:10.x.x.1; port:3260]
Aug  8 08:08:27 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|sm_exec D:cde9ab537ead|xapi] Session.create trackid=fe70c35b2b487bb5913ba0da08903aeb pool=false uname= originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Aug  8 08:08:29 nldxensrv1 iscsid: iscsid shutting down.
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.439510] bnx2x 0000:02:00.0 eth0: Removed CNIC device
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.583652] bnx2x 0000:02:00.1 eth1: Removed CNIC device
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.759365] Loading iSCSI transport class v2.0-870.
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.763550] iscsi: registered transport (tcp)
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.781587] iscsi: registered transport (iser)
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.827030] cnic: Broadcom NetXtreme II CNIC Driver cnic v2.5.19b (Mar 05, 2014)
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.829161] Broadcom NetXtreme II iSCSI Driver bnx2i v2.7.10.31b (Mar 06, 2014)
Aug  8 08:08:29 nldxensrv1 kernel: [8445621.829188] iscsi: registered transport (bnx2i)
Aug  8 08:08:29 nldxensrv1 iscsid: iSCSI logger with pid=21285 started!
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.738220] scsi6 : iSCSI Initiator over TCP/IP
Aug  8 08:08:30 nldxensrv1 iscsid: iSCSI daemon with pid=21286 started!
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.998693] scsi 6:0:0:0: Direct-Access     IBM      2145             0000 PQ: 0 ANSI: 6
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.998756] scsi 6:0:0:0: alua: supports implicit TPGS
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.999103] scsi 6:0:0:0: alua: port group 00 rel port 180
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.999301] scsi 6:0:0:0: alua: rtpg failed with 8000002
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.999510] scsi 6:0:0:0: alua: transition timeout set to 60 seconds
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.999516] scsi 6:0:0:0: alua: port group 00 state A non-preferred supports tolusna
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.999519] scsi 6:0:0:0: alua: Attached
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.999718] sd 6:0:0:0: Attached scsi generic sg2 type 0
Aug  8 08:08:30 nldxensrv1 kernel: [8445622.999951] sd 6:0:0:0: [sdb] 419430400 512-byte logical blocks: (214 GB/200 GiB)
Aug  8 08:08:30 nldxensrv1 kernel: [8445623.001530] sd 6:0:0:0: [sdb] Write Protect is off
Aug  8 08:08:30 nldxensrv1 kernel: [8445623.001883] sd 6:0:0:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
Aug  8 08:08:30 nldxensrv1 kernel: [8445623.004285]  sdb: unknown partition table
Aug  8 08:08:30 nldxensrv1 kernel: [8445623.005834] sd 6:0:0:0: [sdb] Attached SCSI disk
Aug  8 08:08:31 nldxensrv1 multipathd: sdb: add path (uevent)
Aug  8 08:08:31 nldxensrv1 iscsid: Connection1:0 to [target: iqn.1986-03.com.ibm:2145.v5030-iscsi.node1, portal: 10.x.x.1,3260] through [iface: default] is operational now
Aug  8 08:08:36 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|dispatch:PBD.set_device_config D:053880831208|taskhelper] task PBD.set_device_config R:66a0e571e601 forwarded (trackid=ed519702f7981d8c7499b9debaa6e6dc)
Aug  8 08:08:36 nldxensrv1 multipathd: sdb: add path (operator)
Aug  8 08:08:36 nldxensrv1 multipathd: sdb: spurious uevent, path already in pathvec

Aug  8 08:08:36 nldxensrv1 kernel: [8445628.591421] device-mapper: multipath round-robin: version 1.0.0 loaded
Aug  8 08:08:36 nldxensrv1 kernel: [8445628.591839] bio: create slab <bio-1> at 1
Aug  8 08:08:36 nldxensrv1 multipathd: 36005076380818408f000000000000005: load table [0 419430400 multipath 1 queue_if_no_path 0 1 1 round-robin 0 1 1 8:16 1]
Aug  8 08:08:36 nldxensrv1 multipathd: 36005076380818408f000000000000005: event checker started
Aug  8 08:08:36 nldxensrv1 multipathd: sdb [8:16]: path added to devmap 36005076380818408f000000000000005
Aug  8 08:08:36 nldxensrv1 multipathd: Path event for 36005076380818408f000000000000005, request call of mpathcount
Aug  8 08:08:36 nldxensrv1 xapi: [ info|nldxensrv1|1369 UNIX /var/xapi/xapi|session.login_with_password D:add3ca7f1c93|xapi] Session.create trackid=1f01e1c0b95a20b20cd5c385bfd12fca pool=false uname=root originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|sm_exec D:cde9ab537ead|xapi] Session.destroy trackid=fe70c35b2b487bb5913ba0da08903aeb
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|PBD.plug R:dce1431e8a42|xapi] Not registering SM plugin lvmoiscsi (version 1.0 < 2.0)
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|dispatch:PBD.plug D:167c0d65e5bf|taskhelper] task PBD.plug R:a19da61c4243 forwarded (trackid=1057c7a63f0dea972427d42c717551c1)
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|PBD.plug R:a19da61c4243|storage_access] SR 37080e18-91ca-b61d-b54b-66c3f144f8f2 will be implemented by /services/SM/udev in VM OpaqueRef:c369177e-4b1e-3bfa-8f3c-e230b3a76dba
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|PBD.plug R:a19da61c4243|storage_impl] SR.attach dbg:OpaqueRef:a19da61c-4243-e0fd-7e0c-d4a04d7e2d91 sr:37080e18-91ca-b61d-b54b-66c3f144f8f2 device_config:[location:/dev/xapi/cd]
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|sm_exec D:c4a3aa0ec8bc|xapi] Session.create trackid=840427f8b2707481751d46121bc0787f pool=false uname= originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|sm_exec D:c4a3aa0ec8bc|xapi] Session.destroy trackid=840427f8b2707481751d46121bc0787f
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|PBD.plug R:a19da61c4243|xapi] Not registering SM plugin udev (version 1.0 < 2.0)
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|dispatch:SR.scan D:d3adb85566f7|taskhelper] task SR.scan R:0d93276da408 forwarded (trackid=9af7e8b7bd342f7e84263c3ad4f5f536)
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|dispatch:PBD.plug D:fc857b944f89|taskhelper] task PBD.plug R:1d771792dfca forwarded (trackid=42977c24eabf8b9cb100abd01a4b5097)
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|SR.scan R:0d93276da408|storage_impl] SR.scan dbg:OpaqueRef:0d93276d-a408-49f1-9443-cd6467bc4184 sr:37080e18-91ca-b61d-b54b-66c3f144f8f2
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|sm_exec D:59761e7e7012|xapi] Session.create trackid=63de105cfdbaced2f52ddfc5bb6c984f pool=false uname= originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|PBD.plug R:1d771792dfca|storage_access] SR 0f8696b2-8d9d-7f43-72ff-1935fdc93ce0 will be implemented by /services/SM/ext in VM OpaqueRef:c369177e-4b1e-3bfa-8f3c-e230b3a76dba
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|PBD.plug R:1d771792dfca|storage_impl] SR.attach dbg:OpaqueRef:1d771792-dfca-77b8-311f-530e3462b4dd sr:0f8696b2-8d9d-7f43-72ff-1935fdc93ce0 device_config:[device:/dev/disk/by-id/cciss-3600508b1001c13c5cba91283e139006f-part3]
Aug  8 08:08:37 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|sm_exec D:7e1f6e28ae2b|xapi] Session.create trackid=bffc0690ce03b0da641e9e90e8d489c4 pool=false uname= originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Aug  8 08:08:38 nldxensrv1 xapi: [ info|nldxensrv1|1410 UNIX /var/xapi/xapi|session.login_with_password D:be178a2c0c03|xapi] Session.create trackid=24ce7b28b1b65ba721d0c6ae122ee818 pool=false uname=root originator= is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Aug  8 08:08:38 nldxensrv1 kernel: [8445630.350767] bio: create slab <bio-2> at 2
Aug  8 08:08:38 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|sm_exec D:59761e7e7012|xapi] Session.destroy trackid=63de105cfdbaced2f52ddfc5bb6c984f
Aug  8 08:08:38 nldxensrv1 kernel: [8445630.653914] kjournald starting.  Commit interval 5 seconds
Aug  8 08:08:38 nldxensrv1 kernel: [8445630.655395] EXT3-fs (dm-1): using internal journal
Aug  8 08:08:38 nldxensrv1 kernel: [8445630.655400] EXT3-fs (dm-1): mounted filesystem with ordered data mode
Aug  8 08:08:38 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|sm_exec D:7e1f6e28ae2b|xapi] Session.destroy trackid=bffc0690ce03b0da641e9e90e8d489c4
Aug  8 08:08:38 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|PBD.plug R:1d771792dfca|xapi] Not registering SM plugin ext (version 1.0 < 2.0)
Aug  8 08:08:38 nldxensrv1 xapi: [ info|nldxensrv1|1320 INET :::80|dispatch:PBD.plug D:c65b0e893a7a|taskhelper] task PBD.plug R:ca67ae6212ef forwarded (trackid=26a5893a2c2d792464cd2fee01d23813)
Aug  8 08:08:38 nldxensrv1 xapi: [ info|nldxensrv1|1322 INET :::80|dispatch:SR.scan D:66ed72ef6f0a|taskhelper] task SR.scan R:b1d44b67ee0a forwarded (trackid=a9d205a41bd7c4c8b68c665c278cf36e)
 

Link to comment

9 answers to this question

Recommended Posts

  • 0

Hi, 

 

Thanks all, I have taken a look, and though I had configured the multipath.conf I didn't add the udev and didn't add the polling_interval to the default section of my multipath.conf then, now I have.

It's not yet working as expected, and I notice something:

In my multipath.conf I have:

 

defaults {
        user_friendly_names     no
        replace_wwid_whitespace yes
        dev_loss_tmo            30
        polling_interval        30
}

.....

 

        device {
                vendor                  "IBM"
                product                 "2145"
                path_grouping_policy    group_by_prio
                path_selector           "round-robin 0"
                prio_callout            "/sbin/mpath_prio_alua /dev/%n"
                prio                    alua
                path_checker            tur
                failback                immediate
                no_path_retry           5
                rr_weight               uniform
                rr_min_io               1000
                dev_loss_tmo            120
        }
 

Though when I run multipathd -k
show config

 

I see:

 

        device {
                vendor "IBM"
                product "2145"
                path_grouping_policy group_by_prio
                path_selector "round-robin 0"
                path_checker tur
                prio alua
                failback immediate
                rr_weight uniform
                no_path_retry 5
                rr_min_io 1000
                dev_loss_tmo 120
        }
and further 

(        device {
                vendor "IBM"
                product "^2145"
                path_grouping_policy group_by_prio
                getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
                path_selector "round-robin 0"
                path_checker tur
                features "1 queue_if_no_path"
                hardware_handler "0"
                prio alua
                failback immediate
                rr_weight uniform
                rr_min_io 1000
                rr_min_io_rq 1
        } Hopefully a different device )

 

When leaving the polling_interval out of the multipath.conf it defaults to polling_interval 5.

Can it be the way the storage is setup? 


 

 

 

 

Edited by interpharm1
Have added "then, now I have."
Link to comment
  • 0

The polling interval shouldn't make that much of a difference, IMO.  The error "xapi ... Not registering SM plugin lvmoiscsi" sseems to indicate maybe a setting or permission issue on the storage end, but I'm not familiar with that aprticular error nor have I used any fiber channel devices for many years now. Maybe your vendor contact can supply some better information?  I'd certainly try extending the iSCSI timeout as @Alan mentioned above.

 

-=Tobias

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...