Citrix Hypervisor - Help with New Pool iSCSI Direct Attach Storage Setup

[Jonathan Ransom-Flint]

Hopefully this is something simple that I've missed.  I have a brand new Citrix Hypervisor 8.2 CU1 LTSR pool of Dell PowerEdge R450 servers, and a Dell ME5024 storage array configured to use iSCSI with direct attach cables to each R450 server.  I have each iSCSI port on a different subnet 192.168.0.0/24, 192.168.1.0/24, 192.168.2.0/24, 192.168.3.0/24.  Each direct attach port on the corresponding R450 server has an IP on the same corresponding iSCSI port subnet (assigned IP to the R450 connected NIC by clicking the network tab of the server, and then the configure button on the IP address configure tab, then Add new IP address).  Side note - Since it's all direct attach, I do not have a gateway entered for either R450s or the ME5024 on the storage NIC.  In a closed system this should work right?  When I go to add an SR -> iSCSI -> default options, and then click scan target, it fails.  (I am not using CHAP).  The ME5024 configuration status is waiting for initiators, so I assume I needed to attach the R450s before proceeding.  I also can't ping the ME5024 iSCSI port from the hypervisor console, but I have a feeling it's trying to use the management NIC to do so.  Any ideas?  MTU is left at the default, no jumbo frames enabled on the ME5024.  Happy to clarify any info as needed.  Thanks!

[Tobias Kreidl]

See if this is of any help. Unfortunately, my article on the MD3600 with an iSCSI talking directly to a VM seems to have disappeared, sadly.

https://discussions.citrix.com/topic/394780-how-to-use-iscsi-target-directly-in-xenserver/

That said, I have always gone through a network switch! If you try to hook up the cables directly, and assuming that's supported, you'll have to get a cable that flips the receive/transmit wires on one end (a so-called crossover cable).

 

-=Tobias

[Jonathan Ransom-Flint]

Thanks for the reply, Tobias.  Hmmmm I'll have to read more into that thread tomorrow.  This seems pretty wild, am I missing something that it's not able to be done directly through XenCenter?  Any YouTube video I saw with people attaching iSCSI storage looked like a piece of cake, but it never showed the actual network setup on the Hypervisor Host side.  When using XenCenter to add an iSCSI SR, does it try each assigned IP on the host, or does it only try from the management NIC?  I'm using a Broadcom BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller with a Copper SFP+ Direct Attach Cable.  The NIC, ME5024, and R450 are all on the HCL, so I'll be bummed out if I have to use Hyper-V instead if I can't get Citrix Hypervisor to work.

[Tobias Kreidl]

OK, I misunderstood hat you are trying to do initially. You just want to set up an SR (or several) from your ME device to XenServer in a standard way. I somehow thought you wanted to connect a VM directly to storage, bypassing the XenServer internals by and large.

 

Once you have the storage configured on the ME storage device, you'll have to make sure it can be seen on the host through the iSCSI discovery process plus there needs to be permission to access the storage granted from the storage unit itself. Some of that should be possible with Dell's storage management utility (MDSM?). The basic configuration should be possible via XenCenter once you have set up the storage devices, made sure they are exported to your hosts, and you've done the network discovery process.

 

That sat, there is AFAIK no way to directly talk to such a storage device without a network hub in between. I think people have tried crossover cables without success; I have never personally attempted that since I had to work though our network operations group to get ports and addresses worked out ahead of time, anyway.

 

-=Tobias

[Jonathan Ransom-Flint]

Tobias - I found this today:

 

https://www.dell.com/support/manuals/en-gt/powervault-me5024/me5_series_sm/supported-fibre-channel-and-iscsi-direct-attach-configuration-operating-systems?guid=guid-f73aa4b9-4133-4926-9595-2cd64a155072&lang=en-us

 

Everything points to this being a valid config, do you have anyone over there that's worked with an iSCSI direct attach SR before?

[Tobias Kreidl]

Jonathan,

When you have your storage set up, do the iSCSI discovery and then see if you can ping the ports.  Again, I'm pretty sure you can only make this work if you have a network switch between the  storage device and the hosts. Since I left NAU and retired near, all the XenServers have since been disbanded, sadly.

 

-=Tobias

[Jonathan Ransom-Flint]

Tobias - The initial part of this was totally bonehead.  Even though I have two of the same NICs in the R450s, the port numbers were swapped between the two of them.  I rewired the DAC connection between the R450s and the ME5 storage, and now the first host can see the iSCSI target in the XenCenter GUI!  The problem however, is the second R450 still isn't having luck BUT it can ping the iSCSI port.  This is probably a super basic question, but when adding the SR via XenCenter, it's adding it at the pool level.  Is there any way it's only checking if the pool master can see the iSCSI target, and not using the second R450 to probe? This is my first time using shared iSCSI, so I'm sure it's something super basic I'm missing.

 

EDIT:  I found an old post of yours

 

https://discussions.citrix.com/topic/413369-xenserver-pool-iscsi-different-subnets/

 

I have the same issue which section should I be looking at, is this a multipathing enable fix?

[Tobias Kreidl]

Hi, Jonathan:

Without a network switch to go through, I do not see how the second host could know anything about the first host. It sounds to me that you still have just an Ethernet cable between the host and the storage device, correct? If that is the case, you cannot AFAIK create a pool-wide storage that is seen by all pool members without going through a network switch.

 

For specifics on multipathing with iSCSI storage, please read this in detail: https://docs.citrix.com/en-us/citrix-hypervisor/storage/multipath.html

 

HTH,

-=Tobias

[Jonathan Ransom-Flint]

Hi Tobias, so the direct attach cables are only going to the ME5024 storage controller (red and blue in diagram).  The two hosts both have 10Gb NICs that connect to a switch (green in diagram), so they can talk to each other no problem.  Are you saying that all hosts need to be able to connect to all storage ports?  I've attached a very crude diagram of the setup.  Color coded subnets.  So are you saying Vhost1 & 2 would both need to be on red AND blue?  I thought storage subnets were supposed to be isolated to one port only?

[Tobias Kreidl]

Each host should first connect to the switch (green) , them the switch should connect to the storage device. There is no direct cable between a host and a storage device.

[Jonathan Ransom-Flint]

Tobias, are you saying to change to this setup?

 

[Tobias Kreidl]

Jonathan,

Yes! And see the article I linked to above for some help with that configuration.

 

-=Tobias