Jump to content
Welcome to our new Citrix community!

New VPXs, can only ping NSIP, not SNIP, or VIP


Ken Stieers

Recommended Posts

Spun up VPXs connected to 3 subnets...

NSIP on one subnet (

Servers on another ("Servers") with a SNIP

VIPs on a third ("Security") with a SNIP

 

I can ping the NSIP and use the gui consistently, but can't ping either of the SNIPs.

 

I feel like its something simple that I missed... 

 

> sh ip

        Ipaddress        Traffic Domain  Type             Mode     Arp      Icmp     Vserver  State

        ---------        --------------  ----             ----     ---      ----     -------  ------

1)      172.16.98.15     0               NetScaler IP     Active   Enabled  Enabled  NA       Enabled

2)      172.16.3.3       0               SNIP             Active   Enabled  Enabled  NA       Enabled

3)      172.16.15.5      0               SNIP             Active   Enabled  Enabled  NA       Enabled

Done

 

> show interface

 

1)      Interface 0/1 (NetScaler Virtual Interface, VMXNET3) #0

        flags=0xc060 <ENABLED, UP, UP, 802.1q>

        MTU=1500, native vlan=98, MAC=00:50:56:ae:21:e8, uptime 291h47m58s

        LLDP Mode: NONE,                 LR Priority: 1024

 

        RX: Pkts(811037) Bytes(76235304) Errs(0) Drops(748533) Stalls(0)

        TX: Pkts(35464) Bytes(41766662) Errs(0) Drops(0) Stalls(0)

        NIC: InDisc(0) OutDisc(0) Fctls(0) Stalls(0) Hangs(0) Muted(0)

        Bandwidth thresholds are not set.

 

 

 

2)      Interface 1/1 (NetScaler Virtual Interface, VMXNET3) #1

        flags=0xc060 <ENABLED, UP, UP, 802.1q, tagall>

        MTU=1500, native vlan=3, MAC=00:50:56:ae:86:59, uptime 291h47m58s

        LLDP Mode: NONE,                 LR Priority: 1024

 

        RX: Pkts(11705451) Bytes(919595793) Errs(0) Drops(10287344) Stalls(0)

        TX: Pkts(433174) Bytes(18193308) Errs(0) Drops(0) Stalls(0)

        NIC: InDisc(0) OutDisc(0) Fctls(0) Stalls(0) Hangs(0) Muted(0)

        Bandwidth thresholds are not set.

 

 

 

3)      Interface 1/2 (NetScaler Virtual Interface, VMXNET3) #2

        flags=0xc060 <ENABLED, UP, UP, 802.1q, tagall>

        MTU=1500, native vlan=15, MAC=00:50:56:ae:36:a0, uptime 291h47m58s

        LLDP Mode: NONE,                 LR Priority: 1024

 

        RX: Pkts(1447191) Bytes(122016294) Errs(0) Drops(1447185) Stalls(0)

        TX: Pkts(56794) Bytes(2385348) Errs(0) Drops(0) Stalls(0)

        NIC: InDisc(0) OutDisc(0) Fctls(0) Stalls(0) Hangs(0) Muted(0)

        Bandwidth thresholds are not set.

 

 

 

4)      Interface LO/1 (Netscaler Loopback interface) #3

        flags=0x20008020 <ENABLED, UP, UP>

        MTU=1500, native vlan=1, MAC=00:50:56:ae:21:e8, uptime 291h48m09s

        LLDP Mode: NONE,                 LR Priority: 1024

 

        RX: Pkts(91639470) Bytes(12843289063) Errs(0) Drops(0) Stalls(0)

        TX: Pkts(180644125) Bytes(16726358883) Errs(0) Drops(0) Stalls(0)

        Bandwidth thresholds are not set.

 

 

Done

> show vlan

 

1)      VLAN ID: 1

        Link-local IPv6 addr: fe80::250:56ff:feae:21e8/64

        Interfaces : LO/1

 

2)      VLAN ID: 3      VLAN Alias Name: Servers

        Interfaces : 1/1

        IPs :

             172.16.3.3         Mask: 255.255.248.0

 

3)      VLAN ID: 15     VLAN Alias Name: Security

        Interfaces : 1/2

        IPs :

             172.16.15.5        Mask: 255.255.255.0

 

4)      VLAN ID: 98     VLAN Alias Name: Management

        Interfaces : 0/1

        IPs :

             172.16.98.15       Mask: 255.255.255.0

Done

> show route

        Network          Netmask          Gateway/OwnedIP  State   Traffic Domain  Type

        -------          -------          ---------------  -----   --------------  ----

1)      0.0.0.0          0.0.0.0          172.16.15.1      UP      0              STATIC

2)      127.0.0.0        255.0.0.0        127.0.0.1        UP      0              PERMANENT

3)      172.16.0.0       255.255.248.0    172.16.3.3       UP      0              DIRECT

4)      172.16.15.0      255.255.255.0    172.16.15.5      UP      0              DIRECT

5)      172.16.98.0      255.255.255.0    172.16.98.15     UP      0              DIRECT

Done

 

 

 

Here's the relevant config:

#NS12.1 Build 55.18
# Last modified by `save config`, Wed Jan 29 12:21:22 2020
set ns config -IPAddress 172.16.98.15 -netmask 255.255.255.0
set ns config -nsvlan 98 -ifnum 0/1 -tagged NO
set lacp -sysPriority 32768 -mac 00:50:56:ae:21:e8
set ns hostName nsvpx1
set interface 0/1 -autoneg DISABLED -throughput 0 -bandwidthHigh 0 -bandwidthNormal 0 -intftype VMXNET3 -ifnum 0/1
set interface 1/1 -autoneg DISABLED -throughput 0 -bandwidthHigh 0 -bandwidthNormal 0 -intftype VMXNET3 -ifnum 1/1
set interface 1/2 -autoneg DISABLED -throughput 0 -bandwidthHigh 0 -bandwidthNormal 0 -intftype VMXNET3 -ifnum 1/2
set interface LO/1 -haMonitor OFF -haHeartbeat OFF -throughput 0 -bandwidthHigh 0 -bandwidthNormal 0 -intftype Loopback -ifnum LO/1
add vlan 3 -aliasName Servers
add vlan 15 -aliasName Security
add vlan 98 -aliasName Management
add ns ip6 fe80::250:56ff:feae:21e8/64 -scope link-local -type NSIP -vlan 1 -vServer DISABLED -mgmtAccess ENABLED -dynamicRouting ENABLED
add ns ip 172.16.98.15 255.255.255.0 -type NSIP -vServer DISABLED -gui SECUREONLY -mgmtAccess ENABLED -dynamicRouting ENABLED
add ns ip 172.16.3.3 255.255.248.0 -vServer DISABLED
add ns ip 172.16.15.5 255.255.255.0 -vServer DISABLED
bind vlan 3 -ifnum 1/1 -tagged
bind vlan 3 -IPAddress 172.16.3.3 255.255.248.0
bind vlan 15 -ifnum 1/2 -tagged
bind vlan 15 -IPAddress 172.16.15.5 255.255.255.0
set nd6RAvariables -vlan 1
set ns rpcNode 172.16.98.15 -password <encpasswor> -encrypted -encryptmethod ENCMTHD_3 -srcIP 172.16.98.15
bind rewrite policylabel ns_cvpn_v2_url_label ns_cvpn_v2_bypass_url_pol 20000 NEXT
add dns nameServer 172.16.4.212
add dns nameServer 172.16.4.213
add dns nameServer 10.16.3.212
add dns nameServer 10.16.3.213
add route 0.0.0.0 0.0.0.0 172.16.15.1
add ns pbr NSIP-DNS DENY -srcIP = 172.16.98.15 -destPort = 53 -nextHop 172.16.98.1 -protocol UDP -priority 5 -kernelstate SFAPPLIED61
add ns pbr NSIP ALLOW -srcIP = 172.16.98.15 -nextHop 172.16.98.1 -priority 10 -kernelstate SFAPPLIED61
 

Link to comment
Share on other sites

Not sure exactly; but a few things you can look at.

 

You can test connectivity from the ADC to external resources by pinging from the ADC to other destinations. By default, ping will source from the NSIP, use ping <dest ip> -S <source ip> to force a specifc SNIP to be used instead.  Trace route or network trace might help; but I think its simpler l than that.

 

Your show interface is showing DROPS...  I see you've got some PBR's in use; but do you have any ACLs that might be blocking any unexpected traffic?

Do you see any reasons for denied traffic in syslog (nslog is referenced at bottom of article).

 

Are you missing any static routes for the 172.16.3.3 and 172.16.15.5 networks to find their actual gateway?  You have the snips defined, but you don't have any static routes saying where either network's router/default gateway is for external egress?  (But I also saw you have dynamic routing enabled on the NSIP...so there might be a disconnect in some of your settings.)

 

Or is your switch  (or virtual network) not recognizing the vlans you have your interfaces participating on.  It looks like you have vlan 3 and 15 bound as tagged vlans; which I'm not sure if you need here or not.   You might not need "tagged vlans" if these are port based....but it depends on your wider network considerations.

 

 

Verify which modes you have enabled: ensure USNIP mode is enabled.  (Might need to see what is on/off for L3 and L2 mode as well in case there is something unexpected in either value.)

 

You should check nslog if it is complaining about a network issue that's not obvious (such as a switch or hypervisor issue with your vlan config):

shell

cd /var/nslog

nsconmsg -K newnslog -d event

nsconmsg -K newnslog -d consmsg

 

# look for any unexpected events like a switch muting a port or other issue that might indicate other networking problems.

Link to comment
Share on other sites

Easiest things first: Is ICMP enabled? (show ns ip <ip-address> will give you an answer)

 

Next, I's do a network trace. Do you see arp resolution for <ip-address>? Do you see ICMP packets coming in? Do you see ICMP replys going out?

 

If you neither see ARP nor ICMP you can't reach the IP at all due to firewall or routing issues. If you see ICMP packets going in, but none out, you face a routing issue on Citrix ADC. If you see them going out, but don't get an ICMP reply on your workstation, it's an issue somewhere else.

 

 

Link to comment
Share on other sites

On 2/21/2020 at 3:11 PM, Rhonda Rowland1709152125 said:

Not sure exactly; but a few things you can look at.

 

You can test connectivity from the ADC to external resources by pinging from the ADC to other destinations. By default, ping will source from the NSIP, use ping <dest ip> -S <source ip> to force a specifc SNIP to be used instead.  Trace route or network trace might help; but I think its simpler l than that.

 

I can ping from the NSIP, but not from either SNIP. 

 

On 2/21/2020 at 3:11 PM, Rhonda Rowland said:

Your show interface is showing DROPS...  I see you've got some PBR's in use; but do you have any ACLs that might be blocking any unexpected traffic?

Do you see any reasons for denied traffic in syslog (nslog is referenced at bottom of article).

Nothing in the logs that helps, and no ACLs yet... 

 

On 2/21/2020 at 3:11 PM, Rhonda Rowland said:

Are you missing any static routes for the 172.16.3.3 and 172.16.15.5 networks to find their actual gateway?  You have the snips defined, but you don't have any static routes saying where either network's router/default gateway is for external egress?  (But I also saw you have dynamic routing enabled on the NSIP...so there might be a disconnect in some of your settings.)

I'm trying to ping from adjacent IPs, (172.16.15.26)... so I shouldn't need routes... 

 

On 2/21/2020 at 3:11 PM, Rhonda Rowland said:

Or is your switch  (or virtual network) not recognizing the vlans you have your interfaces participating on.  It looks like you have vlan 3 and 15 bound as tagged vlans; which I'm not sure if you need here or not.   You might not need "tagged vlans" if these are port based....but it depends on your wider network considerations.

 

 

I've tried Tagged and not Tagged... but his may be an issue in the VMWare swtich... 

When I spun up a fresh one, added IPs, it didn't break until I applied the vlans... 

 

On 2/21/2020 at 3:11 PM, Rhonda Rowland said:

Verify which modes you have enabled: ensure USNIP mode is enabled.  (Might need to see what is on/off for L3 and L2 mode as well in case there is something unexpected in either value.)

Confirmed that this is enabled. 

 

On 2/21/2020 at 3:11 PM, Rhonda Rowland said:

You should check nslog if it is complaining about a network issue that's not obvious (such as a switch or hypervisor issue with your vlan config):

shell

cd /var/nslog

nsconmsg -K newnslog -d event

nsconmsg -K newnslog -d consmsg

 

# look for any unexpected events like a switch muting a port or other issue that might indicate other networking problems.

Nothing diagnostic... 

 

 

 

 

Thanks so much for taking the time to answer!!

 

 

Ken 

 

Link to comment
Share on other sites

So that looks like an inconsistency between the vpx networking, the vswitch networking (hypervisor), and/or the physical networking will cause some issues.  If the vm is participating in vlans that the physical (or virtual switch doesn't recognize...you will have some problems.)

 

I don't know if this is related to your issue or not, but since you mentioned you were on ESX specifically:  https://docs.citrix.com/en-us/netscaler/12/deploying-vpx/install-vpx-on-esx.html .  There is a note about for vlan tagging to work, the ESX port group vlan id needs to be set to 4095 (to tag all vlans)...so the tagging can be done at the vm level.

But as you found out, you have some inconsistency between the hypervisor network and the vm network (or physical to hypervisor).  (I saw one past thread where someone had to reboot as well...but not sure why, unless they changed the networking at the hypervisor level and needed the vm to see the change...save config first though.)

 

Whether you need tagged vlans or port based vlans or whether there is something else going on...it depends. Johannes gave you some good things to start with as well.  You might want to go back to a simpler network scenario and add only one or two changes at a time until you can pinpoint the problem.

 

 

 

 

 

 

Link to comment
Share on other sites

Hey Rhonda, Johannes, 

 

I ended up deploying fresh VMs, and building the config checkbox by checkbox, and it looks like it boiled down to VLAN tagging, I don't need it anywhere,  based on the configs that were broken vs. what's currently working. 

 

Thanks for your answers!

 

Ken 

 

 

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...