Jump to content
Welcome to our new Citrix community!

Netscaler CPX - How to in host mode bind to a physical ethernet port and or use a physical IP as a VIP?


Recommended Posts

I would like to launch a docker container cpx and use a physical ip as a VIP. That is my goal, follows are my attempts to reach it:

The problem is that it creates this `netscaler` name space and then binds to virtual interfaces, even when I do `NS_NETMODE="HOST". 

Now I may be using some of these environment variables wrong (although the results tend to be similar with or without). So far as I can tell, your documentation doesn't really cover them. I had to pull them from /var/netscaler/bins/docker_startup.sh (which covers them, but I'm not sure I understand their utility still). This is my run command:
 

docker run -dt --privileged=true --net=host --cap-add=NET_ADMIN -e NS_IP="172.20.211.56" -e NS_NETMODE="HOST" --name cpx --ulimit core=-1 -e NS_SSH_PORT="2222" -e EULA=yes -e NS_HTTP_PORT=80 store/citrix/netscalercpx:12.0-56.20


I exec in:

docker exec -ti cpx /bin/bash
and run a netstat:

root@rml4d7y-docker-netscalar:/# netstat -plant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp6       0      0 :::2376                 :::*                    LISTEN      -
tcp6       0      0 :::22                   :::*                    LISTEN      -
tcp6       0      0 172.20.211.59:2376      172.20.211.62:40612     ESTABLISHED -
tcp6       0      0 172.20.211.59:2376      172.20.211.62:40614     ESTABLISHED -

Nothing of note is listening, and on top of that, the script has picked an ip that is not on the server (which is a comment in the script, but I still don't understand why). I think this is the name space mirroring to the veth with 172.20.211.62. 

 

I exec into the netscaler namespace using ip netns exec netscaler bash and run a netstat:

root@rml4d7y-docker-netscalar:/# netstat -plant
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:3010            0.0.0.0:*               LISTEN      206/nsconfigd
tcp        0      0 192.0.0.2:3335          0.0.0.0:*               LISTEN      184/snmpd
tcp        0      0 192.0.0.2:5555          0.0.0.0:*               LISTEN      165/nsaggregatord
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      102/sshd
tcp        0      0 192.0.0.2:36724         192.0.0.1:8777          ESTABLISHED 184/snmpd
tcp        0      0 192.0.0.2:5555          192.0.0.2:41586         ESTABLISHED 165/nsaggregatord
tcp        0      0 192.0.0.2:36728         192.0.0.1:8777          ESTABLISHED 206/nsconfigd
tcp        0      0 192.0.0.2:36722         192.0.0.1:8777          ESTABLISHED 206/nsconfigd
tcp        0      0 192.0.0.2:41584         192.0.0.2:5555          ESTABLISHED 184/snmpd
tcp        0      0 192.0.0.2:5555          192.0.0.2:41584         ESTABLISHED 165/nsaggregatord
tcp        0      0 192.0.0.2:41586         192.0.0.2:5555          ESTABLISHED 184/snmpd
tcp        0      0 192.0.0.2:3335          192.0.0.1:10103         ESTABLISHED 184/snmpd
tcp6       0      0 :::80                   :::*                    LISTEN      108/httpd
tcp6       0      0 :::22                   :::*                    LISTEN      102/sshd

Here's all the things I need to have listening, but I can't actually get to any of them. I can add another veth and manually mirror an eth to it, but then what do I make the VIP? Would it be a 192. or do I pass a physical interface into it and then do something like: add ns ip bla bla bla?

Link to comment
Share on other sites

Hi,

A few points to highlight -

  • In host-mode of networking, CPX creates its own namespace and veth pair for communication between 'default' and 'netscaler' namespaces.
  • Dedicated interface mode: CPX_NW_DEV environment variable can be set at the time of container creation which will configure the interface(s) provided in this variable to be owned by CPX and CPX will receive the traffic directly arriving on the interface(s). The interface(s) can be either physical interface(s) present on host or it can be virtual interface(s) or a mix of two. An example -
    • docker run -dt --privileged=true --net=host -e NS_NETMODE="HOST" -e EULA=yes -e CPX_NW_DEV='eth1 eth2' -e CPX_CORES=5 -e PLATFORM=CP1000 --name cpx_host cpx:12.0-53.x
  • NS_IP and NS_GATEWAY has to be configured if you are using dedicated interface mode. NS_IP becomes the NetScaler IP of CPX and NS_GATEWAY is configured as default route in CPX, so they should be reachable on one of the dedicated interface for communication from outside the host.
  • 'netstat' command will not display any VIP being listened on as it is not configured in Linux kernel. VIP gets configured in CPX's packet engine and can be viewed using nscli (in this case using classic NetScaler CLI commands with cli_script.sh) or nitro APIs. An example -
    • cli_script.sh 'show connectiontable -listen'

For you case, the solution can be like this -

  • While creating CPX set CPX_NW_DEV variable with appropriate physical interface(s) on which your traffic for VIP would be coming. Also NS_IP and NS_GATEWAY has to be set in the same subnet as the network interface for reachability. 
  • If the host has only one physical interface, then a mac vlan interface can be created and should be provided in CPX_NW_DEV variable.
  • VIP can be configured on CPX using nscli/nitro APIs. This VIP will be reachable over the dedicated interface to CPX's packet engine. Here is the syntax -
    • cli_script.sh 'add lb vserver <vserver-name> <protocol> <VIP> <port>'

Note: https://docs.citrix.com/en-us/citrix-adc-cpx/12/deploy-using-docker-image-file.html page talks about the usage of CPX_NW_DEV variable.

 

Thanks & Regards,

Akshay Budhauliya

Link to comment
Share on other sites

  • 1 year later...
Quote

'netstat' command will not display any VIP being listened on as it is not configured in Linux kernel. VIP gets configured in CPX's packet engine and can be viewed using nscli (in this case using classic NetScaler CLI commands with cli_script.sh) or nitro APIs. An example -

cli_script.sh 'show connectiontable -listen'

 

When configuring RHI on a Citrix ADC VPX or MPX, RHI enabled VIP are advertised as kernel routes to the (old) ZebOS routing stack. Which VPX / MPX process handles this route advertisement outside of ADC packet engines ?

 

image.thumb.png.73e20a8e6dce18ee6ddc1c97da2f087c.png

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...