problems with ADC after upgrading to 12.0

[Stefan Sallegger1709159665]

hey

 

i have updated a netscaler from 11.1 to 12.0 62.10

the netscaler is working fine, I enabled the default ssl profiles and did some check up.

but my MAS set the two netscalers to "out of service" and if I rediscover it cancels with error:

 

    Failed to get Citrix ADC version X.X.X.X
    Error: Error in config/nshostname
    Failed to get Citrix ADC version X.X.X.X
    Error: Error in config/nshostname

 

Heres the MPS Log, It seems to me that there is an error with the ssl config:

 

bash-3.2# tail -f mps_inventory.log | grep 172.24.58
{ "errorcode": 0, "message": "Done", "is_user_part_of_default_group": false, "skip_auth_scope": true, "is_user_authorized_all_instances": true, "message_id": "", "resrc_driven": true, "login_session_id": "##2B9040F78B2C84F098D4CE7F7B283CF214345730A90BC83D608E660EA78A", "username": "ssallegger", "tenant_name": "Owner", "tenant_id": "3f43bed4-3903-49ed-a832-398badaf16a4", "mps_ip_address": "mas.mgnt.at", "client_ip_address": "172.24.208.48", "client_protocol": "https", "client_port": 54544, "mpsSessionId": "", "source": "service", "target": "INVENTORY", "version": "v1", "messageType": "inventory", "client_type": "GUI", "resourceType": "inventory", "orignal_resourceType": "inventory", "resourceName": "", "operation": "add", "asynchronous": false, "instance_id": "", "params": { "pageno": 0, "pagesize": 0, "detailview": true, "compression": false, "count": false, "total_count": 0, "action": "", "type": "", "onerror": "EXIT", "is_db_driven": false, "order_by": "", "asc": false, "duration": "", "duration_summary": 0, "report_start_time": "0", "report_end_time": "0" }, "filter_props": [ { } ], "additionalInfo": { "Referer": "https:\/\/mas.mgnt.at\/admin_ui\/mas\/ent\/html\/main.html", "cert_present": "false", "rand_key": "747e4a81faeaacc", "request_source": "NITRO_WEB_APPLICATION", "sessionId": "##2B9040F78B2C84F098D4CE7F7B283CF214345730A90BC83D608E660EA78A" }, "inventory": [ { "devices": [ "X.X.X.X" ] } ] }
Saturday, 14 Dec 19 11:51:22.152 +0100 [Debug] [Main] Rediscovery started for X.X.X.X
Saturday, 14 Dec 19 11:51:22.153 +0100 [Debug] [Main] Rediscovery started for X.X.X.X
{ "errorcode": 0, "message": "Done", "is_user_part_of_default_group": false, "skip_auth_scope": true, "is_user_authorized_all_instances": true, "message_id": "", "resrc_driven": true, "login_session_id": "##2B9040F78B2C84F098D4CE7F7B283CF214345730A90BC83D608E660EA78A", "username": "ssallegger", "tenant_name": "Owner", "tenant_id": "3f43bed4-3903-49ed-a832-398badaf16a4", "mps_ip_address": "mas.mgnt.at", "client_ip_address": "172.24.208.48", "client_protocol": "https", "client_port": 54544, "mpsSessionId": "", "source": "service", "target": "INVENTORY", "version": "v1", "messageType": "inventory", "client_type": "GUI", "resourceType": "inventory", "orignal_resourceType": "inventory", "resourceName": "", "operation": "add", "asynchronous": false, "instance_id": "", "params": { "pageno": 0, "pagesize": 0, "detailview": true, "compression": false, "count": false, "total_count": 0, "action": "", "type": "", "onerror": "EXIT", "is_db_driven": false, "order_by": "", "asc": false, "duration": "", "duration_summary": 0, "report_start_time": "0", "report_end_time": "0" }, "filter_props": [ { } ], "additionalInfo": { "Referer": "https:\/\/mas.mgnt.at\/admin_ui\/mas\/ent\/html\/main.html", "cert_present": "false", "rand_key": "747e4a81faeaacc", "request_source": "NITRO_WEB_APPLICATION", "rest_params_key_present": "true", "sessionId": "##2B9040F78B2C84F098D4CE7F7B283CF214345730A90BC83D608E660EA78A" }, "inventory": [ { "device_ipaddress": "", "device_type": "", "act_id": "9babf674-f235-4adc-9967-228030f49544", "agent_id": "", "inventory_status": "Rediscovery started for X.X.X.X", "devices": [ "X.X.X.X" ] } ] }
Saturday, 14 Dec 19 11:51:22.153 +0100 [Debug] [Inventory[#8]] CCInventoryNSHandler: start for X.X.X.X
Saturday, 14 Dec 19 11:51:22.153 +0100 [Debug] [Inventory[#9]] CCInventoryNSHandler: start for X.X.X.X
Saturday, 14 Dec 19 11:51:22.189 +0100 [Debug] [Inventory[#8]] DeviceSessionManager: sessionid not found in device_session table for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.192 +0100 [Debug] [Inventory[#9]] DeviceSessionManager: sessionid not found in device_session table for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.201 +0100 [Debug] [Inventory[#8]] NITRO Message Body: { "login": { "username": "nsroot", "password": "***********" } }, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.201 +0100 [Debug] [Inventory[#8]] sendMessage: HTTP Request Protocol: https, port: 443, ip_address: X.X.X.X, ContentType: application/x-www-form-urlencoded, Method: POST, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.203 +0100 [Error] [Inventory[#8]] https://X.X.X.X:443/nitro/v1/config, Reason: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
Saturday, 14 Dec 19 11:51:22.203 +0100 [Error] [Inventory[#8]] executeNITROCommand: Send HTTPMessage Failed:Exception: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error for X.X.X.X
Saturday, 14 Dec 19 11:51:22.204 +0100 [Debug] [Inventory[#8]] DeviceSessionManager getHTTPSession: now throwing again exception with errorcode:10001, for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.204 +0100 [Error] [Inventory[#8]] Error appending session to URL: https://X.X.X.X/nitro/v1/config/nshostname?format=json:Exception: DEVICE_NOT_REACHABLE
Saturday, 14 Dec 19 11:51:22.204 +0100 [Debug] [Inventory[#8]] Skipping NITRO call for https://X.X.X.X:443/nitro/v1/config/nshostname?format=json
Saturday, 14 Dec 19 11:51:22.204 +0100 [Error] [Inventory[#8]] https://X.X.X.X:443/nitro/v1/config/nshostname?format=json, Reason: Exception: /nitro/v1/config/nshostname?format=json
Saturday, 14 Dec 19 11:51:22.204 +0100 [Error] [Inventory[#8]] Failed to get hostname for X.X.X.X returning from inventory
Saturday, 14 Dec 19 11:51:22.204 +0100 [Debug] [Inventory[#9]] NITRO Message Body: { "login": { "username": "nsroot", "password": "***********" } }, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.205 +0100 [Debug] [Inventory[#9]] sendMessage: HTTP Request Protocol: https, port: 443, ip_address: X.X.X.X, ContentType: application/x-www-form-urlencoded, Method: POST, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.207 +0100 [Error] [Inventory[#9]] https://X.X.X.X:443/nitro/v1/config, Reason: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
Saturday, 14 Dec 19 11:51:22.207 +0100 [Error] [Inventory[#9]] executeNITROCommand: Send HTTPMessage Failed:Exception: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error for X.X.X.X
Saturday, 14 Dec 19 11:51:22.207 +0100 [Debug] [Inventory[#9]] DeviceSessionManager getHTTPSession: now throwing again exception with errorcode:10001, for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.208 +0100 [Error] [Inventory[#9]] Error appending session to URL: https://X.X.X.X/nitro/v1/config/nshostname?format=json:Exception: DEVICE_NOT_REACHABLE
Saturday, 14 Dec 19 11:51:22.208 +0100 [Debug] [Inventory[#9]] Skipping NITRO call for https://X.X.X.X:443/nitro/v1/config/nshostname?format=json
Saturday, 14 Dec 19 11:51:22.208 +0100 [Error] [Inventory[#9]] https://X.X.X.X:443/nitro/v1/config/nshostname?format=json, Reason: Exception: /nitro/v1/config/nshostname?format=json
Saturday, 14 Dec 19 11:51:22.208 +0100 [Error] [Inventory[#9]] Failed to get hostname for X.X.X.X returning from inventory
Saturday, 14 Dec 19 11:51:22.779 +0100 [Debug] [Stat[#8]] DeviceSessionManager: sessionid not found in device_session table for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.782 +0100 [Debug] [Stat[#5]] DeviceSessionManager: sessionid not found in device_session table for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.795 +0100 [Debug] [Stat[#5]] NITRO Message Body: { "login": { "username": "nsroot", "password": "***********" } }, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.796 +0100 [Debug] [Stat[#5]] sendMessage: HTTP Request Protocol: https, port: 443, ip_address: X.X.X.X, ContentType: application/x-www-form-urlencoded, Method: POST, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.798 +0100 [Error] [Stat[#5]] https://X.X.X.X:443/nitro/v1/config, Reason: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
Saturday, 14 Dec 19 11:51:22.798 +0100 [Error] [Stat[#5]] executeNITROCommand: Send HTTPMessage Failed:Exception: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error for X.X.X.X
Saturday, 14 Dec 19 11:51:22.798 +0100 [Debug] [Stat[#5]] DeviceSessionManager getHTTPSession: now throwing again exception with errorcode:10001, for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.798 +0100 [Error] [Stat[#5]] Error appending session to URL: https://X.X.X.X/nitro/v1/stat/ns?format=json:Exception: DEVICE_NOT_REACHABLE
Saturday, 14 Dec 19 11:51:22.798 +0100 [Debug] [Stat[#5]] Skipping NITRO call for https://X.X.X.X:443/nitro/v1/stat/ns?format=json
Saturday, 14 Dec 19 11:51:22.798 +0100 [Error] [Stat[#5]] https://X.X.X.X:443/nitro/v1/stat/ns?format=json, Reason: Exception: /nitro/v1/stat/ns?format=json
Saturday, 14 Dec 19 11:51:22.832 +0100 [Debug] [Stat[#5]] execute Command Output with error dump : PING X.X.X.X (X.X.X.X): 56 data bytes
--- X.X.X.X ping statistics ---
Saturday, 14 Dec 19 11:51:22.832 +0100 [Debug] [Stat[#5]] Resetting stats for X.X.X.X, icmp_ping_received=1
Saturday, 14 Dec 19 11:51:22.843 +0100 [Debug] [Stat[#8]] NITRO Message Body: { "login": { "username": "nsroot", "password": "***********" } }, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.844 +0100 [Debug] [Stat[#8]] sendMessage: HTTP Request Protocol: https, port: 443, ip_address: X.X.X.X, ContentType: application/x-www-form-urlencoded, Method: POST, URL: https://X.X.X.X:443/nitro/v1/config
Saturday, 14 Dec 19 11:51:22.846 +0100 [Error] [Stat[#8]] https://X.X.X.X:443/nitro/v1/config, Reason: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
Saturday, 14 Dec 19 11:51:22.846 +0100 [Error] [Stat[#8]] executeNITROCommand: Send HTTPMessage Failed:Exception: SSL Exception: error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error for X.X.X.X
Saturday, 14 Dec 19 11:51:22.846 +0100 [Debug] [Stat[#8]] DeviceSessionManager getHTTPSession: now throwing again exception with errorcode:10001, for IP: X.X.X.X, Partition:
Saturday, 14 Dec 19 11:51:22.846 +0100 [Error] [Stat[#8]] Error appending session to URL: https://X.X.X.X/nitro/v1/stat/ns?format=json:Exception: DEVICE_NOT_REACHABLE
Saturday, 14 Dec 19 11:51:22.846 +0100 [Debug] [Stat[#8]] Skipping NITRO call for https://X.X.X.X:443/nitro/v1/stat/ns?format=json
Saturday, 14 Dec 19 11:51:22.846 +0100 [Error] [Stat[#8]] https://X.X.X.X:443/nitro/v1/stat/ns?format=json, Reason: Exception: /nitro/v1/stat/ns?format=json
Saturday, 14 Dec 19 11:51:22.868 +0100 [Debug] [Stat[#5]] OOS Reason: Failed to get statistics of Citrix ADC Instance for IP: X.X.X.X
Saturday, 14 Dec 19 11:51:22.872 +0100 [Debug] [Stat[#5]] Done StatNSHandler for X.X.X.X
Saturday, 14 Dec 19 11:51:22.937 +0100 [Debug] [Stat[#8]] execute Command Output with error dump : PING X.X.X.X (X.X.X.X): 56 data bytes
--- X.X.X.X ping statistics ---
Saturday, 14 Dec 19 11:51:22.937 +0100 [Debug] [Stat[#8]] Resetting stats for X.X.X.X, icmp_ping_received=1
Saturday, 14 Dec 19 11:51:22.945 +0100 [Debug] [Stat[#8]] OOS Reason: Failed to get statistics of Citrix ADC Instance for IP: X.X.X.X
Saturday, 14 Dec 19 11:51:22.949 +0100 [Debug] [Stat[#8]] Done StatNSHandler for X.X.X.X
^C
bash-3.2#

anybody got an idea?

 

best regards

 

 

[Diego Oliveira]

Hi

What version of MAS?

[Stefan Sallegger1709159665]

Hi,

I solved the issue with upgrading to a new 12.1 version of the netscaler