Endless loop of xapi login/outs filling my logs, how to debug?

[Ammar Y]

Hello,

I'm running a pretty simple pool with 2 hosts, and using HA Lizard to synchronize storage between the two servers. Recently, my pool master's log at /var/log/xensource.log has become flooded with endless loops of session.login_with_password, pool.get_all, then session.logout. This is happening around the clock every second (or less). The only other problems I've noticed are that the host is a little slow, and an issue with console connections to linux hosts not taking keyboard input, which is pretty bizarre, but I assume not related.

Sep  9 19:30:46 x1 xapi: [ info|x1|7295 UNIX /var/lib/xcp/xapi|session.login_with_password D:0bec99a68f63|xapi] Session.create trackid=77597fa8da8ba1d284fb32ded69cb6f6 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep  9 19:30:46 x1 xapi: [debug|x1|7295 UNIX /var/lib/xcp/xapi|session.login_with_password D:0bec99a68f63|mscgen] xapi=>xapi [label="pool.get_all"];
Sep  9 19:30:46 x1 xapi: [debug|x1|7296 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:2fd306e09ad8 created by task D:0bec99a68f63
Sep  9 19:30:46 x1 xapi: [ info|x1|7295 UNIX /var/lib/xcp/xapi|session.logout D:8105e976781b|xapi] Session.destroy trackid=77597fa8da8ba1d284fb32ded69cb6f6
Sep  9 19:30:46 x1 xapi: [debug|x1|7297 UNIX /var/lib/xcp/xapi||cli] xe host-list hostname=x1 minimal=true username=root password=(omitted)
Sep  9 19:30:46 x1 xapi: [ info|x1|7297 UNIX /var/lib/xcp/xapi|session.login_with_password D:c1caa7572600|xapi] Session.create trackid=d971ffb74768e2ffcbd775932b5c922c pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep  9 19:30:46 x1 xapi: [debug|x1|7297 UNIX /var/lib/xcp/xapi|session.login_with_password D:c1caa7572600|mscgen] xapi=>xapi [label="pool.get_all"];
Sep  9 19:30:46 x1 xapi: [debug|x1|7298 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:f44131784a60 created by task D:c1caa7572600
Sep  9 19:30:46 x1 xapi: [ info|x1|7297 UNIX /var/lib/xcp/xapi|session.logout D:5ea4eb544aa2|xapi] Session.destroy trackid=d971ffb74768e2ffcbd775932b5c922c
Sep  9 19:30:47 x1 xapi: [debug|x1|7299 UNIX /var/lib/xcp/xapi||cli] xe host-list enabled=true params=name-label username=root password=(omitted)
Sep  9 19:30:47 x1 xapi: [ info|x1|7299 UNIX /var/lib/xcp/xapi|session.login_with_password D:bc09cb500b01|xapi] Session.create trackid=ed01bee9215ef02bec416bd4bc4c4d0b pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep  9 19:30:47 x1 xapi: [debug|x1|7299 UNIX /var/lib/xcp/xapi|session.login_with_password D:bc09cb500b01|mscgen] xapi=>xapi [label="pool.get_all"];
Sep  9 19:30:47 x1 xapi: [debug|x1|7300 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:8735afb9fac2 created by task D:bc09cb500b01
Sep  9 19:30:47 x1 xapi: [ info|x1|7299 UNIX /var/lib/xcp/xapi|session.logout D:7fa9c3dd6b13|xapi] Session.destroy trackid=ed01bee9215ef02bec416bd4bc4c4d0b
Sep  9 19:30:47 x1 xapi: [debug|x1|7301 UNIX /var/lib/xcp/xapi||cli] xe host-list enabled=true minimal=true username=root password=(omitted)
Sep  9 19:30:47 x1 xapi: [ info|x1|7301 UNIX /var/lib/xcp/xapi|session.login_with_password D:97e4b679b8ff|xapi] Session.create trackid=9299cd872f1b92bdff38e243382053c3 pool=false uname=root originator=cli is_local_superuser=true auth_user_sid= parent=trackid=9834f5af41c964e225f24279aefe4e49
Sep  9 19:30:47 x1 xapi: [debug|x1|7301 UNIX /var/lib/xcp/xapi|session.login_with_password D:97e4b679b8ff|mscgen] xapi=>xapi [label="pool.get_all"];
Sep  9 19:30:47 x1 xapi: [debug|x1|7302 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:8dd5dc9967c6 created by task D:97e4b679b8ff
Sep  9 19:30:47 x1 xapi: [ info|x1|7301 UNIX /var/lib/xcp/xapi|session.logout D:51a3f0d13d0f|xapi] Session.destroy trackid=9299cd872f1b92bdff38e243382053c3

 

How could I go about debugging this? How can I identify where those sessions are coming from?

[Tobias Kreidl]

Any chance one of the SSL certificates is expired? How about if the NTP time is way off between hosts? Is there at lest still some space on the logging device?