Jump to content
Welcome to our new Citrix community!
  • 0

XenServer 7.6 crashed and hard reseted itself


Marius Schwarz

Question

Hi all,

 

we have a freshly commissioned XenServer  version 4.7.6-6.1, now fully patched, but when it happend, no hotfixes were applied.

 

This server ran 10 days without a vm to check the hw, network etc. . 2 Days ago, the first VM got onto the server, transfered via snapshot export / import from an old XS6.x .

 

This morning at 4:35 CEST the entire XS crashed and rebooted.

 

Not one logfile shows anything near a warning, hint, suggestion that something is out of the normal day-by-day business of a XS.

 

/var/log/messages and journal files are blank or missing

lastlog shows just the reboot and my logins

no kernel oops of what so ever in dmesg & co.

kern.log.1 blank

 

here the only log that goes near the crash point:

 

...

Apr  3 04:34:15 shortname xapi: [debug|servername.de|274632 UNIX /var/lib/xcp/xapi|session.slave_login D:7b363e9de6be|mscgen] xapi=>xapi [label="pool.get_all"];
Apr  3 04:34:15 shortname xapi: [debug|servername.de|274633 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:68478cee4642 created by task D:7b363e9de6be
Apr  3 04:34:15 shortname xapi: [debug|servername.de|68 ||mscgen] xapi=>xapi [label="event.from"];
Apr  3 04:34:15 shortname xapi: [debug|servername.de|274634 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:82d6dcb9d78b created by task D:614edc33a2a1
Apr  3 04:34:23 shortname xcp-rrdd: [ info|servername.de|6 ||rrdd_main] memfree has changed to 825692 in domain 1
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|4 |events|xenops_server] Received an event on managed VM 96cff72c-1fbb-32db-4bd8-9ca73affa3f6
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|4 |queue|xenops_server] Queue.push ["VM_check_state","96cff72c-1fbb-32db-4bd8-9ca73affa3f6"] onto 96cff72c-1fbb-32db-4bd8-9ca73affa3f6:[  ]
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|28 ||xenops_server] Queue.pop returned ["VM_check_state","96cff72c-1fbb-32db-4bd8-9ca73affa3f6"]
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|28 |events|xenops_server] Task 962 reference events: ["VM_check_state","96cff72c-1fbb-32db-4bd8-9ca73affa3f6"]
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|28 |events|xenops_server] VM 96cff72c-1fbb-32db-4bd8-9ca73affa3f6 is not requesting any attention
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|28 |events|xenops_server] VM_DB.signal 96cff72c-1fbb-32db-4bd8-9ca73affa3f6
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|28 |events|task_server] Task 962 completed; duration = 0
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|28 ||xenops_server] TASK.signal 962 (object deleted)
Apr  3 04:34:23 shortname xapi: [debug|servername.de|310 |org.xen.xapi.xenops.classic events D:a2a7469e0d7d|xenops] Processing event: ["Vm","96cff72c-1fbb-32db-4bd8-9ca73affa3f6"]
Apr  3 04:34:23 shortname xapi: [debug|servername.de|310 |org.xen.xapi.xenops.classic events D:a2a7469e0d7d|xenops] xenops event on VM 96cff72c-1fbb-32db-4bd8-9ca73affa3f6
Apr  3 04:34:23 shortname xenopsd-xc: [debug|servername.de|1998 |org.xen.xapi.xenops.classic events D:a2a7469e0d7d|xenops_server] VM.stat 96cff72c-1fbb-32db-4bd8-9ca73affa3f6
Apr  3 04:34:23 shortname xapi: [debug|servername.de|310 |org.xen.xapi.xenops.classic events D:a2a7469e0d7d|xenops] xenopsd event: processing event for VM 96cff72c-1fbb-32db-4bd8-9ca73affa3f6
Apr  3 04:34:23 shortname xapi: [debug|servername.de|310 |org.xen.xapi.xenops.classic events D:a2a7469e0d7d|xenops] Supressing VM.allowed_operations update because guest_agent data is largely the same
Apr  3 04:34:23 shortname xapi: [debug|servername.de|310 |org.xen.xapi.xenops.classic events D:a2a7469e0d7d|xenops] xenopsd event: Updating VM 96cff72c-1fbb-32db-4bd8-9ca73affa3f6 domid 1 guest_agent
Apr  3 04:34:28 shortname xapi: [debug|servername.de|274635 ||mscgen] xapi=>xapi [label="event.from"];
Apr  3 04:34:28 shortname xapi: [debug|servername.de|274636 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:bd69ff0794e0 created by task D:5dba077b093f
Apr  3 04:34:45 shortname xapi: [debug|servername.de|68 ||mscgen] xapi=>xapi [label="session.logout"];
Apr  3 04:34:45 shortname xapi: [debug|servername.de|274639 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.logout D:b674f6f2ed56 created by task D:614edc33a2a1
Apr  3 04:34:45 shortname xapi: [ info|servername.de|274639 UNIX /var/lib/xcp/xapi|session.logout D:8d002154a523|xapi] Session.destroy trackid=e569a5cc155e2d74631a9aeb9cc51fbd
Apr  3 04:34:45 shortname xapi: [debug|servername.de|68 ||mscgen] xapi=>xapi [label="session.slave_login"];
Apr  3 04:34:45 shortname xapi: [debug|servername.de|274640 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:session.slave_login D:0f2f46d33ba6 created by task D:614edc33a2a1
Apr  3 04:34:45 shortname xapi: [ info|servername.de|274640 UNIX /var/lib/xcp/xapi|session.slave_login D:00f076959080|xapi] Session.create trackid=571b4a0733fb1ea0b3fae245cfa3e45a pool=true uname= originator=xapi is_local_superuser=true auth_user_sid= parent=trackid=9834
f5af41c964e225f24279aefe4e49
Apr  3 04:34:45 shortname xapi: [debug|servername.de|274640 UNIX /var/lib/xcp/xapi|session.slave_login D:00f076959080|mscgen] xapi=>xapi [label="pool.get_all"];
Apr  3 04:34:45 shortname xapi: [debug|servername.de|274641 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:pool.get_all D:23a9561a4816 created by task D:00f076959080
Apr  3 04:34:45 shortname xapi: [debug|servername.de|68 ||mscgen] xapi=>xapi [label="event.from"];
Apr  3 04:34:45 shortname xapi: [debug|servername.de|274642 UNIX /var/lib/xcp/xapi||dummytaskhelper] task dispatch:event.from D:2aa0a6cfed8f created by task D:614edc33a2a1

 

...RESTART...


Apr  3 04:41:39 shortname xcp-rrdd: [debug|servername.de|0 ||rrdd_main] Reading configuration file ..
Apr  3 04:41:39 shortname xcp-rrdd: [debug|servername.de|0 ||xcp-rrdd] Parsing []
Apr  3 04:41:39 shortname xcp-rrdd: [debug|servername.de|0 ||xcp-rrdd] Parsing [debug]
Apr  3 04:41:39 shortname xcp-rrdd: [debug|servername.de|0 ||xcp-rrdd] Parsing [http]
Apr  3 04:41:39 shortname xcp-rrdd: [debug|servername.de|0 ||xcp-rrdd] Parsing [info]
Apr  3 04:41:39 shortname xenopsd-xc: [debug|servername.de|0 ||xenopsd] xenopsd version 0.9 starting
Apr  3 04:41:39 shortname xcp-rrdd: [ info|servername.de|6 ||rrdd_main] Couldn't read path /local/domain/0/data/meminfo_free; forgetting last known memfree value for domain 0
Apr  3 04:41:39 shortname xenopsd-xc: [debug|servername.de|0 ||xenopsd-xc] Parsing []
Apr  3 04:41:39 shortname xenopsd-xc: [debug|servername.de|0 ||xenopsd-xc] Parsing [debug]
Apr  3 04:41:39 shortname xenopsd-xc: [ info|servername.de|0 ||xenopsd-xc] Found 'chgrp' at '/usr/bin/chgrp'
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find xenguest on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find emu-manager on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ info|servername.de|0 ||xenopsd-xc] Found 'tune2fs' at '/usr/sbin/tune2fs'
Apr  3 04:41:39 shortname xenopsd-xc: [ info|servername.de|0 ||xenopsd-xc] Found 'mount' at '/usr/bin/mount'
Apr  3 04:41:39 shortname xenopsd-xc: [ info|servername.de|0 ||xenopsd-xc] Found 'umount' at '/usr/bin/umount'
Apr  3 04:41:39 shortname xenopsd-xc: [ info|servername.de|0 ||xenopsd-xc] Found 'ionice' at '/usr/bin/ionice'
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find setup-vif-rules on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find setup-pvs-proxy-rules on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find convert-legacy-stream on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find vgpu on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find vncterm on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find hvmloader on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find qemu-dm-wrapper on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find qemu-system-i386 on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ warn|servername.de|0 ||xenopsd-xc] Failed to find qemu-wrapper on $PATH ( = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin) or search_path option ( = )
Apr  3 04:41:39 shortname xenopsd-xc: [ info|servername.de|0 ||xenopsd-xc] Found 'pygrub' at '/usr/bin/pygrub'

...

 

Is there something know about 7.6 being instable?

 

cpu: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz

mainboard: Supermicro X10SRi-F

 

 

Link to comment

2 answers to this question

Recommended Posts

XS 7.6 has been very stable on several servers for us for a very long time. Maybe do an extensive POST on your server when it reboots to check for any HW errors. You could look in /var/log/messages for any hints of a crash before it happened, or in /var/log/SMlog for in particular storage I/O issues. You might also consider running a XenServer health check and see what the analysis reports.

 

-=Tobias

Link to comment

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...