Layering Service and Management service "Needs Attention"

[David Ray]

Hi folks,

I am having trouble with my App Layering Appliance.  Have failed to find any mention of this issue here or elsewhere on the internet.  On the appliance under System/Management and Storage  both the Layering service and the Management service status say "Needs attention" with an orange exclamation triangle.   The "running since" on the layering service says 6/30/2025.  The "running since" on the Management service says 11/17/2023, which corresponds to the last time I booted the appliance.  Yes, I have tried rebooting it several times.

The Storage (on the Monitoring and Storage page) shows 308gb free.  Even so I have attempted to expand the storage.  I add some new free space on the disk on the VM but it never shows available when I click Expand storage.  I suspect whatever is cause the Management service to "need attention" probably keeps it from recognizing and offering new space.

 

This would all just be puzzling and annoying except after the appliance is in this state for a while then then Image Publishing tasks stop working.  They sit in state "Pending" forever with a status saying the task will start when more resources are available.

 

I have had a case open with Citrix Support for four days.  There do not seem to be any support articles that apply very specifically and none of the suggestion in the ones we tried have had any effect.  Waiting hopefully for some escalation engineer to have a better idea.

 

Does anyone have experience or suggestions that may help?  Have not been able to publish new images for more than a week now.

 

We are  running ELM version 23.4.0.1007.  I'd be happy to try upgrading to 23.09 but any attempt fails before really starting.  Probably for the same core reason behind the problem.

 

Thanks,
David

[Richard Buffone]

It sounds a lot like internal article CTX291403. It's simple enough to fix but I would also be concerned with if the storage expansion was able to to do any of the steps at all. We may need to fix up the LVM disk if it did.

 

I'll post this on the case for the case owner to look into. If I get freed up I can pull this into escalation.

[Rob Zylowski]

Hi David,

Did this start happening after an event like an upgrade or a failure to create a layer?

Do you have a backup of the appliance from  before you had this issue?

[David Ray]

I've tried to think about that but cannot correlate the start of this issue to anything noteworthy happening on the appliance or our environment.  It is not even super clear when this (the "needs attention" thing) started.  The symptom that brought us to a halt started a week ago.  Any image publish task sits in Pending.  I have not tried a Layer task.

 

I did restore the appliance to date earlier in the month... a date chosen to be before the problematic Pending tasks but after most or all Layer updates (hoping not to lose layer work).   That is the appliance I am working with now.  The Monitoring and Storage info is the same on both (other than the running since date on the Management service).

 

I last updated the appliance (to 23.4) on June 6th this year.   Starting in late October I did get a few of the notification emails from ELM mentioning High swap memory usage with both the Management service and the Layering Service.  Not a lot of mails.. 1 a couple weeks. 2 a couple weeks.  There was one of those emails on the day the tasks went stuck in Pending status.