Jump to content
Welcome to our new Citrix community!
  • 0

Office 365 (click to run) pulling into User layer issue


Question

We are struggling with an issue recently with Office 365 and User layer that I am going to open a ticket for as well but wanted to see if anyone else has run into this or advice.  We are setup w/ full User Layer with MCS Pooled Random setup and have 365 (click to run) setup as a layer and included in the published image that we use.    In the past few weeks we've had 2 random crappy mornings filled w/ calls with users that cannot get Outlook to open and not crash or Outlook (or any other 365 app) will open but will yell that its Unlicensed (we use 365 user activation, not KMS for reference).   As I dug in, what I found is if i mount an affected users User Layer for analysis their c:\program files (x86)\microsoft office\  dir is 2gb (the size roughly of a full 365 install) and you can see all the program exe's which i know should not be. (in some cases it will be closer to 1gb w/out the app exe's but still 1gb worth of dll's and other )     What I can't figure out is if somehow 365 updated itself in the user layer or if the app got pulled into the User layer by some other means (we have that happen w/ another old application that we have to regularly manually mount user layers to clean out files so that it will run again)

 

The odd thing w/ the updates installing theory is I have 365 updates blocked probably 3-4 different ways to try and stop it (it seems to be pretty forceful to try) so that theory i really cant think is whats going on but who knows (In the 365 install xml we have it set to not update, i have another group policy setting to hide the option to update and don't update again and then recently found another reg key i found to block it......i really dont like how Windows and 365 do updates).

 

Anyways this morning we had about 50 people call in w/ the issue and we ended up needing to fully clear user layers as they called in rather than try and manually flush things out since JUST clearing that program files dir doesnt always do the trick

 

SO.....un-long story short i'm curious how people are handling 365 updates to ensure that its not updating in the user layer  and any advice on this weird behavior where the entire program is pulling down into the user layer?  I have a post mortem w/ execs on Thursday to try and explain how this happened twice in 3 weeks so hoping to find something.    (we kept the "broken" user layer in case they are needed for more analysis)

Link to comment

17 answers to this question

Recommended Posts

  • 0

Hi Andrew, nice to chat with you again.

 Any and all changes are copied to the User Layer(UL). It could be an update, but how about a malware, a/v scanner or something opening the files for read/write access. Can you confirm an update did not take place?

Also, we do capture when we have written a file to the UL disk, but not what triggered it.  Looking at those logs, located in the users system, path noted below, will give you an idea as to when the process, it should appear as a move entry, took place. You might be able to figure out what triggered the copying of the data into the UL disk, with that data. But, yes open a case and someone, "else", will work with you.


C:\Program Files\Unidesk\Uniservice\Log

 

Start by looking at log0.

 

Once the time has been identified, for when the files were moved to the UL disk, one might be able to trace the process by looking in the Event Viewer logs around the same time.

Link to comment
  • 0

Thanks Raymond!  So being that we're MCS Random once a VDA is rebooted the event log clears and looks like so does the Uniservice log (dont suppose theres any way to persist that ?  guessing not but cant hurt to ask).  I know there was not any planned updates for Office or A/V (Cylance in our case) since i would have had to be the one to do it.   Do you guys have any recommendations how to completely block 365 (click to run) updates w/ App Layering? i dont think i saw anything in the recipe.  I may have gotten it finally but wanted to ask so i can at least confirm to my team in tomorrows post mortem for the issue

 

I was out of the office Monday when it all hit the fan so i let the team know to just rename user layers to get people going again since it was such a high number of users so all i have is the 'broken' user layers but sounds like thats probably not terribly helpful , hey?   if/when it happens again i'll have to have us copy off that Uniservice log and Windows event logs before rebooting to fix.

 

I actually did open a case Monday evening but still waiting for first contact from anyone.  Unfortunately i'm going out of town stating Friday and was really hoping to start digging in before being out to ease the concerns by higher ups but probably not too feasible at this point.  I appreciate you trying to help here though

Link to comment
  • 0

You could save some of the logs, i.e. Application, System, to a UL disk. The below link shows in possible registry modification to accomplish this or a scheduled copy process. Not knowing when it will happen again, can result in disk space issues

 

https://searchwindowsserver.techtarget.com/answer/Change-the-default-location-for-the-Event-Viewers-log-files

 

 

As for how to block the updates, what we know is posted. In theory, block it as you would in any other vm or a physical machine. Sorry, your case has not been addressed. I will put a note in it, though you will be unavailable, starting tomorrow.

Link to comment
  • 0

I see what happened. Someone did reply to your case, the next day(Jun 11th), but forgot to include your email. I have done that, doh, and not noticed it for a few days until the customer posts again or I check to see why I have not heard back. The same person tried to call you, later that day, but could not reach you. Please update your contact, case with a direct phone number, if one is available.

Link to comment
  • 0

i actually do have my direct number in there but not often at my desk so try to use email as preferred method of contact when i open a ticket. i see i had a couple of missed calls that day but no voicemail.  thanks for looking into it for me! i logged into the portal and don't see an update on that end either but i'll post in there to see what we can try and do today on short notice.  its actually happening again for a lot of people (different people this time it seems so far.....so before rebooted one i grabbed that log0 uniservice log and the app/system/security logs and the user layer disk so i have that to submit if needed

Link to comment
  • 0

Did you ever get this figured out? I seem to have the same issue: user layer shows C:\program files\microsoft office as 2.7GB.  I cleared it out last week and this week it's back. My logoff0.log file lists a ton of C:\program files\microsoft office\ files with a "move" command in the morning (both times that it shows up in the log it was about 40-45 minutes after I logged in). 

 

Thanks.

Link to comment
  • 0

So our process is i use config.office.com to configure our xml and i use ODT on a server and use command line :  setup.exe /download 365.xml   (semi annual channel)

 

once that completes i copy that to our file share and once i open our Office layer packaging machine i run:  setup.exe /configure 365.xml and it installs the latest version, finalize and include in a published image.   

Link to comment
  • 0

I know this isn't supposed to be an issue, but we ran into problems with the SAC builds and switched to Monthly. I do pretty much the same thing, although I pre-stage the downloaded files on a network share. Do you also include these values in your xml file:

 

<Display Level="Full" AcceptEULA="TRUE" /> (this line isn't completely necessary)
<Property Name="AUTOACTIVATE" Value="1" />
<Property Name="SharedComputerLicensing" Value="1" />

 

The last one is pretty much mandatory for VDI, and the second line is for convenience. I added the first line so all I have to do is call the ODT with XML file and away it goes.

 

Do you run the App Layering optimizer to back up the O365 activation in the layer? Also, doing a ngen update seems to help after install. My app layer always will not shut down due to an ngen operation, so I just go ahead and run ngen update for 32 and 64 bit layers, reboot, finalize. Been doing this for over a year now for Win7/10/Server2016 and works perfectly.

Link to comment
  • 0
12 minutes ago, David Thacker said:

I know this isn't supposed to be an issue, but we ran into problems with the SAC builds and switched to Monthly. I do pretty much the same thing, although I pre-stage the downloaded files on a network share. Do you also include these values in your xml file:

 

<Display Level="Full" AcceptEULA="TRUE" /> (this line isn't completely necessary)
<Property Name="AUTOACTIVATE" Value="1" />
<Property Name="SharedComputerLicensing" Value="1" />

 

The last one is pretty much mandatory for VDI, and the second line is for convenience. I added the first line so all I have to do is call the ODT with XML file and away it goes.

 

Do you run the App Layering optimizer to back up the O365 activation in the layer? Also, doing a ngen update seems to help after install. My app layer always will not shut down due to an ngen operation, so I just go ahead and run ngen update for 32 and 64 bit layers, reboot, finalize. Been doing this for over a year now for Win7/10/Server2016 and works perfectly.

interesting....what issues did you have w/ SAC?  we've been having A LOT of issues w/ things like this, blue screens, user layer drops all in the past few months so monthly scares me a bit to add that risk but i'm curious what you found

 

I do have the AcceptEula and SharedComputerLicensing set the same but our AUTOACTIVATE is set to false....what does that one do?     and no we didnt do antyhing w/ the app layer optimizer for the o365 activation.  we used to do something like that w/ 10 when we were on office 2013 but i thought since this is online / user activation that wouldnt be needed but if i'm wrong on it i'm happy to adjust!

 

i actually just found the screenshot below so sounds like you actually shouldnt use that autoactivate maybe?

 

image.thumb.png.860790b3697115c7a73852315b7b3591.png

Link to comment
  • 0
7 minutes ago, Andrew Gresbach1709152664 said:

interesting....what issues did you have w/ SAC?  we've been having A LOT of issues w/ things like this, blue screens, user layer drops all in the past few months so monthly scares me a bit to add that risk but i'm curious what you found

 

I do have the AcceptEula and SharedComputerLicensing set the same but our AUTOACTIVATE is set to false....what does that one do?     and no we didnt do antyhing w/ the app layer optimizer for the o365 activation.  we used to do something like that w/ 10 when we were on office 2013 but i thought since this is online / user activation that wouldnt be needed but if i'm wrong on it i'm happy to adjust!

 

i actually just found the screenshot below so sounds like you actually shouldnt use that autoactivate maybe?

 

image.thumb.png.860790b3697115c7a73852315b7b3591.png

 

Ah, I haven't read the docs for the past couple of versions; my XMLs are over a year old, and at the time, you had to have that AutoActivate property in there. The most important option is to turn off updates, otherwise it will totally bork the VM if it tries to update, especially non-persistent VMs. There was a period of time that I went through hell trying to get Office 365 to install and run correctly, so I feel like Murphy's Law happens with O365.

 

<Updates Enabled="FALSE" Channel="Monthly" /> (change the Channel to the ring you're using)

 

We found that the SAC was missing a LOT of features due to how infrequently they update it, and it seemed less stable overall. You would think it's the other way around, but from my experiences, the Monthly builds turn out a lot better. Almost zero crashes or odd behavior, plus the latest feature set. Depending on how often you rebuild your VDI gold image, there's probably a new Monthly build by that time. I do always capture each version in it's own layer, no "upgrading" from one build to another. We've physically deployed the Monthly build to probably close to 150 machines, and we don't have any complaints from our users.

 

Yes, the O365 activation backup is mandatory. I always do it in the O365 layer; just run the App Layering Optimizer as Admin, select option (K, I think) that says Process Office 365 Activation, Save Changes, and shutdown the layer and finalize it. What are you using for the user layer? We're a VMware shop, not Citrix, so I can't speak as well to Citrix-related issues, but App Layering 4.x pretty much works agnostic of platform, so an issue is an issue., when it comes to app layers.

Link to comment
  • 0

well damn thats interesting....because yup all logic would point to bleeding edge being the riskier move.  granted those monthlys will have bug fixes quicker but also chance of new issues but i'm curious to try

 

our setup is citrix mcs and our user layers are hosted on a Nutanix AFS SMB share

 

so the activation backup is actually mandatory? for some reason i thought that didnt apply anymore but could be wrong. definitely something that we can try.    If anyone on the app layering team see's this can they confirm that and what would we possibly have happen if we DIDN'T do that option (like we're doing now)

 

thank you for all the input by the way!

Link to comment
  • 0

also as far as the UPDATES deal........we definitely did have that off but was having an issue where 365 kept kicking off  a process to TRY and look for updates even though we had it off in multiple places.   a MS vendor suggested we change the content delivery method in the xml to SCCM (even though we dont use it) w/ the thought process that it will try that and wont have anything to update so that would help.  i wonder if that is a smart move or not.   

Link to comment
  • 0
Just now, Andrew Gresbach1709152664 said:

also as far as the UPDATES deal........we definitely did have that off but was having an issue where 365 kept kicking off  a process to TRY and look for updates even though we had it off in multiple places.   a MS vendor suggested we change the content delivery method in the xml to SCCM (even though we dont use it) w/ the thought process that it will try that and wont have anything to update so that would help.  i wonder if that is a smart move or not.   

 

Hmm, I wonder if there isn't something wrong with the XML structure of your config. I don't use MS' tool, I create my XMLs by hand, referencing MS' guide on the available commands, and I've not had an O365 layer try to update yet. However, on some physical machines running the SAC build, they decided to quit activating until I manually checked for updates and forced it to update. After that, it reactivated fine. Yet another reason I use the Monthly builds..

 

There is an activation token that gets copied and backed up when you run the Optimizer and select the Process Office 365 option. I would agree that it would seem like that's pointless and O365 would just activate on it's own, but O365 writes data to weird places, hence the reason App Layering offers a separate O365 Layer option instead of a full User Layer.

 

Since we're licensed, and MS has released the binaries, I switched to FSLogix for my profile management, but I still build my masters in App Layering and use Elastic Layering. I haven't tested the App Layering User Layer in a while, but there used to be a lot of caveats relating to O365, OneDrive, and other products that integrate closely with O365. In fact, I have my XML set to not install OneNote, Groove, and OneDrive, because we use OneDrive extensively, and the ODFB (Groove) is deprecated, and who knows what build of OneDrive it is installing. I use a GPO to install OneDrive and configure it for auto-redirection, and with Win10 + FSLogix, it's worked perfectly. I can't speak to Citrix's UPM, and I haven't tested App Layering's User Layer lately, but last I knew, OneDrive was still an issue, as was search indexing. 

Link to comment
  • 0

yup that was my thinking since its a user logon based activation and we have the shared roaming feature on that wouldnt be needed (again i'm curious if a Citrix person can confirm that if they see this) but i agree that things get sucked into weird spots for this so i wonder if its one of those things that isnt needed but also doesnt hurt to just do it just in case

 

you know its funny because we JUST looked at FSLOGIX and i was ready to go down that path but we have a handful of applications that store there data outside of the user profile (ie. db in the Program Files install dir, programdata)  so we wouldnt be able to persist that data between sessions unfortunately so kind of killed that idea

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...