The Application Streaming system "streams" stuff from the central network server to an execution platform.  The streaming aspects of this are very neat, but it turns out for many customers that the centralized publishing and the isolation provided from Application Streaming are the important parts. Well, different people have their own definition of the important parts, but stick with me on the concept.

The trick though is that a "first time launch" for a large application can be a massive data transfer and this isn't desirable if all you really wanted was centralized publishing and isolation.  How can you move that data transfer to "network idle time"?  RadeDeploy.exe is the utility included with the streaming client for this purpose.  Its is a command line utility that will advance copy the streaming content onto the execution machine.  It is assumed that the admin has facilities in their software management system to control when this utility gets executed so that the streaming content gets copied down to the execution machine before users arrive in the morning and start running applications.

Server side or client side?

Application Streaming client runs in two spaces: Stream to Client or Stream to Server.  For the most part, pre-deployment applies primarily to the stream to client configuration as its assumed that the Presentation Servers have a fast network between them and the file server that holds the streaming content.  

The ultimate mission is to move execution content from the central file server into the RadeCache execution spaces.  These are the "installation layer" of the 3 layers of isolation.  These directories are always a subset fill of the entire installation contents.  No matter what percent populated, the isolation system will lie to the application and tell it everything is present.

Here is the RadeCache space on my machine.

 
Copying from the network server to runtime fill the RadeCache execution space is the streaming aspect of Application Streaming.  In an online case, or more exactly, a non-pre-deployed case, the isolation system does cache fills by extracting content from the CAB file stored on a network server, pulling down the item needed for execution of the application, just in time for use.  This is the part of running the application, particularily first time launch, that generates network traffic.

 
To enable "offline use", the Streaming System also supports a concept called "PreDeploy".  Instead of streaming from a network server to the RadeCache, the streaming system will instead use a local Deploy folder as its source - and never contact the network server for the cache fill operations.  This is good stuff and since the Deploy location holds 100% of the content from the network server needed to run that application, it is possible to disconnect from the network while running a "streamed" application. 
 
Here's a snapshot of the Deploy location

 

Now - is it streaming?
 
The cool part of this implementation of "offline" is that the majority of the streaming system is completely unaware of the concept of offline.  When a cache fill is needed, the caching system asks the streaming service (radesvc.exe) to do the cache fill.  The services' logic is simple: Do I have this in the Deploy location?  Great, use that one. Else - consult the network server.   In the case of pre-deployed, the answer to the first question is always yes, so the traffic to the network server is completely avoided.  The file system driver that initiates the cache fill operation remains completely in the dark that the streaming was from one side of the hard disk to the other rather than across the network.  Keeping it simple, keeps it from breaking.
 
RadeDeploy.exe
Is the utility that populates the Deploy location identified above in graphic.  RadeDeploy copies the CAB file and other contents from the central file server and places it into the Deploy location.
 
Notice that RadeDeploy DOES NOT populate the RadeCache!   It perhaps SHOULD, but it doesn't.  This means that even if you do RadeDeploy to pre-copy execution content onto the client machine, you still have a first time launch penalty to apply to the first user on that machine that runs an application from that profile.  How can this be avoided?  In an ideal world, RadeDeploy.exe would also or optionally also populate the RadeCache.  It doesn't (Streaming Client 1.2).

 
You can do it manually.  How?
1) Do not delete the RadeCache directory.  DACLs are applied there that let the streaming service user account perform cache fills.
2) Create a subdirectory there to match the GUID and Version of the Target to execute.  Easiest way, run it once, see what gets created.
3) Extract Device\C and all dubdirectories from the CAB and place them into the RadeCache manually.
 
"Magic" is an acceptable means of populating the streaming cache.  Software management systems are assumed to be good people for populating the cache.  Ideally, you should do this when the streaming service is not running as it likes to keep track of what is in the cache so it can decide what to delete and otherwise avoid conflicts in cache fills/deletes.   At a minimum, do it when there are no active applications using the same cache content that is being filled.

 
When more stuff is where it will ultimatley need to be, runtime performance is optmized.

 
Final thought - Stream to Server and RadeDeploy

I'm not a big fan of Deploy for server side execution.  The servers are in the data center and they have fast and reliable networks.  Many application only EVER access 25% of what they "install", so avoiding the full CAB file can cut down the overall data transfer required in the data center. There's more...

 
Through HRP1 (Streaming Client 1.1), the non-Deploy update of cache space for Target update is much more efficient for that the Deploy case; only the changes are transferred on upgrade of a Target from V1 to V2.   In the upcoming Delaware release, this preference to not using pre deploy largely goes away as the same differencing is algorithm is given to the Deploy case as for the "online" update case.
 
Enjoy, 
 
Joe Nord

The Streaming Profiler is the application that "captures" installation program content.   A simple view, the profiler's mission is to observe the activities of an installation program (disk/registry), prevent them from really happening and finally, to write down everything that the installation program thought should be done to the machine.   The whole time, the installation program BELIEVES that it has truly installed things just like at runtime under the Streaming Client, the executed application will believe that everything it needs for execution is present, even when only a subset is there.

From an isolation view, this activity of "profiling" is very-very similar to the activity of running isolated applications.   The Streaming Profiler though has many extra jobs such as extracting the icons of the application that was "installed" and making them available for the publishing system.  It also examines the isolated "Start Menu" to see what additions the installation program "thinks" it made to the system.  These additions become the list of detected applications that are part of the profile.  There is lots of other stuff going on such as simulating reboots and otherwise figuring out what the installation program was trying to accomplish.  All of these are neat things, but they are not the focus on this blog.

Where does the profiler put the temporary stuff? 

This is a question I get a lot.  People are well familiar with the "3 layers of isolation" chart that I have shown before in other posts.  During profiling, there are only 2 layers and the layer that is missing is the "user space".   In a way, the layer missing is the middle layer, but it fundamentally doesn't matter, there are only 2 layers during profiling.  The installation image during profiling is "writable".  At runtime, the installation image is read only and there is an extra "pane of glass" laid down to handle the per-user space.

During normal Streaming Client runtime, the installation image is "cached" below the \Program files\Citrix\RadeCache directory.  During profiling, these same directories are not used!  They are not used on-purpose to avoid any messing up by profiling and running applications at the same time. 

Right - where's the stuff?

The Streaming Profiler ALSO deletes everything when it closes profiles, or when the Streaming Profiler application is terminated.  WHY?  Why not?  It was temporary stuff.  This does make the content harder to find if you go looking for it AFTER you close the profiler.  In more realistic scenarios, this deleting of the temporary content is what allows the Streaming Profiler to profile hundreds of installers without rebooting the machine.  This is very handy for conferences where the show floor can "show and tell" profiling all day long without rebooting.

Right, where's the temp space?

Two things are needed to reflect the layers of glass of isolation.  The disk redirection and the registry redirection. 

DISK:    %TMP%\Citrix\Packager
REG:     HKLM\Software\Citrix\AIE 

Here's a snapshot of the disk and registry profiling locations on my notebook.
Registry temporary space during profiling 

Can I mess with it?
Answer: Sure!
Should you mess with it? No!

Long answer: Not unless you have a good reason!   In theory, there is no reason to mess with the content of these directories/registry while profiling.  BUT - People have asked about it, so they must have a good reason - or maybe just be curious.

When running the Streaming Profiler, the profiler GUI will open .profile files off of network servers.  Or, maybe it will create new profiles.  Either way, the GUI sends instructions to the profiler "back end" (radepkg.dll), which does the heavy work of operating on profiles.

When a profile is opened, nothing related to the execution targets is in the temp editing space.  Only when a Target is placed into edit state does the CAB file contents get unzipped into the temp space and the registry contents get extracted.  All the activities of profiling then work to "modify" this temporary space and when the profile is saved, the execution target is then "zipped up" (cabbed up?), into the image that is stored on the network server.  While the Profile/Target is being edited, the contents of the disk and registry are "fair game" for being messed.  Again: Should you?  No!  But, if you do mess with the extracted contents of the profile in the temporary space, while the profile is being edited, the changes you make WILL be captured and saved with the profile.  Like all back doors, don't expect the internal operation of the Streaming Profiler to be the same from release to release, but in concept, this process has worked so far.

Saving your changes... 
Just because you modify things behind the back of the streaming profiler does not mean that the Streaming Profiler will save your modifications.  If the profiler back end believes that "nothing changed", it will optimize the cab up and save portion of the editing to "do nothing" and this will abandon your changes.  The easiest way to convince the profiler that "stuff changed" is to launch an installer that does nothing.  My favorite is CMD (command prompt) - and then "exit".  Having launched an installer, the profiler back end will be convinced that the execution Target has been modified and will include that Target during the save.

A better way 

The better answer is to use the Streaming Profiler SDK.  You're trying to do programmatic stuff anyway, so why not do it from a program?  Even better, using the SDK will insulate you from changes in implementation of the streaming profiler.  More information on the profiler SDK can be found here.  What it comes down to is a COM interface into the profiler back end (the plumbing, not the GUI).   This is an area ripe for more discussion, so I'll put that on the list of things to write about.

What about all those subdirectories?

Right, those are my fault.  Though MSI and other installation tools normally have a limit of one active instance at a time, the Streaming Profiler does not.  That is, multiple instances of the Streaming Profiler *CAN* be opened at once and from the view of the profiler, all don't know about the others.  Eventually, MSI prevents multiple concurrent profiling, but that's a limitation of the installation system, not of the Streaming Profiler.  

Why bother with this?  Don't know, it seemed like a good idea at the time.  The good news is that you can open one profile for reading and open another for really profiling stuff and the streaming profiler back end will deal with keeping all the items separate.  Well, that's the theory at least, this multiplicity of profiling has not seen significant exercise in the field, but the profiler back end supports it. 

Consider that when a profile is created, the back end has to store the data for that profile, but that the profile does not yet have a name.  The name is only assigned during SAVE and this lack of a name is a big headache for the back end.  The PROFILE is edited and the Targets of that profile can be edited.  

When the profile is edited (happens immediately when you open the profile in the Streaming Profiler), the profile contents are placed into edit space (build location) in a PKG_n directory/key below the top locations noted earlier in this post.  (Example: PKG_1).  If PKG_1 already exists, the back end moves on by bumping the number until it finds a whole that isn't being used by some other profiler.  Eventually, it gets a hit and that's the number space for the profile that is being edited.

When the Target of a profile is created, or edited, the back end needs to extract its contents to temporary space so that the "layers of glass" can be constructed and so that the Target can otherwise be edited.  Targets are identified by their GUIDs, so it would make sense that the GUID is the key for storing the Target's contents.  It WOULD make sense, except....   The Target's GUID is assigned when it is created - and then CHANGED when the Profile is SavedAs.  The Profile also has a GUID that receives must less attention, but it is still there.  

On a SAVE, the GUIDs are retained and the version number of the Targets is incremented.
On a SaveAS, the GUIDs are changed.  

The first save of a new profile (a created profile) is always a SaveAs, so the GUIDs are useless for storage of temporary content.

Since the GUID cannot be used as the key for the directory, the same low tech unique name is used as is used for the profile.  Temporary directories for the target are TGT_n and stored below the top space described earlier.

In HRP1, during "Finalizing your profile", the profiler back end would CAB up the contents of the Target and place it into the PKG_n directory and then the PKG_n directory would be copied to the destination location, presumably on a file server.   I just tried this in the upcoming Delaware level code and the intermediate step seems to have been optimized out by the Streaming Development team - I send kudos.

If you go to edit things while they are being edited, then you'll find the good disk stuff below one of the TGT_n directories and the registry contents below the AIE key.  Be sure to close out your editing program or it will prevent the Streaming Profiler from deleting the temporary space when it terminates. 

Cleaning up 

If the Streaming Profiler "dies" mid streaming - right, this would never happen!, but if it did, there can be "stuff" left around in the temp space.  Future executions of the Streaming Profiler will treat those left around directories as "it must be someone elses" and they will go for a bigger number until they get their own space for profiling. 

You can SAFELY obliterate the Disk contents used by the profiler temporary storage, at any time that you are not profiling. 

Example command:

RD /s /q  "%TMP%\Citrix\Packager"

Registry temporary space used in profiling is actually in the same space as AIE execution on PS 4.0 servers and PS 4.5 servers.  Cleaning up these (where the profiler didn't do it for you) is a bit more work.

The keys to whack are below HKLM\Software\Citrix\AIE.  If you're not on a PS server or not on a PRODUCTION server, you can use this command to delete the temporary AIE space.

reg delete hklm\software\citrix\aie

In my experiment, some of these keys aren't going away.  There's a lock held!  Hum.  In theory, they can go away. 

I don't know if stuff like this is useful for the general community.  Mostly I expect it to be useful for the streaming developers, but they already know it already.  If you find this useful or want some more details, let me know.

Joe Nord
Product Architect Application Streaming
Citrix Systems, Fort Lauderdale, FL

Filling the execution cache is an obvious job of the streaming service and is an obvious item to discuss.  Since the obvious item is not as interesting as the unobvious, deleting from the cache is covered in this post in detail.

Warning: Geek speak is turned on for this post.   I hope people find it useful. 

The Application Streaming client has the mission of maintaining its execution caches which support the execution of applications under Application Streaming.  Yes, that's rhetorical.  The streaming client has the same cache management logic regardless of its execution platform and manages the cache in exactly the same manner for both server side execution and client side.

Most administrators understand pretty well that the middle layer of the isolation spaces represents the "installed application" and this layer is "not really present" at runtime.   I've posted a few other blogs that show a graphic of this, so I won't repeat it here but you should be able to find it up or down a few screens.
The installed contents (installation image) are presented to the application as if they are "present" and are only truly brought into the execution machine when the application actually "touches something".  That is, the arduous job of actually copying installation image content from the central server to the execution machine is put off until the last possible moment, or as a more elegant term, "just in time populated" into the cache.  This is classic computer science stuff: If you put work off long enough, you may be able to avoid it completely.  In college, I did my homework with this same methodology.

At runtime, the execution cache starts empty and grows and then retains its contents for future executions.  The cache is maintained even across boots so that the later sessions will have execution content "already there" and avoid the need for a cache fill.  Cache fills equal network traffic, equals slow, so we prefer to have 2nd time launch experience rather than a first time.  Notice that cache fills are also shared across users.  Consider a Presentation Server hosted execution where content is added to the cache for UserA.  The execution cache is shared across the users, so the cache addition for UserA, will benefit UserB.  The "first time launch penalty" of using streaming and isolation really applies only to the first user on the entire server, and there are ways to reduce that impact.

Deleting content from the cache

When running an unlimited number of different applications, for an unlimited number of potential users, from an unlimited number of separate streaming profiles, the streaming service must support them using finite space in which to run the applications.

How much space does it get? 
What is the maximum size of the RadeCache?

The answer is ...  that it depends.  The RadeCache limits are implemented as a size constrained high water, low water system where the low and high limits are determined at streaming client installation and possibly overridden by an administrator.  The max size limit (high water mark) is installation time written to the registry and defaults to 5% of allocated disk space of the primary boot volume ( e.g. C: ), or a minimum of 1GB.  The low water limit defaults to 95% of cache size.

Right - what does that mean?

The Streaming Service (RadeSvc.exe) keeps track of the space being used by the execution cache and where that value grows ABOVE the high water mark, the service unleashes a "purge thread", or "delete thread" to free some space.  Notice that it does not fail or even delay the cache fill for the application that triggered the growth beyond the high water mark.   The application that is trying to run is immediately put back to doing useful work and this means that the cache is temporarily allowed to grow beyond the high water mark.  The delete activity is done by the streaming service too, but it is done on a separate thread from the cache fill logic.   The delete actions are also done at idle priority.  The delete thread is persistent though and will continue chugging along until the deleting is finished.

The streaming service is aware of the cache contents of ALL of the execution caches and the size of each file in each target, and for each file, the age of when that file was most recently accessed.    These execution caches are all of the subdirectories of \Program Files\Citrix\RadeCache.  Notice that there can be many caches and even cache content for applications that are no longer published to any user that is presently logged onto that server or possibly even for applications that are no longer published to anyone anywhere.  The point is that "new stuff" is more important than "old stuff" and eventually, the old stuff will age its way out of the cache. 

If someone runs an application that uses old stuff, the old stuff is new again and is no longer the best candidate for erase.  Low tech, but easy to implement which should read as "it doesn't go boom".

Low water mark

The cache manager delete thread repeatedly deletes old things from the cache until a "low water" mark is achieved.   Once low water mark is achieved, the delete thread goes back to sleep.   New cache fills can be occurring concurrently with cache deletes and all the while, the streaming service has to keep track of what is in the cache.  This is one of the reasons that: If you want to purge things from the cache, you should use tools like RadeCache.exe to do the delete rather than just deleting the files. This lets the streaming service retain its self stated importance as the keeper of the cache and lets it know what is presently occupying space in the cache.  Those are good things, but more importantly, it protects against windows in time where an application can start opening a file and then the file getting erased by the deleter before the file open has occurred.   This would be a long aside, so I'll get back on subject of deleting stuff from the cache.

Default low water mark is 5% below full.  Strangely, it's not implemented as 95% of full, which is bazaar, but that's how it is. This value is configurable via registry, it defaults to "5" and you generally shouldn't mess with it.  If you do, understand that it is how much space should be FREE before the delete thread stops deleting stuff rather than the percent full after the deletion stops.

Low disk avoidance

In addition to filling the cache and deleting stuff from the cache, the cache manager also tries to notice when the disk is crazy full and takes steps to be a good citizen.  The default here is 200MB. If you're down to this limit of disk space, the cache manager will dispatch the purge logic to start freeing up space on the disk and otherwise to try to keep the machine from buckling.  Once the minimum disk space is achieved, the delete thread goes back to sleep.

Thrash avoidance

Notice that deleting stuff for purposes of managing the cache is a good thing.  Deleting lots of things to help the system not run out of disk space is also a good thing.  It's possible though that all this good citizen stuff will result in endless cache deletes followed by fills followed by cache deletes, followed by fills, ...

The cache manager includes thrash detection logic to prevent this scenario which says:  If EVERYTHING in the cache was touched in the last 24 hours, then we're thrashing, don't delete anything. This may not be perfect - actually, it isn't perfect - but it is an attempt to do the right thing and effectively support streamed applications while automating the cache cleanup. 

Registry keys
Here are the registry keys that control the streaming service cache management.  They are all items in HKLM\Software\Citrix\Rade

Registry item     Type       Default and comments

CacheLimitMb      REG_DWORD  5% of disk space or 1GB.
                             Value is in MB (mega bytes) even though
                             key name indicates Mb (mega bits).

CacheLowWaterPct  REG_DWORD  5.
                             Value is percent of cache to make full
                             when start deleting

LowDiskThreshold  REG_DWORD  200.  Value is in MB.

Enjoy. 

Joe Nord
Product Architect, Application Streaming. 
Citrix Systems, Fort Lauderdale, FL

A customer recently reported the desire to use Windows Scripting Files as the "installer" for Application Streaming profiles.  The report was that the installation could not be done under the Streaming Profiler because the profiler refused to trigger the execution of the script.

A quick reproduction confirmed the report.  Solution follows, I hope positing it here can help other folks who hit the same issue.

Experiment installer:  HelloWorld.wsf

<job id="HelloWorld">

  '** Hello world in VB as a Windows Script File

  <script language="VBScript">

        WScript.echo "Hello!  Fantastic, it worked."

  </script>

</job>

First - Confirm the failing behavior.

In the Streaming Profiler, tell it to use this file as the installer and the result is
"This is not a valid Win32 application".  HUM.

Meanwhile, "Double Clicking" on this file is known to work to trigger installations when the Streaming Profiler is not involved.  What's up with .WSF?

How to solve:

Open a command prompt and peek inside .WSF.  Two commands to get the answer

C:\>assoc .wsf

.wsf=WSFFile

C:\>ftype WSFFile

WSFFile=%SystemRoot%\System32\WScript.exe "%1" %*

Okay, the Windows application for installer really WScript.exe, not HelloWorld.wsf.

To get this installer to run under the Streaming Profiler, two steps are required.

1)      The "installer" is wscript.exe

2)      The "parameter" to the installer is HelloWorld.wsf

Click on Next,

Click on Launch Installer

Disclaimer:  This *SHOULD* work for triggering the installers that are commanded via the script.  In writing this blog entry, I have not taken it to the next step of actually installing (profiling) something.

OKAY - How's this work?

When launching installation programs, the Streaming Profiler takes the "installer" parameter and the "parameters" and feeds them to the Win32 CreateProcessEx API, run under the context of isolation.    It would appear that .WSF is not a known extension for CreateProcessEx. 

The solution is to feed CreateProcessEx the WScript.EXE file (which CreateProcessEx knows how to launch) and to give that EXE file a parameter of the script that should be executed.

This same process should be possible for other extensions that provide similar results, possibly perl, rexx and a variety of other scripting languages.

Out of the box, .EXE, .MSI, .VBS, .BAT, .CMD work when launched.   The full list is variable, but this is the general idea.

Joe Nord

Product Architect Application Streaming, Citrix Systems, Fort Lauderdale, FL

Streaming Client 1.0 was the original release of the client that accompanied Presentation Server 4.5.

Streaming Client 1.1 shipped with PS 4.5 HRP1, around September 2007.  It included some performance enhancements that are not widely understood.  This blog post brings a bit of attention to some good work.  Under the hood work, but still good work.  The upcoming Delaware release will include additional performance enhancements. 

The Streaming Profiler has a few jobs.  The primary one is to "observe" the actions of an installation program, prevent the installation items from really occurring and "write down" what the installer tried to do to the machine.  This writing down part includes two primary items, the "file system" and the "registry".

If you peek inside an application streaming profile, you will see a ".profile" file.  This is the highest level file.  It is XML formatted data that describes everything needed for publishing.   There are a series of "CAB files" that hold the execution content for 1 or more execution platforms.  These "CAB files" are the backing store that support execution.  They are often called "execution targets" and they store the registry content and disk content for actually running the program.

The registry data is inside a .TAB file and isn't important for today's conversation.  The disk content is stored below the \Device folder in the CAB file and holds everything that was "installed" to the file system - potentially, lots of stuff.

Now - why did we use CAB files to hold this?

Is an excellent question and one of great debate in the development circles.  Instead of a compressed CAB, the content could just as easily be a directory with everything stored below that directory.  The CAB was used to faciliate offline access and make it a boolean decision on "its there" or "its not there".  Looking back, this was doable without a CAB, but we have what we have right now.  

The trick!  Efficient access to CABs.  Accessing files inside a CAB is not as efficient as accessing files beneath sub-directories. 

When creating an isolation sandbox, the streaming and isolation system needs to "know" what was stored during profiling.  When an application at runtime does a "DIR", the response to that directory enumeration uses information that "isn't there".  The information is provided to the application based on what "was there" at profiling time rather than was "is there" at runtime.  If the application then opens a file that isn't there, the streaming system makes the file "present" before completing the file operation.  This last item is a "cache fill" and is the basis of the "streaming" part of Application Streaming.

Consider: How do you construct a list of all the files in the CAB?

In the Streaming Client 1.0, this list was constructed as a part of creating the isolation sandbox.  That is, created at runtime.

it was assumed that an administrator might update the CAB file "behind the back" of the Streaming Profiler.  With this, the contents of the CAB were scanned at application launch and the directory structure for enumeration was built in memory based on the scan.  Hindsight says that this was a BRUTAL activity for performance.   Actually, it only was a problem for "large" applications, but here it was a real concern.   The problem is that it happens EVERY launch, not just first time launch, so this makes the inefficiency a big thing to be concerned about.  It was a big enough concern that we "fixed it" in 1.1.

In Streaming Client 1.1 (4.5 HRP1), the Streaming Profiler was enhanced to store the directory enumeration data as part of saving the profile (target).  With this data available in a single place, the Streaming Client could "skip" the DIR /S on the CAB file and speed application launch -- Greatly faster!  A side note is that the admin can no longer update the CAB file behind the back of the Streaming Profiler.  No worries, nobody was doing that anyway.

To get the benefit of the meta-data stored with the execution target, the existing profile needs to be loaded into a 1.1 level Streaming Profiler, the Target needs to be edited and then saved.  Done.  The 1.1 Streaming Client looks for the meta-data.  If its there, it uses it.  A 1.0 level Streaming Client will access the files of the CAB "the old way" and this makes the profiles backward and forward compatible even with the change.  Computer Science is fun stuff!

What's the future... 

In Streaming Client 1.2, this is taken a step further to change the compression of the CABs in a manner that's more friendly with WanScaler and happens to give a great benefit to first time launch performance.  I'll save that for a later blog entry, probably "post-Delaware".  For now, you should know that the CAB file in 1.0 and 1.1 is created in a format that allows more efficient single file extracts than does the default formatting from CABARC or similar tools.

In a dream, I'll shoot CAB files completely.  Truth is, won't get there anytime soon.

Joe Nord

Product Architect Application Streaming, Citrix Systems Fort Lauderdale, FL 

Can the Delaware level Application Streaming client be used with PS 4.5 level servers? 

Quick answer: Yes.

Long answer:...I get this question alot.  So much so that this blog will serve a useful reference so I don't have to type the same information over and over.  Part of this is my own fault, having just returned from Citrix Synergy conference where the first question I received AFTER the App Streaming breakout was: Can I use the IIC enabled Streaming Client on PS 4.5 servers?  Argh!  How did I forget to mention that to the room full of hundreds of people?

"Delaware" is Citrix codename for the Presentation Server release targeting Windows Server 2008 as an execution platform.  The Delaware level Streaming Client is the one that will have Inter-Isolation Communication (IIC) and HTTP based streaming, so its a hot item.   The Delaware level Streaming Client is better described as Streaming Client 1.2. 

Since Delaware is specific to Server 2008, the question is really one to say: Do I have to go to Server 2008 to get IIC? 

Answer: No.  You can install the 1.2 Streaming Client on PS 4.5 servers and you get the 1.2 level function on the older PS platform.

Streaming Profiler/Client - Version history 

• 1.0 Presentation Server 4.5 GA
• 1.1 Presentation Server 4.5 HPR1
• 1.2 Delaware (not yet released...)

Hey Joe: IIC is pretty compelling stuff, why isn't the version number 2.0?  Answer: There's no good reason.  2 was bigger than 1 and it makes the numbers different, so that's all we did.  I'm not a huge fan of bumping version numbers just for entertainment.   I have a long history of major.minor changes causing troubles, so I tend to just bump one of the numbers.

The 1.2 Streaming Client can be installed separately from the server (Stream to client).  This concept is easy. 

The 1.2 Streaming Client CAN also be installed on top of PS 4.5 HRP1 Servers (32 and 64 bit). 
This will bring 1.2 streaming client function to PS 4.5 servers!  This is important.

More important: Yes, we intend it to work that way.  Yes, we are even testing it and YES, it works!

The Streaming Client is a client.  Think "ICA Client".  The client does not REQUIRE a Presentation Server release for the client to be updated.  So far, we've shipped the streaming client only when using the ship vehicle of the Presentation Server; but this isn't required.  Also consider that ICA Client can be installed on top of Presentation Servers - enabling pass through.  The same concept applies to Streaming Client. 

What about the data??  (profiles) 

Profiles created with Streaming Profiler 1.0 and Streaming Profiler 1.1 CAN be used with the 1.2 Streaming Client.  This "forward compatibility" works as would be expected.  Performance enhancements were added in 1.1 and additional enhancements added in 1.2 so there is value in "updating them".  But, if 1.0/1.1 profilers are in the field and working the cost of doing the update is probably bigger than the value of the performance boost so this alone isn't a big enough reason to go update everything. 

"Older profiles" can also be Inter-Isolation Communication enabled via IIC function of Streaming Profiler 1.2 with no need to reprofile the existing applications.  That is, 1.0, 1.1 created profiles can participate in IIC - which is pretty cool!

Going backwards, it gets trickier: Streaming Profiler 1.2 created profiles that do not use IIC *CAN* be consumed by the 1.1 Streaming Client.  This is conceptually, forward compatibility, but forward compatibility has limits.  A 1.1 level Streaming Client has no idea what is "IIC", so it cannot consume an IIC enabled profile created by the 1.2 Streaming Profiler.   Well, technically it can consume it enough to figure out that it can't handle it.

Where a streaming client (old) "lacks" capabilities to run applications from a profile (e.g. IIC), the old streaming clients include a polite error message to describe this and recommend an upgrade to a newer streaming client.  This has existed from the beginning (1.0) and will only now be exercised in 1.2 and only where IIC is enabled.  Generally it doesn't come up because admins upgrade the clients, but its there none the less. 

Update: After writing this, one of the streaming developers commented that the HRP1 level AMC will not understand an IIC enabled Streaming Profile.  True.  Solution: Use Delaware level AMC to publish into 4.5 farm.  Hopefully that works.

Joe Nord
Product Architect Application Streaming, Citrix Systems Fort Lauderdale, FL

Supporting administrators using Application Streaming, one of the items that comes up from time to time is RadeRunSwitches.  They can be a good tool for running applications and even more important, for debugging the operation of a streamed application. 

This post describes the undocumented switches...

The Application Streaming client exists as part of Presentation Server 4.5 and later versions.  It works in conjunction with the PNAgent Win32 ICA Client as well as the Web Interface clients that run on the Windows platforms.  This is true both on user machine "client side" as well as server hosted, stream to server.

The icon placement and decision to launch an application happen in PNAgent and/or the Web Interface. These two components eventually toss the execution over the wall to the Streaming Client who does the actual work of running the Streamed application.  In the diagram below, the Streaming Client components are in green and the publishing components are in blue.

.

The executable that receives the launch request is RadeRun.exe.    RadeRun's mission is to carry out the launch request, via a single set of code that may be called from multiple sources.  Classic computer science stuff here; implement the launch logic once and you can be confident that if it works for one, it will work for the other and you get a single point of maintenance.    The "true" flow between the various components is a bit more involved than the above, but in principle, this is how it works.

Back to the beginning

In the earliest days of Application Streaming development, the PNAgent and Web Interface teams were not yet engaged.   All execution of applications started with RadeRun.   Early programming, testing and proof of concept were all done using RadeRun.exe as a command line utility to trigger the execution of streamed applications.  Some of this legacy remains even though PNAgent and the Web Interface are now quite capable of communicating with the streaming service without an intermediary. 

Isolation layers

In a minute, I will go through each of the switches to RadeRun - but first it helps to have an understanding of the isolation layers used in Application Streaming.

The application views the machine from above looking down.  The higher levels are like panes of glass laid on top of a business desk. The desk represents the true disk and true registry of the execution machine.  The application is above looking down and the layers of isolation glass "mask" the application's view of the true machine.    The layer of glass in the middle is read only at application execution, but was writable during profiling.  This layer represents the execution image for the application and is sometimes called the "InstallRoot" in documents describing Application Streaming.

The top layer is a per-user image and is writable at application runtime.  The application view of the true machine is masked from top to bottom, first by the per-user space and then by the installation image.   The application runs from above, looking down through the panes of isolation glass and since the middle layer represents what was written during profiling and since the top layer of glass starts clear, the initial application view of the machine is what existed at profiling.  The application believes it is installed - when it is not.  Each user gets their own top layer of glass and it is this layer that is writable at runtime.  This way, if an application writes to isolated spaces at runtime, that write is held in the per-user space.  Put it all together and applications not written for clean execution on a multi-user system can run without conflict.  More, the per-user pane of glass stays with the user's profile and can follow them from session to session.

The above applies to both disk and registry.  It also applies to COM objects and the systems named addressable items like PIPES and named semaphores.  

Lies, damn lies and statisticians:

The execution image (middle layer) is not really there.  Instead the isolation system lies and tells the application that the installation image is present on the machine and this reduces the amount of stuff that has to be brought down to the execution machine to run the application.  Many applications, particularly big ones, only reference a small portion of the stuff they install.  More classic computer science stuff says that if you can put off copying that data to the execution machine long enough, maybe you can avoid it permanently. 

Now - you'll notice a few important things.  Files that need to be there aren't and the isolation system has to do stuff to make them look like they are there and eventually, when actually needed, it has to make them really there, pausing and resuming the executing application while filling the missing content.  Worse, complicated things like Short File Names exist on the profiling machine and these names need to be maintained all the way to execution machine even if the TRUE short file name on the execution machine doesn't match the one that was used during profiling.  The application must see the SAME name no matter what and there is no reliable system API to make this happen.  The list goes on - bottom line is that there's lots of work to make these layers work and this means that there is lots of testing needed to prove that it works.

Proving it works

Once the execution cache becomes "full", very few runtime cache fills occur.  If you're in the business of testing the cache fill logic, this is no good.   Consider "stress" test in one of the Citrix labs.  The test is 100s of servers, 30 users on each server with all the users all running a variety of streamed applications.   The test then runs for about 24 hours and if anything ever fails, the test stops and your phone rings.  "They don't call it the stress lab for nothing!".

How can testers get the cache empty when the cache is being filled?

Answer: Backdoor logic to tell the streaming service to purge the cache before the application starts. Variations of this exist to control flushing all of the layers in the isolation system and to control "when" the flushing occurs.    This backdoor logic is controlled via command line arguments to RadeRun.exe.  There's one trick.  RadeRun.exe is not directly used for testers or users to launch applications. Instead, the applications launch via the web interface or PNAgent.   

Quandary:  If you don't "run" RadeRun directly, then how do you give it command line arguments? 

Answer: Registry key

HKLM\Software\Citrix\Rade\RadeRunSwitches (Reg_SZ)

RadeRun still accepts directly provided command line parameters, but it also checks a registry key for additional parameters.  With this, you can give command line arguments to the Streaming Client launcher even though you're not directly using RadeRun to trigger the execution of an application.

Evertything so far has discussed "why" RadeRunSwitches exist.  We can finally get to what they are. RadeRun.exe command line parameters

Specify using registry: HKLM\Software\Citrix\Rade\RadeRunSwitches (Reg_SZ)

c    clear execution cache before app opens

C   clear execution cache and per-user cache before app opens

d    clear execution cache after app closes

D    clear execution cache and per-user cache after app closes

e    Pre-fill everything into the execution cache

x    launch cmd.exe inside of isolation when launch the streamed application

Example contents: "-C -x"

*Are they documented or undocumented?

Now that I've written this, they are documented.  That said, in theory, with the exception of -x, they aren't needed much or more precisely - shouldn't be needed much.   The sections below will give more color to each of the switches;  what they do and where they should be used.

-c

Useful for the Citrix test groups to cause high-exercise for the isolation systems cache fill logic. Not really useful for customers as it results in all application launches being a first time launch, and first time launches are "slow" compared to a second time launch.  

-C

Same as the lowercase version, but clears BOTH the execution cache and the per-user cache.  People still tend to code this as -c -C when running the switches.  It actually makes more sense to me that -c should control the execution cache and -C control the per-user cache, but that isn't how it is.

-d

Useful for the Citrix test groups to cause high-exercise for the isolation systems cache fill logic.  In some cases, this one is useful for customers. If you have "secret" stuff that is part of your execution image and you want it "gone" after the application terminates, then -d can be an answer.  I'll add that the execution cache is DACL protected and users who are not running the application "right now" can't see it.  That said, some folks have good reason to be paranoid and this switch tells the streaming client to purge the cache when application terminates.  Notice that -d, like -c will cause the next launch to be "a first time launch" every time, and first time launches are "slow" compared to a second time launch.  

I normally recommend avoiding -c and -d and instead using a post-exit script to delete the just the smallest amount of secret stuff.  This way, the application launches are not first time penalized and the secret stuff is gone after execution.   The post exit script deletes the secret stuff and the next launch brings it back with a runtime cache fill.  Presumably, the secret stuff is small compared to the whole app.

-C

Same as the lowercase version, but clears BOTH the execution cache and the per-user cache.

-e

No longer needed.   In my first rounds of "documenting" RadeRunSwitches, I didn't even put this one on the list.  -e was created during development to allow the isolation system to run applications even before the cache fill logic was coded.   Today, it is sometimes used to diagnose a suspected cache fill error - or more precisely, a suspected "escape" from isolation.  If an application "works" with -e and doesn't work without it, then it implies that something isn't being isolated right and needs to be diagnosed. I do not recommend the use of -e in production systems but it can be useful in debugging applications.

UPDATE (04-Apr-08): I have been told of a worthy use of -e.   If an application is run with -e enabled as a part of a maintenance activity, then the entire cache can be filled; and then -e turned off.  This as a means of ensuring fastest possible launch time for users even if no user has ever logged on that client machine or server.  Application Streaming still does all the central publishing and applicaiton isolation stuff, but the streaming part not really used.  Interestly in this scenario, RadeRun.exe can be used directly.  Its fun to see how stuff gets used.

-x

This is my absolute favorite of the bunch and the real reason for documenting the switches. 

Notice that -x has nothing to do with cache management.   When profiling and testing profiled applications, it is often useful to have a command prompt or other utility running next to the streamed application and "seeing what the app sees".   Adding CMD.exe to the application profile and then publishing it is one way to get this accomplished.  This was common during the early days of Application Streaming development - until the developers got tired of repeatedly publishing a command prompt with every created profile.   Adding a switch to the RadeRunSwitches allows very quick addition of an in-sandbox command prompt for any streamed application.  This simplifies diagnosis of failing systems because you don't have to ask the person that created the profile to go back and change it to add debugging information.  Regedit to add the switch, launch the application and "poof!" a command prompt inside the same isolation environment as the launched application.

TIP: If you're debugging more than one application at a time, it is useful to use the "title" command to label your command prompts.

If you run regedit, from the command prompt, then regedit will see the view of the system that the isolated application sees, handy.  Do know though that you need to not have regedit already running for this to work because a second instance of regedit kicks the execution over to the first and then terminates.

Killing sandboxes: If you're running applications and particularly if you're writing scripts for your profile, you need to know that the sandbox does not terminate until all of the isolated applications in that sandbox terminate.  The started command prompt is "part of the sandbox" so it too must close for the sandbox to end. 

Wrap up

I hope this information is useful.  If you have comments or ideas for other switches, post here or let me know.

Joe  Nord
Product Architect Application Streaming, Citrix Systems, Fort Lauderdale, FL

Application Streaming stores stuff isolation layers.   The main one that holds the executable content is stored in the "RadeCache" directory below the Citrix Streaming Client installation directory.  The sub-directory name is a GUID that uniquely identifies the execution image.  There is a corresponding user layer that is sandwiched on top and the application at runtime views the machine through the 3 layers of isolation. 

Here's a picture of the layers.  
The majority of the applications installation image is in the middle layer.  This layer is "actually" stored below the RadeCache directory, below the installed to directory for the Streaming Client.

It can be big - Customers commonly want to "move it". 

Setting the location of this directory is easy; run the ClientCache.exe utility that is included with the streaming client; done.   It's even documented in KB article: http://support.citrix.com/article/CTX115137

The ClientCache utility does a few jobs

-          Sets a registry string to tell the Streaming Service where the Cache is located

-          Creates the directory

-          Adds a DACL to the directory to give the Streaming Service privilege to write stuff to the cache.

-          Note that the streaming client service actually runs on a dumbed down account and without this DACL, even though it's a service, it lacks privilege to write to anything important.

Deploy folder

Consider offline "streaming".  How do you "Stream" when disconnected from the company network?

Answer: Everything that would normally be on a central store to support the streamed execution of the app, is actually copied onto the execution machine.  At runtime, execution content is "streamed" from this local copy into the execution cache, as it is needed - just like the online case.

The directory, \Program Files\Citrix\Deploy holds all the profiles copied to the execution machine to support offline execution.  Neatly, the streaming client core doesn't distinguish between online and offline.  It just knows where the execution cache is located (Installation/Execution image) and runtime populates stuff into the cache.  The source for the runtime populate when offline is actually a CAB file local to the execution machine; a nit.  It keeps the architecture simple when the client doesn't have to worry about online vs. offline. 

Changing the location of the deploy folder

We already covered changing the location of the RadeCache, this is easy.  Use the ClientCache.exe utility, done.   

Imagine my surprise when someone asked me about setting the location of the Pre-Deploy folder; there's no ClientDeploy.exe utility!  Hum.

Can it be done?  Sure.  There's no utility to do it for you, so it becomes a manual process - documented below.

Configuring the location of the directories

Key directories:

-          The RadeCache location defaults to \Program Files\Citrix\RadeCache.

-          The PreDeploy location defaults to \Program Files\Citrix\Deploy.

The Streaming Service (RadeSvc.exe) queries the location of each of these key directories by reading strings from the registry as part of its startup logic.   

Here are the registry keys that set the location of the RadeCache and PreDeploy folders.  These are both stored below HKLM\Software\Citrix\Rade.

-          CacheLocation

-          PreDeploy

Changing the location of the Deploy folder can also be done, but it is a manual process.

How to change the location of the Deploy directory:

1)      Create a directory

2)      Fix the registry string to point to the new location

3)      Give the Streaming Service Full rights to the created directory

4)      Terminate all presently streamed applications

5)      Unload and Reload the streaming service

Example steps from a command prompt:

-          net stop radesvc

-          Mkdir C:\NewLocation

-          cacls c:\NewLocation /E /G Ctx_StreamingSvc:F

           Alternate to above is to MOVE the existing folder, and its contents.

-          reg add hklm\software\citrix\rade /v PreDeployDir /t REG_SZ /d C:\NewLocation

-          echo y| reg delete hkcu\Software\Citrix\Rade\Offline

-          net start radesvc

Finally, PNAgent  - Right Mouse Button, refresh applications - and the new location will be used for the Deploy folder.  Depending on how the administrator has published the application, it may be necessary to actually run an application before the "bring it to deploy folder" logic kicks off. 

The above will work with Streaming Client 1.0 (Presentation Server 4.5) and with Streaming Client 1.1 (Presentation Server 4.5 HRP1).   There will be some changes in this logic going forward and when we get there, I can update this post with the particulars.
Joe Nord

Product Architect - Application Streaming.  Citrix Systems, Fort Lauderdale, FL

Some customers want to store the RadeCache on USB sticks.    Is it possible?  The answer is: Yes, but it requires jumping through a few hoops.  The background on the problem and the steps to pull it off are described below.

Background 

With Application Streaming, the general idea is to runtime populate execution material onto the machine - and to execute that content from the runtime populated cache.  The cache is called "RadeCache" as that is the base directory where all execution content is stored.  There are actually two; a main one which is shared across all users on the machine and another that holds the per-user isolation layer.  For this discussion, we are most interested in the common one that holds the majority of the execution material.

How is the runtime cache populated?  The execution content is held centrally on a file share, or if you prefer the fancy word, the "Application Hub".  There is no Citrix code running on the file share.

Here's a picture that conveys the general architecture.   The highlighted pieces are specific to Application Streaming.  The rest of the components are the publishing infrastructure common with XenApp (new name for Presentation Server).

Challenge:

Some customers want to store the RadeCache on USB sticks.   

The primary reason I've heard for doing this is to use Citrix for application publishing and as the central point of application updates, but the customer is more interested in isolation than they are in streaming and, here's the kicker, the network link to the home office is a 1200 baud 1985 vintage modem and they don't want to move execution content across that link!  Publishing information is okay, just no gigabyte execution images.   More: They also do not want a file share (Application Hub) at each remote office.  Another possible reason is to have the execution content for an application follow a user as they - and their USB stick - move from machine to machine at the office.  Whatever the reason, folks want to do it, the rest of this post describes the "how".

How to store RadeCache on USB StickFirst thing to know is that most USB sticks pre-formatted when you buy them and are prepared for the FAT32 file system.   This makes them ready to use right out of the box and makes them compatible with the largest set of computer systems. 

The Application Streaming code will not isolate anything formatted FAT32.  It assumes this is user document space and leaves it alone.  If it doesn't isolate that space, it can't store the execution image there and from a "before" view, this means that you can't store the RadeCache on removable media - but that's not the complete story.  The steps below show how to format a USB stick for NTFS and how to tell the streaming system to use that stick for storage of the RadeCache.  Interestingly, even when formatted NTFS, the isolation system will still not isolate user documents stored to the stick as the media is removable and the isolation system leaves removable media alone.

Back on subject - what has to happen to store the RadeCache on a USB stick? 

Step 1:

Format the USB stick NTFS.  The steps to do it are documented rather nicely, here.

Step 2:

Tell the Streaming Client that the RadeCache location is on the USB stick.  Note: This must be done using the utility below and not with registry edits.  A DACL is applied to the directory that gives the Streaming Service user account permission to write to the directory.  Without the DACL, runtime cache populates will not occur and you'll get an error messages on failed cache fills.   The "why" is that the streaming service actually runs on a pretty dumbed-down user account as compared to local system.  It can only write to certain places, like the RadeCache.

Start / Run: C:\Program Files\Citrix\Streaming Client\ClientCache.exe

Browse to the USB stick. Tell it where to create the directory. The utility must CREATE the directory.

Step 3:

Reboot to have the change take effect. 

If you're impatient for reboots, terminate all running streamed applications and from a command prompt issue "net stop radesvc" and "net start radesvc".

That's it!

If you find this useful or can describe other use cases where this can be of value, I'd like to know of them.  Append here for all to share.

Joe Nord

Product Architect - Application Streaming Citrix Systems, Fort Lauderdale, FL 

Application Streaming shares the publishing infrastructure with the rest of XenApp™ (the new name Presentation Server).  Whether the application is published as "Stream to Server", "Stream to Desktop"  or "Streamed for off-line usage",  the publishing infrastructure for Application Streaming is the same as for traditional server hosted applications.   It has been around for years, customers like it and it works. 

SoftGrid® (Microsoft Application Virtualization) can also be published for execution on a Citrix infrastructure.  The server side case is the most common, but SoftGrid can also be executed on a connected desktop system and published via Citrix.    One method of running SoftGrid streamed to desktop is by publishing the package as "content".

• Access Management Console: Publish application, application type as "content".   The ".SFT" file is the content. 
• User launches the Citrix published "app" and the SoftGrid client kicks in via File Type Association to run the sequence.
• There are some "gotchas" such as what if the SoftGrid client isn't available, but it works. 

Dual-mode is what makes things interesting.   Dual-mode streaming

A real value of the Citrix Application Streaming solution is dual-mode streaming.  Since the publishing system and execution system communicate, it is possible to attempt to stream, but automatically fallback to a server side execution if streaming is not possible.   If the client execution system includes the Streaming Client and if all the pre-launch requirements on that machine are met, the application will be streamed.  If client side launch requirements are not met, then the application will be delivered via ICA connection to XenApp.  The user get's their application delivered on the optimal execution platform. 

This capability is normally associated only with the Application Streaming feature of XenApp and is a good plus for administrators.  It is possible to implement dual-mode for SoftGrid as well and the remainder of this post describes how to do it.   The key is scripts. Scripts are your friends

Application Streaming scripts are powerful.   Scripts are not text - they are binary and are "anything" that can be run using the Win32 CreateProcess() API.   This includes BAT, CMD, VBS, EXE and perhaps even MSI, though I can't conclude a good use for that last one.    Scripts can be run BEFORE application execution or AFTER the application terminates and can be run inside or outside of isolation.  There can be numerous scripts per profile and scripts can even be stuff that is not directly executable such as data or DLLs.  The important item here is that scripts travel with the profile and can be run "outside isolation". 

To achieve dual-mode Citrix streaming with the SoftGrid agent, a few steps are needed.

1. A pre-launch script run outside of isolation to confirm the presence of the SoftGrid client
2. An application that does nothing should be used as the application to run.  A BAT file with "exit" in it would do the job though an improved version would be a Windows GUI .EXE which has no actual display and just terminates.   Being a GUI app, no command prompt window will come up and go away, so the flicker will be avoided.  For the remainder of this document, I will call this application "DoNothing.exe".
3. A post-exit script needs to be created that kicks off the SoftGrid client. 

The Streaming Client generally considers scripts to be a short lived operation.  The idea is that prelaunch scripts are quick, the application takes a long time to execute and then the post-exit clean up can occur, which is also quick.   To get dual-mode SoftGrid, we have to change that timeline so we spend almost no time on prelaunch ("1") and running the published application ("2")  and  then most of the time executing the post-exit script.   Conveniently, the streaming client is patient and places no time limits on how long a post-exit script can take to execute.   

The job of the pre-launch script is to verify the presence of the SoftGrid client.  If it's there, return code should be set to 0 and the Streaming Client will proceed to launch DoNothing.exe.   If the SoftGrid client is not present, the Pre-Launch script should return non-zero (aka one) and this will trigger a fallback for dual-mode streaming.  This assumes that the application was published for dual-mode in the Access Management Console. 

The last step is the post-exit script that actually triggers the execution of the SoftGrid agent to run the application.  This script would be added to the profile along with the pre-launch script and both would work to manage the SoftGrid based operation.  In a simple form, the post-exit script would be the .SFT file and file type association can take over to run the sequence.   

Making it elegant

A more elegant solution will pass the name of the sequence to run as a parameter during publishing in the Access Management Console.  This way, a single Application Streaming profile could be used to publish an unlimited number of softgrid sequences.  The name of the application shown to the user and the icon can be adjusted in the AMC and the parameter to specify the SoftGrid sequence to launch can be specified during publishing.  
When the parameter arrives at "DoNothing.exe", it must store the parameter away so it can be accessed during the post-exit script.   I haven't implemented this elegant version, but it should work. 

If sounds interesting please give it a go and post back your results, I'll watch this space to help cover the gotcha cases. 

Joe Nord

Product Architect - Application Streaming Citrix Systems, Fort Lauderdale, FL