I was recently on a panel in Stockholm moderated by Brian Madden and Alessandro Perilli where, with Mike Neil, Avi Kivity, and Lance Berc, I enjoyed an engaging discussion on the future of virtualization.  The event was a great opportunity for the audience to dig into different products and vendor strategies and Brian and Alessandro allowed none of us to wriggle away from the real issues.  Definitely the best panel I've been on - and all due to the two moderators. 

Brian is one of a rare breed that I am seldom lucky enough to meet.   He's about 20 (probably permanently), super smart, and incredibly engaging and friendly. He dresses cool and trendy.  He is an excellent presenter and author. He also knows more about Citrix products than many people at Citrix, me included.  He has found and filled a key niche in the Citrix ecosystem for no-bullshit, hands-on, been-there-and-done-it technology and product expertise that has won him the respect of Citrix and its customers. As an independent advocate to our customer base he has developed a reputation for demanding answers, and getting them. 

So Brian's latest piece predicting the end of Xen should be seen in precisely that light - he's beating the hive with a stick to see whether bees will come buzzing out.   His logic is a little blunt - hence the stick analogy - but I value tremendously his approach  because it gives me an opportunity to clearly state the Citrix virtualization strategy, so that customers understand why an investment in Xen and XenServer is sound.  I guess it also drives clicks to brianmadden.com, which can't be a bad thing, eh Brian?

Brian predicts the end of Xen based on the following claims:

1. Citrix XenServer has "literally zero percent market share"
2. VMware has dominant presence in enterprises,
3. Microsoft Hyper-V will take over SME virtualization, so Citrix will abandon XenServer
4. The open source community will abandon Xen in favor of KVM

His argument is superficially appealing, but unfortunately completely wrong.  That said, I'm grateful that he has highlighted the need for Citrix to clarify its virtualization strategy, and I hope that this long, but rather rapidly prepared response goes some way towards doing so. 

1. XenServer has zero market share:  Setting aside Brian's challenging hyperbole, the XenServer business has been approximately doubling quarter on quarter since its first release - "approximately" only because the transition from XenSource to Citrix meant a new CRM system, new support system, new channel management, wrapping our heads around Citrix licensing and a ton of other overhead that took us a while to figure out, but which will accelerate our business in the back half of 2008. We have somewhere approaching 4,000 enterprise customers, and about 3000 trained channel partners.  (Watch for the inevitable follow up from Mike D at VMware who is still out to prove that he finished high school math).  VMware claims 100,000 customers.  Citrix has about 220,000 customers and about a hundred million users.  The XenServer market share is small, and growing as rapidly as any such product can given the current VMware brand status, and the fact that we started well behind them.  We had a few key blockers for enterprise adoption, four fifths of which are addressed in our forthcoming XenServer 4.2 release.

A recent analyst report that I saw gave us about 4% market share (by customers), which would seem a tad generous, but not far off.   But XenServer is just starting to show up in the market. HP and Dell have just started shipping XenServer embedded into their server hardware.  And with HP stating that XenServer HP Select Edition is HP's preferred embedded virtualization platform there must be something in it.  Yes, this is embedded virtualization done right - In each of our announced OEM deals, our partner gets to offer their customer the tremendous value of  a powerful virtual infrastructure component "built in", while adding their own differentiated value propositions on top.  And uniquely we charge one fixed price per server, not per core or socket.  We are proud to be a completely hidden component of a ProLiant server that is managed using HP's powerful multi-hypervisor VMM 3.5 (also manages VMware and Microsoft), integrated with HP Lights Out management and Systems Insight Manager, and with a compelling graphical ProLiant Virtual Console, whose UI is like my Mac.  After all, a key HP value prop to their customers is out of the box manageability. XenServer is also available integrated into Dell PowerEdge servers and soon with Dell OpenManage, and it is embedded into the NEC Eco Center and other x86 servers, x86 servers from Lenovo, and Egenera PAN Manager for Egenera servers, which is also resold as Dell PAN Manager. Each offers a powerful value-added, XenServer-powered customer experience that is simply not available with VMware or other products. And then there's Marathon EverRunVM and EverRun for XenServer - both of which offer five nines of availability for VMs. (Can others do this? No). Stratus Avance-- a perfect mid market appliance for business continuity on a pair of industry standard servers.  XenServer inside.  Avance is also sold by Dell (gosh, that's right: XenServer is inside three key Dell enterprise virtualization offerings). (Can other vendors do this? No).  Finally, most recently and strategically profoundly importantly, Symantec's Veritas Virtual Infrastructure (VVI) - an incredibly rich software offering from a major enterprise IT vendor that combines the incredible depth of the Veritas storage management suite with the powerful, open storage repository model of  XenServer to create a high-end enterprise offering that application level availability, and performance and management integrated with the industry's leading storage management solution stack. (Can the competition do this? No). 

The army of enterprise class competitors to VMware is growing.  XenServer is inside and it is compatible with Hyper-V. And it's not going away.  It's just that we allow our partners to make money from virtualization, and we don't demand to be the sole value proposition.  What a novel idea!

2. VMware owns the Enterprise: Ignoring Brian's second hyperbole (we all know that VMware has signed up many large enterprises to their ELA), remember that only 10-15% of servers are virtualized, and there is a long way to go yet.  Evidence of imminent change in this regard came a week ago on an ACM sponsored panel with the director of IT for a major F50 who said "We have more Xen in production than VMware.  An open architecture gives us greater scalability and control, and we cannot afford to rely on a single vendor."  There you have it - VMware, for all its success, has inherent weaknesses - the largest of which is its one size fits all, single vendor sells all mentality.  Its arrogance with customers and the channel, and the paucity of rich value-added ISV offerings around it all demonstrate their vulnerability.  It is VMware that will be the Netscape of Virtualization, and not Xen or Citrix.  We have never "wanted it all" and by having an approach that is founded on the notion of a rich, competitive choice of value-added offerings, we explicitly acknowledge that we will partner in key areas, and add value in others.  Citrix is after all a partner-centric vendor.

Our largest partner is of course Microsoft.  And we share with Microsoft and Hyper-V an entirely different view of virtualized infrastructure, one which is embodied in XenServer and Hyper-V: A virtualized infrastructure founded on fast, free, compatible and ubiquitously available hypervisors and a rich ecosystem of value added partners that address all customer needs. So Brian's statement is simply a statement of current market status.  Sounds big and bold, but nothing really new.

3. Microsoft will take the rest of the market:    It is important to state yet again that we are not in a competition for server sockets with Microsoft.   If that were the case, why would we have helped Microsoft to make Hyper-V a better hypervisor, by developing the shims and drivers that will allow Linux to run with optimal performance on Hyper-V?   The founding thesis of XenSource, and the continued strategy at Citrix, is to promote fast, free, compatible and ubiquitous hypervisor based virtualization.  If the hypervisor is free, why worry about who delivers it?  Let the customer pick the implementation method that they want - the real money is in the up-sell with products that make virtualization valuable for customers. 

I personally view Hyper-V as one of the most compatible implementations of the Xen architecture out there, and we guarantee that XenServer and Hyper-V are 100% compatible at the VM level.  So why not drop Xen and go with Hyper-V?  Xen is ahead of Hyper-V in scalability, cross platform support and crucially, support from a rich ecosystem of vendors all of whom stand to benefit from commoditizing the hypervisor, and continuing to make Xen great.   Xen and XenServer offer the industry a ring through the nose of a bull - called Microsoft.  We have a rope through that ring and we tug it from time to time to ensure that the bull goes the right way.  But when the bull charges (as it will now with the RTM of Hyper-V) we will step nimbly aside and watch the effect on our friends at VMware.

Who will use Hyper-V?  Over time, a lot of customers, starting in SME.  Will Hyper-V eclipse XenServer's footprint? Again, yes, over time and often for different use cases.  Microsoft still needs to extend the architecture to deal with live relo, offer better resource pooling than simply re-using Microsoft Cluster Server, and figure out how to deal with storage, but they will do a good job over the next few years. Do I feel threatened by that?  No.  Every implementation of the fast, free, compatible, ubiquitous hypervisor architecture offers us an opportunity to up-sell the customer with rich value-added features. Remember - we're the guys who made the hypervisor free!  Not for nothing have we been beating Microsoft about the ears to get the darn thing to market - it will offer us a terrific opportunity for up-sell.  With what? 

• XenServer Platinum today is already hypervisor agnostic. It can instantly boot and run a VM on Xen, Hyper-V, VMware and even bare metal, with a streamed VHD that offers better performance than local disk.  It even allows us to boot a thousand VMs from a single VHD, which is key to scalability for XenDesktop. 
• Oh, and then there's XenDesktop, our powerful VDI offering that presumes the availability of fast, free, compatible, ubiquitous hypervisors...  I see I'm starting to repeat myself.  XenDesktop is a key area of focus for Citrix, and the inclusion of XenServer as a technology feature is the right way to go about delivering desktops as a service.  No need to be a virtualization administrator - delivering desktops is hard enough already.   Doing it VMware style you have fights between the Desktop guy and the Virtual Center administrator about why a user's desktop disappeared, and no way to figure out why it all went wrong.
• XenApp (Presentation Server) on XenServer already offers a factor of five better performance than on "a leading virtualization vendor's product" (they'd sue me if I used their name, and it doesn't begin with M).
• And yes, there's more coming we haven't spoken about yet.

Bottom line on this point: XenServer today and in the future offers a powerful, Hyper-V compatible enterprise virtual infrastructure that is simply a component of many of the industry's most powerful virtualization offerings, including our own XenDesktop and soon XenApp.  XenServer at the platinum level today extends Hyper-V with powerful value-added features to address some of its key enterprise scalability needs, particularly in dynamic management of storage.  And it will soon be able to be managed by System Center VMM (or any other DMTF equipped management tool).

4. The open source community will abandon Xen for KVM:  Brian, please don't take offence, but I suspect that for many Microsoft MVPs, the workings of the open source community may be a little bit of a mystery. First, there is no such thing as "the (singular) open source community".  [The Xen community] is independent of any vendor, large, growing and vibrant. It is quite different from the [Linux kernel community], though we share a few developers.   Xen is a cross platform, type 1 hypervisor, OS neutral, that runs on ARM based PDAs, x86, PPC and SGI Itanium super computers.  Xen is in BIOS offerings from major vendors, available at point of sale on just about every x86 server, and embedded in network and storage chipsets and appliances. It has tremendous performance and leads in IOV hardware support and performance.  Xen is available from the major Linux Distros, but also in Sun xVM and Oracle VM in their virtualization offerings, and they are very committed to their products.  Xen is or soon will be available embedded in every x86 server.  Xen is used as a standard test on every AMD and Intel CPU, and is the foundation of the planet's largest deployment of virtualization, at Amazon.  Xen is present in or runs every major cloud, including Google (who recently began to host the open source project to develop a Xen equivalent of VM Safe).  And Xen is quietly emerging as an embedded hypervisor on laptops and PCs. At the Xen Summit last week, we had representatives from 12 countries, 14 universities, and 47 vendors - totaling about 175 core developers.  The research and work in progress indicated that the Xen project is stronger than ever, and far deeper and more powerful than "virtualization in Linux" - a role that I'm sure KVM will evolve to fill very well.   The Xen ecosystem is participating in the benefits of secure class 1 hypervisor based virtualization, while still being able to take advantage of the incredible innovation in Linux (which some vendors use as a parent partition / driver domain).  And of course in spite of their marketing fluff, the Red Hat engineering team continues to be a significant contributor to Xen (and we love them for it)  and Red hat has a seat on the project's advisory board. 

KVM is interesting, but rather late to the party - I think the world has already decided that hypervisor based virtualization is the way to go.  That said, KVM is arguably more convenient for a Linux distro - because they don't have to get their hypervisor from xen.org and their kernel from kernel.org, and combine them.  I like KVM a lot.  Great for Linux based developers too. But there's nothing really new in KVM - the technology has been in products such as Microsoft Virtual PC/Virtual Server, Parallels and so on for quite a while.  It's just a VT/AMDV driver added to Linux to allow it to host additional VMs.  Great if your usage model is "first install Linux, then use your Linux skills to install VMs".  None of our customers want that, though I'm sure the adopters of Xen in SLES 10 or RHEL 5 are comfortable with the model.  Unfortunately the KVM project isn't yet addressing any of the other key requirements for virtual infrastructure (virtualization-aware shared storage, snapshotting, cloning, thin provisioning, HA, and much more) it is just another way to do CPU and memory virtualization ... at a time when Xen already offers Linux a typical overhead of under 1% (SPECJBB), and a rich set of value-added features.  To be honest, I'd love to see the two projects working together to have a single extension code base to both core virtualization technologies, but the kernel.org folks are solely Linux focussed (good for them) whereas we tend to look at all OSes with a requirement for OS indepencence.  I also have a strong suspicion too that by the time the KVM folks are done, they will have discovered that they have to re-design many of the core Linux resource management algorithms to take account of VMs, so it's going to be a long and slow road. (There was a new performance comparison of Xen vs KVM at the Summit, which couldn't complete because the KVM test kept crashing, but I've seen earlier work that indicates that KVM performs very respectably).  Can the two projects co-exist? Of  course.   And Linux now contains the key paravirt_ops API for secure dynamic binding to a class 1 hypervisor - Xen, Hyper-V (we've built the shim) and of course VMware, who collaborated on the API development.  Ultimately the open source community will have the benefit of both approaches - type 1 and type 2. 

There is one intriguing aspect of KVM, namely its likely adoption by one or more Linux Distros. I'm looking forward to hearing how those vendors explain the value proposition of a product that contains two incompatible virtualization technologies, and no management framework other than a thin veneer of an API that is incompatible with the ([DMTF]) accepted industry standard for management of virtualization. [Added note, with thanks to follow-up poster: I forgot to mention that there is a project under way to provide the relevant DMTF profile support, but it's a long way off, and there's rumored to be an effort to address the VM incompatibility issue.  It would be a great step forward for open source if these projects would accelerate, and it would be a great way to accelerate the adoption of fast, free, compatible, ubiquitous virtualization by the Linux distros.]

Summary

1. XenServer is growing strongly on its own, and XenServer is inside the industry's most differentiated enterprise virtual infrastructure products from leading vendors.  It offers the greatest price performance, compatibility with Hyper-V, and a rich ecosystem of customer-focused value props for management, availability, dynamism and flexibility.  Get it in your next server, completely built in.
2. VMware has a strong lead, customer confidence, significant presence, and expensive, over priced products that demand customers buy into a single vendor, proprietary, closed, monolithic architecture that has significant architectural and scalability drawbacks.  Next time your VMware sales guy calls, ask him for some of the free stuff, or better, just download XenServer for free.
3. Microsoft Hyper-V is a fine implementation of the Xen architecture, compatible with XenServer.  It has a strong development plan.   XenServer today and in the future will extend Hyper-V to address advanced use cases, and to support specific Citrix/Microsoft product partnerships.  A good example is XenDesktop.  There will be others.  System Center VMM is a great product, and will be able to manage XenServer as well as Hyper-V and VMware.
4. The Xen community is vibrant and growing.   Xen is widely used, massively deployed, core to the product and corporate strategies of some of the industry's largest vendors.  Predicting its demise would be extremely naïve.  KVM is very "in":  interesting, incompatible and incomplete.  The two projects can co-exist independently and happily. 

Now, back to my day job!  And thanks again Brian for banging on the bee hive.  Beer on me.
 

I'm just back from the Burton Catalyst conference in San Diego, which featured a superb track on virtualization.  One of the highlights was a talk on the challenges in security resulting from virtualization, by Alessandro Perilli of virtualization.info.  If you haven't seen him present before, make sure you do.  Unbiased, insightful, technical and superbly articulated.  The Burton team did a great job too, including sessions on storage, licensing, management and other key issues in virtualization. T

oday Hyper-V has finally RTM-ed.  Congratulations to Mike Neil and the Microsoft team on a job well done, and welcome to the world of hypervisor-based virtualization.   With this release comes vindication of a core thesis of the Xen project: that as an OS independent, open source reference standard hypervisor  Xen could transform the architecture of enterprise virtualization through a commitment to fast, free, ubiquitous and compatible virtualization. Microsoft Hyper-V shares a common architectural heritage with Xen , and is thus a welcome addition to the family of products that adopt this approach.  Citrix XenServer guarantees VM compatibility with Hyper-V, and in addition serves a set of use cases that effectively extend the Microsoft platform.  In the near future expect Citrix announcements for its portfolio of value-added products that extend Hyper-V to deliver powerful virtualization enabled solutions to enterprise customers.

Today at the DMTF interoperability bake-off in San Diego we also introduced another component of our virtual infrastructure toolset, Project Kensho.   Kensho showcases our commitment to open standards based virtual infrastructure management using DMTF CIM based interfaces, and will in the not too distant future allow Microsoft System Center VMM to manage XenServer.   It also allows users to quickly and easily export their virtualized workloads to and import them from the new industry standard portable virtual machine format, OVF.   You'll be hearing much more about Kensho and its features in the near future.

The OVF standard, which I was fortunate to be able to help to develop  offers ISVs and enterprise IT staff a hypervisor-independent portable virtual machine format that packages a complete application workload with its  resource requirements, configuration and customization parameters, licebnsand signatures to facilitate appliance integrity and security checking, as an open standard. Virtualized data center workloads captured in OVF format can be installed and run on any DMTF compliant virtualization platform. OVF also supports software license checking for the enclosed VMs, and allows an installed VM to localize the applications it contains and optimize its performance for a given virtualization environment.At the DMTF interoperability event, we used Project Kensho to create VMs from VMware, Hyper-V & XenServer in the OVF format.  We also used Kensho to import and run OVF virtual appliances on XenServer and Hyper-V.  Kensho will  allow application vendors and IT users to produce virtual appliances once as "golden application templates", independent of the virtualization platform used to deploy them - and is a clear demonstration of how Citrix will add value to Hyper-V. 

One other cool feature of Kensho is that it can use Citrix Workflow Studio based orchestration to provide an automated, environment for managing the creation of OVF packages and the import and export of OVFs from any DMTF capable virtualization platform.  The reason this is important is that Kensho will therefore directly plug into Microsoft System Center, with Power Shell bindings, and also that the workflows themselves can be customized to our partners' and customers' environments using WFS, to include additional policies, processes and interfaces.  A technical preview of Project Kensho will be available for download in Q3.  

In a posting on his blog, Chris Hoff laid into some comments I made to security SearchSecurity.com, in which I remarked that "Virtualization vendors [are] not in the security business." 

He quotes me as saying "While virtualization vendors will do their role in protecting the hypervisor, they are not in the business of catching bad guys or discovering vulnerabilities, said Simon Crosby, chief technology officer of Citrix Systems." and then goes on to berate me for that position.  He says "The fact that the "industry" has "decided" that "third party vendors are required to secure any platform" simply points to the ignorance, arrogance and manifest destiny we endure at the hands of those who are responsible for the computing infrastructure we're all held hostage with"

I reckon that Hoff, who is normally fairly clued-in,  has put the smoking end of the cigar in his mouth before thinking through this argument.  He's horribly confused, but as smug as always, so let me clarify what I said, and what it means. 

What I said is that Citrix is not a security vendor for guests of the virtualized infrastructure. We do not spend our days and nights looking for evil types that wish to attack guest OSes by looking for virus signatures or other security techniques.   That is not our business, and never will be. There is a strong and vibrant ecosystem of security vendors whose job it is to protect guest operating systems in physical and now virtualized infrastructure.  There are challenges that arise as a result of virtualization, and we and those vendors will work to fix them, but it is not our role to specifically protect any OS or its applications through OS/app specific knowledge in the virtualization layer.  The industry has long looked to third party vendors to add security to infrastructure deployments.  This is why vendors such as Symantec and McAfee exist - as customers' preferred partners to implement security for their apps/OSes.  The same will be true for virtualized environments.

In terms of the hypervisor, we are manically focussed on security, as is VMware - though they appear to be more retrospectively focussed on security, judging by their incredible rate of patches (more than one per week, on average). Xen supports TPM, AMD SVM, and Intel TXT, and trusted platform boot using platform based attestation is on the roadmap. Xen does not contain drivers, and implements a multi level secure architecture. The Xen community is putting Xen through common criteria level 5 certification, which is way beyond the typical enterprise software EAL 2, or even VMware's EAL 4.  Xen implements the features of IBM sHype, and has benefited from contributions of Xen security modules from the NSA and other key security research groups and agencies.  Xen is open source and is available for inspection and testing by the community at all times, so bugs found are quickly fixed and vulnerabilities, should they exist, are rapidly explored.  Xen is massively and continually tested by the community and there are scores of university research projects related to security that use Xen and work on Xen, including honeyfarms, Xen virtual appliances for security and more.  

The largest virtualization deployment in the world, Amazon, uses Xen, and more Xen hosts face the Internet every day than VMware hosts, simply because Xen is open source and available.  Xen is used in most major clouds too, and those folks really care about security.  The community is are justifiably proud of the security record of Xen and its open approach to security research and vulnerability assessment.

The security of any Xen vendor's product is simply up to them.  Citrix focusses very heavily on the security of XenServer.  it is tiny, often embedded in read only flash on industry standard servers, doesn't run any network services except for a single secure protocol, and enforces security principles of MLS throughout.   We are proud of the fact that we have only ever issued 3 hotfixes for XenServer, two of which were in beta periods.  Compare that to VMware's 48 patches for this year alone!  How anyone can consider software that has to be patched at a rate of more than one patch per week to be enterprise class, let alone secure, escapes me.

But we are not in the business of specifically securing guests or their applications, other than through offering a secure virtualization platform.  Even VMware with VMsafe simply exposes APIs to third party security vendors, so that customers can choose their preferred security partner to secure guests.  I think that the VMware Determina acquisition was very smart, and that hints to me that VMware sees itself having a greater role in the security of guest OSes, since it could choose to be in the vulnerability checking business without 3rd party security vendors, but thus far they are working very openly with the ecosystem. 

In summary an assertion that the virtualization platform vendor has to fix the sad state of the OS/App world by making it secure is demanding too much.  It would mean that we have to be experts in every piece of system software including all of the vulnerabilities of all OSes and their apps.  In my view the reason the state of security is poor now is because of the monolithic approaches of traditional OS and app vendors.  We will focus manically on our layer, make it secure, tiny and bulletproof to attack in its own right.  And we will work closely with experts in security of OSes and Apps to give them an opportunity to implement guest-level security outside the guest, through privileged interfaces that themselves are secure.  

When I talk to customers about their initial experiences with virtual desktop deployments (VDI for VMware users), they have three key concerns:

1. Complexity of the solution
2. Cost per desktop
3. User experience

At the most fundamental level, the ROI of a VDI deployment will be negative if users reject the solution because of poor performance.  Most VMware VDI end users  that I talk to, tell me that their user experience is "nowhere near that of a PC".  We think we deliver a compelling desktop experience with Citrix XenDesktop, which you can download here.  Of course XenDesktop (which includes XenServer) is also optimized for Microsoft Hyper-V and fully supports VMware - so you get the best possible user experience independent of your virtual infrastructure.

But at the virtual infrastructure layer the heat is on, and VMware has made another clumsy attempt to inject FUD into the market in the form of a blog posting by Eric Horschmann of VMware who attacks the ROI of XenServer or Hyper-V based deployments of virtual desktops because, using ESX's memory overcommitment feature, he managed to boot many more VMs on ESX than on XenServer / Hyper-V. 

Roger Klorese (XenServer product marketing, and one-time product manager at VMware) corectly identifies the fallacy underlying the VMware claims:

"What do you think happens when those pages start to un-share or users start to load up different applications, as people start doing real work? How big do you need to expand those balloons, and how much do you have to starve those guests, to keep your 5:1 memory allocation? And if you can't balloon 5:1, how much do you further degrade it when you start using the hypervisor swap file?"

There's no such thing as a free lunch, and in VMware's case there isn't a free hypervisor either.  When you overbook memory excessively, guest performance takes a hit.  Not only will the hypervisor have to start swapping (so much for the  claims that ESX is a lightweight hypervisor - it still contains swapping, which is an OS feature), but the guests will also start to swap.  We have observed many occasions where ESX performance hits the floor because the hypervisor has to swap in memory pages just so that Windows can swap them out!    

Several independent users have chimed in - a welcome addition to the debate.   In a follow up to a CRN article on the topic, Stan Kasper writes:

"My experience has been that the memory sharing features in ESX place a heavy burden on performance. In fact, to optimize performance I disable the PSHARE option and do a fixed allocation of memory for each VMWare guest. PS My initialze test on the beta Hyper-V vs ESX for disk performance is that they are about equal, and maybe Hyper-V is a bit faster. But do not read to much into this as benchmarks are rather a finicky science."

Though overbooking and common code page sharing are different things, even overbooking impacts performance, and causes major headaches and additional complexity and latency in suspend/resume and live relocation operations.  But assuming for a moment that VMware's memory overbooking and PSHARE are flawless and impose no performance overhead, then you can get a good idea of the performance per guest by taking the CPU speed of the server and dividing it by the number of guests.  Though the CPU speed is not offered in the VMware "analysis", let's assume it's a dual core 2GHz server with 4GB RAM.  So each of Eric's Windows Desktops gets about 50 MHz of CPU.  Even with double the CPU, that's only a 100 MHz PC.  No wonder users are underwhelmed by their performance!  

I conclude that VMware's flawed focus on defending the price point of its hypervisor, and thereby maximizing dollar take per server, is in direct conflict with the customer's goal in any Desktop Delivery project - a great user experience with terrific ROI.

Getting back to ROI, it appears that VMware also fails to understand that ROI is a solution-based analysis (not a hypervisor based one).   The right way to calculate ROI for desktop virtualization is to compare the overall cost per desktop of a complete solution that delivers great user experience.  One key piece of the architecture that is missing from VMware's "pseudo ROI claims" is the storage cost. Citrix XenDesktop, with XenServer Platinum, can boot up to 1000 VMs from a single Windows golden image.  That's a factor 1000 less storage than the VMware "VM Sprawl" approach, and a factor 1000 less effort to patch and manage desktop workloads.  And it doesn't have to be stored on a SAN - VMware's typical storage deployment.  A thousand SAN based VMs will cost an awful lot of money.  With XenServer / XenDesktop you can use any storage repository.  For example - in XenServer 4.1 (download the beta here), we have direct integration with NetApp's ONTAP API to leverage array-based snapshots and cloning, and to use their thin provisioning and block dedup technologies. So the real cost of the SOLUTION is what counts. My friends at VMware, heavily addicted to their SAN based storage architecture, drive customer acquisition costs for virtual desktops through the roof. Bottom line: Until you look at an overall solution cost per delivered desktop, you don't have an ROI case.

The bottom line: VMware's "ROI analysis" offers neither an ROI comparison nor any analysis.  But it does offer valuable insight into the mindset of a company that will fight tooth and nail to maintain VI3 sales at the expense of a properly thought through solution that meets end user requirements.  The very fact that the VMware EULA still forbids Citrix or Microsoft or anyone in the Xen community from publishing performance comparisons against ESX is further testimony to VMware's deepest fear, that customers will become smarter about their choices, and begin to really question ROI.

Scott Lowe, while stating that he isn't a Xen expert, has several issues with Xen, it seems. Not that he's inherently anti-Xen, but honestly responding to the wealth of hyperbole that's out there (from all sides).  So his is not an attack, but a reasonable challenge and request for clarification.  Since what I'm about to say is also relevant to some follow up to my previous postings about the enterprise readiness of Xen and XenServer, here is a nutshell version of why we are not simply trying to copy VMware, but deliver virtualization as a component of an application delivery stack.   Before I respond, it's important to note that though Scott phrases his comments in terms of Xen, they have nothing to do with Xen per se, but actually our product, XenServer.  Indeed, some of the comments to my previous blog are about Xen, and not XenServer. 

 Also, before I address specifics, let's just deal with the concept of "enterprise ready" once and for all.  The largest deployment of virtualization on the planet uses Xen, and that is at Amazon.  Is Amazon not an enterprise?  Every enterprise has a unique set of requirements, so when someone says that XenServer is not enterprise ready, my response is to ask what specific features are missing, and to prioritize those for development.   I specifically reject statements by VMware resellers that we are not enterprise ready just because we have a different form factor of the product.  But I do accept that we don't yet have all the bells and whistles that VMware has, and so that means that some people won't buy our product.  We certainly do have every feature that VMware had for its first $1BN of revenue.   But here's a good example of something we don't yet have: certification against EMC Clariion storage arrays.  Anyone care to guess why?  Fortunately with the help of our ecosystem partners, and with the Citrix portfolio of add-on features, we have an incredibly compelling offering - one that is suitable for most enterprises and that addresses both native and virtualized workloads.

Memory: Scott points out that VMware supports up to 256 GB memory per host and says that we support less.  Actually that's not true.  Our testing limit of 128GB/host ($40K of memory!) is just that.  If someone wants to put more memory in the box, will the system work?  Yes. The architecture scales directly to 4TB. Does anyone care?  That would be a very expensive box.  I'm not sure yet, but likely XenDesktop will exercise a lot of host memory in some implementations.

Paravirtualization: Scott states that VMware also supports paravirtualization, and also points out that paravirtualization is useless for legacy guests.  Yes, VMware with the vmxnet driver supports what we'd call a PV driver. So, rather than decry Xen for equivalency, Scott should acknowledge that Xen led the industry in development of paravirtualization, and helping to optimize OSes for virtualization (including, through XenSource and now Citrix) all future Windows kernels, which are "enlightened".  We've always argued that the basic engine over time should be  (a) equivalent (arguably identical and compatible) and (b) commoditized, in all offerings.  So Scott, we take your arguments as a powerful argument for Xen's innovation and creativity, and acknowledge your thanks to the Xen community for helping VMware to find a better model for the future development of its hypervisor.   I think I was  the first to welcome VMware to the era of Paravirtualization.

Where we (XenServer) do have a very significant edge, and Scott doesn't have all the facts is that .. with some careful work one can really optimize the performance of legacy Windows on Xen's PV hypervisor - which we've done. So we do a very good job for Windows virtualization. Not all Xen implementations do, by the way - we regard that as proprietary value add.   I also repeatedly hear about lack of HA etc in XenServer.  Well, I found a customer using NetScaler as an application layer HA tool for XenServer recently, and this is a perfect example of why XenServer, with the Citrix portfolio, offers a richer environment for customers than simply jamming every known IT infrastructure function into the virtual infrastructure layer.    It is also true that XenServer is still a follower to VMware VI3.  Hey, they've been at it 10 years and we've been going for 3.  But at a core functionality level, we are absolutely there, and the relative differences are rapidly diminishing.  Working with a rich ecosystem of partners is the right way to finish off the remaining gaps.

At a fundamental level there is a key difference between the way we build our product and the way it is factored for market, versus VMware's: We have an open, pluggable storage architecture with no clusterfs, but support a range of storage back-ends including tight coupling to virtualization-aware storage infrastructure, such as NetApp arrays and the ONTAP API.  We have a powerful partnership with Symantec, and there are more storage vendors to come. We like it, storage vendors like it, customers like it because it doesn't break existing storage management procedures, values the contribution of the storage infrastructure vendors, and fits in with what customers have today. We don't have a centralized management console that purports to be the god of all things virtual, and therefore represents a single point of failure. We do have a redundant management architecture that explicitly endorses 3rd party plugins, and we go to market with a strong ecosystem of ISV add-ons, rather than excluding them from the opportunity.  And we're not a systems management vendor, so we're delighted to plug into the management console of choice of our customer - System Center, Director/Tivoli, HP VMM/OV, BMC, CA - you name it.  We haven't ever shipped a hotfix, and VMware ships more than one per week.  In my view, the patch rate is a very significant statement about the failure to meet enterprise quality levels on the part of VMware.  I'm not saying we have no issues, just that we haven't needed to ship any patches yet - and we've had the ability based on our relatively smaller footprint, to deliver 2-3 releases per year.

We offer much better price performance, and a portfolio of products that extends the value prop of virtualization up the app delivery stack, and through the network itself. From an end to end perspective, we have more 'stuff' to make the user's use case work than VMware does, and therefore have a very powerful set of offerings. 

Finally, we offer solutions that address both native workloads and virtualized workloads - through XenServer Platinum which includes dynamic provisioning for both native and virtualized workloads, "any-ness", and comprehensive workload management while dramatically reducing VM / image sprawl.  VMware does nothing for the 90% of servers that today are not virtualized, other than promise to virtualize them for an exhorbitant fee.

That's about it.

With both my Citrix XenServer and Xen community hats on at once, I have to salute another piece of creative writing from the VMware competitive misinformation department.  This time it's about CPU architectures and live relocation (which we call XenMotion, just because we can).  The piece states that we don't check CPU compatibility on migration of a VM between servers.   This is ridiculous, because if we didn't VMs would be crashing about our ears when we do live relocations.  Both in XenServer and in open source Xen, we require a match on CPU processor vendor, family and stepping before a migration can be performed. 

I'm beginning to understand why the VMware blog is called "A little truth...".  While I'm all for competitive intelligence in general, when the code is available in open source for everyone to review, it's just not possible to pull the wool over everyone's eyes like this.

And though I know this has hit the wires before, I keep hearing from VMware customers that are buying XenServer why they love the simplicity and robustness of our product and of Xen: The total number of patches shipped to date for XenServer: 0.  Total hot fixes and patches shipped last year for VMware VI3/ESX alone: 68*!* of which 17 were critical.  Silly me! Those were probably only "feature enhancements". 

Mike DiPetrillo, a competitive marketing engineer at VMware has an interesting rant aimed at Citrix, Microsoft and XenSource, reported via virtualization.info.  I have a lot of respect for my friends at VMware, but I'd rather expect those such as Mike who are charged with gathering competitive information to spend less time breathing their own exhaust and more time working on the facts.  With nonsense like this being pushed into the VMware marketing machine as the result of "competitive research", who knows what may come next? 

Mike, I think you should write more "research" like this.  Spend a lot of time doing it.  Every time you or your colleagues begin to foam at the mouth, more of your channel partners come back home to Citrix, fed up with being treated shoddily and competed with,  and tired of the dogma that seems to be VMware's response to the powerful Microsoft Citrix partnership, its breadth of features, its loyal channel, and its solutions centric approach that values ecosystem partners. 

We know you don't get it.  That's fine.  Chin up, and do please keep writing. 

Ashlee Vance of the register has unfortunately mis-told the story about XenServer and Symantec Storage Foundation.  So just to ensure all the facts are out there, this is the story:

XenSource signed an important partnership with Symantec about 9 months ago, that would allow us to integrate Symantec Storage Foundation as an included volume manager in XenSource's XenEnterprise product.    The deal was based on the realities of the XenSource situation at the time - namely we were an independent company with our own routes to market.  The Symantec Storage Foundation would be included for customers at no additional charge, when they purchased XenEnterprise.  Also included in XenEnterprise is a standard Linux based volume manager, LVM, as well as storage repositories for VHD based virtual hard disks on NFS, and iSCSI "LUN per VDI" storage.

When Citrix acquired XenSource, our route to market changed - or rather was amplified - by the Citrix channel partners.  The Symantec agreement, which we still view as critical to our ability to address the massive installed base of storage without requiring customers to change their processes, training or technologies for storage management, is now being re-drafted to expand its scope and to ensure that there is complete alignment between Symantec and Citrix in the market.  A key part of this is ensuring that the channel is trained to distribute, install and support our product and all of its component technologies, including those of our partners where relevant.  Although our integration work against Symantec Storage Foundation is complete, we have not yet completed the work necessary to ramp the channel training, support and certification with the additional Symantec storage management capabilities, so it has been omitted from the XenServer 4.1 release. 

Ashlee inferred that this is all about Citrix trying to make money from the deal.  The contrary is true.  We're keenly interested in helping our partners make money from what we do with XenServer, because it divides the overall value pie more equitably across the ecosystem and thereby strongly incentivizes the channel and our partners to deliver more Citrix product. 

The omission of Storage Foundation from XenServer 4.1 should not be read as any change in the strength or strategic nature of our partnership with Symantec.  On the contrary, Symantec is possibly our most strategic ISV partner, because Citrix is not, and never will be a storage management vendor and Symantec offers one of the industry's most powerful capability sets for managing the diverse storage infrastructure used by our customers today.  Our product is built to take advantage of the powerful storage management products provided by our partners, and in the area of volume management, Symantec is the market leader and a key go-to-market partner.  Suffice it to say that as soon as we have nailed down the specifics of how to go to market with Symantec now that we are part of Citrix, we believe our solution will be much more compelling than the closed, proprietary storage architecture of VMware's, and we are going as fast as we can to get it finalized.

It seems that somehow when we briefed Paula Rooney at ZDNet on the new Citrix positioning and forthcoming product release of XenServer, we managed to confuse her.  Matt Asay appears to have read Paula's blog as verbatim truth and injected further conspiracy theories of his own. So, what should one do when an influential blogger is out by 180 degrees?  Difficult, since Paula has been a great ally to the Xen project and XenSource, and has always stood by us.  So, Paula please accept my apologies for managing to confuse you, and permit me to point you in the right direction.  Reading your post it is clear you still have confidence in the Xen community and the Xen model, which is great to see.  

In summary:  Xen underpins all of our core product initiatives at Citrix. Here's a brief restatement of the core commitments of Citrix to Xen and its role in our product portfolio.

1. The Xen project is in great shape, superbly funded by Citrix and the community, and is operated independently from Citrix, by the Xen project Advisory Board.   Citrix has more than doubled XenSource's open source team size already, and is continuing to develop new initiatives for Xen. At the most recent Xen developer summit in December, we had over 200 attendees, and there was fantastic participation from across the industry.  Our own open source team operates independently from the product groups and has a blank check for headcount and resource.

2. Citrix XenServer is a core foundational product to Citrix.  Specifically, XenApp (formerly Presentation Server) and XenDesktop (formerly Desktop Server, addressing the VDI use case) will both include XenServer in all future releases. Why? Because XenServer is being optimized to run the XenApp and XenDesktop workloads, and provides a fantastic set of manageability, availability, scalability, and flexibility options to the XenApp/XenDesktop administrator, with incredible performance (very significantly better than VMware's). Citrix customers don't like using VMware for virtualizing Presentation Server, because of the very serious performance penalty, but they need to virtualize it  for various reasons: test & dev flexibility, consistency of image management, DR, ease of provisioning etc.  XenServer offers them all they need, at much better price/performance than VMware.  XenServer 4.1 specifically contains optimizations for Presentation Server as a workload.

3. XenServer itself continues to go from strength to strength.  The new release 4.1 boasts over 50 new features and performance optimizations, and a profound and strategic tight coupling between the virtual infrastructure platform and smart virtualization aware storage, such as the NetApp devices.  Expect a range of exciting announcements as we move down this path.

Xen is profoundly important to Citrix, and is changing everything about the way that Citrix conceives of itself, develops and delivers its products, and competes in the market.  Citrix is fully supportive of open source and the community has embraced Xen and the Xen community wholeheartedly, and now offers the most compelling application delivery portfolio in the industry.  Virtualization for us is a capability set that is associated with the delivery of applications.  It is not a mind set as it is with VMware, who view VI3 as a hammer, for which everything in the data center is a nail.  We have a powerful product offering, a rich ecosystem and a deep partnership with Microsoft that makes us the natural choice for existing Citrix customers and any Windows user.  We have an end-to-end product portfolio, and a strong historical commitment to and relationship with our channel.  These differentiators will make us stand out in comparison to VMware, with its one size fits all, proprietary architecture.

Once again (it happens every 6 months or so) VMware's press gangs have pushed the "send" button on their spam engine looking for talent. They have spammed the entire XenServer team, and probably the entire Xen community work list, with promises of fulfilling, gainful employment, and on open source to boot!   Normally I just hit the delete button, but when they sent the exact same spam formatted email to Ian Pratt, who runs the Xen project, it made me laugh. Ian passed it on because he wasn't sure he was well enough qualified.  Keir (Xen project core committer) was also on the list, but similarly felt under qualified for the role...   

-------------------- 

To: Ian Pratt
Subject: VMware open source engineering positions 

Hi Ian,

This is Tom from VMware.  I noticed some of your contributions to the Xen community. I work at VMware in Staffing for the Core Technology Group.  I'm pretty sure you are not actively looking for a new position but I wanted to check with you to see if knew anyone that may be interested in interviewing to join our team.  Most of what I look for in candidates is deep open source experience.  People that know low level C programming typically do well here.  We have an opportunity in one of our core R+D groups for someone that is open source savvy, can read low level C code, and is willing to handle Build Release  for this group.   I also have other opportunities that involve working with the open source community to help make Linux a better OS for virtualization. Ideally, these positions would be located in the US Palo Alto, CA or Cambridge, MA. For the Build release position, we may be willing to consider top talent being located in other facilities we have around the world. Again, I respect that you may not be looking for anything right now but if you're interested of know someone who may be, feel free to let me know. I look forward to hearing from you. 

Thanks again, Ian. 

Tom M

VMware Core Technology

Palo Alto, CA