The argument AGAINST virtualization

It seems a day doesn’t go by that I don’t see another article written on the virtues of virtualization. For those that don’t know what virtualization means it is technology that allows you to run multiple instances of an operating system on a single server or on top of a system of clustered servers. Virtualization has been around forever. This is the method that many mainframes used to deploy software, but virtualization became popular for desktops/workstations in1998/99 when VMware was first released.

While virtualization techniques have improved dramatically in the last 10 years (Think 3D support, para-virtualization for direct access to the hardware layer, etc) there is a fundamental problem with the whole concept of virtualization that no one ever talks about. That is the issue of the HUGE overhead that comes along with having multiple instances of an operating system running at the same time for software that doesn’t doesn’t NEED to be run on different machines. This is best illustrated by an example.

Lets assume there are 100 units of CPU processing power available on 2 servers that are configured identically (From a hardware perspective), and that 10% of the system resources are dedicated to servicing the operating system running on these servers. **10% is a very very low number in my opinion, but I will use it to be on the safe side of this argument.** Lets assume that a given user/customer consumes 2% or 2 units of system resources each.

Server A – 100 Units of CPU
10 Units used for OS (Windows, Linux, OSX, etc)
90 Units for users/customers -

Server A can accommodate 45 users.

Server B – 100 Units of CPU
10 Units used for OS (Windows, Linux, OSX, etc)
2 Units for users/customers

12 Units per customer or a little more than 8 users per server.

45 users vs 8 users… Hmm…. Now I have taken several liberties with regard to my assumptions. To be fair, there are numerous techniques used to speed up the process of virtual systems that I have not explained but I think you get my point. Here are some important reasons to USE virtualization. Listed below are several cases where virtualization may be the best or only option you have.

* If you need to run disparate instances of OSs on the same hardware IE – OSX and Windows on the same machine at the same time.
* Testing purposes – If you want to set up an alpha, beta, live setup on the same server.
* Security reasons – Memory & cpu are segregated pretty well on most virtualized environment (Disk I/O not so much – in fact I think its terrible on vmware, parallels, xen, etc). A lot of progress has been made in this area, but its not even close on a fully loaded machine in my opinion and based on extensive i/o testing.
* Need to migrate virtual machines on the fly – This is a great feature that many vm products support.
* If you OFTEN have a need to dynamically change resources for different OSes then a virtual product may be good for you as you can change cpu/memory/disk resources easily and in many cases make these changes on the fly.

Here are some of the reasons against using a virtualized product.

* Overhead of all the multiple OS installs to deal with before you even run a single program.
* You have to do security updates/maintenance for every OS install you have installed. Just thinking about 10 instances of Windows Anything running on a server is enough to make any botnet operator salivate.
* ITS SLOW
* ITS SLOW
* ITS STILL SLOW

Virtualization has its place. Its a super important piece of technology, but it is being applied in many areas where efficiency is scrapped for convenience. I revile the idea of convenience over efficiency for a long term strategy, yet many companies are doing just that. If you are a company deploying huge numbers of virtual machines to control resources CPU/MEMORY/DISK then you are just throwing money away. In an industry where every penny counts why give your competition any advantage?

29 Responses to “The argument AGAINST virtualization”

  1. Brent2 says:

    These other companies also tend to charge a higher rate, allowing them to pay for less users on the box. This still comes with higher overhead in sysAdmins, data center space and support staff. In the end it’s often slower, even with fewer users.

    Aside from using it for testing, I’ve always avoided they’re use.

  2. Robert says:

    I run VMWare Fusion to test sites in Internet Explorer. Basically, I keep four or five snapshots of the OS: IE6, IE7, IE8, the base install with only service packs, and one that I use for running random programs. Your second argument against virtualization (endless upgrades) seems to be the biggest PITA for me. Not only does VMWare Tools seem to have an update every week, Windows seems to have an update every week at a different time.

    To make matters worse, Windows added a feature that will automatically restart your computer after updates are installed if you don’t click “Restart Later” within a few minutes. Since I’m often in several browsers at once, the virtual machine goes unwatched while I’m reviewing recent changes in Safari and Firefox. I’ll go back to review in IE only to find out the OS has restarted.

    I’m 100% with you. Keeping everything up to date is a major pain, and I’m not even running that many snapshots.

  3. [...] Matt Heaton: ‘While virtualization techniques have improved dramatically in the last 10 years (Think 3D support, para-virtualization for direct access to the hardware layer, etc) there is a fundamental problem with the whole concept of virtualization that no one ever talks about.’ [...]

  4. Russ says:

    I agree that the overhead that comes along with having multiple instances of an operating system running at the same time for software can be cumbersome. Who needs that?

    Efficiency has got to be the number one priority and then convenience – without a doubt.

  5. steve says:

    There is a huge culture being built around virtualization that is terribly misdirected. It seems like every admin I hire has the master plan for converting all our existing infrastructure over to a virtualized solution. The message I regularly work to get across is that while virtualization has its place, that place is not on a piece of hardware that is already maxing out its resources. The answers that I have gotten from my admin’s when I ask they “why” has lead me to believe that it has never occurred to them that installing a virtual solution on a piece of hardware does -not- multiply the functional capacity of the hardware.

    The moral of the story, at its most elemental level is that if performance is the goal, virtualization is not the answer.

  6. Jay R. Wren says:

    Matt,

    Thank you so much for having the guts to speak sanely even when it is not a popular viewpoint. I totally agree, but when one presents what you have presented in a clear and logical way, one is often met with resistence because its not the accepted standard.

    Virtualization does make sense in certain IT organizations where operating system or “servers” are use to segment tasks to control complexity. e.g. a windows Active Directory controller separate from DNS separate from DHCP separate from file services separate from web services. By splitting these across 10 servers that is a lot of hardware and each server probably has less than a 10% load on average. Grouping them together into 2 physical servers running virtual machines probably makes sense…

    That said, as soon as you need performance and capacity instead of a way to control complexity using VMs is simply the wrong solution.

    In many organizations the IT knowledge does not exist (and windows instead of linux makes this harder IMO) to support mixing all the installations. Even microsoft says don’t install SQL Server on your Active Directory controller. They sell Small Business Server specifically for smaller orgs that want to mix the two.

    Luckily, in the Linux world, administrators have full control to integrate everything and anything on a single physical server. Bluehost seems to do an excellent job at this. I commend Matt for doing it well. That said, buying more hardware and just getting things working is easy and it is usually cheaper than hiring a new guru person.

    @robert: I’m pretty sure Matt is talking about server virtualization common in many private it shops and other web hosts. Desktop virtualization definitely has its place and it sounds like it is working reasonably well for you. I recommend you go into windows update and tell it “download but don’t install” to solve your reboot problems.

  7. Ploni Almoni says:

    Matt,

    I too commend you for stating the plain facts. I have stressed to other IT administrators who have jumped head over heels onto the virtualization bandwagon and as a result seen performance plumet to keep their heads screwed on straight.

    Virtualization works well to take several severs that don’t have high performance needs but you for various technical reasons need to/want to keep separate and put them on one physical machine. It does a great job, fantastic. Imagine having a BES server, a GOOD server etc… They don’t need much but they just don’t play nicely together. Great instead of a half dozen machines doing virtually nothing you can merge them all together and still be safe.

    On the other hand (yes I’m a Windoze person) I have seen genius admins put an Exchange Server, a Domain controller, a File Server and a Terminal Server (which I point out is the same basic idea as virtualization in many ways) on the same machine and then wonder what is wrong. Just use your head, some things work well virtualized and others do not. Some items can work well on the same physical hardware where others can not. A file server doesn’t require much CPU but requires a lot of I/O. So it can go nicely with something that requires little I/O but a fair bit of CPU. But toss Exchange (I/O Intensive) and a file server on the same physical box and well you don’t need a PHD to guess the results…

  8. Ryan Gubler says:

    Nerd!

    Reading that was like eating vegetables with wheat germ for flavor. I think I should write the translation of it for the other 98% of bluehost clients that could not understand a thing you said.

  9. Ian L says:

    On the 10% of system rescources thing, it depends on what you’re running. If you’re doing a fancy-schmancy cloud application that’s developed for efficiency, you may be running a stripped-down operating system that requires just a few dozen MB of memory to do its job, when placed on top of, say, OpenVZ.

    The key here is to only run the stuff that NEEDS to be virtualized on a virtual machine. e.g. you aren’t running six coies of CPanel on six VMs when one control panel can manager all of the VMs centrally. The point with VMs seems to be about utilizing servers to the max with a variety of uses per server, versus having a dedicated server for any given task and trying to wring as much efficiency as possible out of that setup.

    There’s no doubt that virtualization for virtualizations sake is a bad idea efficiency-wise, but it’s very preferable to grabbing a power-hog lower-end dedicated server to run your custom application on…

  10. Tomoiaga says:

    I am sorry Matt but I don’t think it’s ok to treat a subject so vast like virtualization with such superficiality.

    As you stated in your post:

    “…there is a fundamental problem with the whole concept of virtualization that no one ever talks about. That is the issue of the HUGE overhead that comes along with having multiple instances of an operating system running at the same time for software that doesn’t doesn’t NEED to be run on different machines.”

    In my opinion no one ever talks about it because this has nothing to do with virtualization and it’s concepts but more with how people used it.
    If a bunch of people use virtualization the wrong way (or because of commodity) it doesn’t mean anything regarding virtualization and it’s fundamental problems. By using fundamental , problems and the virtualization words you need to have (a lot more) arguments that really regard virtualization and it’s concepts and not how people use it.

    As far as I can see you are talking about full OS virtualization, hypervisor based for example which people use in the so called cloud hosting or other names that do not mean anything , just to attract clients. Your superficiality here can be seen regarding this too.
    You do know that we have OS virtualization, like openvz for example where there is no HUGE overhead and there is no software that doesn’t NEED to be run (that software can be ignored).
    Yeah, you still have a little overhead with os virtualization but it’s very small.

    To cut it short, you wanted to tell everyone that you can do it without virtualization, in shared hosting (limit cpu, memory a.s.o.).

    Congratulations, you found cgroups and cfs in the linux Kernel.

    Which proves my point regarding the fact that this is related to how people use virtualization and can be considered an argument against using virtualization in shared hosting (not an argument agains virtualization) since the Kernel already knows how to do things (of course some mods need to be made but it’s no big deal).

    There are endless arguments against and pro virtualization and I do not think it’s ok the way you treated this subject, but I may have understood your words the wrong way.

  11. Zane Perry says:

    Why did you remove my question about Bluehost’s policy on File count? I will return the favor by filing a complaint with the better business bureau as your claims of unlimited hosting space are fraudulent. I tried to be civil :)

  12. Blake says:

    A few points you missed:

    Virtualization is great for silos and privacy. Quite often you can’t share a system for this reason, not to mention the security risks of sharing a single, long-lived operating system.

    If you can rebuild a virtual instance of a server (attaching permanent data at boot time via something like a SAN or Amazon’s EBS) extremely quickly, then you don’t really care about upgrades and patches on all your running servers, all you have to do is keep your set of “golden images” up to date.

    Virtualization is also great for cases where dynamic application scaling, and short-lived systems are used. Let’s say you need to scale an application to massive amounts of CPUs for a short period of time, there’s currently no better way to do this without distributing the application to multiple servers (horizontal scaling). To do this you’d design the application such that you can bring up multiple instances (dedicated to that specific application) and tear them down with little effort. Automation and configuration management are KEY here.

    Shared system resources are awesome for cases where thousands of users have insignificant needs for the majority of the time. Any successful application will out-grow this environment extremely quickly.

  13. Jimmt says:

    Actually your argument against is actually one of the best reason for. Let’s use SQL server as a good analagy. You have a SQL server with a production, archive, and reporting database. Maxed utalization of this server is 45 users average. With virtualization I could double, possibly tripple, that count and host additional databases by dividing the databases up into their own operating environment and allocating dedicated resources; memory, CPU, storage, etc… By doing this I also created an isolated environment incase a adhoc report is executed it only affects that one database and not the other three due to confined resources. Oracle, IBM, and Microsoft have benchmarks by stacking virtual machines on a host outperforms a single host install and increases user connectivity by providing isolation to each instance. IBM holds the world record of Exchange accounts on a single host and it was accomplished by “stacking” virtual guest on a physical server.

    The overhead of the OS is minimal and a good virtual administrator will fine tune each guest configuration and OS to ensure that the application/service is receiving exactly the resources it needs and running only the neccessary services for the application/service; no overutalization (most common) or underutalization.

    The argument against virtualization; actually more of a perception than argument, is that virtualization doesn’t always lowers TCO. In fact, in most cases it increases TCO as it increases software licensing. This is where you have to evaulate your TCO and determine if virtualization is the best fit for the service you are deploying.

  14. TJ Phippen says:

    In many cases multiple OS instances drastically reduce efficiency(your example). We use Parallels’ Virtuozzo which from my experience has done quite a bit better than others, allegedly by “reusing” some identical processes across the node/cluster when possible.

    Like everything, virtualization has it’s pros/cons and they should be carefully weighed. I think many in this industry aren’t seeing the whole picture, increasing their hardware expenses to meet the same performance they could expect without virtualization. The hype is no reason to virtualize :)

    On the other hand there’s many cases where virtualization is a great advantage, or even required to meet your needs. Cloud hosting is a great way to easily scale resources as needed across a redundant cluster. Guaranteed resource allocations and isolation ensure that the requirements of others do not reduce your performance. Being in the VPS business myself I see many who simply need root access without the expense of a dedicated server.

    @Brent2, I assume the “other companies” you mentioned are offering VPS, which is very different than shared hosting. They are more expensive because resources are generally guaranteed/not shared, and meet the needs of those who need more. That being said, Blue Host offers high-cpu hosting comparable to those “other company’s” prices.

  15. Brian Lacy says:

    I am disappointed that your post entirely ignores the end-user perspective. In fact, I see pretty much everything you’ve said as highly irrelevant.

    Essentially, your article discusses the complications and limitations of virtualization as an approach to hosting. But these are ultimately nothing more than cost hurdles, which are overcome simply by charging more for the service provided.

    A successful business may surely built by focusing on a niche, as BlueHost has done with shared hosting; however, a successful business grows by giving customers what they want and need. And not all customers’ needs are met by shared hosting.

    Frankly, VPS hosting should not be compared to shared hosting at all. Rather, it should be compared to Dedicated Server hosting. A dedicated server is an absolute necessity for many organizations, particularly those that:
    - Require greater control and flexibility over their web development environment
    - Tend to use their servers for more general purposes than merely hosting simple web sites, but still need public access to their server
    - Require a more secure environment than shared hosting can provide
    - Do not have the budget to purchase, staff, and maintain an in-house server or high speed connection

    But dedicated servers tend to start around $200 per month, which can still be prohibitive for small businesses. But you said yourself that a good server can host up to 8 VPS customers. Some simple math tells us that charging these customers $50/month (a much more attractive fee for small biz) is a worthy sum for a hosting provider, and you end up giving your customers exactly what they’re looking for at a price they can afford.

    In reality, it makes me wonder why EVERY host doesn’t offer VPS.

  16. eZaroorat says:

    Quite informative article. Was planning to go in for VPS hosting for my site but never thought about this aspect. Thanks.

  17. Georgie says:

    Get yourself a Sun 5140 with 128 GB or more of ram, then you can do some serious virtualization via zones. Sun’s zones don’t suffer from a lot of the issues you are talking about here. Yes, throwing multiple instance of Windows on a crappy x86 box is a losing proposition.

  18. Jon Hill says:

    I came to this site actually hoping for a valid argument against virtualization since I’m currently writing up a plan to move from a physical to a virtualized environment. All I found was a series of out date admins who have not had the gumption to actually try something new in a production environment and as such are opposed to a new, cutting edge technology on the merits of their own bias…

    “It’s slow, It’s slow, and it’s still slow” is not a factual argument. Having used virtualization in a production, development, and office environment I’ve found that your “arguments” so-called do not hold up.

    The key to virtualization is to use the current technology with the current hardware. You would not take a 5 year old machine, throw ten VMs on it, and then expect it to perform at the same level that it would in a 1:1 scenario. Which is the impression that your “arguments” so-called give.

    I can take a HP DL585 with 32 GB of RAM and 4 Physical Processors and run 20-30 VMs without even denting the resources available. I can run 40 VMs before it even appears to be taxing the system. I know this from having run 60+ VMs in a three-node ESX cluster with one of the ESX hosts not hosting any of the VMs. This was a production environment.

    Additionally your argument that “each user takes 2% of your cpu capability”…where are you pulling this from? In reality there is more that goes into serving an application than CPU. You have to account for RAM, Disk Space, Bandwidth, not to mention the application resources. In a real world scenario the application is going to run whether you have 2 users or 2,000 users. But a 2:1 unit comparison for users:cpu is ridiculous.

    So I would suggest to you that if you are going to try to make an educated argument either for or against something you come to the table with a valid argument rather than an opinion based in bias.

  19. Julian says:

    I can tell from your article that you don’t have a lot of experience with virtualization. As a manager of a virtual enterprise environment with well over 1000 VMs running together, I can say your arguments ring hollow in most respects. Virtualization is a tool. It is different from other hosting methodologies, and must be used properly. I have virtualized systems that are running just as quickly and just as efficiently as a physical server. These systems have SQL, AD, Exchange, IIS (and conversely CentOS/Redhat systems with LAMP) and they run great. Enterprise virtualization is not a 1:1 resource translation over physical systems. It is more. Virtualization has many benefits. Some of the primary benefits that we have seen in practice are the incredibly reduced footprint in our data center (on average, virtualized environment is only 33% of the original footprint e.g. 3 42U racks of equipment vs. 9 42U racks of equipment.) Another benefit is more efficient use of IDLE processing resources (Memory & CPU). Our costs in the virtualized environment are a fraction of the costs of the same environment in a physical topology. Virtualization is extremely convenient for real-world scenarios. OS’s can be built, configured, honed and then templatized into gold images which can then be deployed and re-deployed in countless numbers and countless times. This is an exponential improvement over the “traditional way”. Virtualization has afforded my users the ability to SELF-SERVE their desired environments without any interaction from the IT department. In a development environment this capability is invaluable. Once a VM is no longer useful, it is simply archived or deleted. In a physical environment we must re-purpose the physical hardware and re-install a new system , which takes time and costs labor. Labor, in case anyone forgot, is our largest expense (not equipment or memory/CPU or even “speed”). In a virtualized environment old systems are instantly removed from the pool, and the resources are instantly available for new applications of our choosing. In a virtualized environment, a VM can have a snapshot taken to freeze that system in time for forensics, peer review, or as a roll-back function. The benefits of virtualized environments goes on and on and on. It’s not a perfect architecture, but it has a great deal of value in countless ways. Those of us who are still reluctant to advocate its use (and I am STILL one of those people), but who have had the real world, hands-on experience of using these environments, would be equally level-headed about the topic. Now if you want REAL problems with virtualization, you can come back and discuss scenarios like what to do when a production server in the enterprise (that was otherwise completely healthy) is instantly corrupted by your hypervisor, and is unrecoverable (simply because the virtualized technology offering is flawed.) Or we could discuss a REAL problem with what happens to a virtual environment of systems when the storage supporting them gets disconnected or a LUN fills up on a storage array. These are some real heart-burn issues with virtualization. There are real concerns with virtualization that IT personnel should seriously consider before committing to the technology, but most of your suggestions in this article are not among them.

  20. Robert says:

    I like the idea of virtualization but I want to know what place it would have in a typical network. There are two likely reasons it will not be implemented here including lack of professionals and hardware redundancy

  21. dbs says:

    As one who has worked in critical realtime environments, I too am sensitive to speed and bandwidth considerations. (There’s a *reason* for using Realtime Unix OS’s.) Of all the virtualization approaches I’ve seen to date, the in-Linux-kernel KVM variety may be the most responsive.

    An alternative approach for servers is the concept of “physicalization”. FYI, I came across the following two articles from Ars Technica:

    http://arstechnica.com/hardware/news/2009/01/physicalized-servers-may-offer-virtualization-alternative.ars
    http://arstechnica.com/business/news/2009/11/basics-of-physicalization.ars/

  22. joe says:

    Virtualization offers very few advantages. Except for those who don’t understand it’s limitations.

  23. I agree that the overhead that comes along with having multiple instances of an operating system running at the same time for software can be cumbersome. Who needs that?

  24. I quite agree with that virtualization is really great if you know how to use it well enough. It really has a lot of use its endless possibilities.

  25. I am all for the future of technology but things are moving a bit too fast for me. I still don’t have an HD TV … let alone a 3D television! It’s hard to imagine the next step after 3D, for me anyway. While I do like being able to operate on a single system, muti-tasking does make the system quite a bit slower. I do use cloud hosting and I make sure to keep all my hardware updated.

  26. John says:

    The title is very clear. We are talking here about what is “against” Virtualization (not in favor). It is understood that the author should be bias. By the way, the discussion is very informative (both sides).

  27. lazar says:

    VPS versus shared hosting.

    Hi Matt,

    I am struggling these days to decide which one is more cost effective.

    Your pro package costs 24$/month ($20 currently with discount), and moves the account to a 32GB RAM machine shared with ‘only’ 300 other users (I was told by sales person). This means I will get about 100MB of RAM memory for usage after the operating system resources are deducted, right?

    On the other hand, some other companies provide VPS for the same price. I know they may not give SSH and some other stuff included in your pro package (but honestly, I am not interested in those), but this is an example of what I get for the same money
    hetzner.de’s vq12 plan
    (with current EURO/USD conversion rate)
    hetzner.de’s vq19 plan
    (with a bit older conversion rate, when dollar was valued more)

    Can you explain to me, because I am really confused, why I should go for 100MB of RAM, when I can go for the same money for 500MB+ after the operating system resources are deducted.

    I really look forward to your reply.

    Regards

  28. VPS are a basic need lately for me. Many thanks for the info.

  29. [...] have recently found their way into my newsreader:To virtualize or not to virtualize?Matt Heaton: The argument AGAINST virtualizationIt seems a day doesn’t go by that I don’t see another article written on the virtues of [...]

Leave a Reply