Shared hosting is a balancing act of epic proportions. Bluehost/Hostmonster/Fastdomain has thousands of CPU cores, petabytes of information (thousands of terabytes), and half a petabyte of main system memory. We try and distribute these resources as fairly as possible across hundreds of thousands of customers and close to 900,000 domains. Its a monumental task, but we have many tools to help us get up for the challenge.
We have some very good custom software that was developed in house to separate and segregate CPU allocation for users over a given period of time. Memory management is accomplished and monitored effectively by user, but there are a few areas where tools don’t exist or aren’t good enough for to do the job. Disk I/O management by user still falls short, although we are ALMOST there with respect to assigning all activity by customer. The killer for us right now and the bane of most shared hosting companies is MySQL!
MySQL is far and away the most popular database application for shared web hosts. There are many other popular and fast alternatives such as postgres, etc, but MySQL services about 95% of our customers database needs. The problem is that for whatever reason MySQL CAN NOT give us the proper detailed usage statistics we need. Breakdowns by user, and rows affected by user, and ACCURATE cpu time used by user (Not including wait times for blocked IO devices) simply doesn’t exist for MySQL. It is extremely frustrating that a database as popular as MySQL and one that strives be an an enterprise class database would lack these basic features. This one application can literally kill the overall performance of a server. So what can we do?
First, there are some patches that give some of these capabilities to the MySQL server. However, they aren’t actively maintained and they can’t be applied cleanly to the newest code bases of MySQL. As of today I have decided to have these patches picked up by paid developers and cleaned up and maintained so that they will work for current and future versions of MySQL. I will continue to pay to have these maintained and updated to serve the community as a whole. I will release the patches to the public so all can benefit from these changes (Even though they probably should be withheld as a competitive advantage for Bluehost and Hostmonster).
I will also put as much pressure as I can to have these patches added and made a part of the permanent MySQL source tree. It really is astounding to me that MySQL has not addressed these issues since day 1. If anyone has any strings they can pull at MySQL PLEASE have them consider what I have written. Turning their nose away from these problems won’t make them go away.
When these changes go live it will bring us one big step closer to having completely stable and high performing servers available for all our clients.
Matt Heaton / President Bluehost.com / Hostmonster.com