The last several months we have been feverishly testing many different hardware server configurations trying to squeeze every last bit of performance from each of them. One of our main focuses have been towards improving MySQL performance and segregating MySQL users resources from one another so that a user that is abusing the system wont’ affect others on the same server.
The software side of this equation and related custom MySQL patches that make it happen will follow in a few days on this blog along with GPLed source code to help out. However, for this entry I want to focus on the hardware side of things.
How have we increased performance on the hardware side? We are putting in SSDs of course! SSDs or solid state drives have been getting a lot of attention lately – and rightly so! They are EXTREMELY fast. The problem with SSDs are that many aren’t compatible with certain raid controllers, many are slow, and many have extreme write delays. The ones we use have none of these problems. Even random writes which is typically the slowest stat for SSDs outperform any regular platter based drive we have ever tested.
To illustrated how fast our software and hardware combination is compared to just a few months ago I took a server with 500 clients on it and benchmarked it in every way I could. We then took five of these servers and combined ALL the users onto a single new server with our new platform with the SSD and MySQL caching system. With 2,500 users on a single system you would think MySQL would fall flat on its face. Instead it is anywhere from 5x to 100x faster in our multiuser environment.
MySQL used to be the number one process (and corresponding threads) with blocked I/O on our system in a 24 hour period. This had always been the case. On one of our new systems it doesn’t even show up in the top ten. It really is that much faster!
The only downside is that very few of our server setups have our hardware/software MySQL combination rolled out. We are ready to begin rolling this out to most of our servers over the next 2-3 weeks. For those users on servers that can’t be upgraded to this new system you will be migrated to new systems that can take advantage of this setup.
With a MAJOR bandwidth upgrade happening next week, the rollout of our new MySQL solution, and MANY custom kernel patches things are about to get a lot faster and a lot more stable!
I know for many of our users that were originally migrated from our old system to our new system that it was a painful process riddled with downtime. I am so sorry for those issues. The testing process we used was sorely lacking – the responsibility was mine and I fell short for those users. The migration process should be MUCH smoother going forward. If it is any consolation, all of the users that were migrated to our new system will be the first to get the benefit of the new super fast database implementation.
Thanks for supporting Bluehost and Hostmonster!
Matt Heaton / President Bluehost.com – Hostmonster.com