Hosting Nirvana – The Future of Shared Hosting!

Hosting is an interesting business.  It has come a long way from the free website days of Geocities and the like.  Speed, reliability, and features have grow far beyond what they were only a few short years ago.  What you can now buy for under $10 a month is 10x as powerful as what you could buy 5 years ago for the same money.  That is a good thing.

Hosting also has some short comings that still haven’t been addressed that hamper many individuals and companies with lower priced shared hosting.  I would like to discuss some of these short comings and how I believe they can be fixed and show how we think we have finally been able to achieve shared hosting nirvana :)

Listed below are the well known and almost impossible problems to solve with regards to shared hosting –

1) Resource Allocation – Because many users are on a single system it is hard to allocate resources properly.  You may have a very underutilized server (Resources going to waste) or an overcrowded server that can become bogged down with overuse.  It is impossible to determine the usage of a particular user in advance.  This causes unreliability on a server because a server can spike out of control at any time.  Common problems are too much disk i/o consumed in bursts by particular users, extreme short or long term memory usage by a particular user, or spiking or prolonged cpu usage by a small group of users on a particular server.

** What we can do about it –

To us it all comes down to instantaneous real time tracking (About every 15-30 milliseconds in our case).  Advances in general linux kernel tracking and our own propriety tracking have finally allowed us to know which users need the most resources.  We have then tried to automate as much of the process of allocating free resources in real time to these users as possible.  We are constantly updating these automated tools to make the process as seamless as possible.

2) Tracking User Consumed Resources – You can’t “blame” users or even monitor many user activities (Cpu use, or disk i/o usage) on shared servers because tracking user resources means you have to know what user ran what process.  The problem comes into play when you have major applications that don’t run as a particular user and instead are run as their own process with no “real” user tracking.  Let me give you an example of the two biggest problem applications in this area – MySQL, and Apache.  MySQL consumes an enormous amount of CPU and Disk I/O resources, but it normally runs as the “mysql” user.  This makes it very difficult to track resources used by a particular user.  Built in tools such as slow query logs, etc are extremely inaccurate in measuring disk i/o and CPU usage by user.  The other major culprit is Apache.  Apache runs as a separate user as well (At least in our case, and most web hosts have a similar setup).  Apache spawned script process such as PHP, Perl, etc can easily be made to run as a specific user for tracking purposes, but the Apache processes themselves and the corresponding cpu and I/O overhead is never attributed to a single user.  There are many applications that fall into this category, and all of them make tracking inaccurate and problematic for hosting companies.

** What we can do about it –

I have spent thousands of hours over the last 3 years working personally on this problem and I am VERY happy to report that we have nearly solved this problem (MANY thanks to kernel developers all over the world that helped out as well as the talented developers in house!!).  We have spent considerable time and money modifying the linux kernel and userspace applications (MySQL, Apache, etc) to report exactly which user is responsible for cpu and i/o usage in real time.  Lets give an example – Lets say we have user “matt” that does a MySQL query that take 2 minutes to complete (clock time) and 90 seconds of real CPU time to complete (Actually number of CPU seconds required to complete the query).  When MyQL passes the query to specific thread to be serviced we start tracking for that particular user the EXACT cpu time that was used, and the exact number of system reads and writes as well as device specific reads and writes.  We can use this to track and slow down the extremely heavy users in real time so that the server is calm and available for everyone to use.  We are in live testing right now on several boxes and hope to have the CPU portion of this rolled everywhere in the next 3-4 weeks.  The disk I/O portion of our code is already live on 90% of our system and will be live on 100% of our servers in the next week.  The importance of this can’t be overstated.  This is what shared hosting has needed forever, and what will allow it to compete with and in many cases top VPS in performance while maintaining stability in the system.

3) Immediate action on policy enforcement – This is a BIG deal with shared hosting!  When a specific user “violates” a policy like excessive CPU usage or disproportionate disk I/O or memory usage 99.9% of all shared hosting companies will try and alleviate the problem by killing processes or banning a user AFTER the damage has already been done.  It does no good to ban someone after they have consumed so much CPU that the server becomes sluggish.  The sluggishness or downtime has already happened at that point.  Most hosting companies have a very difficult time every determining where these cpu and I/O abuses are coming from let alone mitigating the problem before it happens.  Virtually NO SHARED HOSTING COMPANIES  have good options to actively slow down cpu usage – They usually just stop the offending processes (Not a good option), or kill processes consuming too much disk I/O – not a good option either.

** What we can do about it –

As mentioned above in item #2 we can now track and monitor cpu and disk I/O usage in real time for all our users.  Based on this information we can now do what no other shared hosting company has ever been able to do effectively.  We can limit disk I/O activity in real time and limit CPU activity in real time for all our users.  This allows us to mitigate the effects of sudden spikes in usage that would normally affect all other users.  Here is a good example to illustrate this point – Lets say we have 100 users on a server and that 99% of the time everything runs smoothly but one day one of the users makes it to the front page of digg.com.  One of two things are going to happen.  Either the user that is causing the excessive load is going to be shut down or the server is going to be sluggish or possibly down until traffic subsides to that site.  We have been in this position many times in the past and its never good for anyone.  The disk I/O portion of that problem is now solved for us, and the CPU issue should be ready in the next few weeks.  Instead of shutting the user down we can now “contain” them as if they were on a VPS.  We can isolate them from other users  so they don’t cause problems but still allow them to use any extra CPU cycles or I/O operations that are available.

**What does this mean?

What does this really mean?  It means we will very soon be able to offer the VPS experience for less money and less hassle than every other VPS product out there, and that our shared hosting product is about to become a LOT more stable than everybody else out there.  My opinion is that most VPS users really don’t need or want root access that requires their own time for security updates, Cpanel updates, etc.  They simply want a contained environment and guaranteed resource allocation.  We will be able to offer guaranteed resources just like a VPS or dedicated server solution without requiring any changes for our users.  I am very excited to see all this materialize as this has been my pet project for several years.  If you have made it this far in the blog entry I congratulate you for your tenaciousness in plowing through my technical ramblings!

Thanks,
Matt Heaton / President Bluehost.com – Hostmonster.com

39 Responses to “Hosting Nirvana – The Future of Shared Hosting!”

  1. Dan says:

    Bravo! Nice work Matt. I’m a proud bluehost customer! Keep up the great work.

  2. Jackie says:

    Bluehost is becoming more and more powerful.
    You mentioned in the last passage that most VPS user don’t want root access, I think it’s right. Only a few users who knows Linux will need root access in order to install their own applications.

  3. This will help a lot of other hosts, too, if it was released. I personally think of what your doing is ultimately perfect for any webhost these days. Your time and hard work will pay off with a “better hosting experience” for everyone.

  4. Josh says:

    This is great! Does this mean there will be no more CPU Exceeded errors, just sites that slow down when getting heavy traffice?

  5. I like the idea of tracking user resources for two reasons.

    1. Being on a shared server, I don’t want someone else slowing down the server and interfering with my website.

    2. I want to know if I’m using too much. I want to be able to fix my scripts so that they run better. I want to be able to better optimize my databases so they are faster. With your ability to track resource usage, will we, the users, be able to see a log of our usage/overage? I’d rather fix it based on a log than having my usage limited. (Hopefully I’m understanding everything correctly.)

  6. Matt Millard says:

    Great post, it was a joy to read both as a customer and sysadmin!

  7. This seems really exciting to hear. Can I also suggest that if this works well that you use it too also change the way the excessive CPU usage is handled with the users. I have had a couple of times where the first I hear there is a problem is the support staff are telling me my account has been terminated! That isn’t get Customer support. I know one customer I recommended to you guys left on the first time this happened.

    I think it would be much better if a warning was too take place and even a message to contact them and work through it to some degree. I am pretty stubben and have contacted support and worked it out but I am sure others have just up and left on the first sign of trouble (my friends did) so a little extra preactive care on that front could help to retain customers.
    JMTC
    Keep up the fine work
    Molly
    PS. Your support staff are gold!

  8. Jimbob says:

    Will you share? :)

    Since you use cpanel… maybe consider selling the technology to them, so other cpanel hosts can use it, too?

    What would it take for you to release this technology to other hosting providers?

  9. Renea says:

    This is very exciting news. I look forward to seeing this in action and commend you for the amazing amount of work this must have taken.

  10. Jay R. Wren says:

    I love that you do this kind of work. I look forward to reading more about it.

    I’d love for you to consider the save vigilance toward allowing your customers to legally use their web space for themselves. Your “support” people have disabled my service in the paste because of mp3 files to which I have legal right in my web share. These mp3 files were NOT publicly available. They were accessible via password protection only. There was no copyright violation. There was no DMCA violation.

    Will you ever support the legal use of copyright material via your services?

  11. Kraig says:

    Will the code be made available?

  12. vinicius says:

    Matt, congratulations! It’s very nice to see a CEO working on real problems.

  13. Sat says:

    We are thinking of shiting of data centre from a traditional type data hosting centre to a cloud computing ?

    Is that a good thing or not ?

    If the answer is yes, do you guys provide cloud computing service.

    Cheers

    Track and Locate

  14. [...] See more here: Hosting Nirvana – The Future of Shared Hosting! « mattheaton.com [...]

  15. I says:

    Hi Matt,

    I like reading the technical stuff that you write. It’s been one of the reasons I’ve had account on both bluehost and hostmonster for several years — seeing that you know what you are doing, and that this company is a result of an enthusiasm.

    I do have one concern though: why are our passwords stored in a plain text format? Password should be hashed !? I think I tried to discover an answer through the help center, but have failed.

    Regards.

  16. As a web hoster myself as well, CPU, Memory and Disk I/O are the most important things to deal with when ensuring the server performance and stability is meeting the requirements and at the same time majority of the clients on the same servers been handled equally well.

    These are the issues which we have being researching on and trying to work out some possible solutions as well for many years. Is good to hear that BlueHost managed to work out a solution for this matter.

    Well Done and Congratulations!

  17. Paul says:

    Matt, you are lucky, you have your $7 hosting, but what are you doing beyond your Walmart model? You are sort of a businessman, but not really. Instead of expanding your hosting, you are content in getting rid of clients instead of dicussing options, allowing growth and being a true businessman. Get your team together and find solutions to growth instead of punting clients over simple and insignificant matters.

  18. dave says:

    Hi Matt!!

    Sooo… for the users that consume an inordinate amount of resources, would there ever be a dedicated option so that they don’t flee to a VPS host? Any chance of getting an exclusivity option, where you could be guaranteed to be the only person hosted on that box? I would think it would cost around $100/month for something like that, if it were possible?

    Thanks!

    Dave

  19. Matt – this is great news – and I think a huge step forward – you’re giving us exactly what’s needed in a shared environment: freedom, and protection – at the same time..

    After many years of web developing, and intense use of wordpress, I’ve literally just setup my first bluehost account – and it’s very impressive – very smooth. Well done.

    – keep up the excellent work.

    Paul
    http://paulcoughlin.com

  20. Rishi says:

    Thanks for keeping us posted with this latest update. I’m glad you’re finally getting around to implementing something you and the BH team have spent so much time on. :-)

  21. Tyrone says:

    Thanks for the update. I’m very eager to see your VPS solutions. I’d love to be able to use Blue Host’s quality for Java-based hosting.

  22. Peter Pan says:

    This was really good news Matt, especially the automated resource allocation and sharing! My website only generates moderate traffic MOST of the time, except when some new event is published, and then I worry about everyone hitting the server at once. With my site having so much notoriety, this has had me biting my nails whenever some major new material was put on the site, as there was at least one time in the past when the surge forced your engineers to shut me down for a while. Well I’m putting up a really important update end of this week and expect quite a bit of mounting attention for a while. Hopefully the new methodology will help make it a good experience for my visitors, even if there is a temporary surge!

  23. Ian Littman says:

    Sounds cool. DreamHost actually appears to partition off Apache processes by user, but I know most web hosts don’t do this. Like the one who now hosts my main site, and does a very good job at it.

    As for VPS-like stability etc., how would that work in terms of getting more resources allocated to you, should traffic spike? I’d be willing to pay more for, say, a guaranteed amount of memory (a VPS thing I know, but something to bring up nonetheless) so if traffic did spike my site wouldn’t suffer.

    Additionally, wouldn’t two weak links in the whole performance equation be Apache and CPanel? How many people serve largely static content from their websites, and wouldn’t nginx be a nice option for serving such static content? Or is it mostly dynamic these days, with Apache thus required?

    As to CPanel, I hear it’s a resource hog :/. You guys probably get a screamin’ deal on CPanel and it’s what people know and love, but might DirectAdmin or some homebrew solution push server power just that much farther? Or is CPanel somehow hosted on a centralized system, so CPanel use gets taken somewhat out of the equation? I’d guess that this probably isn’t the case, but just throwing stuff out there.

    At any rate, I’m looking orward to having the option of a “partitioned shared” host with VPS quality with lower overhead and thus lower pricing. Though personally I have now grown accustomed to root access for more power-hungry applications. On the other hand, my personal website and all generally mission-critical stuff is run on a low-density shared host…

  24. Bill Gales says:

    Matt-

    I’m fully looking forward to adding this and promoting it on my hosting website, James told me about it earlier this week and I’m really excited. I’m very happy with the help I have received in starting this site from James.

    Thanks Bill

  25. Amer says:

    Is this new CPU feature available on BLUE HOST right now?

    I need to sign up to a new web hosting company for my online animation magazine. Most videos are embedded from other sources (youtube) and some will be placed on the server in Flash format, site will run on WordPress. Will this be a problem?

    I am ready to sign up right now if you have implemented this new CPU feature.

    Thanks

  26. Aaron Rice says:

    Very impressed Matt!

    Its features like this that keep me with BlueHost even though the latency to you is very high for me as I’m in the UK.

    Well done to you and the team!

  27. J.Urben says:

    May i quote you here Matt?

    “Advances in general linux kernel tracking and our own propriety tracking have finally allowed us to know which users need the most resources. We have then tried to automate as much of the process of allocating free resources in real time to these users as possible. We are constantly updating these automated tools to make the process as seamless as possible.”

    How about we simply suspend all acounts that have more than 50 000 files without warning – Than we tell our support to be especially rude to them so the get lost? Problem solved Matt! At least that’s what Bluehost does.

    Your new file limit that you implemented without even finding the time to make a notice in your tems of service are a joke at best! 50 000 files is nothing nowadays 10 cms will break the borderline without a problem.

    Ans what about the logistics here? I can have 2500 email accounts on my account with with 20 e-mails stored on each of these accounts? That will already crash through the 50 000 files limit! I get a lot more spam than 20 per day! But than i haven’t even start to build a single one of my websites. I only have ten. Ten very good websites, but they where a lot of work and i’ve spent hundreds of hours on them!

    Three of these sites sell our own software, which we have been developing for 4 years. Now we can’t even support our cliens anymore.
    And that all of a sudden without a warning out of the blue. no time to clean out and find out if something is wrong.

    I have 10 websites (CMS) on my unlimited account, that’s all! No crap, no unnecessary files! If you want to limit the accounts to 5 sites, stop calling them unlimited. It’s misleading at best!

    Why doesn’t your TOS tell us anything about the 50 000 file limit? on an unlimited hosting account? I have a copy if you care to read it. There is nothing in there stating any limitation.

    Sorry but i am very frustrated and my account is still suspended and no answer from support….

  28. Thank you for taking the time to keep us updated on what you are trying to do. I look forward to seeing the final product and pricing.

    As you said, I just want something that is more reliable and lets me do a little more, I don’t want to go to get a crash course in server management.

  29. [...] Sunday, June 14 2009 @ 10:39 AM Eastern Daylight Time Contributed by: Admin Views: 0 The post on Matt Heaton’s blog is a little old but it is worth a mention since it is some exciting news for customers of BlueHost [...]

  30. I am a new customer of blue host and till now, everything is great! There were no hassles to set up the wordpress account and the email account.

    Thanks for taking your time to explain in detail about the initiative taken by blue host to give uninterrupted service. Since we are in network infrastructure domain, we can appreciate your efforts. Good job, and hoping to see the same good experience in future too.

    excITingIP.com

  31. VPS is great solution and if implemented will result in higher reliability and robustness of sustem as a whole.

  32. [...] spent thousands of hours (according to himself) developing the system and introduced this so called Hosting Nirvana (One might argue DreamHost has been doing this for some time – but technically, it’s [...]

  33. cheap uggs says:

    Additionally, wouldn’t two weak links in the whole performance equation be Apache and CPanel? How many people serve largely static content from their websites, and wouldn’t nginx be a nice option for serving such static content? Or is it mostly dynamic these days, with Apache thus required?

  34. [...] Posted by Solokron Look at this little snippet I ran into today.. You really should give credit for the source of your quote when you rip an entire quote off a website. Also, here is a link to a discussion about Matt’s blog [...]

  35. Shad Vick says:

    Ok – great article – we need this. Would you mind sharing a tutorial on how to implement – step-by-step? I remember when I read this the first time you mentioned sharing some scripts…or was that another read?

  36. Tina says:

    Resource allocation is still a biggie for me. I am with a pretty great UK hosting provider for one of my larger sites, and they are great in every way. Now the kicker is that due to the way resources are allocated, I am unable to run a back-up via CPanel. Whilst the option is present, the host instantly disables any back-up I start to run. Their answer to this is for me to back-up my site via ftp. That’s really not ideal.

    Now I don’t move because they provide a far better service than many other hosting out there. Thing is, there is nearly always a downside isn’t there. You certainly raise some good points!

  37. Dan says:

    I am a pretty happy bluehost customer, and I am a web developer. So I recommend a lot of clients to bluehost. However lately it has been causing me headaches. The sites have been getting very slow response times. And it is a definite pattern through out the day. I think they will not be satisfied until they move to VPS or Dedicated. I wish that wasn’t the case, but even with the improvements, at least some of your boxes are still very slow. (The box in question is 390) Support tickets have improved it somewhat (from 2000ms+ to 1500ms), but they expect 900ms average.

    Thanks for what you are doing for shared hosting. Any plans to get into VPS?

  38. [...] thousands of hours (according to himself) developing the system and introduced this so called Hosting Nirvana (One might argue DreamHost has been doing this for some time – but technically, it’s not the [...]

Leave a Reply