Technical limitations…

Ok, I’ve done enough personal blog posts that its time to get technical again. Web hosting is a changing animal. Customer needs are becoming more and more demanding as businesses rely on the web to provide critical infrastructure for their success. In order to facilitate these customer needs hosting companies require an EXACT method to track resources needed and consumed by every customer in realtime and historically. Currently these tools and tracking are sorely lacking.

Most web hosting companies use Unix or Linux (many varieties) for the servers that provide service for their customers. Linux is rising star and is what we use for 100% of our hosting services. Linux is GREAT! It is fast, it is stable, and it is only getting better. However, until very recently Linux has not provided the necessary kernel (heart of the operating system) enhancements to effectively track what we need to both guarantee resources for sites that are starved for cpu, disk I/O, etc, and to limit and block the cpu and I/O hogs that are causing problems for everyone else.

I talk with many hosting companies every week and try and assess what others are doing to meet these challenges. In many cases companies are just throwing more servers at a problem without dealing with the underlying cause. This was how it had to be because Linux simply didn’t provide the necessary information. Let me be specific so you know what I am talking about. Below are a list of things that we must have in order to provide guaranteed resources in a “shared” hosting environment. Some of these requirements can be obtained but are difficult to get at and others the Linux kernel either doesn’t report or give access to or is inaccurate to the point of being useless in performance tuning.

*DISK IO REQUIREMENTS**
– Byte counts per process and/or user and group to an IO device in realtime and historically (Taskstats in the kernel does this – FINALLY!!).
– Byte counts including or excluding cached hits to any physical device.
– Disk accesses per process and user per physical IO device tracked in realtime and historically – This is EXTREMELY important in a shared hosting environment. As I talk with other hosting companies this is almost never even looked at and in my view is the #1 performance bottleneck in hosting by a long shot.
– Exact stats of journaled filesystem flushing mechanisms such as kjournald, pdflush, etc.

**CPU REQUIREMENTS**
– Better hooks in the Linux kernel for managing/tracking and if necessary blocking process. “nice”ing processes is like hitting an ant with a sledgehammer. It works but it is not the elegant solution that companies need.
– Ability to reduce CPU and I/O block devices in tandem. The CSF I/O scheduler that is the current default I/O scheduler in the Linux kernel (2.6.17 and above I believe) allows I/O priority scheduling. “ionice” and other software is available to do this scheduling, but isn’t set to work with the CPU so I/O and CPU are difficult to reduce in concert. CPU and I/O don’t increase or decrease in a fixed ratio together, but viewing an individual process with a goal to reduce CPU without controlling the I/O is a waste of time in my opinion.
– Better user space application threading is also a need that all hosting companies have. As multi-core CPUs are now the norm in virtually all servers and 8 core CPUs are on the way (4 core is available now in Intel, and AMD Barcelona 4 core CPUs expected in 2nd half of 2007) more “web hosting” apps needs to be multi-threaded to make better use of these CPUs/cores. Cpanel (Our control panel) still uses many older single threaded apps because it is difficult to upgrade. The company that created Cpanel needs to take this to heart and start improving performance for the real world instead of just what works in their lab.

I am writing this blog entry for a couple of reasons. First, so you know that the CEO of Bluehost isn’t just a business/marketing entity at Bluehost. I understand the technical challenges that hosting poses. Second, I want you to know that we are actively working on and implementing changes that will shield users from other customers CPU and I/O demands. I know customers hate to see “CPU Quota Exceeded” errors. I know sites are sometimes slow or I/O is backed up. These issues are on the top of our list and solutions are coming that will be exclusive to Bluehost. We have some of the most talented and bright developers that have been challenged to solve these issues. Real solutions are coming and performance will continue to increase!

Thanks,
Matt Heaton / President Bluehost.com

19 Responses to “Technical limitations…”

  1. Jun says:

    Talking about “technical challenges”, I am a potential customer and I just got off a live chat at bluehost. I was told that your postgresql installation does not have full text search (tsearch2). I have very high regard for BlueHost, as it comes highly recommended, but I think offering incomplete implementation of postgresql functionality is not in the best interest of potential customers.

  2. Yeah, that are a couple of people at digg.com bashing bluehost since some customer’s bluehosted website imploded under heavy load (see http://digg.com/gadgets/Not_the_Best_Buy_My_Dad_Ever_Made_HDTV_PURCHASE_HORROR_STORY#c5292788). As usual, you seem well aware of the situation and I appreciate the pro active efforts bluehost is trying to put forth.

  3. PoRtCuLLiS says:

    Good to see you’re taking these challenges on, I hope you make some headway. The most worrying thing about being hosted at bluehost is the “CPU Quota Exceeded” screen of death.

    I’ve been a customer for years but one of my accounts was deemed to be too heavy on resources recently and it had to go. A clear way to measure and track resources usage would be a real benefit.

  4. Curtis Long says:

    Matt,
    Thank you for the information. It answers questions I’ve had since I first started my web site in 2002. Bluehost is the 3rd web host I’ve had. The two prior web hosts failed to meet my needs in several ways but I think the problems you mention in this blog explain all the problems I’ve had with the two prior web hosts. I’ve been with Bluehost for about one month now and I’m very happy. Every aspect of Bluehost has exceeded my expectations. Since being with Bluehost, I would rate the two prior web hosts with a 2-3 out of 10 and Bluehost a 10. The speed up and down to my web site is better than I ever expected. The power of the control panel is the total control I should have over my web site. I will continue to highly recommend Bluehost to everyone I talk to. If I can be a reference for Bluehost, I volunteer. Thank you for excellent service. Curtis Long.

  5. […] Posted by Bob Morris on February 22nd, 2007 From the CEO of BlueHost. They host 240,000 sites, run Linux and love it. But it does have some frustrating limitations. […]

  6. Charles Tutt says:

    Does this mean that you would like to punish my business if I’m fortunate enough to get enough traffic to exceed my quota?

    As one considering a move to Bluehost from one of my current hosts, that makes me shudder! And are you also saying that you too have customer slow sites? That’s one of the main reasons I’m looking for a new host.

    I would much prefer to keep my site fast, my visitors served and pay for excess usage if I should be blessed with the good fortune for that to occur.

    Am I (hopefully) simply missing your whole point?

  7. Wes says:

    If cpanel is such a problem, are there any other alternatives to it? If there isn’t surely its something developers can look into and develop a better application than cpanel.

  8. Michael says:

    This isn’t related to this post but I wanted to say it anyway.

    I signed up for BlueHost.com after reviewing some hosting comparison lists because 1. it was usually in the top 10, and 2. because I live in Lindon so I’m geographically very close.

    I’ve completely underutilized the service but am very happy with it. Partly because I’m confident that BlueHost will help me if I need help.

    A little while after I signed up for an account, my coworker also purchased hosting and a domain from BlueHost and quickly threw his business’s website up. He doesn’t really know anything about web development and I helped him with things that I could in designing his website. The problem was that I don’t know much either, and he quickly ran into difficulties he didn’t know how to solve. I suggested that he contact BlueHost to see if they could help him figure something out with Mambo CMS. They did. It seems that even when it didn’t have anything to do with them, they rose to the challenge and helped him out.

    I think it’s awesome to have a company go so above and beyond customer satisfaction and into customer amazement. I’m just starting to read blogs on a regular basis and Matt’s will be at the top of my list. If the man can build a company with a passion for customer service like that, he’s got to have some good ideas to share.

  9. Matt, you guys have to really do something about the Exceeded CPU Quota Error. I like Bluehost and I’ve been slowly transferring my sites to you guys. However, I’m starting to wonder if I’m making a BIG MISTAKE. I have a blog (cellphonedigest.net) and I’m in the process of trying to transfer the site to Bluehost. The blog has articles going back a year. Whenever I rebuild my blog because of an update, I get the CPU error. THAT IS CRAZY. That means my site will be down everytime I update it. The host I’m transferring from (if I still do it) gives me no error. I can rebuild it all I want and I NEVER GET A CPU QUOTA ERROR. You GUYS HAVE TO FIX THIS, or you will force me to take my sites to another host.

    -Patrick @ pbirm@yahoo.com

    ===========================================

    This Account Has Exceeded Its CPU Quota

    Please contact this site’s webmaster.

    Wait a few minutes and use your browser’s “Back” button or click here to try again.

    ——————————————————————————–
    If you are the webmaster, your account may have exceeded for one or more of the following reasons:
    Your site has used more than 20% of the cpu.
    Your account has too many processes running at the same time.
    Your site was consuming too many resources. This happens on occassion to very busy sites that have inefficient scripts running.

  10. Aaron says:

    Thanks for working on the CPU quote exceeded errors.

    I’m mainly running wordpress on my site, and it puzzles me why I’m getting CPU exceeded messages from time to time, even though wordpress is a well-developed and efficient script. I suppose other people on my box are screwing up the server. Hopefully I won’t see this happen in future. So far I’ve been very happy with the service except for this issue. :)

  11. Hello,

    I’m a VERY VERY happy customer of hostmonster and bluehost..

    I had alot of issues to begin with when it came to CPU usage and delays. These have mostly been fixed with the use of file based caching systems to limit the amount of PHP renders and mysql calls made with each visit.

    I have a number of webpages and would love to be able to grow my services without adding to the problems your’re facing.

    my major question is: Can you PLEASE look into making it possible for accounts to monitor the usage they are using and how much they are given… I would love to see peak times/peak services..

    simply to see whats being used and also profile what is lagging and what services I need to install caching with..

  12. Matt L says:

    I am looking forward to seeing hosting get faster with my BlueHost account!

  13. Sh1fty says:

    hi Matt,
    i don’t have any credit cards or a paypal acc (croatian banks aren’t compatible with paypal :S) but i’ve gathered enough money to pay for 2 years of your hosting. why doesn’t bluehost accept direct money transfers? it’s much more accessible then credit cards or paypal (you can just give cash at any post office and transfer the money). you could blog about it or just email me :)

    Best regards,
    Sh1fty

  14. Nabil says:

    I like bluehost, it fast, cheap, secure … and so on.

    BUT it isn’t opened to many webmasters or companies all over the world who can’t pay using Paypal (available in few countries) or CC.
    It would be a great idea to accept Moneybookers like dynadot.com does (I like these guys)

    Nice blog by the way, I feel happy to post on it

    Best Regards
    Morocco

  15. F Duran says:

    Matt, you guys are offering a great service. But im very worried about the “CPU Quota Exceeded” screen.

    I cant use my hosting account the way i want to. I have to be recheking each script to see if i can improve my code.

    THIS IS A VERY IMPORTANT TASK TO FIX.

    In our particular case, we cant work this way.

    We are very dissapointed. we have 8 accounts with bluehost, and we will have to transfer some becouse of this problem.

    We hope you could do something about this.

  16. Gary R says:

    Yesterday, I had over 22734 hits on one page on my site. This site was shut down FIVE times yesterday because of the CPU Quota Exceeded issue.

    I then went to the CPU EXCEEDED LOG – and searched for this page. I found this page 113 times on this log. Out of the 113 times it was on the log, there were a total of SEVEN instances that went over .30 seconds. The maximum time that I found for this script was .46. By the way, this is the largest MYSQL database that I had in operation yesterday.

    Some statistics :

    22,734 hits
    7 times over .30 seconds

    0.31
    0.34
    0.35
    0.36
    0.36
    0.37
    0.46

    You call that an inefficient script? SEVEN out of 22,734?

    Come up with a better reason to shut down my site for 5 minutes – this is not a valid reason.

  17. I found this thread on Google.

    I just got my domain set up at BlueHost and installed phpWeb. Today is the first day I’ve taken a look at the mods and themes. I’m quite certain that I was the only person accessing the site — the URL hasn’t been publicized yet at all. Yet I too got the CPU Quota Exceeded message.

    How could my site have exceeded its CPU quota with just one user accessing it?

    I can’t, simply can’t, allow my users to encounter this message. And they shouldn’t, since we’ll have a few dozen users, tops.

    I see that this has been a problem at BlueHost for several months. Is it insoluble?

    J.D.

  18. java says:

    I too just found this site from googling after an incredibly frustrating day trying to move servers. I chose Bluehost when I decided to move hosts because I’d heard great things and it was constantly on best 10 lists. I had a wonderful experience with the customer service — every single person I spoke to was knowledgeable and helpful. Kudos on that. But just like JD, I had my site up and running and had gotten the dreaded CPU exceeded message no less than three times in the first hour. It’s a fairly straightforward WordPress site with few frills and light on plugins or scripts.

    Five hours later I signed up with a new host and I’m closing my Bluehost account less than twelve hours after opening it. I really, really wanted to like Bluehost, but a fantastic technical support team is nothing if the stuff doesn’t work right.

  19. damu says:

    i have seen these quota errors few times and i am so disappointed as i don’t have many visitors as well. The president’s post was made on february and today is 22 aug, 07 and still i am getting that error. Its so frustrating to see the site go down due to errors like those. I have currently 3 sites running here and 2 more in 2 different companies but i had/have never seen those errors before. I hope u’ll really look into this matter and try to keep all customers. I was planning to migrate here other 2 as well but decided to halt for a while until i know that the error has been solved. For the rest i am so happy to be here.

Leave a Reply