Knowledge to Power Your Website

DreamPress: A Lion in Winter

DreamPress: A Lion in Winter thumbnail

Keeping up with the Joneses is hard work! They get a new car, so you get a new car. They have a dog? You get a lion! A lion trumps a dog, right?

When it comes to comparing what web host is better, though, it’s often a case of “different, but not better or worse.” The reason for that is everyone’s website is different. The way you built it and the way people use it is different than the way they use mine. And that’s as it should be. Diversity rules!

But that makes measuring what host is better for you a little difficult. The only real way to do it is to make the exact same site on multiple hosts and run the same battery of tests on all of them to see which responds better for the site you’re making. Then you need to run the same kinds of tests, but this time measuring load. Once you have all that information, you have to think about what metrics matter most to you.

Who on earth has that kind of free time?!

Oh, hi.

Of course we did all that for you! And just in time for the new year, we have Rudolph’s Shiny New Metrics! No Island of Misfit Blogs here, folks!

First, yes, we did really build the exact same WordPress site, same plugins and everything, on a variety of our coopetition. We picked some of the most well-known Managed WordPress hosts out there: Bluehost, GoDaddy, HostGator, Pagely, Siteground, and WP Engine. Furthermore, we took each host (even ourselves) at the basic defaults. If that host had default plugins, we left them in the state they were installed. Every single one had caching set per the host’s directions. No questions asked, no “but I know better” customizations attempted.

Then, using a tool called Loadstorm, we ran 3 basic tests of 20 minutes each. Each test had a different volume of users. And each test had ‘peak’ users over a different length of time. The peak means the amount of time Loadstorm spent trying to visit the site with as many of the users as possible.

  • 100 users, 10 minutes at peak
  • 500 users, 2 minutes at peak
  • 1000 users, 5 minutes at peak

Those numbers aren’t as arbitrary as it seems. 100 users, peaking for 10 minutes, is pretty much what a lot of blogs get when they publish a new post. 1000 users for 5 minutes is what happens when Chris Lema retweets your post.

The metrics we cared about are as follows:

  • Total Requests: This tells me how many requests that site can handle in the time allocated. (higher is better)
  • Peak Response Time: This is the slowest the site got. (lower is better)
  • Average Response Time: This is the average site speed. (lower is better)
  • Total errors: How many pages errored out. This would tell us if we were hitting security walls or if a server is unable to keep up with the load. (lower is better)

Then we ran them all and compared:

Test 1 – 100 users, 10 min at peak

Hostname Total Req. Peak Res. Time (ms) Avg. Res. Time (ms) Total Errors
Bluehost 6034 632 157 330
GoDaddy 6035 7267 132 0
HostGator 6018 1070 186 0
Pagely 6048 1312 11 0
Siteground 6033 2410 151 0
WP Engine 6042 1415 83 0
DreamPress 6049 1108 16 0

From this test, we suspected that we were hitting Bluehost’s security. Even though Loadstorm does a good job of having not all the traffic come from the same IP, it was possible we’d accidently tripped the spam blockers. The numbers we got from Bluehost were exceptionally reasonable before we started getting errors, and the server was still up and running when we checked, so the logical assumption here was security.

We can also see that everyone has roughly the same volume of requests. Bluehost’s low peak response was related to how many requests were erroring out. The errors happening that quickly certainly made the theory we had about the security blocking us more likely.

How does DreamPress fare in all this? The Joneses better watch their back, because we’re right there behind them. The Peak and Average response times are pretty awesome. 16 milliseconds on average? That’s practically a blink-and-you-miss-it load time.

Test 2 – 500 users, 2 min at peak

Hostname Total Req. Peak Res. Time (ms) Avg. Res. Time (ms) Total Errors
Bluehost 22341 1322 301 1250
GoDaddy 22394 7584 208 0
HostGator 22326 10099 259 1
Pagely 22465 7378 157 0
Siteground 22349 3218 275 0
WP Engine 22308 2621 179 0
DreamPress 22364 2304 165 0

At this point we were sure we were tripping the spam blockers on Bluehost. But this is good to know! The “unnatural” traffic was being blocked, even though we were trying to make it look real.

Everyone manages to handle the same amount of requests, which is good. And we’re within 100 milliseconds of each other for average response time. It’s in the peak response time that you start to see the real gains.

DreamPress is incredibly consistent, neck and neck with the best yet again.

Test 3 – 1000 users, 5 min at peak

Hostname Total Req. Peak Res. Time (ms) Avg. Res. Time (ms) Total Errors
Bluehost 49950 1461 347 2780
GoDaddy 49416 15100 535 311
HostGator 42838 8051 277 0
Pagely 50270 6426 168 0
Siteground 49966 5375 303 0
WP Engine 50193 1480 211 0
DreamPress 49897 13711 189 0

This time we expected a lot more errors from everyone. Boy, were we surprised!

The peak response for DreamPress shot up, due to much the same reasons Bluehost kept dying. We were throttling the users to not crash the server. Obviously we can improve on that. But our average response time means that only a couple hits triggered that. Guess what we get to work on now?

Even with that, our average response time shows that when we’re not throttling, we’re right there, aces over eights high.

Results

While I’d love it if the stats showed that we were top of everything, the results actually demonstrate that every host is pretty consistent and solid. It also shows that different traffic patterns give different results on each host. Sometimes one host was better in peak response while being less wonderful at the average response. Others were faster with errors. This reflects each host’s choices when it came to designing their networks and configuring their servers.

DreamPress held up very well against some of the specialized, WordPress only, web hosts. While we have room to improve, the tests have helped us to isolate a few problem areas and we know where to direct ourselves next year. With improvements to our systems, upgrades to our hardware and software, it can only get better and better for DreamPress in 2016.

Overall this race is just too close to call. But I think we can say this — In the race to keep up with the Joneses, DreamPress is a good choice to bet on.

We’re a damned lion.

A happy DreamHoster since 2012, Mika is one of the WordPress developers at DreamHost. When she's not making DreamPress better or practicing being the person you want with you for the zombie apocalypse, she's slashing bugs in the next version of WordPress.

5 Comments

  1. Mika, congrats on your progress with DreamPress and thx for conducting and posting the results of your benchmark. We’ll accept the “liger” distinction 😉 cheers

  2. Nice writing.
    sexy category widget on the sidebar.

    I have been using DreamPress for a while now and I am quite happy with it. It is nice to read about a test like this, from Mika, makes you feel good about your choice 🙂

    PS: Is there plans of any higher config variant of DreamPress ?

    1. The rumors that we have a giant wheel we spin every quarter, letting us pick a project and allocate our dev resources on it is entirely unfounded.

      We duel with lightsabers and the winner gets to pick the next feature.

      We have a lot of plans, and a lot of things that are in the works. PHP upgrades, HHVM, staging areas, more ‘types’ of DreamPress, plans, integrated backups… Basically we want it all too, but we’re making sure we get it right 🙂 Nothing sucks more than waking up to find out our great idea crashed things for you guys!

Comments are closed.