Last week was an interesting one.
It all began on Tuesday with a post I wrote called, How Does Your Brain See Color. The post wasn’t so much about how we actually see color as it was about how we make associations instead of seeing what’s really in front of us.
As I usually do, I submitted the site to a number of social sharing sites like del.icio.us and StumbleUpon. I also submitted it to Netscape. Well, my little post got hot on Netscape – reaching the front page in a matter of minutes. It kept climbing steadily all the way to the #2 position – where it stayed for about 6 hours – getting about 10,000 hits per hour – and stretching the capacity of my server.
Then it hit #1 on Netscape and flood gates opened. Soon I was getting 20,000 hits per hour and climbing…until…my server account was overloaded and my site crashed.
Even with my site down, my post remained #1 on Netscape for the next 15 hours. My hit rate also stayed above 20,000 that whole time – even with the site down.
Obviously, I wasn’t happy that my hosting wasn’t able to carry the load. Okay, wasn’t happy is a understatement. It actually consumed me on Wednesday – where much of the day I was in the #1 position.
By late Wednesday night, more than 24 hours after I submitted my post to Netscape, I was off the front page. But the hit load didn’t drop off too much. Even being off the front page, I was still drawing 14,000 hits per hour – too much, still for my site to come back up.
My site finally went back up sometime on Thursday. And I had spent two days working with my host, tracking the status of my account. Really, I was exhausted from the whole experience.
And, yet, I’m glad I went through it. Now, I’m working on making adjustments to my hosting account so that I can carry a huge traffic load like that. I’m also re-evaluating each of the WordPress plugins I use, which are pulling system resources with each request to my server (the fewer requests, the better). And so I should soon be ready for Netscape’s homepage again.
So if you were curious, that’s what happened to my blog and I last week. That’s why my site was down. And that’s why I couldn’t communicate about it with you. And, I wasted so much time with my host while my server was down, that I got a bit behind on my client work – which left me with no time to write at the end of last week.
To everyone who emailed or phoned me about my server being down – thank you. I’m truly moved by your caring. Here’s hoping that we don’t go through this again.
So, when have you experienced lots of traffic not being a good thing?
Dawud – Ah the price of fame! Had similar problem earlier this year – but nothing like the problem you had. Frustrating doesn’t even begin to describe the feeling. Glad you’re back in the saddle without too many scars. Later.
haha I don’t know if i could ever complain with too much traffic. Even if i made the front page of digg and had to pay an extra 5 bucks for the bandwidth charges the traffic is worth it!
Check out the new Yahoo YSlow plugin for firefox, it checks 13 different criterion to help speed up website performance.
I know there are a couple wordpress caching plug ins out there that you might want to check out as well, heres one that I know of thats pretty good.
http://wordpress.org/extend/plugins/wp-cache/
Wow, that sucks that your host couldn’t keep up. Are you on a shared hosting account? A dedicated server definitely should’ve handled it, but there could be dozens or hundreds of other sites on your server if you’re using a shared account.
I think there may be services that temporarily set up a mirror site for you if you get on Digg or another large site.
sam,
Yeah, I’d been getting away with a shared hosting account for quite some time. Now, time to move on.
One sort of neat thing that happened as not only did I take down the server I was on, but on Friday my host was forced to reset a bunch of servers partly due to my traffic blitz. Interesting, huh?
MorganLighter,
I think I’ll heal fine. What was frustrating is the effort I made on that story went mostly for naught in gaining readers. But, oh well, live and learn.
Congrats!!
Never ever has too much traffic been a bad thing for me. However, after reading about yours I am looking forward to it. 😉
Shana
Shana,
I thought the same thing. Then it happened to me and I mourned all the missed opportunities to reach readers while my site was down.
Are you taking any steps with your host to be sure your site doesn’t crash if you get a tidal wave?
shaun,
Oh, I’m not complaining. Just want to be sure that I’m thinking about how to contain such a huge traffic surge. And offering the same to everyone else.
How many times have you clicked through a Digg story only to find the site down?
That’s the real bitter sweet of it all, isn’t it?
I’m in the process of adjusting my server and relocating a few of my small time blogs and clients to make room for the possibility of heavy traffic. I’ve tried to fly under radar lately to avoid such traffic until I can get myself positioned properly.
I’ve crashed too many times on small loads of visitors – I certainly don’t want to find myself in your position and not a page to show for it.
Wow, Dawud.
You know, that is not even a consideration that has crossed my mind. Having read your description I can see now how easily that could happen.
Isn’t it uncanny how we begin to think of the readers who can’t get into the conversation?
Keep writing…
Did you learn anything about how to avoid the problem of the site being down next time?
Tom,
Thanks. I’ll check out the Yahoo service.
I’m already using WP-Cache on my blog. Of course, a high number of plugins is one of the reasons this happened. It’s a small one, but it still contributed to the crash.
Steve,
Thank, I definitely will keep writing.
I knew about the pitfalls of shared hosting. I just wasn’t ready for it to happen to me. I do recommend looking into either a dedicated server (read: expensive) or a Virtual Private Server for any business site. I just hadn’t bothered to change mine – until now.
John,
Oh yeah…don’t use shared hosting. I’ve advised clients on that for years. But hadn’t really warranted switching myself – until now.
The other thing is use as few WordPress plugins as you can. Part of this issue had to do with how much memory was being called upon on the server to support all the traffic I was getting. Each plugin has a memory load. So for every plugin you use, the ‘heavier’ your site becomes in sucking up memory.
So, no shared serving and use fewer plugins. That’s my advice.
Also, host with a company you can get personalized service.
Gayla,
It really was. Truthfully, I just didn’t want to pay the higher fees of virtual or dedicated servers. But I guess it’s time. I definitely found the limitations of my shared account.
I’m glad you’re back! And don’t worry too much about not being able to write. We understand that life happens!
Just out of curiosity, are you happy with your host and the way they handled this situation? Would you mind sharing how much you pay per month? I’m thinking about switching because my host has time out problems too often in little spurts throughout the day. I’ve had a couple of wonderful readers let me know, which tells me there are probably more that I’m inadvertently turning away 🙁
LaurenMarie,
Thanks.
My feelings on my host are still up in the air. I use DreamHost and have mostly been happy with them up until the last couple of months. But I can’t find a better alternative right now without going to a dedicated server – which I just find way too expensive of a hosting option.
I do still put many of my clients on DreamHost and yet I’m still searching for another option. Can I whole-heartedly recommend DreamHost – no. But have I found a better host, with similar fees and packages – no, as well.
Am I happy with how they handled the situation – yeah, pretty much. Of course I didn’t want the site to go down at all. And I understand that it had too. But the tech support guy I worked with was great. He answered all my questions, gave me ideas for solutions, and even took my account home with him when his shift was done so he could monitor my site’s traffic levels – he emailed me 3 or 4 times from home. The next day, he was the one who got my site back up.
For shared hosting – I was paying $8/mo for an enormous hosting package.
Well well well, that was the another side of the coin.
I bet you sure love it now !
I can see it now (or, I hope I can)…
An upcoming post on WP plugins – the good, the bad, and the server hogs.
Pretty please? 😉
By no shared hosting what do you mean exactly? No shared at all (I guess a virtual server is ok based on one of your comments). What kind of pricing level are you talking about on a monthly basis? Who would you recommend as hosts (if you didn’t want to post that – an email would be great but I can understand if you don’t want to do that). Part of my issue is I would really gladly pay more to get more but I am not at all sure paying more gets you more – in many cases. Trying to find the right trade-off of paying something extra for a better uptime, service… and not needing something very expensive. I have hosts I have been satisfied with but not delighted with (and over the years I am used to hosts getting worse and having to move…).
Dawud – what an awesome response to your blog! (and well deserved) Thanks for sharing some tips with your readers about how to prepare for / handle a situation like that. May we all have such problems. 🙂
xnepali,
I’m not at all complaining about the traffic spike. I just wish my hosting could have contained it. I’m hear for the people who want to read my blog. So by my blog going down, I knew it inconvenienced people who regularly read my blog. That’s what really bothered me.
Adam,
I am considering a series of posts about unnecessary plugins. My site even had a few that I could very easily hard code into my theme. And every plugin turned off, pulls less system resources.
John,
Oh, I hear you. I don’t want to pay more for hosting than I have too, either.
Currently, I’m with DreamHost. My experience with them has been mostly good. And at times, I’ve not been happy. Yet, how they handled my down time, I’m actually quite happy about. So it’s a catch-22.
Do I recommend DreamHost – yeah, I still do. And I’m actively looking for a host that’s got a package that is specifically geared toward blogger’s needs. I haven’t really found that yet. Most hosts just seem to offer basic packages on shared hosting until you either user a virtual private server or dedicated server. Both are much more expensive than shared servers. What’s different is I don’t see a scalable hosting package, with great customer service, that’s geared toward the ever-changing needs of a blogger. And I’ll keep looking.
In the meantime, go for great service. Phone support is key – which DreamHost doesn’t offer at a basic level of hosting. Probably my one, ongoing complaint.
DreamHost is offering an interesting new service…It’s a VPS that scales on the fly. So you buy what you need and pay monthly – $15 for $150 MB of memory, for instance. As you site needs more, you simple ‘dial up’ your setting to higher resources at about $1/10 MB of memory. No reboot. And you can dial up and down at will. So if you find yourself on the first page of Digg, you can increase your resources temporarily and then dial them back once the wave passes. Then they bill you retroactively. I’m going to give this a try and either it’ll work great and I’ll stay with DreamHost. Or, I’ll more aggressively look for a new host.
One further note, all this about shared hosting only comes into play if you’re constantly pushing the resource limits of your host. If not, don’t change a thing. You can contact your host to find out the details. Then as you site gets more popular, just watch.
Connie,
Thanks. I’ll keep everyone updated too on future changes. I’m not wanting to go dedicated yet – I just can’t justify the expense. So I’ll try VPS for a while and see what happens. You can read the above comment for more…
That sounds like an interesting experience, I hope that I will experience it someday too 🙂
I would really like to hear what sort of plugins you decide to use after you have re-evaluated them.
When my site – with its 1000s of Vbulletin-generated pages – apparently became the target of an automated translation service. At least, that was where the IP address led back to. It devoured my whole bandwidth limit for the month within days.
But that’s not altogether relevant. Judging from its title (which is about as much of it as I can follow), maybe this article is more so?
Jens,
I will definitely post my plugin choices and why I’m using them. I’ll also, likely, post the plugins I’ll stop using and why.
Hilary,
There is a server load issues, at times, with WordPress. Much of it has to do with plugins. And the issues tend to be more about system resources (memory and CPU speed backups) than with bandwidth.
I also know that spam can be a problem, but I’m not sure what can really be done about that. Maybe your article link will shed some light on that one.
It would have been a great thing if your host could keep up. It’s too bad that it didn’t…
Crystal,
I know. I’m trying not to allow that mistake in the future. Then we’ll try again.
At least when you outgrow shared and have to move to dedicated your site should be producing more than enough to pay for it.
I could only hope to ever have this “problem” 🙂 But seriously, moving away from shared hosting was a good move. Now you just need another test of your server to see how it holds up, eh?
mlankton,
That’s the plan. Yet I don’t monetize my site, so at this point I have to rely on increased business.
Michael,
Right on. Haven’t hit the jackpot yet with a post – been too focused on my niche instead of the type of stories that draw mass appeal. But soon.
I have been planning a contest on my blog. After hearing a bunch of stories like this, the first thing I did was call my host and see if my bandwidth could handle the surge. They said no problem and they actually have bandwidth in reserve if needed.
Andy,
It’s not just bandwidth you want to be concerned about. You also want to find out about system resources on your server. Specifically do you have the memory and processor speeds to manage a large hit of traffic. That was my issue on a shared plan (I actually brought down a bank of servers that crashed because I was sucking up all the system RAM). I have massive bandwidth with my host so that didn’t come into play at all.
I’d suggest phoning your host and asking them to evaluate the system load on blog based on your server package for so many hits per hour. Remember, hits aren’t visits, they’re calls to the server. So one visitor can create 5, 10, 30 or more calls to the server. They should be able to help you know what the threshold is for keeping your site up.
[quote comment=”9571″]Wow, that sucks that your host couldn’t keep up. Are you on a shared hosting account? A dedicated server definitely should’ve handled it, but there could be dozens or hundreds of other sites on your server if you’re using a shared account.
I think there may be services that temporarily set up a mirror site for you if you get on Digg or another large site.[/quote]
No S****!!
My former was the same,oopps scratch that, there could not be worse!!
Like a ‘sci-fi’ horror flik by Steven King!! or bettter Stanley Kubrick. (may he R.I.P.!)
My site was taken down over a dozen times! So l never got a chance to get any traffic, when he left alone, l made my sandbox
~~SING ~ & ~ DANCE!
He even addmited he was jealous!
So now here l sit with my new comp.
and y et homeless!
Cause they got the power to things, normal people would’nt do, but this one aint!
Even though l changed hosts, l still seem to be on his server some how?
Can someone tell me please?
My mozzila browser won’t work cause it says ‘no good certificate’!!
and the ip is his and name, how can that be??
lf anyone can give me a lil clue as to how this,and realy could happen?
lf so how do l get opff it??
thanx!
Have a Great Day ALl~!~!
Babe
~~~~
I hate shared hosting, not only do you get the problems mentioned in your post but you get the problem of bad neighbours
I think there comes a point when you have to weigh up the cost of dedicated hosting v the positive impact on your business particularly if your web site is generating sales or the traffic is increasing into the 1000’s.
Well that’s true. That’s why we should always be prepared for things like that, because these things can actually do more harm than even considering that we are getting tens of thousands of traffic to our sites. There is such a thing as too much of a good thing. And this is just that.