1,105,310 Community Members

DaniWeb Downtime ... all my fault

Member Avatar
Reputation Points: 1,356 [?]
Q&As Helped to Solve: 927 [?]
Skill Endorsements: 204 [?]
 
0
 

Well, I just got myself through a major disaster. A couple of days ago, vBulletin, the forum system which powers the core of DaniWeb, released a shiny new version, vB 3.6. Every day since, they've been announcing new features - many of which look just awesome!

This afternoon, they finally released public beta. My first instinct was to naturally wait until stable. However, when 7pm rolled around, my anxiousness got the better of me.

I dutifully backed up the database and the filesystem, and away I went. Only to find that the upgrade wasn't trivial by any means, but rather required many file edits and template edits and all kinds of stuff ... the kinda stuff that you'd be crazy to not have a test server running for at least a week first before attempting the upgrade.

Well ... that was behind me now. So the clock is ticking ... 8 pm rolls around, and I try to crazily scramble to fix one thing after another. Maybe if I pull an allnighter ...

And then I think about that SQL backup I saved! But last time I ran a mysql > source to restore the database, it took about 8 hours, and that was months ago! Imagine how long it would take this time.

On the other hand, once I went through the 8+ hours of downtime, we'd at least be back to normal before the morning. My only other option at this point was to pull an allnighter - or two - scrambling to upgrade all of my hacks and additions and templates and everything else to be 3.6 compatible. Remember, only the CORE of DaniWeb is based on vB - the portal, blogs, code snippet library, tutorials, link directory, and all of the extra features, which are all integrated btw, are my own creation.

So, I sighed, I took a deep breath, and I flushed out the live database, and restored from backup. And ... drumroll please ... it finished in less than five minutes. I promise you. In fact, if I am lying, I wouldn't be able to type this all out right now, one hour after I started this whole disaster (Which turned into about 30 mins on and off of downtime during and after the upgrade when I didn't know what to do with myself).

The moral of the story is don't underestimate the power of a new dual core dual opteron as a dedicated database server. I swear to you guys ... what used to take a full half of a day on a Dual Xeon took 5 minutes max on a dual core Dual Opteron. Aside from the difference in the processor, the other advantage (and it's a big one) is that I now had a dedicated database server which was able to focus entirely on restoring the backup, and didn't have to deal with constant Apache connections and 1000+ users trying to browse a site that's down.

So ... the upgrade will be for another day, after it's already been fully prepared on a test server and ... did I mention ... when it's outta beta?

Member Avatar
Dani Horowitz

I run DaniWeb. I coded (and currently maintain) the site from scratch using PHP/MySQL/jQuery and grew it into a thriving community of over 1 million IT professionals and 10 million visitors a month, making it not only one of the largest social media websites in its niche, but one of the largest publications on the web to date. I sell advertising. I host annual events throughout Manhattan during Internet Week, Advertising Week and Social Media Week which each see an average of over 1000 attendees. I host a bimonthly meetup group for techies on Long Island and eastern Queens. I speak on panels at search engine conferences. I run a brick-and-mortar coworking community for the tech industry in Queens, NY. I multitask. I consider myself a platform creator and a community builder. I'm a super-geeky programmer with a passion for Internet marketing. I am not for hire.

Member Avatar
Dani
The Queen of DaniWeb
20,552 posts since Feb 2002
Reputation Points: 1,356 [?]
Q&As Helped to Solve: 927 [?]
Skill Endorsements: 204 [?]
Administrator
Featured
Sponsor
 
0
 

I just went to check my email ... over 1900 database error alerts:!:

Member Avatar
jwenting
duckman
8,333 posts since Nov 2004
Reputation Points: 1,399 [?]
Q&As Helped to Solve: 445 [?]
Skill Endorsements: 35 [?]
Infraction Points: 5
Team Colleague
 
0
 

welcome to the wonderful world of beta software :mrgreen: :twisted:

Member Avatar
Dani
The Queen of DaniWeb
20,552 posts since Feb 2002
Reputation Points: 1,356 [?]
Q&As Helped to Solve: 927 [?]
Skill Endorsements: 204 [?]
Administrator
Featured
Sponsor
 
0
 

The problem isn't the fact that it's beta software - it's not really buggy. The problem is that I completely understimated the amount of time it would take to upgrade my heavily modified and hacked vBulletin.

Member Avatar
iamthwee
Posting Sage
7,038 posts since Aug 2005
Reputation Points: 1,307 [?]
Q&As Helped to Solve: 592 [?]
Skill Endorsements: 74 [?]
Featured
 
0
 

Where do you learn how to do all that. Is it information you have amassed on the fly from experience?

Member Avatar
Xenon
Technical Admin
2 posts since Jul 2005
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
Team Colleague
 
0
 

Lool

i'd say that never happened to me before, but it did, so i feel with you dani ;)

Member Avatar
Dani
The Queen of DaniWeb
20,552 posts since Feb 2002
Reputation Points: 1,356 [?]
Q&As Helped to Solve: 927 [?]
Skill Endorsements: 204 [?]
Administrator
Featured
Sponsor
 
0
 

Thanks for the support, Stefan. I actually set up a test installation this second time around and it's near ready to be launched. Everything is working fine - it's just that I am going to wait until at least RC before I go live, after this scare.

Member Avatar
joe12
Newbie Poster
1 post since Jun 2006
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

All our puppies are top quality english bulldog puppy, Full AKC registration With Multiple Champion Bloodlines, wonderful with small children.Excellent Companion. And they are ready for Thier new loving home.for more information you can visit our website at:www.allpuppies.net

Member Avatar
Zaigham
Newbie Poster
1 post since Jul 2006
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Indeed you under-estimated the time to upgrade :P ...
Thanks for posting your experience with upgrade, it will surely help to consider the things when upgrading anything.

Member Avatar
Dani
The Queen of DaniWeb
20,552 posts since Feb 2002
Reputation Points: 1,356 [?]
Q&As Helped to Solve: 927 [?]
Skill Endorsements: 204 [?]
Administrator
Featured
Sponsor
 
0
 

You're welcome.

You
Post:
Start New Discussion
View similar articles that have also been tagged: