Recent Downtime

[Dani]

I would like to thank everyone for supporting us during our recent downtime. Unfortunately, it was inevitable. On Saturday morning, at 9:30 a.m., I tried accessing the site only to find it was down. I called my host immediately and they were not only unaware of the problem, but they didn't know how to fix it. After a few hours on the phone with technical support, I was able to conclude that the machine had actually lost its network connection entirely. It could not even see other machines in the same datacenter. Unfortunately, my host didn't agree with me, insisting that the sluggishness of the machine was due to me running too many concurrent MySQL connections, despite a CPU load of 0.01. Therefore, they refused to do anything about it. Meanwhile the site was mostly down and there were fewer than 30 people logged into the forums. It took until 6 p.m. for me to finally convince them that it was a network issue. It turns out that the previous night, my host mistakingly assigned another machine my block of IP addresses. It actually took them 10 hours to figure this out, despite seeing that everytime they SSH'ed into the daniweb server they were taken to an entirely different server.

Frustrated, I signed up with a different host, which was set up late Saturday night. (I would like to graciously thank them for putting me ahead of the queue of new servers to be set up.) However, I had to wait another day for the DNS to propagate. In addition, I had to wait for my old host to get their act together, and figure out some way for me to get a recent backup of the database off of a machine with no functioning network. (Yes, after two days, they still couldn't figure it out.)

Needless to say, we are now on a new, stable network, which a much nicer server. Thank you for your patience and continued support.

Unfortunately, the latest backup I have was from Saturday at noon. However, because very few people were able to connect to the site at all, we only lost about 15 posts.

I also apologize to those people who registered over the weekend. Unfortunately, due to no network connection on the server, the activation email was unable to be sent out to anyone. I have therefore manually activated all new members who registered this weekend.

In addition, I am giving all current advertisers 10% extra banner impressions of however amount they ordered.

[i686-linux]

It is truly amazing how a lot of these hosting companies stay in business. I pushed out a real quick website for a "friend-of-a-friend" type of person and the hosting provider that this company was using was terrible. I can't tell you how many times there host went down for hours or even a day at a time over the course of several months. It seemed that every couple of telnet sessions I would run would cause the site to be down. I am not even talking heavy hitting site. Low visits with static html, and the only thing done inside of the telnet session was vi!

[DMR]

Sorry you had to spend your weekend that way Dani. It really is frustrating to deal with a situation like that, especially when you can't go to the site personally to analyze the situation.

It really does amaze me how stubborn some of these technical service providers can be; I've spent days wrestling with cable/DSL providers and hosting services. They invariably instist that everything is working perfectly on their end, and they are invariably wrong. Grrrr!!

[Dani]

I just amended my original post :)

[DMR]

Ah, so I wasn't crazy- one of my posts really did get eaten.
No biggie though, could have been worse! :)

[Dani]

Even when the site was accessible (although soooooow) via the web, I was unable to ssh into the server. My host was able to ssh in from within the datacenter, but once in, not able to ftp any files out. For that reason, I was able to make a mysql dump of a more recent version of the database. But no way to access it from the outside!

[kc0arf]

Hello Dani,

Congrats on your successful migration. If you are keeping a scrapbook on the "life of daniweb", make sure you keep entries. Through the process, you learned a lot of things, and responded to a lot of pressure.

I hope this week goes much smoother for you. So far, so good!

Take care,

Christian

[johnroach1985]

Thank you CSCGAL for your dedication to this site .The INTERNET needs people like you.People who actually KNOW THE JOB.And I am sorry for your weekend being a disaster.Think you should accept a pay raise after this :)(hope this new server thecs don't have issues :))

[Dani]

Thank you for sticking by despite all of the timeout errors you guys probably got over the weekend. I still have a bunch of server tweaks to work out (including customizing the error pages, etc.) But all is good to go so far!

This problem actually turned out to be a very good thing. We moved up from a P4 2.4 ghz on an unstable network to a dual 2.4 ghz Xeon on a stable network :) Hopefully this server will stay with us for a long time and we will be able to grow into it!

[oalee]

It is truly amazing how a lot of these hosting companies stay in business. I pushed out a real quick website for a "friend-of-a-friend" type of person and the hosting provider that this company was using was terrible. I can't tell you how many times there host went down for hours or even a day at a time over the course of several months. It seemed that every couple of telnet sessions I would run would cause the site to be down. I am not even talking heavy hitting site. Low visits with static html, and the only thing done inside of the telnet session was vi!

You should try my DSL provider... you wouldn't know sumthing's wrong until they got it fixed. Try and call them up and this annoying answering machine keeps on telling you that customers are important to them yet you have to wait for at least an hour and a half to get your call answered. And that's not all of it, they'll even tell you to try and call back later... those @#$!.... all they think about is profit...