Upgrades and Outages

[John Adams]

All,

The hardware upgrades are here and in place, so I will be performing migrations and maintenance throughout the weekend. This spells outages for all services intermittently. I'll try to keep it to a minimum, so if you get errors or Page Not Found's, don't panic. It's just me, tweakin'

[shargash]

Did you do the hearts and valentines? Or was that something automatic in the client?

[John Adams]

Attn: SVN Users

Sometime in the afternoon (PST) over the next few days, I will be migrating our SVN Repositories to their new server, and upgrading VisualSVN in the process. One of 2 things will happen; 1) Everything will go swimmingly and the new service will start up and read our existing repos (I have 27 repos) without a hitch, or 2) catastrophe will strike and all data will be lost.



Just kidding with #2. While anything is possible, the upgrade should go flawless, and I always have backups and SVN dumps, just in case. Devs can commit until the moment the service is shut down, so don't worry about synchronizing with me. The change-over shouldn't take a few minutes (hopes).

[John Adams]

The SVN migration is complete, almost flawless For those hitting the new server, the only thing to note is you'll need to accept the new self-signed certificate when you first hit it.

Please report any issues immediately.

[John Adams]

The last machine to be replaced is this webserver. It is the one most noticeable as having damage, as the sites are growing slower every week, so before things get much worse, I need to move to the new webserver in the new cluster. I have everything configured and ready to go, I am just vacillating over whether or not to use the convoluted, clunky, over-engineered ISPConfig 3 CP, or just go native Apache (like we currently have). There's some advantages to the CP, like single-point management - I just hate the product, even if it's free.

I'll probably just go with apache and set up load balancing web heads just cuz it's cool. Plus, I have to move 50 databases over, and I am not looking forward to that task.

[Scatman]

HTTP 2.0 is finished and on the way! I don't think apache supports it *yet* but it will and it'll be a lot faster with multiplexing and the other features!

[John Adams]

Since this is my only "active" community I will only post this here... but we're going down for a few hours while I (painstakingly) move all of my websites to a new host, so we can ditch this dying VM.

Should take a couple of hours, pending disaster.

[John Adams]

Okay, I think I got the bugs worked out. It is [color=#FF0000:30r7pwtc][b:30r7pwtc]imperative[/b:30r7pwtc][/color:30r7pwtc] that you report [u:30r7pwtc]ANY[/u:30r7pwtc] page or module that reports an error. Thanks for your patience.

[Odinzz]

none so far, but i must say its super frigging fast now holy crap!

[John Adams]

I do have to share the one scare I had today.


Was just telling Jabantiz, that it's taken me almost 6 hours straight to try and recover the EQ2Emulators Forums database.... yes, the entire thing. All their posts, were gone. The bad spot on the drive sat right under their Posts table, of course. Almost 29,000 posts on that site spanning almost 9 years. I panicked.

After trying every thing I knew, and searching the stupid ass google/blogger/armchair "experts" on stackoverflow for hours, I finally walked away for a moment... when it suddenly hit me; what if I read the Posts table in chunks, and skipped over the bad stuff?

So I started doing "insert into new_table select * from old_table limit 0,1000" until I hit the bad spots. I got about 9,000 rows into the corrupt table. Then, it started looking bleak again. 10,000, 11,000, and so on... all bad.

Of course I had a backup, duh. But it was from the last time I moved everything (bad JA, bad!) which was Aug 2014. So I loaded up the backup. Low and behold, there were over 28,000 posts in that table... so worst come to worst, EQ2 would only lose their latest posts. That was not good enough for me!

So I had another brainstorm... when there's a bad "spot" on a drive, it's just a spot right? So, I saw the highest record ID in the backup and went to the corrupted table and did a DESC sort on that ID and what do ya know? 175 posts was all that was missing. And the corrupted table let me have them.

Peace and Tranquility has been restored to [url=http://eq2emulator.net:2h88eags]EQ2Emulator[/url:2h88eags].

F*#k I need a beer.