Server Migration (Tentatively scheduled for 11/12)

ISO · Post Post #0 (ISO) » Mon Oct 31, 2016 5:52 am

Howdy all,

I am still working on our hosting provider to get some definitive answers regarding this weekend's downtime. It seems likely that our server was rebooted in recovery mode to diagnose a non-critical issue (regarding cPanel access - I noticed a problem there when I was working on getting a information page up for the cards we recently shipped). I did not schedule this, and I have also not received confirmation that this is why the server went down, but I doubt the timing is just a coincidence.

This is the third significant downtime we have experienced with this host, all in 2016. We were told that there may be an issue with the specific datacenter where our server is located, and they have offered to help us migrate to their main datacenter. We are considering that option, as well as the possibility of moving to a different hosting provider (possibly cloud based, which would potentially enable us to react more quickly to issues like this and fix them ourselves, and should also be more cost effective). Either way, there will likely be another downtime involved when we do migrate. I will post updates here when I have more information.

As always, when mafiascum is unavailable we have the "fallout shelter" forum on my personal server, located at pnewman.net/shelter. Kison has suggested configuring our DNS to point shelter.mafiascum.net there as well, which I will look into doing soon.

Klingoncelt · Post Post #1 (ISO) » Mon Oct 31, 2016 6:32 am

You might want to block that adspambot that's posting Viagra ads all over the fallout shelter...

Davsto · Post Post #2 (ISO) » Mon Oct 31, 2016 7:22 am

idk they seem like a stand up guy

McMenno · Post Post #3 (ISO) » Mon Oct 31, 2016 7:27 am

seems reasonable

ISO · Post Post #4 (ISO) » Mon Oct 31, 2016 3:30 pm

Ok, I got some details on the issue. It seems the cPanel access issue was a symptom, and the downtime was (probably) only a matter of time.

In the short term, we are going to be moving to a different server with the same host. I'll try to give everyone some warning before we go down for that, but bookmark the fallout shelter (pnewman.net/shelter) as I'll be posting updates there while we migrate. In the medium term, we will still be looking at moving to a new provider.

ISO · Post Post #5 (ISO) » Wed Nov 02, 2016 4:54 am

The new server is provisioned; we will be setting it up this week and likely migrating this weekend sometime.

ISO · Post Post #6 (ISO) » Wed Nov 02, 2016 9:22 am

How long do you think the site will be down during the migration?

ISO · Post Post #7 (ISO) » Fri Nov 11, 2016 10:14 am

Migration is tentatively planned for tomorrow evening; Kison says the migration itself will take 1-2 hours. The server is already set up (with a two week old backup), so the time consuming part will be the transfer itself. DNS propagation may take a little more time, but we should be able to put up a redirect on this server during the transition.

zakk · Post Post #8 (ISO) » Fri Nov 11, 2016 3:37 pm

GL mafiascum

ISO · Post Post #9 (ISO) » Sat Nov 12, 2016 4:20 pm

We are back up.

I am waiting for mith to point the domain to the new server. Until that is done, the old server will continue to process your requests, but will communicate with the database on the new server. The process of transferring the domain should be unnoticeable.

Several pieces of software were upgraded as a result of this move. If you notice anything odd or not functioning properly, be sure to let me know.

Lastly, a bunch of new features for the site have been released. I will post about those in another thread shortly.

ISO · Post Post #10 (ISO) » Sat Nov 12, 2016 4:44 pm

Domain should now be pointing to the new server, propagation can take 24-48 hours (but is usually faster than that for most people). Assuming there are no problems I will be letting our host know they can shut down the old server on Monday.

Annadog40 · Post Post #11 (ISO) » Sun Nov 13, 2016 1:09 am

Can we get some moderation on back up, there is spam everywhere.

ISO · Post Post #12 (ISO) » Mon Nov 14, 2016 6:08 am

For some reason I can access the site through my iPad but not PC

chamber · Post Post #13 (ISO) » Mon Nov 14, 2016 3:18 pm

They likely are using different DNS's. It will propagate to whatever your PC uses soon if it hasn't already.

Yume · Post Post #14 (ISO) » Tue Nov 15, 2016 12:26 am

The forum search engine returns a SQL error code.

Cheery Dog · Post Post #15 (ISO) » Tue Nov 15, 2016 1:41 am

In post 14, Yume wrote:The forum search engine returns a SQL error code.

It worked for me searching for posts, but not topics.

ISO · Post Post #16 (ISO) » Tue Nov 15, 2016 3:37 am

finally corrected after i reset router and modem. Not sure if that's the reason or just time, but I'm back in action.

zakk · Post Post #17 (ISO) » Sat Nov 19, 2016 9:18 pm

WB mafiascum

Website · Post Post #18 (ISO) » Tue Nov 22, 2016 4:50 am

Ok this happened 3 times in a row arround 8:53-9:05am est if your an admin that can check logs

I am logged in. I try to make or edit a post

Before my edit or new post goes through it says I need to sign in

I sign at given prompt. I try to make or edit that post

Before my my edit or new post goes through it says I need to sign in

I believe you see the pattern at this point. Is the website logging me out randomly and or not signing me in correctly? Anyone else having this issue? I started noticing it after site breach/server migration that happened recently. Thought I was crazy but given it just happened 3 times in a row I not think I crazy. Any admin with a suggestion on a fix would be great