• Basilisk extended downtime 08/09/2017

    Basilisk extended downtime

    Posted August 9th 2017
    The SWGEmu Staff




    Basilisk extended downtime


    While doing a routine backup before the merge one of the HDD's died, unfortunately that was the second one we lost in a matter of weeks. Right now TheAnswer and the rest of development staff are discussing the options and trying to figure out how best to try to recover all Basilisks data. Of course, with two HDD's gone, there is a strong possibility that what they recover is too corrupt and unusable or they may not even manage to recover all the necessary data to restore Basilisk in its original state before merging it with Publish 9 code and we will be forced to do a full database wipe. Of course its still early to give any predictions but one thing is for sure, we are not at a point where we can definitely say that we will be able to move forward one way or the other.


    We apologize for the extended downtime that is probably going to last for days.


    No matter the the outcome, we thank you for your dedication and your continuous support. Of course we will keep you posted as things progress. As soon as we have any info, you will have it too.


    Update 5:


    Quote Originally Posted by TheAnswer
    Its a 48 hour test, so its ongoing. But, no bad news so far.
    UPDATE 4:

    Quote Originally Posted by Miztah View Post
    Alright folks, update time. We've managed a recovery of the database and we've got everything copied over to our new SSD's. We're letting Bas run for the weekend as a stress test on the database to ensure there is no corruption that didn't pop up initially. This means that for this weekend, Basilisk is going to remain on Publish 8. We're looking to give the Publish 9 merge another shot on Monday, assuming all goes well over the next 48 hours. We'll update regarding Pub9 closer to Monday when we know for sure.

    In the meantime, continue on Basilisk as you normally would. We'll be keeping an eye out on our side for any potential stability issues.

    Keep in mind that we're still doing integrity checks on the database, if we run into any database corruption between now and then it's very likely that any progress you make this weekend will be lost when we revert to a backup. This is not guaranteed to occur, but be aware that it is definitely a possibility.

    UPDATE 3:

    Quote Originally Posted by TheAnswer
    We finished uploading 1.1TB of data and now we're going though it.
    UPDATE 2:

    Quote Originally Posted by Miztah View Post
    To clarify a bit, Basilisk was up but still in the process of validating the integrity of most of the database objects. The server is currently running through our backup HDD's, which is causing the server's loading to take a lot longer than it normally would on SSD's. We shut down the login server because any actions triggered on Bas will delay the database check, and it's going to take long enough as it is.
    Quote Originally Posted by Miztah View Post
    The good news is, the server didn't explode using the recovered database. There's no current bad news, but as of right now, that's the only good news. We still have to verify that all of the objects in the database are actually valid and load properly, and it will take time. The possibility of a wipe still remains if we find too much corrupted data. Time will tell, but in the meantime you'll see Basilisk go online and offline multiple times, but you won't be able to access it. Those still connected will not be able to reconnect when they disconnect, and in the interest of getting this testing done as quickly as possible, they should disconnect on their own anyway.


    UPDATE:


    Quote Originally Posted by Ivojedi View Post
    About the loading status: We are attempting to load a test db instance to check for data integrity. This should give us an indication of whether we'll need to wipe or not. It's loading on the HDDs so it's slow and will take time. Thanks for the continued patience.


    ~The SWGEMu Staff
    This article was originally published in forum thread: Basilisk extended downtime 08/09/2017 started by Vlada View original post