Server downtime 10/18/11

[reconroth88]

[14:13] <%Vlada> You can recover Lib?
[14:13] <@oru> yeah we are working on it
[14:13] <@oru> there is only a slight chance though
[14:13] <%Vlada> OK, how long?
[14:13] <%Vlada> tonight, tomorrow, 2 days
[14:14] <@oru> no idea
[14:19] <@oru> it takes so long because we want to make sure we will have more attempts to fix it


I wonder what Oru means by "slight" chance? haha
Looks like its back to char creation. **** happens I'll play regardless what emu staff do with server.

Good luck, Hope server is up for the weekend!

[StJohn]

I have never heard of multiple drives in a RAID array going bad @ the same time.

I have. Apart from the reasons you mentioned some drives will fail due to components failure. Not much to do about that when the drive/drives are faulty from the factory.

[pseudonim]

Okay folks after reading alot of posts I have a few thoughts. None of this is aimed at any of the SWGEmu Dev's or anyone else.

1. People seem to be more concerned with losing their toon than WHY the HDD crashed in the first place.
Being a computer junkie with network knowledge I have never heard of multiple drives in a RAID array going bad @ the same time. I have spoken with people who have MANY years in the server/network enviroment and they have never heard of such a thing either. They have stated that IF 1 drive shows signs of starting to fail you should replace all the drives just to be safe. They also stated that every server they have ever worked on runs some form of HDD monitoring software and servers running this software have never had an actual crash. SO..... Your toons should be the lease of your worries. If the Dev's can save them then fine, If not O-Well.
2. This should be bringing up some serious concerns about our current provider.
Not only HOW did we manage to lose both HDD's BUT how were there no issues detected by our host??

And finally.. After spending 30 min or so doing search after search concerning multiple HDD failures at the same time the only logical answer seems to be a physical issue. More specifically a power issue. The most common (as we all should know) either a surge, brown out or a faulty power supply would all result in possible multiple failures. Multiple failures in a defect free functioning system are nearly unheard of.

Just my thoughts. I know many of these topics have been discussed in this thread but I hope the SWGEmu team takes a serious look at how this could have possibly happened.

Thank you to all the Dev's and support teams. I know you are doing your best. Thank you for all your hard work. Especially the messangers, yours is probably the hardest job dealing with us in here.

There might be another explanation: The information we get here is not 100% correct, similar to the old game of Chinese Whispers (telephone, grapevine, or whatever you called it). I'd be much more inclined to believe that something has been lost in the information than what you propose.

In the end, we pretty much all need to chill out, and wait for the fix and full explanation.

[Ajuma]

I know it kills some people the game is down lol the fear of actually getting away from the computer and attempting a social life is just horrible

[jameski]

I think, Stop wasting time trying to save char info and just let us all start over again?

[pseudonim]

This was actually a great week for it to be down for me. I had a ton of work to do until Friday.

[Domo321]

[14:13] <%Vlada> You can recover Lib?
[14:13] <@oru> yeah we are working on it
[14:13] <@oru> there is only a slight chance though
[14:13] <%Vlada> OK, how long?
[14:13] <%Vlada> tonight, tomorrow, 2 days
[14:14] <@oru> no idea
[14:19] <@oru> it takes so long because we want to make sure we will have more attempts to fix it


I wonder what Oru means by "slight" chance? haha
Looks like its back to char creation. **** happens I'll play regardless what emu staff do with server.

Good luck, Hope server is up for the weekend!

We all have access to that thread, dont think there is any need with all the "oh no libs down" threads. Yes we all know we all cant log on either but I am sure everyone is doing all they can when they can to stop this stand still.

[Ajuma]

I think, Stop wasting time to trying save char info and just let us all start over again?

um negative ghost rider

[EvilGrass]

Okay folks after reading alot of posts I have a few thoughts. None of this is aimed at any of the SWGEmu Dev's or anyone else.

1. People seem to be more concerned with losing their toon than WHY the HDD crashed in the first place.
Being a computer junkie with network knowledge I have never heard of multiple drives in a RAID array going bad @ the same time. I have spoken with people who have MANY years in the server/network enviroment and they have never heard of such a thing either. They have stated that IF 1 drive shows signs of starting to fail you should replace all the drives just to be safe. They also stated that every server they have ever worked on runs some form of HDD monitoring software and servers running this software have never had an actual crash. SO..... Your toons should be the lease of your worries. If the Dev's can save them then fine, If not O-Well.
2. This should be bringing up some serious concerns about our current provider.
Not only HOW did we manage to lose both HDD's BUT how were there no issues detected by our host??

And finally.. After spending 30 min or so doing search after search concerning multiple HDD failures at the same time the only logical answer seems to be a physical issue. More specifically a power issue. The most common (as we all should know) either a surge, brown out or a faulty power supply would all result in possible multiple failures. Multiple failures in a defect free functioning system are nearly unheard of.

Just my thoughts. I know many of these topics have been discussed in this thread but I hope the SWGEmu team takes a serious look at how this could have possibly happened.

Thank you to all the Dev's and support teams. I know you are doing your best. Thank you for all your hard work. Especially the messangers, yours is probably the hardest job dealing with us in here.

Just a possible explaination to your 1 I work as an systems engineer for multiple customers and well I saw it happen multiple times that the RAID controller gives a powerpeak that all the discs on the RAID are broken. I hope the guys in the datacenter are aware of this and check the RAID controler, else the problem will be back in no time. (DELL-servers have this problem alot :'( )
Also lack of monitoring could be the problem and sadly I see most of the hosting companies overhere (Belgium) ask a fee for monitoring your hardware. In other words monitoring is an option not something that is done from start. You have to imagin that a big datacenter has over 1000s of disks and the chance that somebody goes into the server room to check the lights of all the discs is very small.

Possible solution but is very costly is to run 2 servers in cluster, if one has hardware failure the other takes it over. Same counts for the sql databases.

I totally agree on point 2

And I really appreciate the hard work everybody is doing on getting back online

[pre-cu ftw]

I think, Stop wasting time trying to save char info and just let us all start over again?

I think.. I agree with you.

[ledvan]

I would rather they fix Lib up and get it back online as fast as possible.

The Liberator server is a vital cog in the overall picture that is the SWGEMU Brand. The experience the SWGEMU team are gaining from running Liberator is very worthwhile, in regards to maintaining the server on a daily basis, implementing Event structures, fixing issues like this one, better understanding on how to come back from Crashes etc etc.

The experienced gained here will be vital when we launch 1.0, the team will have a better understanding on how to react and save data etc when were live with 1.0, which is vital for the continuity of the Brand as a whole package.

[pre-cu ftw]

I would rather they fix Lib up and get it back online as fast as possible.

The Liberator server is a vital cog in the overall picture that is the SWGEMU Brand. The experience the SWGEMU team are gaining from running Liberator is very worthwhile, in regards to maintaining the server on a daily basis, implementing Event structures, fixing issues like this one, better understanding on how to come back from Crashes etc etc.

The experienced gained here will be vital when we launch 1.0, the team will have a better understanding on how to react and save data etc when were live with 1.0, which is vital for the continuity of the Brand as a whole package.

Are you sure its not just because you got jedi not too long ago?

If a majority of people want a wipe to begin with, than your point is invalid anyways
Either way, there is little hope for data to be recovered.

[mikki369]

Okay folks after reading alot of posts I have a few thoughts. None of this is aimed at any of the SWGEmu Dev's or anyone else.

1. People seem to be more concerned with losing their toon than WHY the HDD crashed in the first place.
Being a computer junkie with network knowledge I have never heard of multiple drives in a RAID array going bad @ the same time. I have spoken with people who have MANY years in the server/network enviroment and they have never heard of such a thing either. They have stated that IF 1 drive shows signs of starting to fail you should replace all the drives just to be safe. They also stated that every server they have ever worked on runs some form of HDD monitoring software and servers running this software have never had an actual crash. SO..... Your toons should be the lease of your worries. If the Dev's can save them then fine, If not O-Well.
2. This should be bringing up some serious concerns about our current provider.
Not only HOW did we manage to lose both HDD's BUT how were there no issues detected by our host??

And finally.. After spending 30 min or so doing search after search concerning multiple HDD failures at the same time the only logical answer seems to be a physical issue. More specifically a power issue. The most common (as we all should know) either a surge, brown out or a faulty power supply would all result in possible multiple failures. Multiple failures in a defect free functioning system are nearly unheard of.

Just my thoughts. I know many of these topics have been discussed in this thread but I hope the SWGEmu team takes a serious look at how this could have possibly happened.

Thank you to all the Dev's and support teams. I know you are doing your best. Thank you for all your hard work. Especially the messangers, yours is probably the hardest job dealing with us in here.

some few years ago i used to own a computer hardware retail outlet, there was an occasion where i bought from the manufacturer (maxtor) a box of 120gb hdd's 20 in total. i sold them to various customers for various applications such as the main drive in their systems or backups to the main drive it could have been that they were used in raid arrays also but i don't know, My point is that without exception every single one of those drives failed with the same defect after a very short period of running. The fault was a mechanical problem inside the drive that was not detected by any monitoring software and failure was instant rather than degradation over time. All of these drives were replaced by Maxtor who told me that they had a problem with one batch of this size drives so i can quite easily believe that there could have been a similar incident with the 2 drives in our server. It's not beyond the realms of possibility is all i'm saying.

[Merkor]

I would rather they fix Lib up and get it back online as fast as possible.

The Liberator server is a vital cog in the overall picture that is the SWGEMU Brand. The experience the SWGEMU team are gaining from running Liberator is very worthwhile, in regards to maintaining the server on a daily basis, implementing Event structures, fixing issues like this one, better understanding on how to come back from Crashes etc etc.

The experienced gained here will be vital when we launch 1.0, the team will have a better understanding on how to react and save data etc when were live with 1.0, which is vital for the continuity of the Brand as a whole package.

thats wrong, liberator uses a old dead code. Its Incompatible in any way to OR Code. Also Adding Structures is not possible and they doenst gain anything from fixing liberator because they use for lib an old dead code. Its Just a FUN Server, nothing more. And BTW. Liberator does not have even Live Support. Its Just a FUN Server, also dont expect a high Priority for Liberator.

They Should Wipe it, so we have more testers on Nova, last time (during deco event) on nova was not even two hands full testers online on Prime Time. The Projekt Plan is to have a 1.0 Playable Version of SWG, not an OLD Coded FUN Server. More tester, faster fixing, faster Developing.

[MasterLFD]

I have. Apart from the reasons you mentioned some drives will fail due to components failure. Not much to do about that when the drive/drives are faulty from the factory.

Faulty from the factory??? We've been playing on those drives for over 6 months..