1

Unexpected Outage (Read 281 times)

    Hi all,

    RA was offline at around 4 PM EDT for about an hour.  I was able to reach the servers.  I contacted the data center and they did a remote reboot of the servers, which brought everything back online.  I am looking into the cause of the outage, which I suspect is a hardware problem.  I will keep you posted when I get more info.

     

    eric Smile


    Interval Junkie --Nobby

      And there was much rejoicing! . . . followed by trepidation about hardware problems . . . then fear of gremlins . . . then people turned to blame a North Korean cyber attack . . . but once they figured out it was all Justin Bieber's fault, there was much rejoicing again!

      2014 Goals:  sub-3 Marathon ("Congrats! It's tough to race with poop in the mind" --Wing)

      Current Status 03/17: Drinking beer and eating crap -- all the things I couldn't do before the marathon

        I was not happy that the data center personnel rebooted the servers, even though that "fixed" the problem.  I asked that they confirm it is a server and not a network problem.  They could have just checked the consoles but I suppose hooking up a monitor is too much work and opted for the reboot solution.  Thus far, I am unable to find the problem because apparently the server in question only keeps the logs in memory and not disk...  Who's bright idea is that?!


        Interval Junkie --Nobby

          Yeah, I have some Ops folks who don't understand that "we need to catch it in the act!"

           

          In memory logs -- that's a new one on me though.  BRILLIANT!

          2014 Goals:  sub-3 Marathon ("Congrats! It's tough to race with poop in the mind" --Wing)

          Current Status 03/17: Drinking beer and eating crap -- all the things I couldn't do before the marathon

            If I wasn't trapped in the office it would have been a good time for a run.

              I went through all the logs but could not glean any info regarding the outage.  I believe it's a hardware problem, which might only be displayed on the console.  This means it might happen again.

                I was not happy that the data center personnel rebooted the servers, even though that "fixed" the problem.  I asked that they confirm it is a server and not a network problem.  They could have just checked the consoles but I suppose hooking up a monitor is too much work and opted for the reboot solution.  Thus far, I am unable to find the problem because apparently the server in question only keeps the logs in memory and not disk...  Who's bright idea is that?!

                 

                I actually started to get nervous after about 30 minutes. Is it standard procedure for these guys not to write the server logs to disk? That sounds a little wacky.

                   

                  I actually started to get nervous after about 30 minutes. Is it standard procedure for these guys not to write the server logs to disk? That sounds a little wacky.

                   

                  It depends on the type of server.  The firewalls (that I use) do not write to disk because of the volume of traffic.  I'm not keen about this because if there's an attack, I would like to go through the logs for additional info.  The web servers and database servers do log to disk but they all stopped logging at around 3:30 PM EDT, indicating the problem is upstream of them.  The firewalls do log the (hourly, daily, weekly, monthly and yearly) traffic trends, but all I get is a gap caused by the outage.  I know the exact start time of the outage but that doesn't help me.


                  day after day sameness

                    It was my fault, I just thought it was the light switch....not the on/off for the internet. Sorry 'bout that.

                    I've done my best to live the right way; I get up every morning and go to work each day...

                      I don't know if it's related to the "unexpected outage" yesterday, but the user group forums have a formatting error.

                      Anybody else have this problem?

                      (It was ok yesterday afternoon before the outage).

                       

                      MTA.... I think it's my browser.  It's working normal on my iphone, but not on my PC.  Disregard, I think.

                      2014 Goals:

                      #1: Do what I can do. <DOING>

                      #2: 365 Hours training

                       

                        Sorry, but every time I see this thread title I read it as "Unexpected Outrage"...

                        "I can do 440 in 220"    Half Fanatic #846    "90% of running is half mental"    If I collapse, please pause my Garmin