12

RA Status: Update 7 (Read 563 times)

    Hi all,

    Hopefully this will be the last update.  As of 7 AM EDT today, RA's primary file server finally came back online and is now serving the web servers.  It means the workout importer is no longer affected by the performance of the link to the backup file server and can process your GPS uploads nearly instantaneous again.

     

    As I indicated in a previous update, there was some data loss because file backups were done on a nightly basis.  GPS data uploaded on June 19 to June 20, depending on your time zone, cannot be recovered.  You'll need to upload them again.

     

    The data loss is the first in RA history.  I am looking for backup/replication solutions that will allow for more frequent backups without impacting the website's responsiveness.

     

    Thank you all for your patience and understanding through this ordeal.

     

    eric Smile

      And thank you for your monumental patience and diligence through this ordeal.

      Smile

      “Everything you need is already inside.” -- Bill Bowerman

        And a big thank you Eric, you da man!

          My avatar is back!  I missed the swan.  I have no idea where that photo is on my computer, so I couldn't get it back on my own.

            This was a tough few days for you.  Thanks so much for your diligence and hard work!

            Lou, (aka Mr. predawnrunner), MD, USA | Lou's Brews | lking@pobox.com

              Yes, thank you so much Eric for your diligence and expertise in resolving this huge problem!!!  We appreciate you sooooo much!!!!!

              Use your momentum...keep going.  You know you can make it.

              Enric Hilversum


                Great job mate!


                Interval Junkie --Nobby

                  The data loss is the first in RA history.  I am looking for backup/replication solutions that will allow for more frequent backups without impacting the website's responsiveness.

                   

                  All things considered, this is actually very impressive.  The data loss, being confined to a few days' logs, is so small.  Great job.

                   

                  I don't think anyone expects 5x9s from a free Running Log. Wink

                  2014 Goals:  sub-3 Marathon ("Congrats! It's tough to race with poop in the mind" --Wing)

                  Current Status 03/17: Drinking beer and eating crap -- all the things I couldn't do before the marathon

                     

                    All things considered, this is actually very impressive.  The data loss, being confined to a few days' logs, is so small.  Great job.

                     

                    I don't think anyone expects 5x9s from a free Running Log. Wink

                     

                    Any data loss is unacceptable.  While nightly backups are standard practice in the IT world, the window is still too large given that users are uploading data 24 hours a day.  This episode exposed weaknesses in my backup procedures.  The recently added backup file server makes RA more resilient to equipment failures.  It also reduces the amount of down time in the event of a failure.

                     

                    On a related note, I am testing a backup solution that can make backups every 15 minutes without impacting performance.  There are solutions that can make nearly instantaneous backups.  I haven't ruled them out yet, although some of them have very high resource requirements while others seem to be no longer maintained.

                      Is a Google or Amazon data store a possibility?

                      Well at least someone here is making relevance to the subject.

                        Is a Google or Amazon data store a possibility?

                         

                        I would love to move all my servers into the cloud.  The biggest benefit is I don't have to maintain the hardware.  I can pay as I go, and only pay for what I need.  I have to create servers that are more powerful than what I need at the moment so that they can handle future growth.  That money could have been saved using the cloud, but...

                         

                        The biggest drawback with the cloud is price.  I looked at Amazon's ECS, Microsoft's Azure, and Google's Compute Engine every so often, hoping that their pricing models have changed.  Their virtual machine options do not fit my usage.  It seems that the amount of memory for a given machine is proportional to the disk size.  I want a machine with lots of memory and a small disk as my database server.  The file server has the opposite requirement: large disks and (relative) little memory.  RA has about 1 TB worth of compressed data, all of which must be persistent.  Persistent storage is very expensive.

                         

                        In the end, owning the hardware and paying for space and bandwidth in a data center is still more cost effective than all other options I've explored.  It's much more work for me should something goes wrong, but it is what I can afford with the ad revenue and your financial support.

                          Thanks Eric. You rock.

                          Run, Walk, Crawl, just Finish.

                          HF # 1189

                             

                            I would love to move all my servers into the cloud.  The biggest benefit is I don't have to maintain the hardware.  I can pay as I go, and only pay for what I need.  I have to create servers that are more powerful than what I need at the moment so that they can handle future growth.  That money could have been saved using the cloud, but...

                             

                            The biggest drawback with the cloud is price.  I looked at Amazon's ECS, Microsoft's Azure, and Google's Compute Engine every so often, hoping that their pricing models have changed.  Their virtual machine options do not fit my usage.  It seems that the amount of memory for a given machine is proportional to the disk size.  I want a machine with lots of memory and a small disk as my database server.  The file server has the opposite requirement: large disks and (relative) little memory.  RA has about 1 TB worth of compressed data, all of which must be persistent.  Persistent storage is very expensive.

                             

                            In the end, owning the hardware and paying for space and bandwidth in a data center is still more cost effective than all other options I've explored.  It's much more work for me should something goes wrong, but it is what I can afford with the ad revenue and your financial support.

                             

                            Obviously, you've done your homework (I should have known).  Thanks again for all your dedication to the members of this site!

                            Well at least someone here is making relevance to the subject.


                            A Saucy Wench

                              Thanks for the explanation on the cloud eric.  Otherwise my dh would have worn your ear off about it .

                              I have become Death, the destroyer of electronic gadgets

                               

                              "When I got too tired to run anymore I just pretended I wasnt tired and kept running anyway" - dd, age 7

                                Thank you Eric!

                                12