May 7, 2013 |
Early this morning we experienced a major outage of the main database running our WordPress E-Commerce and Dreamweaver websites. This outage resulted from a problem with the main datastore used by this database. Modern Retail and Rackspace engineers were engaged immediately when the server went down around 2:00 am CST.
For those of you who've been a long-time customer, you know Modern Retail tries to avoid downtime at all cost. This is our promise to you and we're committed in having one of the the fast, safest and most secure infrastructures for your website. Unfortunately due to human error several safeguards used to protect your data became corrupt. This resulted in us only being able to restore your data from 9:00 pm CST last night. However, Modern Retail is manually reconstructing all missing orders from 9:00 pm to 2 am CST. These orders will show up in Store Manager some time in the next 24-48 hours.
Here's what's being done about today's outage:
- Rackspace Visit - Three key Modern Retail employees (Todd Myers, Roy Plum & Jeff Leyser) will be traveling to San Antonio to meet with Rackspace in person on Friday. The goal of the trip is to get a detailed explanation of tonight's issues, get a better understanding of why the safeguards failed and come up with a plan to minimize potential future problems.
- Open Communication - We will be reporting back to you after our visit with Rackspace to give you a detailed plan on what's being done to avoid this kind of problem in the future. Rackspace will also be issuing a release regarding today's outage.
- Restoring Orders - Modern Retail is working diligently to restore all orders placed between 9 pm and 2 am CST. This is a very manual process which will take many hours to complete.
We know you rely on Modern Retail for your online success and we know this outage has seriously dampened your confidence. We are committed to restoring your confidence in Modern Retail and reaching our up-time availability promise to you of 99.99%.
If you have any questions please Submit a Request in Store Manager or call us at (800) 640-1826 option #2 for Support. We're here to help.
Thank you,
Todd Myers
Modern Retail