[Rainbow] We have seen a full disk failure on our Server hosting the Rainbow Reseller Server

Resolved
Resolved

We are seeing all webistes recovered.

If your site is still down please reach out ASAP via any support channel for us to diagnose the issue.

Caching as of 08:00 has been enabled and site speeds are starting to return to normalised speeds. We have added extra resources to Rainbow during this time to help cope with Resellers using their sites more than useless this Morning.

We again apologies for any inconvenience this may have caused.

Both teams are now working with hardware Manafacture to find the root cause.

Avatar for

We are seeing around 90% of Accounts Restored.

Avatar for

Restoration is still in progress but we are seeing over 50% of accounts restored successfully.

Avatar for

We are still seeing Website successfully recover from Backup. We are around 20% of the way through the backup and will continue to update as things progress.

Avatar for
Recovering

Team 1 Update - Restoration in progress, accounts slowly starting to restore in alphabetical order.

Team 2 Update - Confident that MySQL Database will be recoverable, Strong belief Home Files are sadly corrupted beyond repair.

Avatar for

Team 1 Update - Small Hold up on the /root Partition, currently resolving this.

Team 2 Update - Partition Restoring for home still in progress

Avatar for

Team 1 Update - Faulty Hardware replaced, cPanel Installed. Configuring CloudLinux and JetBackup to start Restore process from a previous backup (1AM this morning)

Team 2 Update - Faulty Hardware Received, confident recovery of MySQL Partition as table looks intact, Home Files partition is causing more of a problem.

Avatar for

We have two teams working on getting things resolved -

Team 1 - Replace faulty Hardware and reinstall with last known backup

Team 2 - Pull faulty hardware and attempt to recover old data

Avatar for
Investigating

We regret to inform you that a bad RAID Controller has caused a full disk failure on the Rainbow Server.

Our team are currently redeploying Rainbow from the last available backups.

We really do apologies for all the inconvenience on rainbow over the last few days and we have all resources focused on getting things back online

Avatar for
Began at:

Affected components
  • Hosting
    • Reseller Hosting