- Priority - Critical
- Affecting Server - Game US 3 | Kansas City
Node is offline again. We're trying to get this rebooted and looking in to the cause. We're doing one more normal reboot before looking into it and see if there's any issues with hardware. We'll update you once we have additional information.
EDIT 10:06 pm: This was rebooted from KVM 40 minutes ago but the server is still not responding to pings, and no ssh, since rebooted. I've asked data center technicians to take a look at the display and network connectivity, and a network technician is now checking on the issue. If they don't find an issue with the network we'll be booting it into rescue mode.
EDIT 10:19 pm: The node can not find a bootable OS. We are now going to boot it into a rescue iso. We'll attempt to make backups of your data to an offsite storage, if possible, and reinstall the node.
EDIT 11:50 pm: During the process of trying to boot the server into the rescue CD environment, the server locks up. While troubleshooting why the server was locking up, the technicians discovered that the PSU had gone bad and had caused the power to the SSDs to short out. As a result the SSDs are not detected at all, and when they are connected they start to overheat due to the short on the power circuitry of the SSD. Data is completely non-recoverable. The failed SSDs have been replaced, and we are looking in to restoring data from our backups.
- Date - 16-06-2020 19:40
- Last Updated - 16-06-2020 23:54
Below is a real-time overview of our servers where you can check if there's any known issues.