Hyonix - AMS1 Partial Power Outage – Incident details

All systems operational

AMS1 Partial Power Outage

Resolved
Partial outage
Started almost 2 years agoLasted about 20 hours

Affected

Virtualization Hosts

Partial outage from 7:01 PM to 7:53 PM, Operational from 7:53 PM to 2:44 PM

Amsterdam, Netherlands (AMS2)

Partial outage from 7:01 PM to 7:53 PM, Operational from 7:53 PM to 2:44 PM

Updates
  • Resolved
    UTC
    Resolved

    Both side of powers have been restored at the Datacenter, all Asus servers have been re-installed with redundant PSU at this time.

    We will now close this status as the issue is resolved.

    Please let us know via ticket if you still have any issues!

  • Monitoring
    UTC
    Monitoring

    We have temporarily detached the failed PSU as a workaround to this issue, the CPU has stopped throttling and all servers are reachable at this time.

    Please note that the servers were not down, just the performance of the servers were heavily degraded.

    Our team has reported this to Asus engineer and they are aware of the issue.

    We will keep monitoring the situation until the Datacenter power fully restored.

    Our apologize for any inconvenience caused!

    Please let us know via ticket if you still having issues.

  • Update
    UTC
    Update

    Our technician has arrived on-site and is checking on the issue.

  • Identified
    UTC
    Identified

    We are aware of the issue in our Amsterdam Datacenter (AMS1) causing 4 nodes to be unreachable.

    Our current Datacenter (Equinix AM5) is losing 1 power feed, and all our host nodes have started using redundant power supply, however a few Asus-based host nodes are currently heavily throttled due to missing power supply, it still remains online but the performance has been heavily degraded to the point where it's unresponsive.

    Our technician has been dispatched to the Datacenter to check on the issue. We've also contacted Asus for explanation as to why this behavior occurred.

    Further update will follow as we have more information to share.

    Our apologies for any inconvenience caused.