Hyonix - NYC2 Outage – Incident details

This incident has been marked as resolved as there has been no further issues after 7 days!

The Datacenter Operators have confirmed that maintenance has been completed, there is no further works being done, all powers have been restored.

Our team will still monitor the situation for 1-2 days before closing this incident.

We apologizes for any issues or inconvenience may have caused.

Thank you for your patience and understanding during this process.

Please let us know via ticket if you still have any issues!

We've completed assessment on this incident.

Root cause: The Datacenter is performing maintenance related to electrical system upgrade. During the maintenance window, the power circuits of the cabinet row hosting our main network cabinet has tripped, causing our routers to lose both main and redundant power sources, leading to all other cabinets losing network access.

Solution: Upon arrival, our technician noticed that many electricians were aware of the problem and were already actively working on it. The power was restored shortly after, however due to the power getting completely tripped, a few of our cabinets were affected. Our tech team started to go over every cabinet to ensure that all servers were brought online.

Current state: All servers are back online and operational at this time. The Datacenter is still under maintenance but don't expect any further issues. We were given ETA for completion to be at 09/18/2024 04:00:00 am EDT. Our team will update once we heard confirmation from the Datacenter.

Customers with servers restarted due to the power loss will be notified via ticket. Our team will be monitoring the issue closely.

Our apologies for any inconvenience caused.

Please reach out to us via ticket if you still having issues with your servers.

Datacenter Operators confirmed the power outage in a section of the Datacenter hall. Affecting many rows of cabinets. Electricians are also on site and bringing the power back up.

Our team is currently assessing the impact of the power outages and will post further update shortly.

Our technician is currently on-site at this time, it appears the Datacenter is suffering a major power outages. we are still checking with Datacenter Operators on the root cause.

More update to follow!