Hyonix - Los Angeles Outage – Incident details

It appears that due to the recent power outages, one of our network optics to the main network were affected and has became faulty causing the entire connection to constantly flaps, due to the link keeps flapping and not going down entirely, the auto fallback procedure did not kick in.

We've now disabled this faulty port and re-enabled our main network. Everything remains stable at this time. Our technician is scheduled to go to the Datacenter to perform replacement of the network optics by end of tomorrow, no further network impact is expected on our end at this time.

Our apologies for any inconvenience caused. We understand that the recent incidents have raised concerns about our service quality. Please rest assured that we are working actively internally to make sure these kind of incidents to not resurface again.

Our team is still pending Reason for outage (RFO) from the Datacenter in the recent power outage and steps taken to prevent this from happening again as they mentioned that they are conducting a throughout review before issuing an official statement. We will post update here once we hear back from them

Thank you for your patience and understanding on this matter!

We've been able to narrow this down to network-only event and not power event. Our team is looking into this.

We are observing very high packet losses toward our network, suspecting transport fault to our core network location.

Our team is forcing fall back to backup network at this time. Elevated latency is expected until we restore the main network.

It appears that our infrastructure is down again just now. We are contacting Datacenter for more information at this time.

Further updates will follow as we receive information from Datacenter staff.

Our apologies for any inconvenience caused. Please rest assured that our team is actively working on this issue at the moment.

Power and network have been restored completely. Our team will keep this incident open until we receive an official update from the datacenter.

Datacenter electrical engineers are on-site and working to restore power to racks.

Uninterruptible Power Supply (UPS) could not be turned on due to the fire alarms went off.

There is no official estimation as to when this will be completed yet. Our team is still on-site and actively working to get more information at this time.

Our in-house technician has arrived on site and confirmed that all our devices are not affected. We still haven't heard back from Datacenter site team about estimated time of restoration at this time.

Our team is still working actively on this and will provide more updates soon!

The Datacenter has confirmed that the fire alarm went off, which is why all power within the suite was cut.

Currently, the power is being restored to the suite as the site team is clearing the alarm and assessing the damages. However, it will take time to restore power to the racks. We are still trying to get an estimation from the Datacenter on when our racks can be restored.

We will provide more updates as we receive more information from the Datacenter! Thank you for your patience and understanding on this issue.

We've received notification from the Datacenter that the entire suite is experiencing power outages. Datacenter operator is aware of this situation and is investigating into it.

Our in-house technician has been dispatched on site at this time. Estimated 30-40 minutes for arrival.

We are still actively working with the Datacenter on this issue and will provide updates as more information is available to us.

Our apologies for any inconvenience caused.