Hyonix - Los Angeles degraded performance – Incident details

All systems operational

Los Angeles degraded performance

Resolved
Degraded performance
Started about 1 year agoLasted 19 days

Affected

Core Networks

Degraded performance from 11:14 PM to 12:53 PM, Operational from 12:53 PM to 10:41 AM

Los Angeles, California (LAX1)

Degraded performance from 11:14 PM to 12:53 PM, Operational from 12:53 PM to 10:41 AM

Updates
  • Resolved
    UTC
    Resolved

    This incident has been marked as resolved as there was no further issues.

  • Monitoring
    UTC
    Monitoring

    After extended troubleshooting with all parties, we've identified that the issue is caused by a faulty cross-connect from Datacenter side. The cross-connect has been replaced at this time and our network is now back at full capacity. We will continue to monitor this issue for another 48 hours before closing.

    If you have any further questions about this incident, please let us know via ticket!

    Thank you for your patience and understanding.

  • Update
    UTC
    Update

    Currently our team is engaging Zayo technician but we still haven't been able to identify the root cause yet. We are suspecting an cross-connects fault on either Datacenters. Once Zayo has fully tested the fiber and the issue still not resolved, our team will start engaging with both Datacenters to troubleshoot on-site cross-connects.

  • Identified
    UTC
    Identified

    Our team is aware of a connectivity issue in our LAX1 location and has identified that our main transport link (dark fiber) has lost connectivity, our network has automatically fell back to on-site backup upstream. We've observed a brief inbound network loss for up to 15 minutes as networks start propagating routes globally. Outbound network was fallen back immediately.

    Currently, all service remains fully operational, however some network routes might not be optimal and might have higher latency than normal as our network is currently running on backup link.

    Our team has reached out to dark fiber provider (Zayo) for clarification and is escalating the issue.

    We will provide updates as more information become available to us.

    Thank you for your patience and understanding!