5th May 2022

Intermittent inbound call issue

Latest update:

Update: No new incidents have been spotted through the course of today. We are confident that the issue is resolved and will keep monitoring the platform closely for any re-occurance.


[2022-05-06 08:38:39 NZST] Update: Lightwire engineers have still not seen new occurrences of the issue after the maintenance overnight. We will continue monitoring the system throughout the day, however the incident is tentatively considered solved. We will update this event if we notice any anomalies.

[2022-05-05 21:01:29 NZST] Update: Services were restarted and Lightwire engineers are monitoring for any more occurrences of the issue. As of writing we have seen no new incidents, but we will keep monitoring as call load ramps up.

We will update this event once we're confident the incident has been resolved.

[2022-05-05 13:23:39 NZST] Update: Vendor recommendation is to restart service-affecting services. This will cause a small (1-2 minute) loss of service. We have scheduled this for 20:30 NZT, 18:30 AEST.

Due to the low impact of the fault (approximately 1% of inbound calls affected) we are waiting until after business hours to cause the more disruptive action of the services restart, however as we wish to restore the fault we are doing it earlier than our normal window of 1am to 5am.

Once done, we will confirm if the issue was resolved. Thank you for your patience.

[2022-05-05 12:35:09 NZST] Update: Lightwire engineers have cleared caches and restarted services in an attempt to resolve the issue which was not effective. As of writing we're still seeing a very low percentage of inbound calls intermittently failing. We have escalated the case with our vendor and will report back once we have a resolution.

[2022-05-05 10:18:35 NZST] Lightwire engineers are investigating a fault that has impacted a small number of inbound calls to New Zealand numbers hosted by Lightwire. Affected inbound calls are failing to route to customer PBXes, and timing out. Less than 1% of calls are affected, and a repeat call would generally succeed.

The fault has been reported to the vendor, and this event will be updated with more detail when available.