Past Incidents

The hardware that had an issue last night, was rebooted last night and seemed stable, but unfortunately it faulted again this morning around 7:40 this morning. Last night we had worked out a DR plan if this happened again which we invoked and had the majority of services back online within 7 minutes. We have since visited the hardware and migrated physically as many fibers as possible off the failing line card.Unfortunately at the same time a second fault occurred at another site that appears unrelated, but due to the first issue being ongoing it took longer to resolve than usual. This second issue effected a small number of WAN's that used the Lightwire Central FortiGate firewall platform.We will continue to work with the Vendor to resolve the issue permanently and a have spare card on standby if required.Again, we apologies for inconveniences these issues have caused.

The outage was caused by a device which control plane failed. While the New Zealand National network recovered in milliseconds, and Hamilton services or services relient on Hamilton continued to have trouble for about 10 more minutes until the device reset itself. All Hamilton based services then recovered other than those directly connected to that device, which took another 5 minutes to recover when the device reboot was complete.We will be investigating the root cause of the control plane failure with the Vendor and apologise for any inconvenience this incident may have caused. An incidenet report will be attached to this even in the next couple of days.