The cause of last night's outage has been positively isolated to a software defect.  Specifically, a false over-temperature report from a fabric module sensor triggered an emergency power-down of all fabric modules and linecards on the core switch at 36 NE 2nd, which subsequently recovered without intervention.

The shutdown occurred at 22:06:07 and forwarding resumed at 22:29:53.

To prevent any recurrence, we will apply a patch at 1am EDT on Sunday, May 15.  The patch is non-disruptive; no impact to forwarding is anticipated.

Thank you,

Ryan Woolley

On Saturday, May 14, 2016 at 12:21:58 AM UTC-4, randy wrote:

FL-IX Members,

We have received reports from a small number of members that are not seeing
link at the 36 NE 2nd location.

Our engineers are investigating this issue and we hope to have some answers
shortly as to the root cause of the problem and a resolution.

Thank you for your  understanding.

Regards,

Randy Epstein
Community IX / FL-IX

_______________________________________________
Members mailing list
Members@lists.fl-ix.net
https://lists.fl-ix.net/mailman/listinfo/members