October 2024 - Members - lists.fl-ix.net

RFO Attached: (was: Outage on edge01.mia002 (Equinix-MI1) / Investigation in progress)
by Randy Epstein 29 Oct '24

29 Oct '24

Hello FL-IX Members, Please see attached RFO regarding this incident. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail From: Randy Epstein <randy(a)communityix.org> Date: Sunday, October 20, 2024 at 8:52 PM To: <members(a)lists.fl-ix.net> Subject: UPDATE: Outage on edge01.mia002 (Equinix-MI1) / Investigation in progress Hello FL-IX Members, This matter is still under investigation. We ended up reloading the switch (edge01.mia002) and it’s now normalized and passing traffic again. We’re opening a ticket with Arista to investigate further to determine root cause. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail From: Randy Epstein <randy(a)communityix.org> Date: Sunday, October 20, 2024 at 8:34 PM To: <members(a)lists.fl-ix.net> Subject: Outage on edge01.mia002 (Equinix-MI1) / Investigation in progress Hello FL-IX Members, We’re currently investigating an outage on one of our edge switches at Equinix-MI1. We’ll report back shortly once we determine the cause. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail

1 0

COMPLETE: Emergency Maintenance at Equinix-MI1 - Tuesday, October 29th, at 12:01am (ET) - Switchcard replacement on edge01.mia002
by Randy Epstein 29 Oct '24

29 Oct '24

Hello FL-IX Members, The maintenance at Equinix-MI1 has been completed. Please note that our portal (my.fl-ix.net) is still down. This should return within the next 4 hours. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail From: Randy Epstein <randy(a)communityix.org> Date: Monday, October 28, 2024 at 11:56 PM To: "members(a)lists.fl-ix.net" <members(a)lists.fl-ix.net> Subject: REMINDER: Emergency Maintenance at Equinix-MI1 - Tuesday, October 29th, at 12:01am (ET) - Switchcard replacement on edge01.mia002 Hello FL-IX Members, This maintenance is about to start in 5 minutes. BGP will be blocked on edge01.mia002. Members directly connected to this switch will see their ports go down for about 10 minutes while we replace the switch card. Members that have bilateral sessions with members connected to edge01.mia002 will see those sessions go down. I’ll send another notice when we are hands off. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail From: Randy Epstein <randy(a)communityix.org> Date: Thursday, October 24, 2024 at 12:26 PM To: "members(a)lists.fl-ix.net" <members(a)lists.fl-ix.net> Subject: Emergency Maintenance at Equinix-MI1 - Tuesday, October 29th, at 12:01am (ET) - Switchcard replacement on edge01.mia002 Greetings FL-IX Members, As a follow-up precautionary action related to the outage that occurred on October 20th, 2024, we will be performing an emergency maintenance to replace the switchcard in edge01.mia002 on Tuesday, October 29th, 2024 at 12:01am ET. To note, there’s been no further issues with the switch. Arista has determined it’s most likely hardware related and wanted to take a wait and see approach, but we felt it would be in our member’s best interest to just go ahead and replace the switchcard in a controlled setting. For the duration of the maintenance, we will block BGP traffic only to those members affected by the maintenance as described in BCP 214. Sessions will re-establish after the maintenance has concluded. No action is required on your part. To those members not directly connected to this switch, you will see sessions go down to the members that are directly connected, so please keep that in mind. Additionally, while we are on-site, we will be inserting a new 100G card into one of our Dell servers that serves as route-server 1 (206.41.108.4/2001:504:40:108::1:4). We don’t expect this to impact members as all members are built to both route servers. We expect to conclude this maintenance by 1:30am ET. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail

1 0

REMINDER: Emergency Maintenance at Equinix-MI1 - Tuesday, October 29th, at 12:01am (ET) - Switchcard replacement on edge01.mia002
by Randy Epstein 29 Oct '24

29 Oct '24

Hello FL-IX Members, This maintenance is about to start in 5 minutes. BGP will be blocked on edge01.mia002. Members directly connected to this switch will see their ports go down for about 10 minutes while we replace the switch card. Members that have bilateral sessions with members connected to edge01.mia002 will see those sessions go down. I’ll send another notice when we are hands off. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail From: Randy Epstein <randy(a)communityix.org> Date: Thursday, October 24, 2024 at 12:26 PM To: "members(a)lists.fl-ix.net" <members(a)lists.fl-ix.net> Subject: Emergency Maintenance at Equinix-MI1 - Tuesday, October 29th, at 12:01am (ET) - Switchcard replacement on edge01.mia002 Greetings FL-IX Members, As a follow-up precautionary action related to the outage that occurred on October 20th, 2024, we will be performing an emergency maintenance to replace the switchcard in edge01.mia002 on Tuesday, October 29th, 2024 at 12:01am ET. To note, there’s been no further issues with the switch. Arista has determined it’s most likely hardware related and wanted to take a wait and see approach, but we felt it would be in our member’s best interest to just go ahead and replace the switchcard in a controlled setting. For the duration of the maintenance, we will block BGP traffic only to those members affected by the maintenance as described in BCP 214. Sessions will re-establish after the maintenance has concluded. No action is required on your part. To those members not directly connected to this switch, you will see sessions go down to the members that are directly connected, so please keep that in mind. Additionally, while we are on-site, we will be inserting a new 100G card into one of our Dell servers that serves as route-server 1 (206.41.108.4/2001:504:40:108::1:4). We don’t expect this to impact members as all members are built to both route servers. We expect to conclude this maintenance by 1:30am ET. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail

1 0

Emergency Maintenance at Equinix-MI1 - Tuesday, October 29th, at 12:01am (ET) - Switchcard replacement on edge01.mia002
by Randy Epstein 24 Oct '24

24 Oct '24

Greetings FL-IX Members, As a follow-up precautionary action related to the outage that occurred on October 20th, 2024, we will be performing an emergency maintenance to replace the switchcard in edge01.mia002 on Tuesday, October 29th, 2024 at 12:01am ET. To note, there’s been no further issues with the switch. Arista has determined it’s most likely hardware related and wanted to take a wait and see approach, but we felt it would be in our member’s best interest to just go ahead and replace the switchcard in a controlled setting. For the duration of the maintenance, we will block BGP traffic only to those members affected by the maintenance as described in BCP 214. Sessions will re-establish after the maintenance has concluded. No action is required on your part. To those members not directly connected to this switch, you will see sessions go down to the members that are directly connected, so please keep that in mind. Additionally, while we are on-site, we will be inserting a new 100G card into one of our Dell servers that serves as route-server 1 (206.41.108.4/2001:504:40:108::1:4). We don’t expect this to impact members as all members are built to both route servers. We expect to conclude this maintenance by 1:30am ET. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail

2 1

Outage on edge01.mia002 (Equinix-MI1) / Investigation in progress
by Randy Epstein 21 Oct '24

21 Oct '24

Hello FL-IX Members, We’re currently investigating an outage on one of our edge switches at Equinix-MI1. We’ll report back shortly once we determine the cause. Regards, Randy Epstein Executive Director Email: randy(a)communityix.org Mobile: +1 561-756-4475 Office: +1 888-925-4678 Fax: +1 561-431-0437 P Please consider the environment before printing this e-mail

1 1

UPGRADED: Cisco (AS36692) has upgraded to 2x10 Gigabit Ethernet!
by Randy Epstein 07 Oct '24

07 Oct '24

Members, Cisco has upgraded to 2x10 Gigabit Ethernet. If you don’t already peer with them, please contact them. They are also connected to the route servers. AS: 36692 v4: 206.41.108.105 v6: 2001:504:40:108::1:105 Further details can be found in the portal at: https://my.fl-ix.net/switch/configuration Best, The FL-IX Team

1 0