All - As of 12:30pm, we believe that all servers and services are restarted. If you are still experiencing issues that you believe to be related to this event, please contact the US:IT Helpdesk. Steve. On Tue, Mar 24, 2020 at 11:17 AM Steven Premeau <[log in to unmask]> wrote: > DCO has restarted all underlying infrastructure and SysAdmins are now > checking individual servers and services. The next update will be at 12:30 > p.m. or when the process completes, whichever is sooner. > > Steve. > > On Tue, Mar 24, 2020 at 9:54 AM Steven Premeau <[log in to unmask]> > wrote: > >> All - >> >> As of 9:50am, Data Center and System Administration teams are are >> beginning to recover infrastructure services, starting with storage and >> virtualization services. >> >> Given the nature of the service crash, we are anticipating we will >> discover additional issues as we restart server systems. >> >> The next update will be no later than 1100. >> >> Steve. >> >> On Tue, Mar 24, 2020 at 9:41 AM Jeffrey Letourneau <[log in to unmask]> >> wrote: >> >>> *9:30 - *Work continues on installing and configuring replacement >>> equipment. Compute and storage resources are being brought back online. We >>> anticipate remaining work will take at least another hour to complete. >>> >>> >>> -- >>> Jeff Letourneau >>> Executive Director, Networkmaine >>> US:IT >>> University of Maine System >>> t: 207-581-3507 >>> >>> >>> >>> On Tue, Mar 24, 2020 at 8:39 AM Jeffrey Letourneau <[log in to unmask]> >>> wrote: >>> >>>> *8:30 am - *Data Center network switching fabric is back online. >>>> However, we have experience equipment failure that is preventing services >>>> based in the Portland Data Center from coming online. Replacement >>>> equipment has been identified and is in the process of being installed and >>>> configured. >>>> >>>> >>>> -- >>>> Jeff Letourneau >>>> Executive Director, Networkmaine >>>> US:IT >>>> University of Maine System >>>> t: 207-581-3507 >>>> >>>> >>>> >>>> On Tue, Mar 24, 2020 at 7:08 AM Jeffrey Letourneau <[log in to unmask]> >>>> wrote: >>>> >>>>> Some equipment has not come back up cleaning after power was restored. >>>>> The network switching fabric in servicing the compute and storage >>>>> resources in the Data Center remain down as do a small number of wide area >>>>> network services. Staff continue to work at bringing all services back up >>>>> to full operational status. >>>>> Jeff >>>>> -- >>>>> Jeff Letourneau >>>>> Executive Director, Networkmaine >>>>> US:IT >>>>> University of Maine System >>>>> t: 207-581-3507 >>>>> >>>>> >>>>> >>>>> On Tue, Mar 24, 2020 at 5:35 AM Jeffrey Letourneau <[log in to unmask]> >>>>> wrote: >>>>> >>>>>> Root cause was determined to be a loss of power in the Portland Data >>>>>> Center. Power has been restored. Network and computing equipment booting >>>>>> up. All services should be restored shortly. >>>>>> Jeff >>>>>> -- >>>>>> Jeff Letourneau >>>>>> Executive Director, Networkmaine >>>>>> US:IT >>>>>> University of Maine System >>>>>> t: 207-581-3507 >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Mar 24, 2020 at 3:33 AM Jeffrey Letourneau <[log in to unmask]> >>>>>> wrote: >>>>>> >>>>>>> We have lost network communications with the Portland and Gorham >>>>>>> campuses including the Portland Data Center at approximately 2:40am. Staff >>>>>>> are on route to investigate. Current suspect storm relate power problem or >>>>>>> fiber cut. >>>>>>> >>>>>>> Sites that are singly home to Portand are also unreachable at this >>>>>>> time. Wide area network connectivity, Internet access, and telephone >>>>>>> services remain up on secondary paths for all other campuses. >>>>>>> >>>>>>> We will post more information as it becomes available. >>>>>>> Jeff >>>>>>> >>>>>>> -- >>>>>>> Jeff Letourneau >>>>>>> Executive Director, Networkmaine >>>>>>> US:IT >>>>>>> University of Maine System >>>>>>> t: 207-581-3507 >>>>>>> >>>>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "US:IT Leadership Team" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [log in to unmask] >>> To view this discussion on the web visit >>> https://groups.google.com/a/maine.edu/d/msgid/usit-leadership-team-group/CAA9QOjXrN2Rz-MvaFXHfKs7noWF48FCdA_M0mF%2B3U6EXsXu3Rw%40mail.gmail.com >>> <https://groups.google.com/a/maine.edu/d/msgid/usit-leadership-team-group/CAA9QOjXrN2Rz-MvaFXHfKs7noWF48FCdA_M0mF%2B3U6EXsXu3Rw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> >> >> -- >> Steven Premeau, Director of Enterprise Systems Architecture & >> Administration >> University Services: Information Technology >> University of Maine System | (207) 581-5836 (desk) | (207) 944-9391 >> (mobile) >> > > > -- > Steven Premeau, Director of Enterprise Systems Architecture & > Administration > University Services: Information Technology > University of Maine System | (207) 581-5836 (desk) | (207) 944-9391 > (mobile) > -- Steven Premeau, Director of Enterprise Systems Architecture & Administration University Services: Information Technology University of Maine System | (207) 581-5836 (desk) | (207) 944-9391 (mobile)