All -
As of 12:30pm, we believe that all servers and services are restarted.
If you are still experiencing issues that you believe to be related to
this event, please contact the US:IT Helpdesk.
Steve.
On Tue, Mar 24, 2020 at 11:17 AM Steven Premeau <[log in to unmask]>
wrote:
> DCO has restarted all underlying infrastructure and SysAdmins are now
> checking individual servers and services. The next update will be at 12:30
> p.m. or when the process completes, whichever is sooner.
>
> Steve.
>
> On Tue, Mar 24, 2020 at 9:54 AM Steven Premeau <[log in to unmask]>
> wrote:
>
>> All -
>>
>> As of 9:50am, Data Center and System Administration teams are are
>> beginning to recover infrastructure services, starting with storage and
>> virtualization services.
>>
>> Given the nature of the service crash, we are anticipating we will
>> discover additional issues as we restart server systems.
>>
>> The next update will be no later than 1100.
>>
>> Steve.
>>
>> On Tue, Mar 24, 2020 at 9:41 AM Jeffrey Letourneau <[log in to unmask]>
>> wrote:
>>
>>> *9:30 - *Work continues on installing and configuring replacement
>>> equipment. Compute and storage resources are being brought back online. We
>>> anticipate remaining work will take at least another hour to complete.
>>>
>>>
>>> --
>>> Jeff Letourneau
>>> Executive Director, Networkmaine
>>> US:IT
>>> University of Maine System
>>> t: 207-581-3507
>>>
>>>
>>>
>>> On Tue, Mar 24, 2020 at 8:39 AM Jeffrey Letourneau <[log in to unmask]>
>>> wrote:
>>>
>>>> *8:30 am - *Data Center network switching fabric is back online.
>>>> However, we have experience equipment failure that is preventing services
>>>> based in the Portland Data Center from coming online. Replacement
>>>> equipment has been identified and is in the process of being installed and
>>>> configured.
>>>>
>>>>
>>>> --
>>>> Jeff Letourneau
>>>> Executive Director, Networkmaine
>>>> US:IT
>>>> University of Maine System
>>>> t: 207-581-3507
>>>>
>>>>
>>>>
>>>> On Tue, Mar 24, 2020 at 7:08 AM Jeffrey Letourneau <[log in to unmask]>
>>>> wrote:
>>>>
>>>>> Some equipment has not come back up cleaning after power was restored.
>>>>> The network switching fabric in servicing the compute and storage
>>>>> resources in the Data Center remain down as do a small number of wide area
>>>>> network services. Staff continue to work at bringing all services back up
>>>>> to full operational status.
>>>>> Jeff
>>>>> --
>>>>> Jeff Letourneau
>>>>> Executive Director, Networkmaine
>>>>> US:IT
>>>>> University of Maine System
>>>>> t: 207-581-3507
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Mar 24, 2020 at 5:35 AM Jeffrey Letourneau <[log in to unmask]>
>>>>> wrote:
>>>>>
>>>>>> Root cause was determined to be a loss of power in the Portland Data
>>>>>> Center. Power has been restored. Network and computing equipment booting
>>>>>> up. All services should be restored shortly.
>>>>>> Jeff
>>>>>> --
>>>>>> Jeff Letourneau
>>>>>> Executive Director, Networkmaine
>>>>>> US:IT
>>>>>> University of Maine System
>>>>>> t: 207-581-3507
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Mar 24, 2020 at 3:33 AM Jeffrey Letourneau <[log in to unmask]>
>>>>>> wrote:
>>>>>>
>>>>>>> We have lost network communications with the Portland and Gorham
>>>>>>> campuses including the Portland Data Center at approximately 2:40am. Staff
>>>>>>> are on route to investigate. Current suspect storm relate power problem or
>>>>>>> fiber cut.
>>>>>>>
>>>>>>> Sites that are singly home to Portand are also unreachable at this
>>>>>>> time. Wide area network connectivity, Internet access, and telephone
>>>>>>> services remain up on secondary paths for all other campuses.
>>>>>>>
>>>>>>> We will post more information as it becomes available.
>>>>>>> Jeff
>>>>>>>
>>>>>>> --
>>>>>>> Jeff Letourneau
>>>>>>> Executive Director, Networkmaine
>>>>>>> US:IT
>>>>>>> University of Maine System
>>>>>>> t: 207-581-3507
>>>>>>>
>>>>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "US:IT Leadership Team" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [log in to unmask]
>>> To view this discussion on the web visit
>>> https://groups.google.com/a/maine.edu/d/msgid/usit-leadership-team-group/CAA9QOjXrN2Rz-MvaFXHfKs7noWF48FCdA_M0mF%2B3U6EXsXu3Rw%40mail.gmail.com
>>> <https://groups.google.com/a/maine.edu/d/msgid/usit-leadership-team-group/CAA9QOjXrN2Rz-MvaFXHfKs7noWF48FCdA_M0mF%2B3U6EXsXu3Rw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
>>
>> --
>> Steven Premeau, Director of Enterprise Systems Architecture &
>> Administration
>> University Services: Information Technology
>> University of Maine System | (207) 581-5836 (desk) | (207) 944-9391
>> (mobile)
>>
>
>
> --
> Steven Premeau, Director of Enterprise Systems Architecture &
> Administration
> University Services: Information Technology
> University of Maine System | (207) 581-5836 (desk) | (207) 944-9391
> (mobile)
>
--
Steven Premeau, Director of Enterprise Systems Architecture & Administration
University Services: Information Technology
University of Maine System | (207) 581-5836 (desk) | (207) 944-9391 (mobile)
|
