PSA - UK2 & UKLR Zones - Service unavailable or long loading time - Under Investigation
Incident Report for Datto
Postmortem

On September 04, 2020 starting at 12:05 UTC and lasting until 13:15 UTC, partners with PSA databases in the LON datacenter experienced a service interruption which caused an error page to be displayed instead of the normal login page. Users with established connections would have received errors and would have been disconnected from their session. 

The root cause for this service interruption was a Distributed Denial of Service (DDOS) attack on an IP block that is advertised, but not owned by SunGard Availability. SunGard Availability does maintain active DDOS filters to mitigate against DDOS attacks, but the existing process only accounts for IP blocks that SunGard owns. While traffic on the affected links returned to a normal state prior to SunGard Availability performing any remediation actions, the affected IP block was added to the DDOS filters. Additionally, SunGard Availability is reviewing all of their DDOS filters against advertised IP blocks in all of their data centers to ensure that IP ranges are added as needed. SunGard Availability will also review their Change Management process to ensure that there are no gaps in configurations moving forward.

Posted Sep 16, 2020 - 13:35 UTC

Resolved
This incident has been resolved.
Posted Sep 04, 2020 - 14:36 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Sep 04, 2020 - 13:13 UTC
Update
We are continuing to investigate this issue.
Posted Sep 04, 2020 - 13:13 UTC
Update
We are continuing to investigate this issue.
Posted Sep 04, 2020 - 12:25 UTC
Investigating
Our teams are currently investigating connectivity issues for PSA in the UK2 & UKLR zones. An update will be posted here with the status of this investigation.

We apologise for the inconvenience.
Posted Sep 04, 2020 - 12:24 UTC
This incident affected: Autotask PSA (UK Limited Release (United Kingdom), UK 2 (United Kingdom)).