AWS outage

Incident Report for Foresight Bank

Postmortem

Between 11:49 PM PDT on October 19 and 2:24 AM PDT on October 20, Amazon experienced increased error rates and latencies for AWS Services in the US-EAST-1 Region. Additionally, services or features that rely on US-EAST-1 endpoints such as IAM and DynamoDB Global Tables also experienced issues during this time. At 12:26 AM on October 20, Amazon identified the trigger of the event as DNS resolution issues for the regional DynamoDB service endpoints. After resolving the DynamoDB DNS issue at 2:24 AM, services began recovering but Amazon had a subsequent impairment in the internal subsystem of EC2 that is responsible for launching EC2 instances due to its dependency on DynamoDB. As Amazon continued to work through EC2 instance launch impairments, Network Load Balancer health checks also became impaired, resulting in network connectivity issues in multiple services such as Lambda, DynamoDB, and CloudWatch. Amazon recovered the Network Load Balancer health checks at 9:38 AM. As part of the recovery effort, Amazon temporarily throttled some operations such as EC2 instance launches, processing of SQS queues via Lambda Event Source Mappings, and asynchronous Lambda invocations. Over time Amazon reduced throttling of operations and worked in parallel to resolve network connectivity issues until the services fully recovered. By 3:01 PM on October 20th, all AWS services returned to normal operations.

Posted Oct 21, 2025 - 15:12 CDT

Resolved

This incident has been resolved.
Posted Oct 21, 2025 - 06:09 CDT

Update

Adding Finastra Mortgagebot to this incident. Most recent update from Amazon -- "...mitigations to resolve launch failures for new EC2 instances continue to progress and we are seeing increased launches of new EC2 instances and decreasing networking connectivity issues in the US-EAST-1 Region. "
Posted Oct 20, 2025 - 13:41 CDT

Update

Adding Verafin to this.
Posted Oct 20, 2025 - 11:07 CDT

Update

Currently, approximately 79 AWS services are experiencing degraded performance. The root cause has been traced to issues within AWS’s Identity and Access Management (IAM), Security Token Service (STS), and Secrets Manager — all of which are essential for secure authentication and service functionality. We’re also aware that some vendors are unable to initiate failover to their Disaster Recovery environments, as the impacted services are critical to that process.
Posted Oct 20, 2025 - 11:04 CDT

Update

Amazon AWS has reported additional widespread API errors and connectivity disruptions affecting multiple services. Currently, we are not observing any direct impact on end users or infrastructure.
Posted Oct 20, 2025 - 09:45 CDT

Monitoring

Amazon is reporting the underlying DNS issue has been fully mitigated, and most AWS Service operations are succeeding normally.
Posted Oct 20, 2025 - 08:04 CDT

Identified

Amazon web services is experiencing an outage impacting multiple vendors. Known services impacted thus far are Apiture and mimecast.
Posted Oct 20, 2025 - 06:32 CDT
This incident affected: Webmail/E-mail, Digital Banking, 3rd Party (Mimecast, Verafin), and Lending Services (Finastra - Mbot).