Service Disruption - 20 Oct

Incident Report for Dynamics Mobile

Postmortem

Incident Postmortem — Global Cloud Service Interruption (October 21, 2025)

Summary
On October 21, 2025, starting around 07:00 UTC, Dynamics Mobile experienced partial service degradation caused by a major global outage at our public cloud provider (AWS) affecting the us-east-1 region. This incident impacted our backend processing systems, web portal, and helpdesk site accessibility.

Throughout the event, mobile applications remained largely functional in offline mode, allowing field users to continue their daily operations without disruption. Data synchronization to ERP systems was delayed but preserved.

Impact

Some customers experienced temporary unavailability of the Dynamics Mobile Portal and Website.
Mobile app users could continue working offline, with automatic synchronization once the connection was restored.
ERP packet processing was delayed for several hours while the affected cloud services were recovering.

Timeline (EEST)

13:51 – Elevated error rates detected. Investigation confirmed dependency outage in the cloud provider’s infrastructure.
14:48 – System began collecting mobile transactions; ERP synchronization remained delayed.
14:54–15:17 – Services gradually recovered; backlog of mobile packets began processing.
15:20 – All systems confirmed fully operational.

Resolution and Follow-Up
Once the cloud provider restored stability, all delayed transactions were automatically processed and synchronized with ERP systems. No data loss occurred. Our team has since completed a full review of system performance and confirmed the platform’s full recovery.

We are implementing additional regional redundancy measures to reduce the impact of similar third-party outages in the future.

Current Status: ✅ Fully Operational
Offline Operations: ✅ Maintained During Incident
Data Integrity: ✅ No Loss Detected

We appreciate our customers’ patience and understanding during this event. The resilience of the Dynamics Mobile platform — particularly its offline-first architecture — allowed field operations to continue seamlessly despite global cloud disruptions.

Posted Oct 22, 2025 - 12:57 EEST

Resolved

The issue is resolved - the system is up and running and full operational.
We will provide issue overview once we summarize the the information from all sources.

Posted Oct 20, 2025 - 15:20 EEST

Update

The system operations are recovered. All delayed mobile packets are processed and sent to the ERP systems. It might take a while for each individual ERP system to catch up the backlog.

Posted Oct 20, 2025 - 15:17 EEST

Update

The service related to packet processing started to recover.

Clients across the affected regions shall see successfully processed packets in their ERP sync log.

Posted Oct 20, 2025 - 14:54 EEST

Update

We are monitoring the system across the region.

Our services are "collecting" mobile packages ( E.g. mobile transactions) but the transaction sending to the ERP systems is still severely delayed due to service interruption from our global public cloud provider. We will provide more updates later.

Posted Oct 20, 2025 - 14:48 EEST

Monitoring

Today at 7:00AM UTC we started to see increased error rates in our infrastructure and reports from the field that our mobile app and web portal are not accessible.
Our team started to investigate the issue and quickly determined the root cause being global outage for specific managed infrastructure services at our public cloud provider. Our teams started to work on improving the overall system health of the impacted resources.

The global issue affected also additional part of our operations:
- Dynamics Mobile Helpdesk Website ( we were still serving support requests via email, phone and other channels)
- Dynamics Mobile Status/Health page

Update - 11AM UTC
------------------------
Most services are restored. We are still seeing major delay in mobile app packet processing - e.g. packets sending to the ERPs are delayed significantly. We are working with the cloud provider on resolving the issue.

Posted Oct 20, 2025 - 13:51 EEST

This incident affected: Dynamics Mobile Portal, Dynamics Mobile Service, Dynamics Mobile Website, and Android App.