Resolved -
This incident is now closed.
After a thorough investigation and continuous monitoring, we can confirm that no live customer data was lost. The issue was isolated to a single sandbox site exhibiting highly irregular behavior, and it did not impact any production environments.
Thank you for your patience and trust during this incident.
Jun 18, 15:31 CEST
Identified -
We have identified that all alerts and errors related to this incident originated from a single customer’s sandbox instance. At this time, no other customer environments are impacted, and we have found no evidence of data loss beyond this isolated case.
Current Status:
The issue is confined to one sandbox environment.
All other customer instances are operating normally.
Notifications and email processing are functioning as expected for the rest of the EU production environment.
We are continuing our investigation to ensure there are no hidden issues.
Thank you for your patience and understanding as we work toward a comprehensive resolution.
Jun 18, 11:35 CEST
Investigating -
Unfortunately, despite our earlier mitigation efforts and the implementation of additional monitoring and safeguards, the issue has reoccurred in our EU production environment. We are treating this with the highest urgency.
Impact:
A spike in system load has once again led to the failure of some Jira events being processed.
As a result, notifications triggered by these events were not sent from our application.
Manual email sending and incoming email processing continue to function, though some delays may still be observed.
This issue is intermittent and does not affect all customers.
Current Actions:
Our Engineering and DevOps team is actively working on this with the highest priority.
We understand how critical event notifications are to your workflows and sincerely apologize for the inconvenience. Thank you for your continued patience and trust as we work to fully resolve this issue.
Jun 18, 11:13 CEST