Bitbucket has an increased error rate
Incident Report for Atlassian Bitbucket
Postmortem

SUMMARY

On September 25, 2023, between 10:07 UTC and 10:31 UTC, Atlassian customers using Bitbucket Cloud faced significant degradation to its website, APIs, and Git functionality. This event was triggered when our storage vendor experienced an outage at their data center. The incident was detected immediately by our internal monitoring, escalated to the vendor, and fully resolved after 24 minutes.

IMPACT

Customers who were impacted were unable to access the bitbucket.org website, APIs, Pipelines, and Git over HTTPS and SSH during the duration of the incident.

ROOT CAUSE

The incident was caused by an outage at our storage vendor’s data center. This affected our ability to communicate with our storage volumes, backing up our internal services and preventing them from continuing to serve requests.

REMEDIAL ACTIONS PLAN & NEXT STEPS

We know that outages impact your productivity. We are prioritizing the following improvement actions to reduce recovery time, limit impact, and avoid repeating these types of incidents in the future:

  • Working with our storage vendor on preventative measures as well as improved detection on their end.
  • Improve resilience to reduce widespread customer impact when a subset of infrastructure is impacted.

We apologize to customers whose services were impacted during this incident; we are taking immediate steps to improve the platform’s performance and availability.

Thanks,

Atlassian Customer Support

Posted Oct 04, 2023 - 00:50 UTC

Resolved
The issue has been resolved and all services are operating normally.
Posted Sep 25, 2023 - 11:36 UTC
Update
We've identified the cause of the issue and remediated the impact, the Bitbucket Cloud team will continue to monitor the issue now the impact has been resolved.
Posted Sep 25, 2023 - 10:42 UTC
Update
We are continuing to investigate this issue.
Posted Sep 25, 2023 - 10:39 UTC
Investigating
The bitbucket team has detected an issue with multiple bitbucket cloud services, we're currently investigating the issue and will provide an update shortly.
Posted Sep 25, 2023 - 10:27 UTC
This incident affected: Website, API, and Pipelines.