Elevated latency for EU instance API

Incident Report for Sardine AI

Postmortem

Summary

On June 2, 2026, some customers experienced elevated latency on /v1/issuing/risks and /v1/customers requests in prod-eu. We’re sorry for the disruption and for any delays this caused in customer risk decisions.

What Happened

A subset of issuing risk requests became slower than expected due to delayed customer data lookups. During the main impact window, affected requests could take up to approximately 5 seconds before continuing through a soft-fail path.

Why It Happened

The issue was caused by a customer lookup path not enforcing its intended short timeout. When some customer lookups stalled, they waited on the broader request deadline instead of failing fast at the operation-level timeout.

What We’re Doing About It

We deployed a hotfix to reduce the customer-load timeout and limit how long issuing requests can wait on stalled lookups. We are also adding follow-up improvements to reduce Datastore dependency in low-latency paths and investigate the underlying cause of the lookup stalls.

Posted Jun 08, 2026 - 20:41 UTC

Resolved

This incident has been resolved.
Posted Jun 02, 2026 - 18:34 UTC

Investigating

We are currently investigating this issue.
Posted Jun 02, 2026 - 17:43 UTC
This incident affected: Issuing API and Customer APIs.