On January 27th, we experienced a service disruption in Mentor affecting APAC and EMEA regions. Users encountered elevated error rates and a loss of functionality for Mentor features.
The incident was triggered by a third-party service outage, which hosts the models supporting our Data & AI services. While a fallback system was in place, it also became unavailable during the outage. Furthermore, the impact of the external service disruption also prevented our team from implementing a manual configuration fix immediately.
We are also working toward an automated fallback mechanism to minimize the need for manual intervention. Additionally, we are implementing agent-level fallbacks to different models at the product level.