An outage occurred after Sunday’s deployment affecting accessibility on Totango’s Web application such as failure to view account profiles as well as integration connection errors with code 9004.
The issue occurred between 08:12 and 08:47 AM UTC and was resolved at 08:55 AM UTC.
Code defect with Totango’s new API throttling caused it to enter a deadlock and failed the user to access the UI API request or connection.
Issue resolved by an immediate Hotfix.
Will be adding new alerts on the test environment to identify these incidents before reaching production and prevent them from happening.
Created an internal new tool that should help capture new API errors on the test environment before deployment.
Posted Jan 09, 2023 - 18:09 UTC
This incident has been resolved.
Posted Jan 08, 2023 - 09:19 UTC
A fix has been implemented and we are monitoring the results.
Posted Jan 08, 2023 - 09:02 UTC
The issue has been identified and a fix is being implemented.