Cannot access account profile
Incident Report for Totango
Postmortem

Event Description:
An outage occurred after Sunday’s deployment affecting accessibility on Totango’s Web application such as failure to view account profiles as well as integration connection errors with code 9004.
The issue occurred between 08:12 and 08:47 AM UTC and was resolved at 08:55 AM UTC.

Root Cause:
Code defect with Totango’s new API throttling caused it to enter a deadlock and failed the user to access the UI API request or connection.
Issue resolved by an immediate Hotfix.

Preventive actions:

  • Will be adding new alerts on the test environment to identify these incidents before reaching production and prevent them from happening.
  • Created an internal new tool that should help capture new API errors on the test environment before deployment.
Posted Jan 09, 2023 - 18:09 UTC

Resolved
This incident has been resolved.
Posted Jan 08, 2023 - 09:19 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Jan 08, 2023 - 09:02 UTC
Identified
The issue has been identified and a fix is being implemented.
Posted Jan 08, 2023 - 08:55 UTC
This incident affected: Totango Web Application.