As of date November 26th - All data from November 25th, 2020 is fully restored with the exception of information that was recorded through the SDR gateway (usage events) for several hours on November 25th. We are in the process of recovering and restoring this data as well. As of date December 11th - SDRs are fully restored. The restoration process is completed.
Posted Nov 26, 2020 - 17:27 UTC
Update
The system is fully operational including real-time data processing. We are working on recovering all data for Nov 25.
Posted Nov 26, 2020 - 09:30 UTC
Update
Today, Totango’s service in the US was impacted by an Amazon outage. Specifically, the AWS Kinesis service was down in the US and Totango services using this component of AWS were affected. We have been actively working with Amazon and will report when this issue has subsided.
The impact on your Totango service may have been as follows (depending on your specific implementation):
Error messages in the UI in certain data update flows: bulk attribute update, setting forecast and latency in timeline updates. Integrations - Zendesk and Salesforce (legacy) are delayed and will resume automatically once resolved. SDR gateway was impacted (used by API sdr.totango.com or Totango javascript library). The impact is a delay in processing events which will recover in the next few days. Segmentation results that filter on usage data from the last day (Nov 25) may be incorrect.
Posted Nov 26, 2020 - 01:06 UTC
Update
We are currently experiencing issues in the system due to external issues on the AWS end
AWS Kinesis (data stream) that Totango relies on is not performing. We expect to resume real-time data processing as soon as it is resolved.
Posted Nov 25, 2020 - 16:55 UTC
Identified
We are currently experiencing issues in the system due to external issues on the AWS end
Posted Nov 25, 2020 - 16:51 UTC
Update
We are currently investigating this issue.
Posted Nov 25, 2020 - 16:42 UTC
Investigating
We are currently experiencing issues as AWS is down