Unauthorized errors
Incident Report for Tymeshift
Resolved
Description:
Some accounts may have seen an increased amount of failed to authorize errors on Tymeshift's Zendesk application due to an internal bug on our data pipeline.

Reason:
After deploying an update to our data pipeline that consumes data from Zendesk, we've introduced a bug that caused an increased amount of calls to some accounts. As a fallback mechanism, Tymeshift stops collecting information if that threshold is surpassed and after a period of time with the same error, disables the communication with Zendesk producing an error seen by some accounts displaying "Unauthorized" on Tymeshift's Zendesk application. The threshold of this trigger depends on the amount and frequency of interactions with tickets and thus why only a small percentage of our accounts was affected.

Actions:
We're putting in place a mechanism to stress test internally our API communication with Zendesk to avoid further situations in the future, as well as a failsafe mechanism that does not completely invalidate the account upon the error threshold. We apologise for the inconvenience caused to the accounts affected today and we will be working immediately on preventing this situation from happening in the future.
Posted Aug 10, 2022 - 18:30 CDT