US2 degraded performance
Incident Report for Tymeshift
Postmortem

Description:

Some customers/accounts were not able to access Tymeshift or they experienced slowness when accessing the site.

Reason:

There was an unexpected increase at compute resource usage level by a component of our application.

As a result, this component started to answer more slowly to the incoming requests, which caused the issues mentioned in the description.

Actions:

We have identified the root cause of this issue and we will start working on the fix immediately.

In the meantime we put in place a mechanism that will monitor this component and, in a similar situation, will automatically perform some actions in order to prevent this from happening in the future.

We apologise for the inconvenience caused to the customers/accounts affected today.

Posted Jan 27, 2023 - 12:38 CST

Resolved
The incident has been resolved and we will follow up with a post mortem.
Posted Jan 27, 2023 - 09:35 CST
Monitoring
We've identified the root cause stemming from the database resources and the system should be recovered. We're currently monitoring and we will update with full resolution soon.
Posted Jan 27, 2023 - 09:11 CST
Investigating
We're seeing reports of degraded performance on our US2 environment and we're currently investigating.
Posted Jan 27, 2023 - 08:59 CST
This incident affected: US2 (Zendesk App, Time & Attendance, Schedule, Forecast, Admin function, Reporting).