We recently addressed issues affecting timeliness of the Events API. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.
On Thursday, July 8, 2021 between 1:15PM PT and 4:05PM PT, some users may have experienced difficulties while working in Box. During this time, notifications may have slowed down for some users, and some Drive users may have experienced delays. This issue occurred as a result of an unexpected surge in traffic because of an internal migration. We were able to resolve the issue by adding more capacity to the affected service. In addition, we have added alerts to prevent similar issues from occurring in the future.
A migration involving one of Box's backend services caused a sudden uptick in the incoming traffic to our messaging pipeline. Due to a preset protective limit, some of that incoming traffic was restricted, causing certain events to slow down. The engineers investigating the issue were able to add additional capacity to the messaging service, and the issue resolved soon thereafter.
The following corrective actions have been completed or are planned:
Adding additional capacity to the messaging pipeline
Adding new alerts to page the on-call engineers when traffic to the messaging pipeline is close to hitting the preset limit
We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter.
The Box Team