We recently addressed an issue affecting multi-factor authentication logins. We would like to further explain the issue and action items we had taken to prevent it from happening in the future.
Between 9:00 AM PST and 12:30 PM PST on November 6, 2023, some users experienced latency and difficulty logging into Box. During this time, users logging in to Box with multi-factor authentication may have encountered slow responses or failures during the login process. Upon investigation, we determined that the cause of this issue was the result of a single enterprise repeatedly retrying programmatic session termination calls for all its users, which temporarily overburdened the multi-factor authentication service. The issue was resolved when the user was temporarily prevented from using the session termination API.
Analysis
The issue occurred due a confluence of factors relating to our session termination API and multi-factor authentication service. Specifically, at the time of this issue, our session termination API did not have a rate limiter in place, which allowed a customer to repeatedly call the API with an excessive number of requests to the multi-factor authentication service. The multi-factor-authentication service additionally did not rate-limit the specific call made to it by the session termination API. Finally, the internal retry logic for timed-out requests continued to load the multi-factor authentication service. As a result, due to the unusually high session termination calls, our multi-factor authentication service became overburdened and resulted in the temporary impact to users.
Corrective Actions
The following corrective actions have been completed or are planned:
We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter.
Sincerely,
The Box Team