[Minor] Delayed information in User Details Report
Incident Report for Box
Postmortem

We recently addressed issues affecting admin reports. We would like to take the opportunity to further explain these issues and the steps we have taken to keep them from happening in the future.

Between December 16, 2021 at 9:00 am PT and January 5, 2022 at 8:08 am PT, some users may have experienced difficulties while working in Box. During this time, some admin reports reflecting data about the current state of a Box deployment may have been out of date. Reports on actions taken, such as the User Activity Report, were not affected. The issue occurred due to a faulty maintenance operation that temporarily prevented us from updating the data used to generate the affected reports. We were able to resolve the issue by adjusting the system to be more reliable and handle extra data. In addition, we have revised our maintenance procedures to prevent similar issues from occurring in the future. 

Analysis 

This issue was triggered by a routine maintenance operation that was incorrectly executed. This left the process that provides updated data from our database to the admin reporting feature in an unusual state that our monitoring failed to identify as problematic. Though the available data became increasingly recent as the issue was remediated, it took time to fully recover given the backlog of updates that this issue created.
Corrective Actions
The following corrective actions have been completed:

  • Hardened monitoring to handle edge cases that can arise during known maintenance procedures and immediately alert if deviations are detected

  • Tuned the affected process to be resilient in the face large backlogs

  • Increased capacity to process data well in excess of normal operations

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience we can. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter. 

Sincerely,

The Box Team

Posted Jan 20, 2022 - 12:57 PST

Resolved
We’ve addressed the underlying delay that led to the User Details Report not being up to date. While the issue has been remediated, there is still a backlog to be processed.
As a workaround, Admins can run a User Activity Report and filter for the information that’s available in the User Details Report. The information from the User Activity Report is not affected, as the delay is only limited to the User Details Report.
We're considering this issue resolved. Please contact Box Support at https://support.box.com if you are currently seeing any issues.
Posted Dec 27, 2021 - 17:14 PST
Monitoring
A fix has been implemented to address the delayed information contained in the User Details Report and we are seeing improvement. Users may see delayed information present in the User Details Report until the backlog is processed. We will continue to monitor the issue to as the backlog is processed.
Posted Dec 27, 2021 - 10:04 PST
Identified
Box continues remediation efforts to restore full functionality to the User Details Report.

Users of this service may continue to see delays in the information provided within the User Details Report while remediation is in process. We will provide additional information as it becomes available.
Posted Dec 27, 2021 - 07:26 PST
Monitoring
A fix has been implemented to address the delayed information contained in the User Details Report. We continue to see improvements in regards to processing of the associated data. Users may see delayed information present in the User Details Report until the backlog is processed. We will continue to monitor the issue to assure full recovery.
Posted Dec 26, 2021 - 22:30 PST
Identified
A fix has been implemented for the delayed information contained in User Reports. We are seeing some improvements in regards to processing of the data backlog for User Details Reports. We will continue to monitor the issue to assure recovery.
Posted Dec 26, 2021 - 21:36 PST
Investigating
We are currently investigating an issue related to User Reports. Users may not see recent information reflected when running a User Details Report. We will return with more information soon.
Posted Dec 26, 2021 - 19:12 PST
This incident affected: Box Web Application (Admin Console & Functionality).