[Minor] Issue Accessing Content for Specific Users
Incident Report for Box
Postmortem
NOTICE AND DISCLAIMER: CUSTOMER UNDERSTANDS AND AGREES THAT THIS IS PRELIMINARY INFORMATION SUBJECT TO FURTHER REVIEW AND ANALYSIS BY BOX. TO THE BEST OF OUR KNOWLEDGE, THIS IS THE CURRENT STATE AND WILL BE UPDATED AS MORE INFORMATION IS CONFIRMED.
LAST UPDATED: NOVEMBER 25, 2019

Summary Impact

Between 1:58 PM PST and 6:17 PM PST on November 19th, a small percentage of users may have experienced difficulties while working in Box. During this time, the affected users were not able to access content on Box services. The issue occurred due to a recent code change in our database tier to add functionality to Box. The change introduced a misconfiguration on one of our user databases and, inadvertently, caused the affected user's data to be inaccessible. We were able to resolve the issue by rolling back the change and making the data available from a redundant copy.

Analysis

A full investigation determined that, as a result of a process gap, a recently added database instance contained a mis-matched configuration in the schema migration tool. This caused the database schema to be incorrectly updated on this database instance. In order to ensure the integrity of the data, it was determined the safest, surest recovery was to failover to a redundant copy of the database which took approximately 4 hours.

Defect Remediation

The following remediation actions have been completed or are planned:

  • Update procedure for adding a new database instance to address missing tool configuration steps.
  • Add additional validation code in the database schema migration tool to abort the schema change operation in this kind of scenario.

We are continuously working to improve Box and want to make sure we are delivering the best product and user experience. We hope we have provided some clarity here and we would be happy to answer any questions you may still have regarding this matter.

Sincerely,
The Box Team

Posted Nov 22, 2019 - 10:34 PST

Resolved
No additional impact has been observed and this incident is now considered Resolved. Please contact Box Support at https://support.box.com if you experience any additional issues.
Posted Nov 19, 2019 - 20:03 PST
Monitoring
Our teams have completed actions to remediate this issue and services should be restored at this time. We are currently monitoring to ensure no additional impact is observed.
Posted Nov 19, 2019 - 19:01 PST
Update
Our teams continue to project this to be remediated within the next hour. We will provide additional information as it becomes available.
Posted Nov 19, 2019 - 17:24 PST
Update
Our teams are continuing to take action to remediate this behavior and currently project this to be resolved within the next two hours. We will provide additional information as it becomes available.
Posted Nov 19, 2019 - 16:23 PST
Update
Some content belonging to affected users may appear to be temporarily inaccessible. This may affect external collaborator access to this content as well. Our teams are continuing to take action to remediate this behavior. This post has been amended to include additional components.
Posted Nov 19, 2019 - 15:28 PST
Update
Our teams have identified an issue that may cause some content for a subset of users to appear temporarily inaccessible. We are currently taking actions to remediate this and will provide additional information as it becomes available. This post has been amended to include additional components.
Posted Nov 19, 2019 - 15:09 PST
Identified
Our teams have identified an issue that may cause some content for a subset of users to appear temporarily inaccessible. We are currently taking actions to remediate this and will provide additional information as it becomes available.
Posted Nov 19, 2019 - 15:07 PST
This incident affected: Box Notes (Web Application), Desktop Applications (Box Sync, Box Drive), Box Web Application (Uploads/Downloads, Collaboration, Search, Preview, Sharing (Shared Links), Comments and Tasks), and Mobile Applications (Preview, Search, Uploads/Downloads).