Following a recent global failure due to an unsuccessful update of Crowdstrike, Grant Thornton servers also experienced issues with the Windows operating system, displaying the blue screen of death (BSOD). This problem impacted hundreds of computers and over 100 servers within the company, resulting in a lengthy restoration process.
The challenge faced by the company’s IT specialists was the need to decrypt BitLocker encryption on the affected computers. This required not only the standard system rollback to a restore point or the deletion of problematic files from the directory “C: Windows System32 Drivers Crowdstrike” in safe mode, but also the input of 48-digit keys required for BitLocker on each computer.
Due to the large number of devices that required restoration, manually entering these keys would have been a time-consuming process. However, an ingenious solution was devised by engineer Rob Volts and infrastructure manager Ben Watson to streamline the recovery process.
Volts proposed creating a script that could convert the Bitlocker keys into barcodes, which could then be displayed on the server screen. By using barcode scanners, the keys could be automatically inputted into the computers, speeding up the restoration process significantly.
After acquiring barcode scanners for just $36 each, the Grant Thornton team began scanning the barcodes displayed on the server screens when prompted for BitLocker keys. This innovative approach allowed the team to restore all computers by Monday, July 22, within a matter of minutes for each device. While servers took slightly longer to recover, with approximately 20 minutes needed for each, the team successfully managed to restore all systems promptly.
Ben Watson commended the team’s innovative problem-solving approach, describing it as a “wonderful example of optimizing workstation restoration.” Rob Volts expressed satisfaction with the rapid recovery process, noting that further automation could be achieved using QR codes in the future. However, the use of barcode scanners proved to be a highly effective solution for the company’s restoration efforts.