search
top

Where’s the undo key!? Business owner deletes entire company with one keystroke

Your week was not worse than this guy’s. That is, of course, unless you also managed to delete your entire company, and the work of hundreds of customers, with a single line of code. 

Marco Marsala, who runs a web hosting company, made a fatal flaw that managed to delete the data of not only his own service, but that of his clients as well. Included in his code was the command “rm -rf,” which effectively tells a computer to delete just about everything. “Rm” refers to remove, “r” means everything within a directory, and “f” forces the command to push past the normal blockers that would prevent such a disaster from occurring. 

Once he realized what he’d done, Marsala posted his mistake to Server Fruit, a server forum. “I run a small hosting provider with more or less 1,535 customers and I use Ansible to automate some operations to be run on all servers,” he wrote. “Last night, I accidentally ran, on all servers, a Bash script with a rm -rf {foo}/{bar} with those variables undefined due to a bug in the code above this line.”

Marsala had actually backed up his servers. However, because of how he had his backups connected, the command pushed through to those as well, even though they were off-site. That mis-configuration cost him dearly.

“All servers got deleted and the offsite backups too because the remote storage was mounted just before by the same script (that is a backup maintenance script). How I can recover from a rm -rf / now in a timely manner?” he asked other server experts. Sadly, he was offered little in the way of consolation. “If you really don’t have any backups I am sorry to say but you just nuked your entire company,” wrote one expert. Others lambasted Marsala for not taking the proper precautions to protect his customers’ data, while one user offered a particularly sobering response: “You’re going out of business. You don’t need technical advice, you need to call your lawyer.”

Related: Intel’s new server chips have up to 22 cores. 22 cores!

But don’t worry — we wouldn’t leave you on such a depressing note for the weekend.

Leave a Reply

Captcha image


top