Google has come clean with a blog post explaining in a little detail how their Gmail cloud-based e-mail service managed to remove all of the e-mails of about 0.02% of their users yesterday.
The problem arised not from an issue with Gmail itself, but in a software update to the underlying storage technology. Of course Google’s infrastructure is massively distributed, and their storage infrastructure is separated from the main Gmail application. It was when user’s data started disappearing that they realised the storage layer’s update was causing issues, and they rolled back the release.
Unfortunately, due to the bug being in the storage system rather than Gmail itself, the deletes affected multiple copies of the data in different datacenters. This means that Google engineers are still working to restore some lost e-mails from offline tape backups.
This somewhat goes to underscore the problems that may be raised by entrusting all a user’s data in the cloud on a single service. Just yesterday, Microsoft made an unconnected announcement that they are upping the amount of compute capacity you get when you sign up for a free Azure trial, in an effort to get more developers building on the cloud.
Comments