T-Mobile and Microsoft susidiary Danger have restored the data Sidekick customers lost when Danger experienced a massive server failure last week. The company announced the news via an online forum used to keep customers apprised of developments.
The companies also provided a few sparse details about the meltdowns. The system failure caused data to be lost in both the core database and the backup system, according to Roz Ho, vice president of mobile experiences at Microsoft.
T-Mobile and Microsoft did not get back to TechNewsWorld in time for publication.
Those interested in delving deeper than the official explanation can do some reading between the lines.
“Microsoft/Danger must have had the data backed up in at least one physical location,” Robert Brownstone, law and technology director at Fenwick & West, told TechNewsWorld. “Perhaps they had to restore the data from backup tapes, such as DLT format.”
That would explain the delay, he continued. “Tape restoration is often a relatively slow process and can be hindered by a tendency for some of the data to be corrupted.
There are other possible explanations for why it took several days for the data to be restored. Perhaps no one in a position of authority at Microsoft or Danger readily knew the backup regimen, Brownstone speculated — or the backups were not properly labeled or organized.
Another possibility is that the backups were stored off-site and the vendor could not readily find them, suggested Brownstone.
Lessons Learned
This episode provides a good opportunity to go over best practices for data storage — not that the industry should have needed it. Minimally, backup copies of data should be kept in at least two places, Brownstone said.
“If the use of tape were the culprit, then perhaps Microsoft or Danger may need to modernize its approach by making greater use of ‘disk-based’ backup — in essence, keeping backup data on hard drives of servers.” Restoration from disk is typically many times faster than from tape, he added.
To prep for data-loss disasters, companies need to make sure they have adequate day-to-day protocols and communication between business leaders and IT leaders, advised Brownstone.
Companies should also have disaster-recovery and business-continuity protocols in place, he said — and periodically check to make sure that there is substantial “in-the-trenches compliance with those protocols.”
Reconstructing the System
T-Mobile and Danger appear to have taken these lessons to heart. Danger has rebuilt the system “component by component,” recovering data along the way, according to Ho.
It is tough to guess what really happened based on their statements, Charles King, principal of Pund-IT told TechNewsWorld. “Frankly, I was surprised when they announced they lost all of the data — that can be very hard to do with adequate backup in place.”
A total loss would suggest “a catastrophic failure of some kind,” he said.
At first, it seemed that T-Mobile was smart in being upfront about the losses, noted King, but in retrospect, “there was a lot of anger over this, and maybe it wasn’t necessary.”
Indeed, two Sidekick customers have filed separate class action lawsuits against T-Mobile, Microsoft and Danger for failing to safeguard customer data after promising that it would be secure.
Even if the lawsuits don’t go anywhere, this event will have a lasting impact on T-Mobile and cloud computing as a whole, Greg Sterling, principal of Sterling Market Intelligence, told TechNewsWorld. “It definitely is a black cloud for cloud computing — at least for consumer applications that rely on it. I don’t think the average user realized his or her data could be so easily lost.”
This could be a watershed moment for cloud computing, forcing providers to make sure their data backup and general security practices are rigorous enough, he said.
A few years ago, AOL’s inadvertent release of individuals’ search data — which was detailed enough that some of the people could actually be identified by name — forced the debate about privacy and search to a new level, noted Sterling, ultimately bringing about some changes in the industry.
Loading ...
Especially AM ong those who lost significant data, I think there’s a fair AM ount of outage outrage AM ong Sidekick-ers. I wanted to add a strategy to reduce your cell bill by around 20 percent for those T-Mobile customers (or even people with other carriers, for that matter) who want aren’t satisfied and want to go farther than the discounts, coupons and freebies. I’ll cut to the chase: I work in the consumer advocacy division of the company Validas, where we electronically audit and subsequently reduce the average cell bill by 22 percent through our website, http://www.fixmycellbill.com . Put simply, Validas guards against frivolous and unnecessary charges that inflate your cell bill more than it should be for your usage. You can find out for free if fixmycellbill.com can modify your plan to better suit your needs by going to the website.
For more info, check out Validas in the media, most recently on Fox News at http://www.myfoxtampabay.com/dpp/consumer/conlaw/lower_cell_phone_bills_072409 .
Good luck to everyone on further reducing those wireless costs.
Dylan
Consumer Advocacy, fixmycellbill.com
Anyone who would depend on anyone else for back-up is an idiot!
Why would you do that? Depending on some external company for a SECONDARY backup is likely a good idea, but if you are not responsible for your own primary back-ups you are an IDIOT!
Back up locally, back up on drives you move off site, then back up on on-line services.
Backing up on ONLY on-line services is just stupid.
The two words that came to mind as I read the article and the comments from the "experts" were sheer ignorance.
I’m appalled that anyone in IT would refer to this as a failure of "the cloud". What, precisely, does that mean? Zip, zero, nada, nothing. Frankly, I don’t like the term "cloud" either…it’s too broad to be meaningful. But let’s not distort reality.
Cloud terminology aside, the Danger catasrophe should compel all of us to re-examine the custodial competency, responsibility and accountability of third party service providers. And that transcends "the cloud".