Skip to content

April 14, 2011

13

Why no company that values their data should EVER “Go Google”

by e1ven

I’ve been working with Google Apps for your Domain for years. Most recently, I set up my current company, to use Google Apps- We’ve been running flawlessly for 6 months, paying them $50/user to avoid handling it ourselves.

The problem is, Google is entirely useless whenever there finally IS a flaw.

Incident #1- Last week, we had a file in Google Docs become “Unavailable”.

We have quite a few documents in Google Docs that our Sales team uses to help coordinate. On Friday, one of these files became unopenable.
No users were able to open the file- They couldn’t open it inline, email it, save it, or access it in any way.

Using the “Recovery” option in the app helpfully gave us the error message “We were not able to find any version of the spreadsheet that could be safely recovered.”

Why Gee, thanks.

We’re not the only ones who have had this problem, either.

We emailed Google’s support, and explained that as paying customers, we’d kindly like access to our file please.
I received a generic message back-

Thank you for your message. I understand that you are experiencing an issue uploading documents.

I believe the nature of your issue is that you are attempting to upload some existing documents to your Google Apps accounts, if I’m incorrect I apologise and please clarify by replying to this message.

When uploading documents there are a few limitation that you should be aware of, such as:
   * Each user in your domain has 1 GB of storage for Google Docs. Usage information is shown to users when they upload a document.
   * Google Apps for Premier users can upload files up to 1 GB.
   * If a file is converted to a Google Doc then the following limits apply:
<Convert documents - up to 500KB per file.
<Convert presentations - up to 10MB per file.
<Convert spreadsheets - up to 1MB per file.
* These are the file types that can be converted to Google Docs:
For spreadsheets: .xls, .xlsx, .ods, .csv, .tsv, .txt, .tab
For documents: .doc, .docx, .html, plain text (.txt), .rtf
For presentations: .ppt, .pps (.pptx and .ppsx is not supported)
For drawings: .wmf
For OCR: .jpg, .gif, .png, .pdf.

For more information on uploading documents, please visit http://docs.google.com/support/bin/answer.py?answer=37603 .
and also

http://docs.google.com/support/bin/answer.py?answer=186466.

If after reviewing the information above you are still experiencing the issue please don’t hesitate to reply to me and I will be delighted to assist you and if necessary I’ll consult with our Docs specialist.

Sincerely,
[Name Withheld]

Why Thanks! That’s… Not helpful at all.

I sent them another email, asking that they kindly help us access the file. I’ll take an emailed version of it as a xls. I’ll take a CSV. I’ll take an online version. Heck, I’d take a screenshot of the file. Nothing.

We got back several emails explaining that they are very sorry, but while they are working on retrieving the file, they can’t promise any dates or times.
It’s now been over 4 days, and still no file.
Luckily, in our case, while the file was very important, we were able to recreate it by using various other internal tools.
For other files, we may not have been so lucky.

We back up all files on all user desktops, but anything that’s created directly at Google is never stored on the local desktops, so doesn’t get backed up by us.
We’ve now started emailing ourselves copies of every file that’s stored at Google and then saving them locally, but it’s an annoying manual process.

The lesson we’ve learned is that storing documents in Google Docs is not safe, and should only be done for truly replaceable/temporary documents.
If you want to store your Holiday Shopping list there, Sure. But you’re nuts to store anything there that your business relies on.

Incident #2- Today, Google broke our Email.

If there’s one team you’d expect to know how to handle email reliably, it’d be the Gmail guys. When we first signed up for Google Apps email, the one concern I had was that Google is nearly impossible to get a hold of when things go wrong.. Today, we’ve seen how true that was.

This morning, we had a technical problem.. The exact details don’t much matter, but it caused email to entirely go down for us. [1]

The exact technical details of the problem don’t matter much- I’m sure Google can fix this bug, as well as the document bug in Incident #1-
The problem with both was that there was effectively no way to escalate.
We called their technical support, we emailed, we explained that email was absolutely MISSION CRITICAL, and we needed it or we were in danger of losing large deals.

Google continually assures us that they’re working on it, but 5 hours later, There was no change, and no update. Everything was “In process”.
Working with them before, I’ve had them say that for up for 48 hours before the domain was fixed.

Since we couldn’t afford to wait that long, I used DynDNS to setup a mail forward for the domain, redirected the MX records, and have mail arriving again. Google continues not to have any useful answer to fixing things on their side.

I love Google Apps, particularly for email. I’ve run Exchange Servers, I’ve run Zimbra servers, and I know it’s a pain in the butt to back it all up, deal with routing, recovery and ensure it stays reliable. I’m happy to pay Google to do this.

But keeping your company on Google Apps just isn’t safe. When things go wrong, no matter how critical it is to you, Google just doesn’t care.
Your email may be mission critical to you, but to Google, you’re just customer number 10 billion and four that they don’t have much time for.

Sadly, I’d strongly recommend that all companies that care about their email and files move off of Google Apps immediately.


[1] I wanted to transition a ‘domain alias’, to become a ‘secondary domain’.. What that means is that we have a second domain that currently matches up every email address, 1:1, with our primary domain- User@domain1.com and User@domain2.com go to the same backend mailbox.
This has worked fine for us for the last few months, but when we tried to remove domain2.com as a domain alias, and re-add it as a secondary domain, google crashed.


After this happened, we were unable to re-instate the domain alias, unable to remove it..
We couldn’t add it as a secondary domain, we couldn’t do anything with it.

The domain still showed up as if it were active as a domain alias in Google’s control panel, but mail delivery for that domain started failing.
Every message to that domain was generating a bounce.


Postfix- After posting, Google was able to help my company resolve both issues. Total time was 5 days for the corrupted file, and 14 hours for email outage.

About these ads
Read more from Uncategorized
13 Comments Post a comment
  1. Apr 14 2011

    You should definitively switch to Google ContactOffice : http://www.contactoffice.com

    In addition of supporting most of Google Apps features (and even more), ContactOffice support WebDAV access which work great on iPad, MacOS X and Windows and ease backup/syncing.

    There is a free trial available.

    Reply
    • e1ven
      Apr 14 2011

      Interesting, I’ll look into it. For now, we’re using DropBox, which syncs the machines pretty nicely. The best part with Dropbox is that it’s stored locally on every machine. Unlike a WebDAV share, it *can’t* go down.. Worst case, it stops syncing, but we still have our files.

      Reply
  2. keshet
    Apr 14 2011

    I cobbled together a fairly simple shell script which uses the GDocBackup utility:

    http://gs.fhtino.it/gdocbackup

    The script runs automatically once a day, downloading all documents from the company google account into a directory whose name is the day of the month. That way there are versions of all the docs going back a month. No brainer.

    Reply
    • e1ven
      Apr 14 2011

      Keshet- Thanks! I like the app, and I’ll try to get users to use it here for their personal Gdocs.
      I wish the GUI option had the ability to add itself as a scheduled task.

      We’re moving off of Google Docs, because the escalation is so poor. I understand they’re going to lose files, but it’s the email problem that on top of that that really pissed me off. In both cases, I feel like no one is listening.

      Reply
  3. Apr 14 2011

    Hi Colin-

    We operate Spanning Backup (http://spanningbackup.com), which continually backs up Google Docs, Calendar, and Contacts to the cloud, so if something is accidentally (or maliciously) deleted or corrupted, you can restore it immediately.

    I invite you to try it to see how well it works for you. The first three seats are free for any Google Apps domain, and additional seats come with a 30-day free trial. It’s been a lifesaver for hundreds of companies.

    Thanks,
    Charlie

    Reply
  4. Apr 14 2011

    This, for better or worse, is a function of Google’s scale. There are 180 million Gmail users. Even with five nines of data integrity, that’s still 1800 victims of Gmail data loss per reporting interval. Think of it as a reverse lottery. Most of the time, you’ll win. But, statistically speaking, somebody is going to lose, and it might be you (especially if you’re an envelope-pusher that does advanced domain mapping).

    Now for the shameless plug.

    Backupify exists in no small part as Google Apps Insurance. We keep independent, daily backups of all your apps domain data. $3 per user per month. Unlimited storage. Fully searchable (as in, all your domain data available from a single search query). Accessible even when Google Apps isn’t. One-click restore of any Gmail or Google Docs content. We also archive the useful parts of your Twitter feed, Facebook Page and Flickr or Picasa albums.

    Google brags about 99.9% uptime. We’re there for the 0.1%. http://www.backupify.com

    Thus ends the shameless plug.

    Reply
  5. Apr 14 2011

    And your point is that if you did this all yourself, it would be cheaper more efficient and less error prone right?

    Good luck.

    Reply
    • e1ven
      Apr 14 2011

      Cheaper/Easier? No way. But in the 9+ hours without email being fixed by the Big G, I could have restored from TAPE by now if necessary.
      I’ve built quite a few Zimbra and Exchange servers in the past. They’re a time sync, but at least if something goes wrong it within your domain to fix. You can put it at the top of the queue.
      Here, it’s entirely outside my control, and getting a low priority at Google.

      Reply
  6. Ketil
    Apr 27 2011

    Did you try calling them? As a paying customer you get access to 24/7 phone support.

    I have been using Google Apps for Business for a while and not had any problems yet. But I do have their support number on my phone just in case.

    I am looking forward to the promised offline access for Docs that disappeared with the termination of Gears.

    Reply
    • e1ven
      Apr 27 2011

      I did call them ;) They basically wrote down my information, entered it into the ticket, and promised to get back to me.

      Reply
  7. Aakash
    Aug 2 2012

    My GMAIL address is in the form of YX@gmail.com. I have been receiving E-mails directed to another person with E-mail of the form Y.X@gmail.com. Normally I would not have even noticed something like it and ignored it as a SPAM, except this time I found an E-mail from United Airlines with travel confirmation for this other person sent to me. We share the same name so I got really worried as I was not making the travel at all. I called United first and reported it. Then I looked closely at the E-mai and realized it was not a mix up on United’s part but rather a goof-up by Google.

    This appears to be a serious breach of privacy by GMail. I use GMail for our business and this issue makes me feel very concerned and I wonder if some of the rather confidential business information may be forwarded by GMail to an unintended recipient.

    Has any one else experienced any thing like it ever? I would have liked to call Google to report this issue, but I can’t figure out how to reach Google except to post on Support Forums.

    Reply

Trackbacks & Pingbacks

  1. User Error Accounts for Most Google Docs Data Loss | Technology NZ

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Note: HTML is allowed. Your email address will never be published.

Subscribe to comments

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: