Google’s Transfer Appliance Simplifies the Transfer of Huge Amounts of Data to Google Cloud

NOTE: I doubt if very many genealogists have a need for this service at home. However, I suspect that several genealogy societies, genealogy libraries, museums, and perhaps a few genealogy book publishers and magazine publishers could use this. Those organizations often have huge databases or collections of data files. Those organizations are the intended audience for this article.

Do you need to back up a lot of data to the cloud for safekeeping? In this case, I am talking about terabytes of data. (One terabyte is equal to one million million bytes, the same thing as 1,024 gigabytes, or 1,024,000 megabytes.)

We tend to think of our normal Internet connections as being “high speed.” Indeed, they are much faster than the old dial-up modems we all used a number of years ago. Still, “high speed” is a relative term.

If you or your employer has a typical in-home or small business Internet connection of 10 megabits per second, backing up your file server’s most critical terabyte of data might require several months to accomplish! Do you have many terabytes of data, such as videos and images? Multiply the time required as needed.

A few organizations may have several terabytes of data, especially if they store a lot of video and graphics files, such as is common with genealogy book publishers and magazine publishers or some web sites. Uploading those on a 10-megabit per second connection might require a year or more of constant uploads!

High speed? Yes, but not nearly high enough.

Most organizations that have that much data in their servers probably have much higher-speed Internet connections. But even a 100-megabit-per-second connection might require a few months to upload everything. To go to truly high-speed connections, the organization’s I.T. (Information Technology) director probably is looking at OC3 Optical Carrier transmission rates (transmission data rate of up to 155.52 megabit/second) or even OC24 (transmission speeds of up to 1244.16 megabits/second). Those connections cost thousands of dollars per month but are worth the expense for corporations and non-profits with big data requirements. Even with an OC24 connection directly to an Internet backbone, uploading a few terabytes of data might require several weeks.

But wait! There is a better way.

A number of companies that provide large amounts of file storage in the cloud offer small servers, often called “appliances,” that you can install temporarily in your own data center. These appliances include internal or external disk drives of 1 terabyte (1,024 gigabytes) to perhaps 500 terabytes (512,000 gigabytes). The appliances are designed to be plugged into the organization’s internal local area network (LAN). Data is then copied from the organization’s servers to the appliance at the highest speed possible. Internal LAN networks typically run at 100 megabits/second or 1 gigabit/second while a very few run on even faster optical network connections.

Even at these higher speeds, copying everything to an appliance will require many hours, perhaps a few days.

One of Google’s smaller Transfer Appliances

Once the copy has been completed, the appliance is powered down, disconnected, packed back into the shipping container, and sent back to the company that provides large amounts of file storage in the cloud. Once received, the appliance is unpacked and connected to that file service company’s internal network. The data residing on the appliance is then transferred to the file storage company’s servers at very high speed.

I spent several years working for one of the leading companies (of that time) that provided online storage space on our servers to our customers. We called this process of copying data onto a server (appliance) at high speed and then sending the appliance to our company as “salting the backup.” That phrase was derived from the old phrase of “salting a mine,” the process of adding gold or silver to an ore sample to change the value of the ore with the intent to deceive potential buyers of the mine.

NOTE: See https://scams.wikispaces.com/Salted+Mine for an explanation of the old scam of “salting a mine” that was common in the American Wild West.

Once all the data has been pre-loaded from the appliance to the cloud-based file storage and backup service’s servers, normal backups are started across the Internet. Since the older data is already available on the backup servers at the file storage company, only the NEW DATA needs to be transferred online.

Some of Google’s larger backup Appliances

This process with an appliance can save your organization days, months, perhaps even a year or more in time and will provide secure online backups during that time as well. For organizations with large data requirements, the use of a transfer appliance can actually reduce expenses significantly.

A number of the larger companies that provide online backup services in the cloud offer appliances. I will focus on one of them: Google Cloud. However, the other services are loosely similar but with some variations in the process and sometimes major variations in pricing.

Google Cloud’s present customers include such well-known companies as Netflix, Charles Schwab, HSBC Bank, Spotify, AccuWeather.com, Marks & Spencer, Northrop Grumman, Harley-Davidson, Motorola, Georgetown University, Stanford University, Woolworth, Domino’s, and many others. If Google Cloud works for those companies and universities, it probably will work well for your organization as well.

Configuring and implementing a backup solution with a Google Cloud Transfer Appliance does require technical expertise. I would not recommend this as a weekend project at home by a novice computer user. However, any organization that has a need for large backup services that will back up terabytes of data probably already has technology experts on staff.

The Google Cloud Transfer Appliance also is not cheap. The customers do not purchase the appliances. Instead, they rent them for the time it takes to transfer all the data to the appliance and then the time required to ship the appliance back to Google. Prices range from $800 to $2,700 for the rental time plus shipping charges.

You can learn more about Google’s Cloud Transfer Appliance at https://cloud.google.com/transfer-appliance/.

If you do install one of these appliances in your home for a few days, please take a picture or two, OK? I’d love to see your in-home network!

 

Leave a Reply

Name and email address are required. Your email address will not be published.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

You may use these HTML tags and attributes:

<a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <pre> <q cite=""> <s> <strike> <strong> 

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: