Cloud seeding your data: When the internet isn’t fast enough for your migration


“Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.”–Andrew Tanenbaum, 1981

That was true then, and it’s true now. Moving data within a cloud is easy — that’s what 40 Gigabit Ethernet is for — but getting that data there in the first place… not so much.

Sneakernet may date from the days of floppy-disks, but it’s still useful today. Sure, the internet is getting faster, but it’s not fast enough once you start working with terabytes of data.

As Randall Munroe, creator of the popular xkcd cartoon, explained,

Cisco estimates that total internet traffic currently averages 167 terabits per second. FedEx has a fleet of 654 aircraft with a lift capacity of 26.5 million pounds daily. A solid-state laptop drive weighs about 78 grams and can hold up to a terabyte. That means FedEx is capable of transferring 150 exabytes of data per day, or 14 petabits per second—almost a hundred times the current throughput of the Internet.

Of course with a ping time that Munroe estimates at 80,000,000-milliseconds, we’re not going to be video-conferencing over FedEx anytime soon.

Still, while we may no longer load up 9-track backup tapes to move data, cloud companies often use similar techniques to upload massive amounts of data. The 21st century term for this ancient way of moving data is cloud seeding.

What happens is the cloud provider sends a disk drive or storage appliance. You then back up your local data and ship it back to the provider. Your data is then uploaded to your provider’s servers. With that done, future incremental data uploads and backups are sent over the internet.

The public cloud seeding offers include:

Amazon Web Services’ Snowmobile cloud seeding service literally uses a semi-trailer truck with a 45-foot long ruggedized shipping container. These containers can hold up 100 petabytes. The Snowmobile is driven to your data center and hooked up to your servers with a high-speed switch, then you use it as one heck of a big network storage device. After your data is loaded, it’s driven to an AWS data center where it is imported into Amazon S3 or Amazon Glacier.

If you want to put your data into Google Cloud or Seagate Cloud System, Iron Mountain offers its Iron Mountain Cloud Seeding and Migration service. Here, Iron Mountain comes to your place of business and loads up your own drives and then takes care of transferring the data to your cloud of choice.

Microsoft Azure offers the Azure Import/Export service to transfer data to Azure Storage. In this plan, you load your data into SATA HDDs or SSDs. Then, the drives are shipped to the nearest Azure data center.

If all this sounds a little elaborate. Consider that, to transfer all the data a single Snowmobile can handle, it would take over 20 years over a 1Gbps internet connection.

It may not be a station wagon or tapes these days, but to really transfer massive amounts of data to the cloud, you still can’t beat the carrying of physical storage from one place to another. Sneakers are optional.


Migration considerations for the cloud

How many servers do you need for your cloud?

Speak Your Mind


This site uses Akismet to reduce spam. Learn how your comment data is processed.