I've written often about cloud computing and the possibilities available when using rented disk space and rented computing power. Writing in his "Storage Bits" column on ZDNet, Robin Harris describes a recent experience by a large biotech research group that needed a lot of CPU cycles to run some simulations.
The article is full of technical buzzwords and terminology. If you are interested in the technical details, I'll refer you to Robin's article at http://goo.gl/HJDqQ. However, I am more interested in the business details. Robin Harris' article describes the flexibility and low costs of cloud computing. There are lessons to be learned here for individuals and large corporations alike.
In this case, the large biotech research group needed to use a very powerful computer, called a super computer, for a short period of time. Ten years ago, a 2,000-plus core computer would have been one of the five most powerful computers in the world. Today, it qualifies as a "run of the mill" commercial system, hardly worth noting. Thousands of similar, or even more powerful, computers are in use today in companies, universities, and governments around the world.
Purchasing their own multi-thousand-dollar computer to use for a couple of hours didn't make sense for the biotech research group. Instead, the group turned to Amazon Web Services (AWS), a major provider of cloud computing services.
They started setting up a 2,048-core cluster at 10:30 AM, and by 11:15 they were maxing out over 2,000 cores. (That is roughly the equivalent of 2,000 simultaneous processors although the numbers don't scale exactly one-to-one.) By 2 PM, the job was completed, and it took another 10 minutes to bring the cluster down. Total price: $525, charged to an employee's credit card. That's far more cost-effective than spending tens of thousands of dollars purchasing your own hardware and requiring weeks to get it installed, configured, and operational.
So what does this have to do with the individual genealogist or other computer user?
Ten years ago, those of us who used computers were typically using Intel model 486 processors or perhaps one of the early Pentium processors. If we had brand-new computers, we probably had 256 megabytes of memory and maybe a 10-gigabyte hard drive. We also did not have today's software. We could not run today's Photoshop and edit huge digital images in the manner we do today. We could not edit and store video and then make the video available to others on the web (most computer users were on 56k baud dial-up connections at that time). Our personal genealogy databases did not include multimedia scrapbooks. We could not access any online genealogy databases that contained images of original records.
All those capabilities have appeared in the past decade due to the availability of cheap hardware and software. Today, we can do things that were unheard-of only ten years ago.
Now, let's fast-forward another ten years, to the year 2020. Will the genealogist of 2020 be able to search databases of 200 terabytes or more? Will he or she be able to use artificial intelligence software to simultaneously examine every census record and every tax record and every military pension file and every land transaction since the nation's founding? (That might be one huge database, or it could be dozens of individual databases on different servers, owned by different companies, all accessed at once by one computer program running on dozens of super computers.) Can the genealogist of 2020 specify a search of "Show me all the men who were born between 1830 and 1850 who left some sort of record indicating possible Civil War service? Men who had surnames spelled WILLIAMS or some spelling that is roughly similar to WILLIAMS? And also see if there is a digital picture available someplace of the men that fit the criteria I specify? If so, show me the pictures of all the men who have high cheekbones, similar to what many of us in the family have today."
Such a search will not only require huge databases by today's standards, but also processing power that is far beyond what we can dream of today. Ten years ago, we could not envision the computing power required to process video files. Today, that computing power is relatively cheap and is commonplace. Today, we cannot envision the computing power required to perform sophisticated searches. I still cannot appreciate the capabilities of artificial intelligence software running simultaneously on several thousand processors.
The rapidly dropping prices of hardware, plus the capabilities of cloud computing and perhaps other technologies that we haven't yet heard of, will result in extremely powerful computers being available to any of us, whenever we want, wherever we want, at prices any of us can afford. Whether we will own the hardware and software or whether we will simply rent the capability will be an interesting detail, but not a major concern. The fact is that we will have options. Simply charge a few dollars to a credit card – or whatever has replaced credit cards ten years from now.
As I mentioned earlier, the computer cluster described in Robin Harris' article would have qualified as one of the five most powerful computers in the world only ten years ago. Today, it is common-place. Can you imagine the computing power that you and I will be able to use ten years from now?