What I Use Now for Backups

I write frequently about the need to make frequent backups of any computer information. After all, you don’t want to lose what you worked so hard to create, do you? I guess I haven’t written about it in a while as a newsletter reader sent a note today asking, “How about an update on what you use for backup now??”

This is my reply:

Well, I never backup to ONE thing! Having only one backup is almost as dangerous as having none. I don’t believe in placing all my eggs in one basket. I always make at least three backups of my desktop system and store them in at least three different locations. I probably backup more than that on my laptop computers.

I back up everything on my desktop computer’s hard drive to an external hard drive that sits beside the computer. I use TimeMachine software for that, a great backup program that is included with every Macintosh. That’s the fastest backup I have and it works automatically all day and night, making backups of all new files automatically within minutes after I create them. I never have to remember to make backups as TimeMachine does all that automatically. Several other companies produce similar software for Windows.

If I need to later retrieve a file, using that full backup of everything on a hard drive attached directly to the computer will be the fastest way to restore it. It even saves all versions of all files. If I need to retrieve a file version as it existed last month or last year, I can do that within seconds using Time Machine and that external hard drive.

HOWEVER, I also know that a fire, flood, hurricane, tornado, or even a burst water pipe might damage or destroy my house, my computer, AND that hard drive that sits beside my computer. That would also destroy more than 30 years’ of genealogy work if that were my only backup copy. Storing backups only in the house is high risk. Therefore, I also make off-site backups.

I have experimented with all sorts of online backup services. I now back up all of my documents, family photos, and much more to Amazon S3 and Amazon Glacier. I like Amazon S3 but am reluctant to recommend it for others. It is quite difficult to get it set up and working if you haven’t spent your life supporting computer systems of all sorts of operating systems, as I have. However, there is an easy method of using Amazon S3 or Amazon Glacier: a piece of software simplifies the job.

Arq (pronounced “ark”) is software available for both Windows and Macintosh computers that performs backups automatically, all day and all night (if you leave your computer running). The first backup probably will take days to complete, as there will be a lot of data to be sent to Amazon’s file storage services. The exact time required depends upon the amount of data to be saved and the speed of your Internet connection. In this case, the important speed is the UPLOAD speed as you are uploading files. Most Internet providers like to quote their DOWNLOAD speeds as those are bigger numbers. However, when copying files to any remote service on the Internet, download speed is unimportant. It is UPLOAD speed that will control the file transfers.

Once the first upload is completed, subsequent backups will only require a few seconds. The software only has to transfer the new or changed files, not everything. Assuming backups are being made hourly, these incremental backups should not last more than a few seconds each. Most users won’t even know when the backups occur.

If you need to restore a file, Arq or any other good backup product will not only allow you to retrieve the current version of a file. It will also restore previous versions! This is very useful if you make an error on a document and don’t discover the error until a week or two later, perhaps after updating the file many more times. Arq, or any other good backup product, will allow you to retrieve the file as it existed two weeks ago or two months oagp or even two years ago.

Arq is available for Macintosh and Windows computers and will back up whatever files you speify to Amazon S3, Amazon Glacier, Google Drive, Google Cloud Storage (including “Nearline”), Dropbox, OneDrive account, your own SFTP server, or a folder on your own network attached server.

You can learn more about Arq at https://www.arqbackup.com/.

I haven’t used it myself but I am told that Cloudberry is a similar program although it is only available for Windows. You can find more information about Cloudberry at https://www.cloudberrylab.com/.

Finally, I still do not trust any one online file storage service, not even Amazon S3. Any company might go bankrupt tomorrow or be bought out by another company that isn’t interested in the file storage service. While highly unlikely in a cloud storage service, it is still theoretically possible that a combination of hardware and software failures in that service’s data centers might mean the loss of your data.

The solution to this is simple: Use two or even more online file services. They won’t all disappear at the same time! The loss of data on any one service should be a minor inconvenience if you have multiple backups. Such a loss should never be a catastrophe.

I use a second online file storage in addition to Amazon S3 and Arq. The service is located overseas, in a country that prohibits release of private data to anyone, not even to the country’s own government. All data being stored is first encrypted in my own computer, then sent to the file service’s servers overseas. The encryption is so good that the employees of the file storage service cannot read my files. In fact, they cannot even see that NAMES of my files nor can they see how big any file is.

Of course, there are still other solutions. While I don’t use these myself, I still will recommend Mozy or BackBlaze or CrashPlan or Carbonite or any of the other dozen or so online backup services. They are easier to set up than is Amazon S3 or Amazon Glacier although they cost more than Amazon S3 and a lot more than Amazon Glacier.

Next, I make copies of all my documents to my off-shore file storage service. If privacy and encryption wasn’t so important to me, I might use Dropbox or Google Drive or a similar service. Many file storage services, including Dropbox, Google Drive, and others (but not all of them) will automatically copy files to and from the desktop computer, as well as to and from the laptops. The same files also become available on request in the handheld cell phone, and any other place I specify. When I turn the laptop on, all my new Dropbox, Google Drive, or other services files are immediately copied to the laptop and are available there within a minute or two; I never have to manually copy them. I stopped using the /Documents folder on my hard drive (Windows used to call it \My Documents.) Instead, I can move all those files from /Documents to /Dropbox and all new documents I create can now be saved in /Dropbox. The /Documents folders on my computers are empty.

My 20,000+ .mp3 music files are on my computer’s hard drive and also are backed up on the external hard drive beside the computer as well as copied onto an iPod music player and to a hard drive attached to the hi-fi in the living room.

I often copy various files to flash drives which I keep in my backpack/briefcase although that’s a manual process and I don’t keep everything there. Just my Powerpoint presentations, all my genealogy data, a lot of recent photographs I have taken, and a few other things that I might need at any time when I meet a friend or relative.

For instance, I took several dozen pictures and videos recently at a party celebrating a relative’s high school graduation. While those are already backed up on my external hard drive and on Amazon Glacier and in the off-shore service, I also keep copies on flash drives in my pocket or backpack. If I meet another relative who would like copies of everything, I can quickly insert the flash drive into his or her computer and copy the several gigabytes of files over. (Those videos are large!) That’s easier and cheaper than using CD-ROM disks and it works as additional backups for me.

So, yeah, I think I am backed up. But I change backup procedures frequently as I experiment with new things. My only rule is: “You can never have too many backups!”

15 Comments

Dick I would like to use some type of on-line back up. I now use TimeMachine and have another usb hard drive that I back up to and then swap the usb drive with another one in a bank vault. This works but I could loose as much as a months work. I also copy a few files to Google Drive and Dropbox when any change is made. These are the really important files.

I have about 2tb of data to backup. What would the cost be to use something like Amazon S3? Is that the right direction for someone like me to use? It seems like the amount of data I have is pretty small for something like Amazon. I like the instant back up but would like to be able to access the data and download files as needed.

Thanks
Roger

Like

    Your present plan sounds like a good one to me!

    However, if you want to add Amazon S3 or any other online service to the mix, my first question would be “How fast is your Internet UPLOAD speed?” (Download speed doesn’t count in this case.)

    Two terabytes is a lot of data for most in-home broadband Internet connections although it would be trivial for many businesses that have OC3 or similar fiber optic connections to the Internet.

    For instance, where I live, I have 60 megabit-per-second download speeds (again, download doesn’t count) and I typically see 20 or 25 megabit-per-second UPLOAD speeds. At that speed it probably will take 2 or 3 weeks to compete the first backup. Later incremental backups would require only a few seconds each time.

    Some of my neighbors pay more than I do and they have 200 megabit-per-second upload AND download speeds. Using that speed theoretically would cut the time required by 90% or more. However, I don’t have a need for that much speed so I have never signed up for the more expensive service. I presently backup up about 250 gigabytes (not terabytes) to Google S3 and to the other off-shore service I use. 20 or 25 megabit-per-second connections are reasonable for me. The local cable company keeps sending me ads suggesting that I upgrade to a higher speed but, so far, I have resisted. However, if I had to back up 2 terabytes or more, I would investigate a faster upload connection.

    200 megabits-per-second is a reasonable number for households that have four or more teenagers and adults in the house and each person frequently streams movies online, watches live television, and things like that. Those families can easily consume a lot of megabits per second when multiple family members are online simultaneously. However, that does not describe my household. The slower speed connection works well for me, even when I am streaming Netflix movies or live television programming or live news. Nobody else in the house is competing for my bandwidth.

    Good luck!

    Like

OK, Dick, I’ll nibble. What IS your off-shore super-secure service? And how does one get in contact with them? Dunno that I have anything that’s so sensitive that I’d need such a service – I don’t keep banking, investment or the like on-line – but I’m intrigued.

So, cough it up, man!

Like

    —> OK, Dick, I’ll nibble. What IS your off-shore super-secure service?

    There are at least 5 or 6 such services available. I prefer not to tell which one I use but I would trust any of them, although I also would never use any cloud-based service as my ONLY backup. I always keep at least three backup copies of my more important files and at least two of those copies must be off-site where a disaster in my home won’t destroy the backups as well.

    While cloud storage services such as Dropbox, Google Drive, and SugarSync are convenient and efficient, they are also less than super secure. I prefer to use a service where files are first encrypted in my own computer with high-quality encryption (AES-256 or equivalent), then sent across the Internet to a secure file storage service. Even the employees of these file storage services are unable to view the contents of my files simply because the files were encrypted BEFORE they left my computer(s). With many of the services, the employees cannot even see the file names or file sizes.

    I haven’t evaluated and tested each of them but I believe the list includes: Tresorit, pCloud, SpiderOak, MEGAsync, Sync.com, DigitalSafe, woelkli, SecureSafe, SwissDisk, and probably some more that I cannot remember right now or perhaps never heard of.

    Also, a number of hosting services simply run software called NextCloud or ownCloud. In fact, for the ultimate in security, you can even obtain NextCloud or ownCloud software yourself (available free of charge for one user although paid versions are available for larger, multi-user installations) and install it on a server in your home or on a remote server out in the cloud someplace. In that case, YOU are the the owner and the system administrator.

    If you prefer to let someone else run the NextCloud or ownCloud servers for you, some of the NextCloud or ownCloud providers can be found at https://nextcloud.com/ and at https://owncloud.com/

    To my knowledge, all of these are high-quality, secure services although I will again state that I haven’t personally tested each one of these in the above list.

    There is one more alternative. I don’t do this myself but I would trust it. It is possible to first encrypt the files in your own computer, using any of a dozen or so encryption products available for Windows, Macintosh, Linux and other operating systems. Then, you can transfer those files to one of the less-secure services, such as Dropbox, Google Drive, Amazon Drive, or OneDrive. Since only ENCRYPTED files are sent, the only person who could ever read them is you. The downside of this method is that you lose convenience (more steps to go through, unable to read the file on your iPad, Android tablet, or smartphone, etc.) but they would remain secure.

    Like

Dick, The one thing not covered and protected against is a hard drive crash which seems like one of most likely reason for the loss of data. None of these backups will restore a hard drive.
I use a program called Macrium Reflect to create incremental disc images to a separate USB hard drive once a month. I can update any out of date files generated during the prior month from the other normal backups once a new hard drive is partitioned and restored.
Happy Data Storage & Protection,
Howard

Like

    —> None of these backups will restore a hard drive.

    TimeMachine does restores of an entire hard drive. I have done that twice, once on a Macintosh system I owned and once on a client’s Macintosh.

    Like

I have been using Arq for several months now, automatic ongoing complete backups of several computers in the household. Apart from a few teething issues it has been working perfectly. This is in addition to automatic data backups to Dropbox and other similar online services. And occasional backups to external hard drives and USB sticks.

Like

Dick,
Thanks for the article. You address data backup. Could you write
something about software recovery? The software itself but more
importantly the config files. Filezilla for example has a config file
which stores FTP addresses, etc. And, how do you keep track
of passwords?

Best, George Waller

Like

One note of caution about using a backup drive that is always connected to your computer: if you are successfully attacked by a virus or malware it can quickly infect connected drives. Always-on backups, such as TimeMachine (Mac) and File History (Windows), are great for creating up-to-the-minute backups, but you should also have disconnected backups you can restore when your computer and the drives plugged into it are infected or held hostage by ransomware.

I keep a separate backup, updated weekly, stored on a portable hard drive in my desk drawer and another, updated monthly, at an off-site location. The cost and upload time of storing a full backup in the cloud are not worth it to me, but I do maintain cloud storage of critical files.

Like

    —> Always-on backups, such as TimeMachine (Mac) and File History (Windows), are great for creating up-to-the-minute backups, but you should also have disconnected backups you can restore when your computer and the drives plugged into it are infected or held hostage by ransomware.

    Correct. As I stated in the above article, I always have at least THREE backups: one in a local USB plug-in hard rive that IS connected to my computer all the time, plus at least two different services in the cloud that are NOT directly connected to my computer all the time.

    Like

Another technique not to overlook is the physical security especially if you don’t like a cloud based storage. I use portable USB hard drives which are the weekly or monthly archives. These are stored in a locked fire proof container. A fire safe bolted to the floor would be even better and less likely to be carried off, should that be a concern. The safe will also mitigate any EMP/lightning susceptibility (esp. for all preppers to counter). Other suggestions presented are very good but don’t overlook physical security.

Like

I also use an external hard drive that backs up automatically as well. I use a flash drive to back up my files to my wife’s desktop and external as well once a month and vice versa, so four copies in the house. I also backup to a laptop for travel purposes.
The cloud services sound just fine, but I have two small external drives. I have one in a drawer, and four times a year, I backup everything and take it to our bank’s lockbox and exchange it with the one that is there. Three months later, I do the same thing, always rotating an external hard drive out of the bank lock box. If the house blows up, the very worst thing is I lost three months data.

Like

Julian de la Garza August 2, 2017 at 11:33 am

Hi Dick, excellent article and thanks for sharing your personal backup plan. I have always used Dropbox and Time Machine for my backups and have just recently added Backblaze as a new option. I can say I am satisfied but still asking myself if this is fair enough. Anyway, I just have one question. What do you recommend to keep track of which files are backed up in which service? How do you know if you have already backed up those photos taken some weeks back? You can say to check it manually but when dealing with lots of files and folders, it can get really tricky and confusing.

Like

    —> What do you recommend to keep track of which files are backed up in which service?

    I am sure there must be 4 or 5 ways of doing that. Here are the methods I can think of at this moment:

    1. Most of the cloud-based file backup services create a log file in your computer, listing all files that have been copied. Simply look in the log file to see if a file was copied. With the services I have used, that is simple to do with any files backed up recently. However, files backed up weeks or months ago might require a bit more effort to find in the log file but still should be listed.

    2. With almost all the cloud-based file backup services, you can open a web browser, connect to that service, and see the files that are there. It will be a bit like looking at Windows Explorer or Macintosh Finder. Assuming you know the name of the file and the folder in which it was stored in your computer, you should be able to locate it within seconds.

    3. Do a RESTORE of the file or files in question. All backup services have menus to restore files. If it was me, I probably would restore a file to a different place on my computer’s hard drive, such as to a subdirectory named WORK or DOWNLOADS. If it restores, obviously a copy of it is on the cloud-based file backup service.

    4. The method I use the most is probably the dumbest but it works well for me: look on my other computer, such as the laptop. Many of the cloud-based services, but not all of them, automatically copy files from one computer of yours to another. For instance, files stored on your desktop computer are automatically backed up to the file service, then copied to your laptop. This automatic copying is often called “file replication.” Dropbox does that, so does Google Drive, SpiderOak, SugarSync, and a bunch of other services. Amazon S3, Amazon Glacier, and some other services do not replicate files automatically, however. You have to learn the capabilities of the cloud-based service you are thinking of using.

    In my case, if I question whether a file was copied from my desktop system to the cloud-based service, I simply look on my laptop computer. If the file is there, I know a copy is also sitting on the cloud-based file service. It also works in the opposite direction: a file saved on the laptop will be copied to the cloud-based file service and soon after is replicated to the desktop computer.

    For that last method, you might have to wait a few minutes or an hour or more for the replication to complete, depending upon how often your cloud-based file service makes copies. If the laptop is powered off at the time, it will connect to the cloud-based file service the next time it is powered on and connected to the Internet. The file will then be replicated to the other computer(s) typically within a few minutes after that.

    I actually use two different desktop computers, located in two different homes in two different states. I spend my summers up north and winters in the sunbelt. I don’t haul the big desktops systems back and forth with me on the airplane when I travel between homes; it is easier to have two computers and they also provide additional backup locations. File replication works on all of them. Files are automatically copied to both desktop and the laptop computers.

    If necessary, I can remotely connect to one of the desktop systems from anyplace in the world and retrieve any file that is stored on that desktop computer. That is in addition to the ability to connect to the cloud-based file service and retrieve files there.

    I try to distribute my eggs amongst multiple baskets.

    Any new or changed file(s) saved on the desktop computer I am using at this moment will be automatically copied to the cloud-based file service within minutes, then are copied to the desktop computer in the other house and to the laptop shortly after that (assuming they are turned on at the moment). Again, if they are turned off, the files will be copied within minutes after the other computer is powered on and connected to the Internet.

    So far, that last method has worked 100% of the time for me. There are probably are other methods as well.

    Like

Good advice re. cloud storage. As I recall, when Kodak digital services was bought by another company, clients immediately were notified that their files stored by Kodak wouldn’t be accessible for six months while the new company wrote software to make them readable in its own system. If this is correct, you might cite it as an example.

Like

Leave a Reply

Name and email address are required. Your email address will not be published.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

You may use these HTML tags and attributes:

<a href="" title="" rel=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <pre> <q cite=""> <s> <strike> <strong> 

%d bloggers like this: