Rsync with Google Drive
DISCLAIMER: Uploading to a cloud storage system implies that you trust the maintainers of that system and everyone in-between to not mess with/read your data. If this is a concern for you, but you want to upload to Google Drive, anyway, please consider using some of the encryption methods mentioned here (or another encryption method of your choosing): http://how-to.linuxcareer.com/using-openssl-to-encrypt-messages-and-files (Thanks to mostlyharmless for this info.)
UPDATE Feb. 12 2016: The gsync project has had some problems with show-stopping bugs, so you might want to try using rclone, instead. This Linux user has had good experiences with rclone. It's worth noting that, although this HowTo is Google Drive-specific, rclone works with about a dozen cloud storage services, including Amazon S3 and Dropbox. --DaneM
RCLONE INSTRUCTIONS
First, get rclone from http://rclone.org. Download the appropriate file for your operating system and architecture. Decompress the file and either run it from the directory it creates or copy it to somewhere in your PATH, like "/usr/local/bin/" and run it from any location on the command-line. Like gsync, this is a command-line program, so you will need to either run it from the terminal or write a shell script (BASH/SH recommended) to make it do what you want via a double-click. If you choose to do the latter, remember to make your shell script executable with "chmod +x <scriptname>", or by using your preferred desktop environment's Properties dialogue.
When you run "rclone config", you will get to pick a name and type for your remote storage medium (referred to as "drive" in this HowTo), and it will ask you to do a few steps to authenticate with your Google account. Follow the instructions! If you run into trouble, go here:
The most basic (but probably not very desirable) way to put all the stuff from one directory on your computer onto Google Drive, overwriting older versions, is to do this:
- rclone sync "<source>/" "drive:/<destination>" -v
This will upload everything once, then only upload changes (like rsync does), later on. However, you probably don't want to execute the command in exactly this way, unless you want all your data to dump into the root folder of the remote location (like a tar bomb). So, instead, format the command like this:
- rclone sync "<source>/" "drive:/MyBackupFolder/<destination>" -v
This will put all the stuff from "<source>/" into the Google Drive folder called, "MyBackupFolder/<source>", creating subdirectories as needed. "-v" provides you with information about what the program is doing, while it executes, and is optional.
Note the the trailing slash on the source but not the destination (see below, in the gsync section).
If you want to put everything from the remote drive onto your computer, erasing old files on your local machine, and replacing them with newer ones from the remote source, just put the source--the remote storage location--first in the command; and put the destination--your local machine--second. (THIS IS DANGEROUS, AND WILL ERASE YOUR LOCAL DATA!) Also, you can transfer data from one place on remote storage to another place on remote storage using the same method: "rclone <from> <to> [options]". Refer to the documentation for instructions specific to each cloud.
Finally, if you want to backup a whole lot of directories in an automated fashion, write out a bunch of these commands--one for each directory to backup--and put it into a text file starting with "#!/bin/bash" or "#!/bin/sh" on the first line. (All of your rclone commands belong on lines AFTER the first, each on separate lines.) Then save the file, make it executable (as above), and run it from the command line or double-click it in a GUI environment (like KDE, Gnome, XFCE, etc.).
Happy cloning!
GSYNC INSTRUCTIONS
(No longer recommended.)
It took me a while to find a good, simple, reliable way to backup my stuff to Google drive, using rsync; so I've decided to share my method with the good people at LQ. In short, the answer is to use "gsync" (NOT "grsync", which is different and broken/incomplete). It supports (so far as I can tell) ALL the same options as rsync (glee!), and lets you do it with Google Drive! You can upload to, and download from GD in this way, by picking which to use as SOURCE/DESTINATION folders.
First, go here and follow the installation instructions: https://github.com/iwonbigbro/gsync/blob/master/README.rst
Do not neglect the bit about authenticating!! If you do, none of the gsync stuff, below, will work!
For reference, here's the command I use to backup my stuff between my LOCAL hard drives--from "/mnt/DATA/" to "/mnt/DATA2":
- sudo rsync -c -r -t -p -o -g -v --progress --delete -l -s /mnt/DATA/ /mnt/DATA2
You can check what the options do using "man rsync." I don't always use the "-c" option, since it's slow (but more thorough for checking the data). This command will delete files that are missing from the destination, and overwrite duplicates. Use with care! Note the trailing slash on the source folder; this is important! A trailing slash on the source folder means to copy the stuff IN the folder into the destination (resulting in "/mnt/DATA2/filesandstuff"). No slash means to copy the folder, itself, into the destination (which would result in "/mnt/DATA2/DATA/filesandstuff", which is probably not what you want). The destination folder ignores trailing slashes. (Thanks to suicidaleggroll for this clarification.)
The "gsync" version is this--from "/mnt/DATA/IMPORTANTSTUFF" to "drive://IMPORTANTSTUFF":
- sudo gsync -c -r -t -p -o -g -v --progress --delete -l -s /mnt/DATA/IMPORTANTSTUFF/ drive://IMPORTANTSTUFF
Please note that you should probably not upload an entire 1TB+ drive to GD unless you have and want to use up all that storage space on the cloud. Therefore, I've specified the subdirectory of "/mnt/DATA/IMPORTANTSTUFF" to represent the important files/folders that I absolutely have to have backed-up remotely. You'll need to run a separate command for each folder (including subdirectories) that you want to upload in this fashion; make sure to change both the source and destination in the command when you do. (I haven't yet figured out how to do them all as a batch job, short of writing a script for it.) Also, I use root (sudo) for this and the rsync command because it helps manage permissions properly--but if you're certain that the current user/login owns all the files involved, you don't need it (and probably shoudn't use it, as a general security/safety precaution).
Finally, if you want to be able to walk away from it and know how long it actually took when you come back, you can prepend the "time" command to the beginning of the gsync or rsync command, like so:
- sudo time gsync -c -r -t -p -o -g -v --progress --delete -l -s /mnt/DATA/IMPORTANTSTUFF/ drive://IMPORTANTSTUFF
If you would like to automate this process using a desktop-clickable script, and are having trouble getting it to work with sudo, check out Bash Tips: Acquire root access from within a BASH script using sudo.
Enjoy!
Update 1-20-15: The following gsync bug is causing problems for uploading large amounts of data: https://github.com/iwonbigbro/gsync/issues/69 In short, all gsync users are currently using the same Google API key, which has a limit as to how much data can be uploaded by gsync in a single day. Patches are in the works to resolve this problem. This bug may also result in only empty directories being created, but the comments have a workaround for that. Please consult the github bug report for further updates.