Debian / Ubuntu: Install Duplicity For Encrypted Backup In Cloud
My cloud based backup service provider has ssh access to the backup system. How do I install and configure duplicity software for remote backups in incremental and encrypted format on Debian or Ubuntu Linux based desktop / laptop / server?
Duplicity is a piece of software which provides easy encrypted versioned remote backup of files requiring little of the remote server. It does this using GnuPG, tar, and rdiff. To transmit data it can use ssh/scp, local file access, rsync, ftp, and Amazon S3.
You also need to setup a passphrase. A passphrase is a sequence of words or other text used to control access to a computer system, program or data.
A passphrase is similar to a password in usage, but is generally longer for added security. Passphrases are often used to control both access to, and operation of, cryptographic programs and systems.
Passphrases are particularly applicable to systems that use the passphrase as an encryption key. I strongly recommend that you setup a passphrase for your ssh-keys.
Install duplicity on Debian / Ubuntu Linux
Open the terminal and type the following command (Red hat and friends see our previous faq “Install duplicity on RHEL/CentOS based systems” for more information):
$ sudo apt-get install duplicity
OR
# apt-get install duplicity
Sample outputs:
Reading package lists… Done
Building dependency tree
Reading state information… Done
The following extra packages will be installed:
librsync1 python-crypto python-gnupginterface python-paramiko
Suggested packages:
python-boto ncftp python-pexpect python-cloudfiles python-gdata tahoe-lafs
python-crypto-dbg python-crypto-doc
The following NEW packages will be installed:
duplicity librsync1 python-crypto python-gnupginterface python-paramiko
0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.
Need to get 1,735 kB of archives.
After this operation, 10.4 MB of additional disk space will be used.
Do you want to continue [Y/n]? y
Get:1 http://mirrors.kernel.org/debian/ stable/main librsync1 amd64 0.9.7-9 [72.1 kB]
Get:2 http://mirrors.kernel.org/debian/ stable/main python-gnupginterface all 0.3.2-9.1 [21.0 kB]
Get:3 http://mirrors.kernel.org/debian/ stable/main duplicity amd64 0.6.18-3 [309 kB]
Get:4 http://mirrors.kernel.org/debian/ stable/main python-crypto amd64 2.6-4 [524 kB]
Get:5 http://mirrors.kernel.org/debian/ stable/main python-paramiko all 1.7.7.1-3.1 [809 kB]
Fetched 1,735 kB in 6s (262 kB/s)
Selecting previously unselected package librsync1:amd64.
(Reading database … 36532 files and directories currently installed.)
Unpacking librsync1:amd64 (from …/librsync1_0.9.7-9_amd64.deb) …
Selecting previously unselected package python-gnupginterface.
Unpacking python-gnupginterface (from …/python-gnupginterface_0.3.2-9.1_all.deb) …
Selecting previously unselected package duplicity.
Unpacking duplicity (from …/duplicity_0.6.18-3_amd64.deb) …
Selecting previously unselected package python-crypto.
Unpacking python-crypto (from …/python-crypto_2.6-4_amd64.deb) …
Selecting previously unselected package python-paramiko.
Unpacking python-paramiko (from …/python-paramiko_1.7.7.1-3.1_all.deb) …
Processing triggers for man-db …
Processing triggers for python-support …
How do I create SSH keys?
To run automated backups, you must set password-less SSH connections using an SSH keys. Use ssh-keygen command to create ssh-keys:
ssh-keygen -t rsa
Skip a passphrase (not recommend)
If you trust your local system/server/latop/desktop do not enter a passphrase. Just hit enter twice and set an empty passphrase. See the following step-by-step guide for detailed information on setting up ssh keys:
Howto Linux / UNIX setup SSH with DSA public key authentication (password less login)
Howto use multiple SSH keys for password less login?
Setup a passphrase for ssh keys (recommend)
If you are paranoid, set a passphrase when prompted and install keychain to setup password less login:
$ sudo apt-get install keychain
Edit your ~/.bash_profile, enter:
vi $HOME/.bash_profile
Append the following keychain config directives:
Note –clear option is a security feature ##
/usr/bin/keychain –clear $HOME/.ssh/id_rsa
source $HOME/.keychain/$HOSTNAME-sh
Save and close the file. See our faq: “keychain: Set Up Secure Passwordless SSH Access For Backup Scripts” for more information.
How do I create a GPG keys?
Install GNU privacy guard – a free PGP replacement:
# apt-get install gnupg
OR
$ sudo apt-get install gnupg
Type the following command to create a GPG key to encrypt backups:
# gpg –gen-key
Sample outputs:
gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
gpg: keyring `/root/.gnupg/secring.gpg created
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048) 4096
Requested keysize is 4096 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y
You need a user ID to identify your key; the software constructs the user ID
from the Real Name, Comment and Email Address in this form:
Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>
Real name: Home Nas Server
Email address: root@nas01
Comment: Home Nas Server Backup
You selected this USER-ID:
Home Nas Server (Home Nas Server Backup) <root@nas01>
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.
To list your gpg keys, run:
# gpg –list-keys
Sample outputs:
/root/.gnupg/pubring.gpg
pub 4096R/4AABBCC7 2013-10-04
uid Home Nas Server (Home Nas Server Backup) <root@nas01>
sub 4096R/12345678 2013-10-04
You need to note down the public key 4AABBCC7.
How do I backup files?
The following example use scp to backup /home/nixcraft/ to ~/backups on the cloud.example.com system:
duplicity /home/nixcraft/ scp://user@cloud.example.com/backups
Exclude the files from backup
The following command will backup the / (whole root file system) but excludes /tmp, /proc, /nas, /jails, and /mnt directories from backup:
duplicity –exclude /tmp/ –exclude /proc/ –exclude /nas/ \
–exclude /jails/ –exclude /mnt/ / scp://user@cloud.example.com/backups
Include the files in backup
The following command will backup only the /home/, /root/, /etc/, and /var/spool/cron/ directories under root (/) file system using the –include option:
duplicity –include /home/ –include /root –include /etc/ \
–include /var/spool/cron/ –exclude ** / scp://user@cloud.example.com/backups
Encrypted backup commands
When backing up, encrypt to the given public key, instead of using symmetric (traditional) encryption. You need to use the –encrypt-key option as follows. To find out your gpg key-id use the following command:
gpg –list-keys
Full backup duplicity command
To make full encrypted backup, enter:
duplicity full –encrypt-key= 4AABBCC7 /home/nixcraft/ scp://userNameHere@cloud.example.com/backups
Incremental backup duplicity command
To make incremental encrypted backup, enter:
duplicity incr –encrypt-key= 4AABBCC7 /home/nixcraft/ scp://userNameHere@cloud.example.com/backups
List the files stored in the archive
To see the files currently backed up in the archive, enter:
duplicity list-current-files –encrypt-key= 4AABBCC7 scp://userNameHere@cloud.example.com/backups
Verify backup duplicity command
You can verify backup with the following command:
duplicity verify –encrypt-key= 4AABBCC7 scp://userNameHere@cloud.example.com/backups /home/nixcraft
Rotate backup duplicity command
You can delete all backup sets older than the given time. Now suppose you want to retain 60 days of backup and remove files older than 60 days, enter:
duplicity remove-older-than 60D –encrypt-key= 4AABBCC7 –force scp://userNameHere@cloud.example.com/backups
Pass the remove-all-but-n-full count to delete all backups sets that are older than the count:th last full backup. In other words, keep the last count full backups and associated incremental ets). The count must be larger than zero. A value of 1 means that only the single most recent backup chain will be kept. Note that the option –force will be needed to delete the files rather than just list them.
duplicity remove-all-but-n-full 10 –encrypt-key= 4AABBCC7 –force scp://userNameHere@cloud.example.com/backups
Restore (recover) last backup duplicity command
Now suppose you accidentally delete /home/nixcraft/ and want to restore it the way it was at the time of last backup, enter:
mkdir /home/nixcraft/
duplicity –encrypt-key= 4AABBCC7 scp://userNameHere@cloud.example.com/backups /home/nixcraft/
Restore (recover) specific file duplicity command
If you wanted to restore just the file “Documents/resume.doc” in /home/nixcraft/ as it was seven days ago into /home/nixcraft/Documents, type:
duplicity -t 7D –file-to-restore= Documents/resume.doc –encrypt-key= 4AABBCC7 scp://userNameHere@cloud.example.com/backups /home/nixcraft/Documents
Cleaning up backups
You can delete the extraneous duplicity files. Non-duplicity files, or files in complete data sets will not be deleted. This should only be necessary after a duplicity session fails or is aborted prematurely. Note that the –force option will be needed to delete the files rather than just list them:
duplicity cleanup –force –encrypt-key= 4AABBCC7 scp://userNameHere@cloud.example.com/backups
Putting it all together
A sample shell script:
#!/bin/bash
# A simple backup script wrapper for duplicity.
# Author – nixCraft <www.cyberciti.biz> under GPL v2+
# —————————————————–
Define VARS ##
_gpg_key= 4AABBCC7
_target= scp://userNameHere@cloud.example.com/backups
_duplicity= /usr/bin/duplicity
_src= /home/nixcraft/
Unlock ssh/scp/sftp ##
source $HOME/.keychain/$HOSTNAME-sh
Cleanup ##
$_duplicity –force –encrypt-key= ${_gpg_key} ${_target}
Rotate old backups than 60 days ##
$_duplicity remove-older-than 60D –encrypt-key= ${_gpg_key} ${_target}
Backup our home sweet home i.e. /home/nixcraft/ ##
Note: full backup if older than 60 day else do incremental backup ###
$_duplicity ${_src} –encrypt-key= ${_gpg_key} –full-if-older-than 60D ${_target}
Setup cron job as follows:
@daily /path/to/your/awesome/backup.script.sh
Recommend readings:
Man pages: ssh(1),scp(1),duplicity(1),cron(8),bash(1)
duplicity home page