Debian / Ubuntu: Install Duplicity For Encrypted Backup In Cloud

My cloud based backup service provider has ssh access to the backup system. How do I install and configure duplicity software for remote backups in incremental and encrypted format on Debian or Ubuntu Linux based desktop / laptop / server?

 

Duplicity is a piece of software which provides easy encrypted versioned remote backup of files requiring little of the remote server. It does this using GnuPG, tar, and rdiff. To transmit data it can use ssh/scp, local file access, rsync, ftp, and Amazon S3.

You also need to setup a passphrase. A passphrase is a sequence of words or other text used to control access to a computer system, program or data.

A passphrase is similar to a password in usage, but is generally longer for added security. Passphrases are often used to control both access to, and operation of, cryptographic programs and systems.

Passphrases are particularly applicable to systems that use the passphrase as an encryption key. I strongly recommend that you setup a passphrase for your ssh-keys.

Install duplicity on Debian / Ubuntu Linux

Open the terminal and type the following command (Red hat and friends see our previous faq “Install duplicity on RHEL/CentOS based systems” for more information):

$ sudo apt-get install duplicity

 

OR

# apt-get install duplicity

 

Sample outputs:

Reading package lists… Done

Building dependency tree

Reading state information… Done

The following extra packages will be installed:

librsync1 python-crypto python-gnupginterface python-paramiko

Suggested packages:

python-boto ncftp python-pexpect python-cloudfiles python-gdata tahoe-lafs

python-crypto-dbg python-crypto-doc

The following NEW packages will be installed:

duplicity librsync1 python-crypto python-gnupginterface python-paramiko

0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded.

Need to get 1,735 kB of archives.

After this operation, 10.4 MB of additional disk space will be used.

Do you want to continue [Y/n]? y

Get:1 http://mirrors.kernel.org/debian/ stable/main librsync1 amd64 0.9.7-9 [72.1 kB]

Get:2 http://mirrors.kernel.org/debian/ stable/main python-gnupginterface all 0.3.2-9.1 [21.0 kB]

Get:3 http://mirrors.kernel.org/debian/ stable/main duplicity amd64 0.6.18-3 [309 kB]

Get:4 http://mirrors.kernel.org/debian/ stable/main python-crypto amd64 2.6-4 [524 kB]

Get:5 http://mirrors.kernel.org/debian/ stable/main python-paramiko all 1.7.7.1-3.1 [809 kB]

Fetched 1,735 kB in 6s (262 kB/s)

Selecting previously unselected package librsync1:amd64.

(Reading database … 36532 files and directories currently installed.)

Unpacking librsync1:amd64 (from …/librsync1_0.9.7-9_amd64.deb) …

Selecting previously unselected package python-gnupginterface.

Unpacking python-gnupginterface (from …/python-gnupginterface_0.3.2-9.1_all.deb) …

Selecting previously unselected package duplicity.

Unpacking duplicity (from …/duplicity_0.6.18-3_amd64.deb) …

Selecting previously unselected package python-crypto.

Unpacking python-crypto (from …/python-crypto_2.6-4_amd64.deb) …

Selecting previously unselected package python-paramiko.

Unpacking python-paramiko (from …/python-paramiko_1.7.7.1-3.1_all.deb) …

Processing triggers for man-db …

 

 

 

 

 

 

Processing triggers for python-support …

How do I create SSH keys?

To run automated backups, you must set password-less SSH connections using an SSH keys. Use ssh-keygen command to create ssh-keys:

ssh-keygen -t rsa

Skip a passphrase (not recommend)

If you trust your local system/server/latop/desktop do not enter a passphrase. Just hit enter twice and set an empty passphrase. See the following step-by-step guide for detailed information on setting up ssh keys:

Howto Linux / UNIX setup SSH with DSA public key authentication (password less login)

Howto use multiple SSH keys for password less login?

Setup a passphrase for ssh keys (recommend)

If you are paranoid, set a passphrase when prompted and install keychain to setup password less login:

$ sudo apt-get install keychain

 

Edit your ~/.bash_profile, enter:

vi $HOME/.bash_profile

 

Append the following keychain config directives:

Note –clear option is a security feature ##

/usr/bin/keychain –clear  $HOME/.ssh/id_rsa

source $HOME/.keychain/$HOSTNAME-sh

Save and close the file. See our faq: “keychain: Set Up Secure Passwordless SSH Access For Backup Scripts” for more information.

How do I create a GPG keys?

Install GNU privacy guard – a free PGP replacement:

# apt-get install gnupg

 

OR

$ sudo apt-get install gnupg

 

Type the following command to create a GPG key to encrypt backups:

# gpg –gen-key

 

Sample outputs:

gpg (GnuPG) 1.4.12; Copyright (C) 2012 Free Software Foundation, Inc.

This is free software: you are free to change and redistribute it.

There is NO WARRANTY, to the extent permitted by law.

 

gpg: keyring `/root/.gnupg/secring.gpg  created

Please select what kind of key you want:

(1) RSA and RSA (default)

(2) DSA and Elgamal

(3) DSA (sign only)

(4) RSA (sign only)

Your selection? 1

RSA keys may be between 1024 and 4096 bits long.

What keysize do you want? (2048) 4096

Requested keysize is 4096 bits

Please specify how long the key should be valid.

0 = key does not expire

<n>  = key expires in n days

<n>w = key expires in n weeks

<n>m = key expires in n months

<n>y = key expires in n years

Key is valid for? (0) 0

Key does not expire at all

Is this correct? (y/N) y

 

You need a user ID to identify your key; the software constructs the user ID

from the Real Name, Comment and Email Address in this form:

Heinrich Heine (Der Dichter) <heinrichh@duesseldorf.de>

 

Real name: Home Nas Server

Email address: root@nas01

Comment: Home Nas Server Backup

You selected this USER-ID:

Home Nas Server (Home Nas Server Backup) <root@nas01>

 

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O

You need a Passphrase to protect your secret key.

To list your gpg keys, run:

# gpg –list-keys

 

Sample outputs:

/root/.gnupg/pubring.gpg


pub   4096R/4AABBCC7 2013-10-04

uid                  Home Nas Server (Home Nas Server Backup) <root@nas01>

sub   4096R/12345678 2013-10-04

You need to note down the public key 4AABBCC7.

How do I backup files?

The following example use scp to backup /home/nixcraft/ to ~/backups on the cloud.example.com system:

duplicity /home/nixcraft/ scp://user@cloud.example.com/backups

Exclude the files from backup

The following command will backup the / (whole root file system) but excludes /tmp, /proc, /nas, /jails, and /mnt directories from backup:

duplicity –exclude /tmp/ –exclude /proc/ –exclude /nas/ \

–exclude /jails/ –exclude /mnt/ / scp://user@cloud.example.com/backups

Include the files in backup

The following command will backup only the /home/, /root/, /etc/, and /var/spool/cron/ directories under root (/) file system using the –include option:

duplicity –include /home/ –include /root –include /etc/ \

–include /var/spool/cron/ –exclude  **  / scp://user@cloud.example.com/backups

Encrypted backup commands

When backing up, encrypt to the given public key, instead of using symmetric (traditional) encryption. You need to use the –encrypt-key option as follows. To find out your gpg key-id use the following command:

gpg –list-keys

Full backup duplicity command

To make full encrypted backup, enter:

duplicity full –encrypt-key= 4AABBCC7  /home/nixcraft/ scp://userNameHere@cloud.example.com/backups

Incremental backup duplicity command

To make incremental encrypted backup, enter:

duplicity incr –encrypt-key= 4AABBCC7  /home/nixcraft/ scp://userNameHere@cloud.example.com/backups

List the files stored in the archive

To see the files currently backed up in the archive, enter:

duplicity list-current-files –encrypt-key= 4AABBCC7  scp://userNameHere@cloud.example.com/backups

Verify backup duplicity command

You can verify backup with the following command:

duplicity verify –encrypt-key= 4AABBCC7  scp://userNameHere@cloud.example.com/backups /home/nixcraft

Rotate backup duplicity command

You can delete all backup sets older than the given time. Now suppose you want to retain 60 days of backup and remove files older than 60 days, enter:

duplicity remove-older-than 60D –encrypt-key= 4AABBCC7  –force scp://userNameHere@cloud.example.com/backups

Pass the remove-all-but-n-full count to delete all backups sets that are older than the count:th last full backup. In other words, keep the last count full backups and associated incremental ets). The count must be larger than zero. A value of 1 means that only the single most recent backup chain will be kept. Note that the option –force will be needed to delete the files rather than just list them.

duplicity remove-all-but-n-full 10 –encrypt-key= 4AABBCC7  –force scp://userNameHere@cloud.example.com/backups

Restore (recover) last backup duplicity command

Now suppose you accidentally delete /home/nixcraft/ and want to restore it the way it was at the time of last backup, enter:

mkdir /home/nixcraft/

duplicity –encrypt-key= 4AABBCC7  scp://userNameHere@cloud.example.com/backups /home/nixcraft/

Restore (recover) specific file duplicity command

If you wanted to restore just the file “Documents/resume.doc” in /home/nixcraft/ as it was seven days ago into /home/nixcraft/Documents, type:

duplicity -t 7D –file-to-restore= Documents/resume.doc  –encrypt-key= 4AABBCC7  scp://userNameHere@cloud.example.com/backups /home/nixcraft/Documents

Cleaning up backups

You can delete the extraneous duplicity files. Non-duplicity files, or files in complete data sets will not be deleted. This should only be necessary after a duplicity session fails or is aborted prematurely. Note that the –force option will be needed to delete the files rather than just list them:

duplicity cleanup –force –encrypt-key= 4AABBCC7  scp://userNameHere@cloud.example.com/backups

Putting it all together

A sample shell script:

#!/bin/bash

# A simple backup script wrapper for duplicity.

# Author – nixCraft <www.cyberciti.biz> under GPL v2+

# —————————————————–

 

Define VARS ##

_gpg_key= 4AABBCC7

_target= scp://userNameHere@cloud.example.com/backups

_duplicity= /usr/bin/duplicity

_src= /home/nixcraft/

 

Unlock ssh/scp/sftp ##

source $HOME/.keychain/$HOSTNAME-sh

 

Cleanup ##

$_duplicity –force –encrypt-key= ${_gpg_key}  ${_target}

 

Rotate old backups than 60 days ##

$_duplicity remove-older-than 60D –encrypt-key= ${_gpg_key}  ${_target}

 

Backup our home sweet home i.e. /home/nixcraft/ ##

Note: full backup if older than 60 day else do incremental backup ###

$_duplicity ${_src} –encrypt-key= ${_gpg_key}  –full-if-older-than 60D ${_target}

Setup cron job as follows:

@daily /path/to/your/awesome/backup.script.sh

Recommend readings:

Man pages: ssh(1),scp(1),duplicity(1),cron(8),bash(1)

duplicity home page

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *