====== ZFS Snapshot Send/Receive ====== Local ZFS snapshots are uploaded to a remote server (backup) over SSH, using the utility znapzend to manage the snapshot schedule. http://www.znapzend.org/ 1. znapzend is installed as a deb package (systemd service) using: https://github.com/Gregy/znapzend-debian 2. Create a shared key for SSH to allow password-less login from source to dest server as the receiver backupuser Do not specify a password on the shared key, just press enter when prompted. ssh-keygen ssh-copy-id -i /root/.ssh/id_rsa.pub root@destserver 3. Create the remote ZFS volume to receive the snapshots and delegate permissions to a non-root user. The ZFS volume of the given name must exist on the remote node *** WARNING: destination 'root@destserver:data/test' does not exist, will be ignored! *** destserver:~# zfs create data/test 'zfs allow' does not yet work with ZFS on Linux < 0.7.0 to allow a non-root destination user destserver:~# zfs allow -u backupuser create,receive,rename,mount,send data/test destserver:~# zfs allow data/test ---- Permissions on data/test ---------------------------------------- Local+Descendent permissions: user backupuser create,mount,receive,rename,send 3. Create a snapshot schedule consisting of a local SRC schedule, and the remote DST schedule to send the snapshots. Keep snapshots for 1week, with a snapshot every 30 minutes # znapzendzetup create SRC '1week=>30min' data/test DST '1week=>30min' root@destserver:data/test Reload the config of the znapzend daemon # systemctl reload znapzend.service 4. The backup schedule is saved as ZFS volume properties # zfs get all data/test | grep org.znapzend data/test org.znapzend:tsformat %Y-%m-%d-%H%M%S local data/test org.znapzend:pre_znap_cmd off local data/test org.znapzend:dst_0 root@destserver:data/test local data/test org.znapzend:mbuffer off local data/test org.znapzend:dst_0_plan 1weeks=>30minutes local data/test org.znapzend:enabled on local data/test org.znapzend:zend_delay 0 local data/test org.znapzend:recursive off local data/test org.znapzend:mbuffer_size 1G local data/test org.znapzend:post_znap_cmd off local data/test org.znapzend:src_plan 1weeks=>30minutes local Watch the output of the znapzend daemon for any errors: :~# journalctl -u znapzend.service -f & Watch for local snapshots to be created: # zfs list -t snap NAME USED AVAIL REFER MOUNTPOINT data/test@2017-08-27-110000 0 - 128K - destserver:~# zfs list -t snap NAME USED AVAIL REFER MOUNTPOINT data/test@2017-08-27-110000 0 - 19K - ~# znapzendztatz -r data/test USED LAST SNAPSHOT DATASET 85.2K 2017-08-27-110000 data/test 0 2017-08-27-110000 root@destserver:data/test 5. You can see how the files and directories looked at snapshot time by accessing the hidden .zfs directory :~# find /data/test/.zfs -type f -ls 7 1 -rw-r--r-- 1 root root 0 Aug 27 11:08 /data/test/.zfs/snapshot/2017-08-27-113000/a.txt destserver:~# find /data/test/.zfs -type f -ls 7 1 -rw-r--r-- 1 root root 0 Aug 27 14:08 /data/test/.zfs/snapshot/2017-08-27-113000/a.txt :~# zfs diff data/test@2017-08-27-113000 M /data/test/ M /data/test/a.txt + /data/test/b.txt :~# zfs diff data/test@2017-08-27-110000 data/test@2017-08-27-113000 M /data/test/ + /data/test/a.txt