Local ZFS snapshots are uploaded to a remote server (backup) over SSH, using the utility znapzend to manage the snapshot schedule.

http://www.znapzend.org/

1. znapzend is installed as a deb package (systemd service) using: https://github.com/Gregy/znapzend-debian

2. Create a shared key for SSH to allow password-less login from source to dest server as the receiver backupuser

ssh-keygen
ssh-copy-id -i /root/.ssh/id_rsa.pub root@destserver

3. Create the remote ZFS volume to receive the snapshots and delegate permissions to a non-root user.

*** WARNING: destination 'root@destserver:data/test' does not exist, will be ignored! ***

destserver:~# zfs create data/test

destserver:~# zfs allow -u backupuser create,receive,rename,mount,send data/test

destserver:~# zfs allow data/test
---- Permissions on data/test ----------------------------------------
Local+Descendent permissions:
        user backupuser create,mount,receive,rename,send

3. Create a snapshot schedule consisting of a local SRC schedule, and the remote DST schedule to send the snapshots.

# znapzendzetup create SRC '1week=>30min' data/test DST '1week=>30min' root@destserver:data/test

Reload the config of the znapzend daemon

# systemctl reload znapzend.service

4. The backup schedule is saved as ZFS volume properties

# zfs get all data/test | grep org.znapzend
data/test  org.znapzend:tsformat       %Y-%m-%d-%H%M%S                         local
data/test  org.znapzend:pre_znap_cmd   off                                     local
data/test  org.znapzend:dst_0          root@destserver:data/test               local
data/test  org.znapzend:mbuffer        off                                     local
data/test  org.znapzend:dst_0_plan     1weeks=>30minutes                       local
data/test  org.znapzend:enabled        on                                      local
data/test  org.znapzend:zend_delay     0                                       local
data/test  org.znapzend:recursive      off                                     local
data/test  org.znapzend:mbuffer_size   1G                                      local
data/test  org.znapzend:post_znap_cmd  off                                     local
data/test  org.znapzend:src_plan       1weeks=>30minutes                       local

Watch the output of the znapzend daemon for any errors:

:~# journalctl -u znapzend.service -f &

Watch for local snapshots to be created:

# zfs list -t snap
NAME                                 USED  AVAIL  REFER  MOUNTPOINT
data/test@2017-08-27-110000             0      -   128K  -

destserver:~# zfs list -t snap
NAME                          USED  AVAIL  REFER  MOUNTPOINT
data/test@2017-08-27-110000      0      -    19K  -

~# znapzendztatz -r data/test
USED    LAST SNAPSHOT       DATASET
85.2K   2017-08-27-110000   data/test
    0   2017-08-27-110000   root@destserver:data/test

5. You can see how the files and directories looked at snapshot time by accessing the hidden .zfs directory

:~# find /data/test/.zfs -type f -ls
              7      1 -rw-r--r--   1 root     root            0 Aug 27 11:08 /data/test/.zfs/snapshot/2017-08-27-113000/a.txt

destserver:~# find /data/test/.zfs -type f -ls
              7      1 -rw-r--r--   1 root     root            0 Aug 27 14:08 /data/test/.zfs/snapshot/2017-08-27-113000/a.txt

:~# zfs diff data/test@2017-08-27-113000
M       /data/test/
M       /data/test/a.txt
+       /data/test/b.txt

:~# zfs diff data/test@2017-08-27-110000 data/test@2017-08-27-113000
M       /data/test/
+       /data/test/a.txt