btrbk: fix duplicate cmdline option -l in lsbtr

After 7 years and a lot of warnings we can drop this code. Note that
most of those refer to `upgrade_to_v0.23.0.md`, which was removed in
16364e2.

.github/workflows/codespell.yml

@@ -0,0 +1,26 @@
+---
+name: Codespell
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+    branches: [master]
+
+permissions:
+  contents: read
+
+jobs:
+  codespell:
+    name: Check for spelling errors
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+      - name: Codespell
+        uses: codespell-project/actions-codespell@v2
+        with:
+          check_filenames: true
+          skip: ".git,*.pdf,*.svg"
+          ignore_words_list: uptodate

ChangeLog

@@ -1,3 +1,22 @@
+btrbk-0.32.6
+
+  * Fix backup of unrelated (by parent_uuid) snapshots (close #339).
+  * Remove echo -e for portability (close #506).
+  * Support btrfs send protocol v2 (send_protocol and
+    send_compressed_data config options).
+  * Add bzip3 support.
+  * Convert ssh_filter_btrbk.sh to POSIX sh, and harden it.
+  * Slight change in ssh_filter_btrbk.sh logging output.
+  * Minor bugfixes, stability and documentation improvements.
+
+btrbk-0.32.5
+
+  * Correct handling of zero-size raw info file (close #491).
+
+btrbk-0.32.4
+
+  * Fix regression: wrong deprecation warnings in some cases.
+
 btrbk-0.32.3
 
   * Fix deletion of many subvolumes at once (close #476).
@@ -28,7 +47,7 @@ btrbk-0.32.1
 btrbk-0.32.0
 
   * MIGRATION
-    - If timestamp_format is not configured, explicitely set
+    - If timestamp_format is not configured, explicitly set
       "timestamp_format short" to revert old behavior.
     - Update ssh_filter_btrbk.sh on remote hosts.
   * Change default for timestamp_format to "long".
@@ -62,7 +81,7 @@ btrbk-0.31.2
   * MIGRATION
     - Update ssh_filter_btrbk.sh on remote hosts.
   * ssh_filter_btrbk.sh: Fix security vulnerability.
-    Specialy crafted commands may be executed without being propely
+    Specially crafted commands may be executed without being properly
     checked. Applies to remote hosts filtering ssh commands using
     ssh_filter_btrbk.sh in authorized_keys.
   * Warn if no subvolume defined in config (close #378).
@@ -229,7 +248,7 @@ btrbk-0.27.0
     - Allow snapshot_dir to be a mountpoint.
     - Search complete target tree for correlated subvolumes.
     - Include snapshots from all mountpoints as candidates (disabled
-      due to uptream bug: github.com/kdave/btrfs-progs/issues/96).
+      due to upstream bug: github.com/kdave/btrfs-progs/issues/96).
     - Read /proc/self/mountinfo instead of /proc/self/mounts.
     - Always read /proc/self/mountinfo.
     - Resolve realpath using readlink(1).
@@ -300,7 +319,7 @@ btrbk-0.25.0
   * Allow trailing comments in btrbk.conf (close #129).
   * Bugfix: rate limiting must be done after compression (close #134).
   * raw_target_encrypt: Always set "gpg --no-random-seed-file":
-    prevents creation of "~/.gnupg/random_seed" with slight perfomance
+    prevents creation of "~/.gnupg/random_seed" with slight performance
     penalty.
 
 btrbk-0.24.0

README.md

@@ -82,7 +82,7 @@ with this package. For a detailed description, please consult the
 [btrbk.conf(5)] man-page.
 
 After a configuration change, it is highly recommended to check it by
-running btrbk with the `-n,--dryrun` option:
+running btrbk with the `-n,--dry-run` option:
 
     # btrbk -c /path/to/myconfig -v -n run
 
@@ -153,7 +153,7 @@ If you don't want to mount the btrfs root filesystem to
     snapshot_dir /btrbk_snapshots
     subvolume    /home
 
-Start a dry run:
+Start a dry run (-n, --dry-run):
 
     # btrbk run -n
 
@@ -161,6 +161,10 @@ Create the first snapshot:
 
     # btrbk run
 
+Print schedule (-S, --print-schedule):
+
+    # btrbk run -n -S
+
 If it works as expected, configure a cron job to run btrbk hourly:
 
 /etc/cron.hourly/btrbk:
@@ -246,7 +250,7 @@ For a quick additional snapshot of your home, run:
 Example: Host-initiated Backup on Fileserver
 --------------------------------------------
 
-Let's say you have a fileserver at "myserver.mydomain.com" where you
+Let's say you have a fileserver at "myserver.example.org" where you
 want to create backups of your laptop disk. The config could look like
 this:
 
@@ -255,11 +259,11 @@ this:
     volume /mnt/btr_pool
       subvolume rootfs
         target /mnt/btr_backup/mylaptop
-        target ssh://myserver.mydomain.com/mnt/btr_backup/mylaptop
+        target ssh://myserver.example.org/mnt/btr_backup/mylaptop
 
 In addition to the backups on your local usb-disk mounted at
 `/mnt/btr_backup/mylaptop`, incremental backups would also be pushed
-to `myserver.mydomain.com`.
+to `myserver.example.org`.
 
 
 Example: Fileserver-initiated Backups from Several Hosts
@@ -270,17 +274,17 @@ fileserver, the config would be something like:
 
     ssh_identity /etc/btrbk/ssh/id_rsa
 
-    volume ssh://alpha.mydomain.com/mnt/btr_pool
+    volume ssh://alpha.example.org/mnt/btr_pool
       target /mnt/btr_backup/alpha
       subvolume rootfs
       subvolume home
 
-    volume ssh://beta.mydomain.com/mnt/btr_pool
+    volume ssh://beta.example.org/mnt/btr_pool
       target /mnt/btr_backup/beta
       subvolume rootfs
       subvolume dbdata
 
-This will pull backups from alpha/beta.mydomain.com and locally
+This will pull backups from alpha/beta.example.org and locally
 create:
 
   * `/mnt/btr_backup/alpha/rootfs.YYYYMMDD`
@@ -381,7 +385,7 @@ running btrbk. Something like:
     rsync -az --delete \
           --inplace --numeric-ids --acls --xattrs \
           -e 'ssh -i /etc/btrbk/ssh/id_rsa' \
-          myhost.mydomain.com:/data/ \
+          myhost.example.org:/data/ \
           /mnt/btr_backup/myhost_sync/
 
     exec /usr/bin/btrbk -q run
@@ -408,7 +412,7 @@ compressed and piped through GnuPG.
     raw_target_compress   xz
     raw_target_encrypt    gpg
     gpg_keyring           /etc/btrbk/gpg/pubring.gpg
-    gpg_recipient         btrbk@mydomain.com
+    gpg_recipient         btrbk@example.org
 
     volume /mnt/btr_pool
       subvolume home
@@ -424,7 +428,7 @@ host. For each backup, two files are created:
   * `/backup/home.YYYYMMDD.btrfs.xz.gpg.info`: sidecar file containing
     metadata used by btrbk.
 
-I you are using raw _incremental_ backups, please make sure you
+If you are using raw _incremental_ backups, please make sure you
 understand the implications (see [btrbk.conf(5)], TARGET TYPES).
 
 
@@ -445,7 +449,7 @@ will need the `btrfs` executable from the [btrfs-progs] package.
 On the client side, create a ssh key dedicated to btrbk, without
 password protection:
 
-    # ssh-keygen -t rsa -b 4096 -f /etc/btrbk/ssh/id_rsa -C btrbk@mydomain.com -N ""
+    # ssh-keygen -t rsa -b 4096 -f /etc/btrbk/ssh/id_rsa -C btrbk@example.org -N ""
 
 The content of the public key (/etc/btrbk/ssh/id_rsa.pub) is used for
 authentication in "authorized_keys" on the server side (see [sshd(8)]
@@ -551,14 +555,14 @@ to run it whenever the key is used for authentication. Example
 "/root/.ssh/authorized_keys":
 
     # example backup source (also allowing deletion of old snapshots)
-    command="/backup/scripts/ssh_filter_btrbk.sh -l --source --delete" <pubkey>...
+    command="/backup/scripts/ssh_filter_btrbk.sh -l --source --delete",restrict <pubkey>...
 
     # example backup target (also allowing deletion of old snapshots)
-    command="/backup/scripts/ssh_filter_btrbk.sh -l --target --delete" <pubkey>...
+    command="/backup/scripts/ssh_filter_btrbk.sh -l --target --delete",restrict <pubkey>...
 
     # example fetch-only backup source (snapshot_preserve_min=all, snapshot_create=no),
     # restricted to subvolumes within /home or /data
-    command="/backup/scripts/ssh_filter_btrbk.sh -l --send -p /home -p /data" <pubkey>...
+    command="/backup/scripts/ssh_filter_btrbk.sh -l --send -p /home -p /data",restrict <pubkey>...
 
 
   [ssh_filter_btrbk(1)]: https://digint.ch/btrbk/doc/ssh_filter_btrbk.1.html

contrib/cron/btrbk-mail

@@ -4,6 +4,9 @@
 
 now=$(date +%Y%m%d)
 
+declare -A rsync_src rsync_dst rsync_log rsync_rsh rsync_opt
+declare -A sync_fs_onchange
+
 #####  start config section  #####
 
 # Email recipients, separated by whitespace:
@@ -30,7 +33,9 @@ rsync_opt[example_data]="-az --delete --inplace --numeric-ids --acls --xattrs"
 # If set, add "rsync_dst" to "sync_fs" (see below) if rsync reports files transferred
 #sync_fs_onchange[example_data]=yes
 
-# Enabled rsync declarations (whitespace-separated list)
+# Enable all rsync declarations (all indices of rsync_src array)
+#rsync_enable=${!rsync_src[@]}
+# Explicitly enable rsync declarations (whitespace-separated list)
 #rsync_enable="example_data"
 rsync_enable=
 
@@ -55,6 +60,8 @@ btrbk_opts="-c /etc/btrbk/btrbk.conf"
 #mail_cmd_block_prefix='\\u200B' # zero-width whitespace
 #mail_cmd_block_prefix=". "
 
+# Newline character
+BR=$'\n'
 
 #####  end config section  #####
 
@@ -83,47 +90,47 @@ send_mail()
         body+="$info"
     fi
     if [[ -n "$detail" ]] && [[ -n "$has_errors" ]] || [[ "${mail_detail:-no}" = "yes" ]]; then
-        [[ -n "$body" ]] && body+="\n\nDETAIL:\n"
+        [[ -n "$body" ]] && body+="${BR}${BR}DETAIL:${BR}"
         body+="$detail"
     fi
 
     # skip sending mail on empty body
     if [[ -z "$body" ]] && [[ -n "$has_errors" ]]; then
-        body+="FATAL: something went wrong (errors present but empty mail body)\n"
+        body+="FATAL: something went wrong (errors present but empty mail body)${BR}"
     fi
     [[ -z "$body" ]] && exit 0
 
     # send mail
-    echo -e "$body" | mail -s "$subject" $mailto
+    echo "$body" | mail -s "$subject" $mailto
     if [[ $? -ne 0 ]]; then
-        echo -e "$0: Failed to send btrbk mail to \"$mailto\", dumping mail:\n" 1>&2
-        echo -e "<mail_subject>$subject</mail_subject>\n<mail_body>\n$body</mail_body>" 1>&2
+        echo "$0: Failed to send btrbk mail to \"$mailto\", dumping mail:${BR}" 1>&2
+        echo "<mail_subject>$subject</mail_subject>${BR}<mail_body>${BR}$body</mail_body>" 1>&2
     fi
 }
 
 einfo()
 {
-    info+="$1\n"
+    info+="$1${BR}"
 }
 
 ebegin()
 {
     ebtext=$1
-    detail+="\n### $1\n"
+    detail+="${BR}### $1${BR}"
 }
 
 eend()
 {
     if [[ $1 -eq 0 ]]; then
         eetext=${3-success}
-        detail+="\n"
+        detail+="${BR}"
     else
         has_errors=1
         eetext="ERROR (code=$1)"
         [[ -n "$2" ]] && eetext+=": $2"
-        detail+="\n### $eetext\n"
+        detail+="${BR}### $eetext${BR}"
     fi
-    info+="$ebtext: $eetext\n"
+    info+="$ebtext: $eetext${BR}"
     return $1
 }
 
@@ -139,10 +146,10 @@ run_cmd()
 {
     cmd_out=$("$@" 2>&1)
     local ret=$?
-    detail+="++ ${@@Q}\n"
+    detail+="++ ${@@Q}${BR}"
     if [[ -n "${mail_cmd_block_prefix:-}" ]] && [[ -n "$cmd_out" ]]; then
         detail+=$(echo -n "$cmd_out" | sed "s/^/${mail_cmd_block_prefix}/")
-        detail+="\n"
+        detail+="${BR}"
     else
         detail+=$cmd_out
     fi
@@ -159,7 +166,7 @@ mount_all()
         if [[ $? -eq 0 ]]; then
             eend -1 "already mounted"
         else
-            detail+="\n"
+            detail+="${BR}"
             run_cmd mount --target $mountpoint
             eend $? && mounted+=" $mountpoint"
         fi

contrib/cron/btrbk-verify

@@ -31,7 +31,7 @@
 #   NOTE: Depending on your setup (hardware, btrfs mount options),
 #   btrbk-verify may eat all your CPU power and use high bandwidth!
 #   Consider nice(1), ionice(1).
-
+#
 #   Incomplete resource eater list:
 #    - rsync: checksums, heavy disk I/O
 #    - btrfs: decompression, encryption
@@ -185,6 +185,8 @@ while [[ "$#" -ge 1 ]]; do
     shift
 done
 
+BR=$'\n'
+
 log_line()
 {
     echo "$@" 1>&2
@@ -207,13 +209,13 @@ tlog()
     [[ -n "$dryrun" ]] && [[ "$status" == "starting" ]] && status="dryrun_starting"
     local line="$(date --iso-8601=seconds) verify-rsync ${status} ${target} ${source} - -"
     [[ -n "$comment" ]] && line="$line # $comment";
-    tlog_text+="$line\n"
+    tlog_text+="$line${BR}"
     log_debug "$line"
 }
 tlog_print()
 {
     # tlog goes to stdout
-    echo -e "\nTRANSACTION LOG\n---------------\n${tlog_text:-}"
+    echo "${BR}TRANSACTION LOG${BR}---------------${BR}${tlog_text:-}"
 }
 
 # parse "rsync -i,--itemize-changes" output.

contrib/crypt/kdf_pbkdf2.py

@@ -54,7 +54,7 @@ salt_hex = "".join(["{:02x}".format(x) for x in salt])
 dk_hex = "".join(["{:02x}".format(x) for x in dk])
 
 print("KEY=" + dk_hex);
-print("algoritm=pbkdf2_hmac");
+print("algorithm=pbkdf2_hmac");
 print("hash_name=" + hash_name);
 print("salt=" + salt_hex);
 print("iterations=" + str(iterations));

contrib/migration/raw_suffix2sidecar

@@ -42,7 +42,7 @@ my $compress_format_alt = 'gz|bz2|xz|lzo|lz4';
 my $file_match = qr/[0-9a-zA-Z_@\+\-\.\/]+/;  # note: ubuntu uses '@' in the subvolume layout: <https://help.ubuntu.com/community/btrfs>
 my $uuid_match = qr/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/;
 my $timestamp_postfix_match = qr/\.(?<YYYY>[0-9]{4})(?<MM>[0-9]{2})(?<DD>[0-9]{2})(T(?<hh>[0-9]{2})(?<mm>[0-9]{2})((?<ss>[0-9]{2})(?<zz>(Z|[+-][0-9]{4})))?)?(_(?<NN>[0-9]+))?/;  # matches "YYYYMMDD[Thhmm[ss+0000]][_NN]"
-my $raw_postfix_match = qr/--(?<received_uuid>$uuid_match)(\@(?<parent_uuid>$uuid_match))?\.btrfs?(\.(?<compress>($compress_format_alt)))?(\.(?<encrypt>gpg))?(\.(?<split>split_aa))?(\.(?<incomplete>part))?/;  # matches ".btrfs_<received_uuid>[@<parent_uuid>][.gz|bz2|xz][.gpg][.split_aa][.part]"
+my $raw_postfix_match = qr/--(?<received_uuid>$uuid_match)(\@(?<parent_uuid>$uuid_match))?\.btrfs?(\.(?<compress>($compress_format_alt)))?(\.(?<encrypt>gpg))?(\.(?<split>split_aa))?(\.(?<incomplete>part))?/;  # matches ".btrfs_<received_uuid>[@<parent_uuid>][.gz|.bz2|.xz|...][.gpg][.split_aa][.part]"
 
 my $dryrun;
 

contrib/tools/btrbk_restore_raw.py

@@ -196,7 +196,7 @@ def main():
     parser.add_argument('restore_dir', help="target directory for restored subvolumes"
                                             " (path argument for \"btrfs receive\")")
     parser.add_argument('-n', '--dry-run', action='store_true',
-                        help="print commmands that would be executed")
+                        help="print commands that would be executed")
     parser.add_argument('--ignore-missing', action='store_true',
                         help="do not fail on missing parent snapshots")
 

doc/Makefile

@@ -1,5 +1,4 @@
-DOCS       = FAQ.md \
-             upgrade_to_v0.23.0.md
+DOCS       = FAQ.md
 MAN_MAN1   = btrbk.1 \
              lsbtr.1 \
              ssh_filter_btrbk.1
@@ -18,8 +17,19 @@ ifeq ($(COMPRESS), yes)
 endif
 
 # convert using "asciidoctor": <https://asciidoctor.org>
-ASCIIDOCTOR_MANPAGE = asciidoctor -d manpage -b manpage
-ASCIIDOCTOR_HTML    = asciidoctor -b html5 -d article
+# fallback to "a2x" from asciidoc package: <http://asciidoc.org>
+ifneq (, $(shell command -v asciidoctor 2> /dev/null))
+  ASCIIDOC_MANPAGE = asciidoctor -d manpage -b manpage
+  ASCIIDOC_HTML    = asciidoctor -b html5 -d article
+else ifneq (, $(shell command -v a2x 2> /dev/null))
+  # NOTE: using -L (--no-xmllint), as xmllint is a separate package on many distros.
+  ASCIIDOC_MANPAGE = a2x -L -d manpage -f manpage
+  ASCIIDOC_HTML    = asciidoc -b html -d article
+else
+  ASCIIDOC_ERR = $(error "please install either asciidoc or asciidoctor")
+  ASCIIDOC_MANPAGE = $(ASCIIDOC_ERR)
+  ASCIIDOC_HTML    = $(ASCIIDOC_ERR)
+endif
 
 # reproducible builds: reference date is ":date:" attribute from asciidoc source
 date_attr = $(shell sed -rn 's/:date:\s*//p' $(1))
@@ -50,10 +60,10 @@ clean:
 	gzip -9 -n -c $< > $@
 
 %.1 : %.1.asciidoc
-	SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_MANPAGE) -o $@ $<
+	SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_MANPAGE) $<
 
 %.5 : %.5.asciidoc
-	SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_MANPAGE) -o $@ $<
+	SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_MANPAGE) $<
 
 %.html : %.asciidoc
-	SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_HTML) -o $@ $<
+	SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_HTML) -o $@ $<

doc/btrbk.1.asciidoc

@@ -1,7 +1,7 @@
 btrbk(1)
 ========
-:date: 2022-06-25
-:release-version: 0.32.2
+:date: 2023-03-25
+:release-version: 0.32.6
 :man manual: Btrbk Manual
 :man source: Btrbk {release-version}
 
@@ -88,8 +88,15 @@ OPTIONS
     commands that would be executed.
 
 --exclude <filter>::
-    Exclude configured sections matching '<filter>'. See
-    <<_filter_statements,FILTER STATEMENTS>> below.
+    Exclude configured sections matching '<filter>' (see
+    <<_filter_statements,FILTER STATEMENTS>> below), or any specific
+    snapshot from being backuped or deleted, or any specific backup
+    from being deleted.
++
+Note that excluding specific snapshots from being backuped has impact
+on scheduling: e.g. if the "first snapshot of the day" is excluded,
+the "second snapshot of the day" shifts to "first", creating a backup
+as "first backup of the day".
 
 -p, --preserve::
     Preserve all snapshots and backups. Skips deletion of any
@@ -136,7 +143,7 @@ OPTIONS
     space-separated, quoted key=value pairs (machine readable).
 +
 If set to "col:", prints only the <columns> specified (comma-separated
-list). Header lines are ommitted if the "h:" modifier is present.
+list). Header lines are omitted if the "h:" modifier is present.
 Columns prefixed with "-" are collapsed if empty. Columns postfixed
 with ":RALIGN" are right-aligned.
 
@@ -154,9 +161,9 @@ with ":RALIGN" are right-aligned.
     command (version >= 20180505) installed on the host running btrbk.
 
 --lockfile <file>::
-    Create lockfile <file> on startup; checks lockfile before running
-    any btrfs commands (using perl "flock"), and exits if the lock is
-    held by another btrbk instance. Overrides configuration option
+    Place an exclusive lock on <file> during program execution, using
+    flock(2). If the lock is held by another process, exit before
+    running any actions. Overrides configuration option
     "lockfile". Ignored on dryrun ('-n', '--dry-run').
 
 --override <config_option>=<value>::

doc/btrbk.conf.5.asciidoc

@@ -1,7 +1,7 @@
 btrbk.conf(5)
 =============
-:date: 2022-06-25
-:release-version: 0.32.2
+:date: 2023-03-25
+:release-version: 0.32.6
 :man manual: Btrbk Manual
 :man source: Btrbk {release-version}
 
@@ -274,7 +274,7 @@ set to ``all'' (the default).
     remote locations. Defaults to ``no''. If enabled, make sure that
     '<compress_command>' is available on the source and target
     hosts. Supported '<compress_command>': gzip, pigz, bzip2, pbzip2,
-    xz, lzo, lz4, zstd.
+    bzip3, xz, lzo, lz4, zstd.
 
 *stream_compress_level* default|<number>::
     Compression level for the specified '<compress_command>'. Refer to
@@ -289,11 +289,11 @@ set to ``all'' (the default).
 
 *stream_compress_threads* default|<number>::
     Number of threads to use for <compress_command>. Only supported
-    for "pigz", "pbzip2", "zstd" and recent versions of "xz".
+    for "pigz", "pbzip2", "bzip3", "zstd" and recent versions of "xz".
 
-*stream_compress_adapt* default|<number>::
+*stream_compress_adapt* yes|no::
     Enable adaptive compression for <compress_command>. Only supported
-    for "zstd" (version >= 1.3.6).
+    for "zstd" (version >= 1.3.6). Defaults to ``no''.
 
 *stream_buffer* <size>|no::
     Add a buffer to the btrfs send stream (locally, on uncompressed
@@ -359,10 +359,10 @@ constraints.
     daemon, auth, lpr, news, cron, authpriv, local0..local7.
 
 *lockfile* <file>|no::
-    Create lockfile <file> on startup; checks lockfile before running
-    any btrfs commands (using perl "flock"), and exits if the lock is
-    held by another btrbk instance. Ignored on dryrun ('-n',
-    '--dry-run'). See also '--lockfile' command-line option.
+    Place an exclusive lock on <file> during program execution, using
+    flock(2). If the lock is held by another process, exit before
+    running any actions. Ignored on dryrun ('-n', '--dry-run'). See
+    also '--lockfile' command-line option.
 
 *backend* <backend>::
     Backend filesystem utilities to be used for btrfs specific
@@ -484,6 +484,18 @@ sources for all (!) known candidates on the filesystem.
     snapshot or backup deletion (sets '--commit-each' option for
     "btrfs subvolume delete"). Defaults to ``no''.
 
+*send_protocol* <number>|no  _*experimental*_::
+    Use btrfs send protocol version N. If enabled on 'target', btrbk
+    adds "--proto <number>" to the btrfs-send(8) command. Defaults to
+    ``no'' (btrfs default).
+
+*send_compressed_data* yes|no  _*experimental*_::
+    Send data that is compressed on the filesystem directly without
+    decompressing it. This requires protocol version 2 or higher
+    (btrfs-progs >= 5.19), and implies "send_protocol 2". If enabled
+    on 'target', btrbk adds "--compressed-data" to the btrfs-send(8)
+    command. Defaults to ``no'' (btrfs default).
+
 *snapshot_qgroup_destroy* yes|no  _*experimental*_:: {blank}
 *target_qgroup_destroy* yes|no  _*experimental*_:: {blank}
 *archive_qgroup_destroy* yes|no  _*experimental*_::
@@ -593,15 +605,11 @@ TARGET TYPES
     btrfs-send(8), with optional compression and encryption.
 +
 --
-Note that the target preserve mechanism is currently disabled for
-incremental raw backups (btrbk does not delete any incremental raw
-files)!
-
 Raw backups consist of two files: the main data file containing the
 btrfs send stream, and a sidecar file ".info" containing metadata:
 
-  <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg]
-  <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg].info
+  <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg]
+  <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg].info
 
 For 'incremental' backups ("incremental yes"), please note that:
 
@@ -612,6 +620,11 @@ For 'incremental' backups ("incremental yes"), please note that:
   make sure that a non-incremental backup is triggered from time to
   time.
 
+* The scheduler will never delete dependent parents of backups
+  preserved by the retention policy (run btrbk with the '-S',
+  '--print-schedule' option to get a comprehensive output of the
+  scheduler results).
+
 * There is currently no support for rotation of incremental backups:
   if 'incremental' is set, a full backup must be triggered manually
   from time to time in order to be able to delete old backups.
@@ -621,8 +634,8 @@ Additional options for raw targets:
 
 *raw_target_compress* <compress_command>|no::
     Compression algorithm to use for raw backup target. Supported
-    '<compress_command>': gzip, pigz, bzip2, pbzip2, xz, lzo, lz4,
-    zstd.
+    '<compress_command>': gzip, pigz, bzip2, pbzip2, bzip3, xz, lzo,
+    lz4, zstd.
 *raw_target_compress_level* default|<number>::
     Compression level for the specified <compress_command>.
 *raw_target_compress_long* default|<number>::

doc/install.md

@@ -9,16 +9,18 @@ file, choose one of the following methods:
 
 ### Generic Linux System
 
+Install [asciidoctor] or [asciidoc] if you want to build the
+documentation.
+
 Download and unpack the latest [btrbk source tarball] and type:
 
     sudo make install
 
+#### Try latest master from Github:
 
-### Gentoo Linux
-
-btrbk is in portage:
-
-    emerge app-backup/btrbk
+    wget https://raw.githubusercontent.com/digint/btrbk/master/btrbk
+    chmod +x btrbk
+    sudo ./btrbk ls /
 
 
 ### Debian Based Distros
@@ -30,7 +32,7 @@ Packages are also available via NeuroDebian: http://neuro.debian.net/pkgs/btrbk.
 
 ### Fedora Linux
 
-btrbk is in the official Fedora repos: https://apps.fedoraproject.org/packages/btrbk
+btrbk is in the official Fedora repos: https://src.fedoraproject.org/rpms/btrbk
 
     sudo dnf install btrbk
 
@@ -47,6 +49,13 @@ btrbk is in the community repository
     apk add btrbk
 
 
+### Gentoo Linux
+
+btrbk is in portage:
+
+    emerge app-backup/btrbk
+
+
 ### Void Linux
 
 btrbk is in Void's `current` repository
@@ -55,3 +64,5 @@ btrbk is in Void's `current` repository
 
 
   [btrbk source tarball]: https://digint.ch/download/btrbk/releases/
+  [asciidoctor]: https://asciidoctor.org
+  [asciidoc]: https://asciidoc.org

doc/lsbtr.1.asciidoc

@@ -1,7 +1,7 @@
 lsbtr(1)
 ========
-:date: 2022-06-25
-:release-version: 0.32.2
+:date: 2023-03-25
+:release-version: 0.32.6
 :man manual: Btrbk Manual
 :man source: Btrbk {release-version}
 
@@ -67,7 +67,7 @@ OPTIONS
     space-separated key="value" pairs (machine readable).
 +
 If set to "col:", prints only the <columns> specified (comma-separated
-list). Header lines are ommitted if the "h:" modifier is present.
+list). Header lines are omitted if the "h:" modifier is present.
 Columns prefixed with "-" are collapsed if empty. Columns postfixed
 with ":RALIGN" are right-aligned.
 

doc/ssh_filter_btrbk.1.asciidoc

@@ -1,7 +1,7 @@
 ssh_filter_btrbk(1)
 ===================
-:date: 2022-06-25
-:release-version: 0.32.2
+:date: 2023-03-25
+:release-version: 0.32.6
 :man manual: Btrbk Manual
 :man source: Btrbk {release-version}
 
@@ -39,13 +39,13 @@ The following commands are always allowed:
  - "readlink"
  - "test -d" (only if "compat busybox" configuration option is set)
  - "cat /proc/self/mountinfo"
- - pipes through "gzip", "pigz", "bzip2", "pbzip2", "xz", "lzop",
-   "lz4", "zstd" (stream_compress)
+ - pipes through "gzip", "pigz", "bzip2", "pbzip2", "bzip3", "xz",
+   "lzop", "lz4", "zstd" (stream_compress)
  - pipes through "mbuffer" (stream_buffer, rate_limit)
 
 Example line in /root/.ssh/authorized_keys on a backup target host:
 
-    command="ssh_filter_btrbk.sh --target --delete --restrict-path /mnt/btr_backup" ssh-rsa AAAAB3NzaC1...hwumXFRQBL btrbk@mydomain.com
+    command="ssh_filter_btrbk.sh --target --delete --restrict-path /mnt/btr_backup",restrict ssh-rsa AAAAB3NzaC1...hwumXFRQBL btrbk@example.org
 
 
 OPTIONS
@@ -82,6 +82,11 @@ OPTIONS
 -p, --restrict-path <path>::
     Restrict commands to <path>. Note that "btrfs subvolume show",
     "btrfs subvolume list" are NOT affected by this option.
++
+It is not possible to restrict commands to exact subvolume names, as
+btrfs-receive(8) takes a <path> as argument (directory, not including
+the subvolume file name to be created, this is encoded in the
+send-stream).
 
 -l, --log::
     Log ACCEPT and REJECT messages to the system log.

doc/upgrade_to_v0.23.0.md

@@ -1,92 +0,0 @@
-Upgrading to btrbk-v0.23.0
-==========================
-
-In order to keep btrbk simple and intuitive while adding new features,
-it became inevitable to change the semantics of the "retention policy"
-related configuration options.
-
-
-What has changed?
------------------
-
-### Preserve *first* instead of *last* snapshot/backup
-
-btrbk used to *always* transfer the latest snapshot to the target
-location, while considering the *last* snapshot/backup of a day as a
-daily backup (and also the last weekly as a monthly). This made it
-very cumbersome when running btrbk in a cron job as well as manually,
-because the last manually created snapshot was immediately transferred
-on every run, and used as the daily backup (instead of the one created
-periodically by the cron job).
-
-The new semantics are to consider the *first* (instead of *last*)
-snapshot of a hour/day/week/month as the one to be preserved, while
-only transferring the snapshots needed to satisfy the target retention
-policy.
-
-
-### Preserve snapshots for a minimum amount of time
-
-In order to specify a minimum amount of time in which *all* snapshots
-should be preserved, the new "snapshot_preserve_min" and
-"target_preserve_min" configuration options were introduced. This was
-previously covered by "snapshot_preserve_daily", which caused a lot of
-confusion among users.
-
-
-Upgrading the configuration file: /etc/btrbk/btrbk.conf
--------------------------------------------------------
-
-Please read the description of the "run" command in [btrbk(1)], as
-well as the "RETENTION POLICY" section in [btrbk.conf(5)] for a
-detailed description. Make sure to understand the new concept, and run
-`btrbk --print-schedule dryrun` after updating the configuration.
-
-
-### Upgrade retention policy
-
-If you want the same behaviour as before:
-
-    # replace this:
-    snapshot_preserve_daily   <daily>
-    snapshot_preserve_weekly  <weekly>
-    snapshot_preserve_monthly <monthly>
-
-    # with:
-    snapshot_preserve_min  <daily>d
-    snapshot_preserve      <weekly>w <monthly>m
-
-    # ... do the same with "target_preserve_*" options
-
-
-But what you probably want is something like:
-
-    snapshot_preserve_min  5d
-    snapshot_preserve      <daily>d <weekly>w <monthly>m
-
-    target_preserve_min    no
-    target_preserve        <daily>d <weekly>w <monthly>m *y
-
-This states:
-
-  * Keep all snapshots for five days (no matter how many there are)
-  * Transfer only the first snapshot of a day to the target
-  * Keep all "first snapshots of a day" for `<daily>` days, etc.
-
-
-### Upgrade "resume_missing"
-
-If you have a line: "resume_missing yes" somwhere in your config,
-simply remove it. btrbk always resumes missing backups.
-
-If you have "resume_missing no", you can imitate this behaviour by
-setting:
-
-    target_preserve_min  latest
-    target_preserve      no
-
-This states: "always transfer the latest snapshot to the target".
-
-
-  [btrbk(1)]: https://digint.ch/btrbk/doc/btrbk.1.html
-  [btrbk.conf(5)]: https://digint.ch/btrbk/doc/btrbk.conf.5.html

ssh_filter_btrbk.sh

@@ -1,9 +1,11 @@
-#!/bin/bash
+#!/bin/sh
 
-set -e
-set -u
+# initialise and sanitise the shell execution environment
+unset -v IFS
+export LC_ALL=C
+export PATH='/sbin:/bin:/usr/sbin:/usr/bin'
 
-export PATH=/sbin:/bin:/usr/sbin:/usr/bin
+set -e -u
 
 enable_log=
 restrict_path_list=
@@ -12,48 +14,57 @@ allow_exact_list=
 allow_rate_limit=1
 allow_stream_buffer=1
 allow_compress=1
-compress_list="gzip|pigz|bzip2|pbzip2|xz|lzop|lz4|zstd"
+compress_list='gzip|pigz|bzip2|pbzip2|bzip3|xz|lzop|lz4|zstd'
 
 # note that the backslash is NOT a metacharacter in a POSIX bracket expression!
 option_match='-[a-zA-Z0-9=-]+'   # matches short as well as long options
-file_match_sane='/[0-9a-zA-Z_@+./-]*' # matches file path (equal to $file_match in btrbk < 0.32.0)
+file_match_sane='/[0-9a-zA-Z_@+./-]*' # matches file path (equal to ${file_match} in btrbk < 0.32.0)
 file_match="/[^']*" # btrbk >= 0.32.0 quotes file arguments: match all but single quote
 file_arg_match="('${file_match}'|${file_match_sane})" # support btrbk < 0.32.0
 
 log_cmd()
 {
-    if [[ -n "$enable_log" ]]; then
-        logger -p $1 -t ssh_filter_btrbk.sh "$2 (Name: ${LOGNAME:-<unknown>}; Remote: ${SSH_CLIENT:-<unknown>})${3:+: $3}: $SSH_ORIGINAL_COMMAND"
+    local priority="$1"
+    local authorisation_decision="$2"
+    local reason="${3-}"
+
+    if [ -n "${enable_log}" ]; then
+        logger -p "${priority}" -t ssh_filter_btrbk.sh "${authorisation_decision} (Name: ${LOGNAME:-<unknown>}; Connection: ${SSH_CONNECTION:-<unknown>})${reason:+: ${reason}}: ${SSH_ORIGINAL_COMMAND}"
     fi
 }
 
 allow_cmd()
 {
-    allow_list="${allow_list}|$1"
+    local cmd="$1"
+
+    allow_list="${allow_list}|${cmd}"
 }
 
 allow_exact_cmd()
 {
-    allow_exact_list="${allow_exact_list}|$1"
+    local cmd="$1"
+
+    allow_exact_list="${allow_exact_list}|${cmd}"
 }
 
 reject_and_die()
 {
-    local reason=$1
-    log_cmd "auth.err" "btrbk REJECT" "$reason"
-    echo "ERROR: ssh_filter_btrbk.sh: ssh command rejected: $reason: $SSH_ORIGINAL_COMMAND" 1>&2
+    local reason="$1"
+
+    log_cmd 'auth.err' 'btrbk REJECT' "${reason}"
+    printf 'ERROR: ssh_filter_btrbk.sh: ssh command rejected: %s: %s\n' "${reason}" "${SSH_ORIGINAL_COMMAND}" >&2
     exit 255
 }
 
 run_cmd()
 {
-    log_cmd "auth.info" "btrbk ACCEPT"
-    eval " $SSH_ORIGINAL_COMMAND"
+    log_cmd 'auth.info' 'btrbk ACCEPT'
+    eval " ${SSH_ORIGINAL_COMMAND}"
 }
 
 reject_filtered_cmd()
 {
-    if [[ -n "$restrict_path_list" ]]; then
+    if [ -n "${restrict_path_list}" ]; then
 	# match any of restrict_path_list,
 	# or any file/directory (matching file_match) below restrict_path
 	path_match="'(${restrict_path_list})(${file_match})?'"
@@ -66,7 +77,7 @@ reject_filtered_cmd()
     # btrbk >= 0.32.0 quotes files, allow both (legacy)
     path_match="(${path_match}|${path_match_legacy})"
 
-    if [[ -n "$allow_compress" ]]; then
+    if [ -n "${allow_compress}" ]; then
         decompress_match="(${compress_list}) -d -c( -[pT][0-9]+)?"
         compress_match="(${compress_list}) -c( -[0-9])?( -[pT][0-9]+)?"
     else
@@ -76,8 +87,8 @@ reject_filtered_cmd()
 
     # rate_limit_remote and stream_buffer_remote use combined
     # "mbuffer" as of btrbk-0.29.0
-    if [[ -n "$allow_stream_buffer" ]] || [[ -n "$allow_rate_limit" ]]; then
-        mbuffer_match="mbuffer -v 1 -q( -s [0-9]+[kmgKMG]?)?( -m [0-9]+[kmgKMG]?)?( -[rR] [0-9]+[kmgtKMGT]?)?"
+    if [ -n "${allow_stream_buffer}" ] || [ -n "${allow_rate_limit}" ]; then
+        mbuffer_match='mbuffer -v 1 -q( -s [0-9]+[kmgKMG]?)?( -m [0-9]+[kmgKMG]?)?( -[rR] [0-9]+[kmgtKMGT]?)?'
     else
         mbuffer_match=
     fi
@@ -87,31 +98,39 @@ reject_filtered_cmd()
     stream_in_match="(${decompress_match} \| )?(${mbuffer_match} \| )?"
     stream_out_match="( \| ${mbuffer_match})?( \| ${compress_match}$)?"
 
+    # `grep`’s `-q`-option is not used as it may cause an exit status of `0` even
+    # when an error occurred.
+
     allow_stream_match="^${stream_in_match}${allow_cmd_match}${stream_out_match}"
-    if [[ $SSH_ORIGINAL_COMMAND =~ $allow_stream_match ]] ; then
+    if printf '%s' "${SSH_ORIGINAL_COMMAND}" | grep -E "${allow_stream_match}" >/dev/null 2>/dev/null; then
         return 0
     fi
 
     exact_cmd_match="^(${allow_exact_list})$";
-    if [[ $SSH_ORIGINAL_COMMAND =~ $exact_cmd_match ]] ; then
+    if printf '%s' "${SSH_ORIGINAL_COMMAND}" | grep -E "${exact_cmd_match}" >/dev/null 2>/dev/null; then
         return 0
     fi
 
-    reject_and_die "disallowed command${restrict_path_list:+ (restrict-path: \"${restrict_path_list//|/\", \"}\")}"
+    local formatted_restrict_path_list="$(printf '%s' "${restrict_path_list}" | sed 's/|/", "/g')"
+    reject_and_die "disallowed command${restrict_path_list:+ (restrict-path: \"${formatted_restrict_path_list}\")}"
 }
 
 
 # check for "--sudo" option before processing other options
 sudo_prefix=
-for key; do
-    [[ "$key" == "--sudo" ]] && sudo_prefix="sudo -n "
-    [[ "$key" == "--doas" ]] && sudo_prefix="doas -n "
+for key in "$@"; do
+    if [ "${key}" = '--sudo' ]; then
+        sudo_prefix='sudo -n '
+    fi
+    if [ "${key}" = '--doas' ]; then
+        sudo_prefix='doas -n '
+    fi
 done
 
-while [[ "$#" -ge 1 ]]; do
+while [ "$#" -ge 1 ]; do
     key="$1"
 
-    case $key in
+    case "${key}" in
       -l|--log)
           enable_log=1
           ;;
@@ -161,7 +180,7 @@ while [[ "$#" -ge 1 ]]; do
           ;;
 
       *)
-          echo "ERROR: ssh_filter_btrbk.sh: failed to parse command line option: $key" 1>&2
+          printf 'ERROR: ssh_filter_btrbk.sh: failed to parse command line option: %s\n' "${key}" >&2
           exit 255
           ;;
     esac
@@ -173,16 +192,18 @@ done
 allow_exact_cmd "${sudo_prefix}btrfs subvolume (show|list)( ${option_match})* ${file_arg_match}";
 allow_cmd "${sudo_prefix}readlink"                    # resolve symlink
 allow_exact_cmd "${sudo_prefix}test -d ${file_arg_match}" # check directory (only for compat=busybox)
-allow_exact_cmd "cat /proc/self/mountinfo"            # resolve mountpoints
-allow_exact_cmd "cat /proc/self/mounts"               # legacy, for btrbk < 0.27.0
+allow_exact_cmd 'cat /proc/self/mountinfo'            # resolve mountpoints
+allow_exact_cmd 'cat /proc/self/mounts'               # legacy, for btrbk < 0.27.0
 
 # remove leading "|" on alternation lists
-allow_list=${allow_list#\|}
-allow_exact_list=${allow_exact_list#\|}
-restrict_path_list=${restrict_path_list#\|}
+allow_list="${allow_list#\|}"
+allow_exact_list="${allow_exact_list#\|}"
+restrict_path_list="${restrict_path_list#\|}"
 
-case "$SSH_ORIGINAL_COMMAND" in
+case "${SSH_ORIGINAL_COMMAND}" in
     *\.\./*)  reject_and_die 'directory traversal'  ;;
+    *'
+'*)           reject_and_die 'unsafe character LF'  ;;
     *\$*)     reject_and_die 'unsafe character "$"' ;;
     *\&*)     reject_and_die 'unsafe character "&"' ;;
     *\(*)     reject_and_die 'unsafe character "("' ;;
@@ -191,7 +212,7 @@ case "$SSH_ORIGINAL_COMMAND" in
     *\<*)     reject_and_die 'unsafe character "<"' ;;
     *\>*)     reject_and_die 'unsafe character ">"' ;;
     *\`*)     reject_and_die 'unsafe character "`"' ;;
-    *\|*)     [[ -n "$allow_compress" ]] || [[ -n "$allow_rate_limit" ]] || [[ -n "$allow_stream_buffer" ]] || reject_and_die 'unsafe character "|"' ;;
+    *\|*)     [ -n "${allow_compress}" ] || [ -n "${allow_rate_limit}" ] || [ -n "${allow_stream_buffer}" ] || reject_and_die 'unsafe character "|"' ;;
 esac
 
 reject_filtered_cmd