Compare commits

..

No commits in common. "master" and "v0.32.2" have entirely different histories.

18 changed files with 962 additions and 789 deletions

View File

@ -1,26 +0,0 @@
---
name: Codespell
on:
push:
branches: [master]
pull_request:
branches: [master]
permissions:
contents: read
jobs:
codespell:
name: Check for spelling errors
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Codespell
uses: codespell-project/actions-codespell@v2
with:
check_filenames: true
skip: ".git,*.pdf,*.svg"
ignore_words_list: uptodate

View File

@ -1,28 +1,3 @@
btrbk-0.32.6
* Fix backup of unrelated (by parent_uuid) snapshots (close #339).
* Remove echo -e for portability (close #506).
* Support btrfs send protocol v2 (send_protocol and
send_compressed_data config options).
* Add bzip3 support.
* Convert ssh_filter_btrbk.sh to POSIX sh, and harden it.
* Slight change in ssh_filter_btrbk.sh logging output.
* Minor bugfixes, stability and documentation improvements.
btrbk-0.32.5
* Correct handling of zero-size raw info file (close #491).
btrbk-0.32.4
* Fix regression: wrong deprecation warnings in some cases.
btrbk-0.32.3
* Fix deletion of many subvolumes at once (close #476).
* Allow disabling ssh_identity and ssh_user options.
* Minor bugfixes and documentation improvements.
btrbk-0.32.2 btrbk-0.32.2
* Fix regression: archive on missing target directories. * Fix regression: archive on missing target directories.
@ -47,7 +22,7 @@ btrbk-0.32.1
btrbk-0.32.0 btrbk-0.32.0
* MIGRATION * MIGRATION
- If timestamp_format is not configured, explicitly set - If timestamp_format is not configured, explicitely set
"timestamp_format short" to revert old behavior. "timestamp_format short" to revert old behavior.
- Update ssh_filter_btrbk.sh on remote hosts. - Update ssh_filter_btrbk.sh on remote hosts.
* Change default for timestamp_format to "long". * Change default for timestamp_format to "long".
@ -81,7 +56,7 @@ btrbk-0.31.2
* MIGRATION * MIGRATION
- Update ssh_filter_btrbk.sh on remote hosts. - Update ssh_filter_btrbk.sh on remote hosts.
* ssh_filter_btrbk.sh: Fix security vulnerability. * ssh_filter_btrbk.sh: Fix security vulnerability.
Specially crafted commands may be executed without being properly Specialy crafted commands may be executed without being propely
checked. Applies to remote hosts filtering ssh commands using checked. Applies to remote hosts filtering ssh commands using
ssh_filter_btrbk.sh in authorized_keys. ssh_filter_btrbk.sh in authorized_keys.
* Warn if no subvolume defined in config (close #378). * Warn if no subvolume defined in config (close #378).
@ -248,7 +223,7 @@ btrbk-0.27.0
- Allow snapshot_dir to be a mountpoint. - Allow snapshot_dir to be a mountpoint.
- Search complete target tree for correlated subvolumes. - Search complete target tree for correlated subvolumes.
- Include snapshots from all mountpoints as candidates (disabled - Include snapshots from all mountpoints as candidates (disabled
due to upstream bug: github.com/kdave/btrfs-progs/issues/96). due to uptream bug: github.com/kdave/btrfs-progs/issues/96).
- Read /proc/self/mountinfo instead of /proc/self/mounts. - Read /proc/self/mountinfo instead of /proc/self/mounts.
- Always read /proc/self/mountinfo. - Always read /proc/self/mountinfo.
- Resolve realpath using readlink(1). - Resolve realpath using readlink(1).
@ -319,7 +294,7 @@ btrbk-0.25.0
* Allow trailing comments in btrbk.conf (close #129). * Allow trailing comments in btrbk.conf (close #129).
* Bugfix: rate limiting must be done after compression (close #134). * Bugfix: rate limiting must be done after compression (close #134).
* raw_target_encrypt: Always set "gpg --no-random-seed-file": * raw_target_encrypt: Always set "gpg --no-random-seed-file":
prevents creation of "~/.gnupg/random_seed" with slight performance prevents creation of "~/.gnupg/random_seed" with slight perfomance
penalty. penalty.
btrbk-0.24.0 btrbk-0.24.0

View File

@ -82,7 +82,7 @@ with this package. For a detailed description, please consult the
[btrbk.conf(5)] man-page. [btrbk.conf(5)] man-page.
After a configuration change, it is highly recommended to check it by After a configuration change, it is highly recommended to check it by
running btrbk with the `-n,--dry-run` option: running btrbk with the `-n,--dryrun` option:
# btrbk -c /path/to/myconfig -v -n run # btrbk -c /path/to/myconfig -v -n run
@ -153,7 +153,7 @@ If you don't want to mount the btrfs root filesystem to
snapshot_dir /btrbk_snapshots snapshot_dir /btrbk_snapshots
subvolume /home subvolume /home
Start a dry run (-n, --dry-run): Start a dry run:
# btrbk run -n # btrbk run -n
@ -161,10 +161,6 @@ Create the first snapshot:
# btrbk run # btrbk run
Print schedule (-S, --print-schedule):
# btrbk run -n -S
If it works as expected, configure a cron job to run btrbk hourly: If it works as expected, configure a cron job to run btrbk hourly:
/etc/cron.hourly/btrbk: /etc/cron.hourly/btrbk:
@ -250,7 +246,7 @@ For a quick additional snapshot of your home, run:
Example: Host-initiated Backup on Fileserver Example: Host-initiated Backup on Fileserver
-------------------------------------------- --------------------------------------------
Let's say you have a fileserver at "myserver.example.org" where you Let's say you have a fileserver at "myserver.mydomain.com" where you
want to create backups of your laptop disk. The config could look like want to create backups of your laptop disk. The config could look like
this: this:
@ -259,11 +255,11 @@ this:
volume /mnt/btr_pool volume /mnt/btr_pool
subvolume rootfs subvolume rootfs
target /mnt/btr_backup/mylaptop target /mnt/btr_backup/mylaptop
target ssh://myserver.example.org/mnt/btr_backup/mylaptop target ssh://myserver.mydomain.com/mnt/btr_backup/mylaptop
In addition to the backups on your local usb-disk mounted at In addition to the backups on your local usb-disk mounted at
`/mnt/btr_backup/mylaptop`, incremental backups would also be pushed `/mnt/btr_backup/mylaptop`, incremental backups would also be pushed
to `myserver.example.org`. to `myserver.mydomain.com`.
Example: Fileserver-initiated Backups from Several Hosts Example: Fileserver-initiated Backups from Several Hosts
@ -274,17 +270,17 @@ fileserver, the config would be something like:
ssh_identity /etc/btrbk/ssh/id_rsa ssh_identity /etc/btrbk/ssh/id_rsa
volume ssh://alpha.example.org/mnt/btr_pool volume ssh://alpha.mydomain.com/mnt/btr_pool
target /mnt/btr_backup/alpha target /mnt/btr_backup/alpha
subvolume rootfs subvolume rootfs
subvolume home subvolume home
volume ssh://beta.example.org/mnt/btr_pool volume ssh://beta.mydomain.com/mnt/btr_pool
target /mnt/btr_backup/beta target /mnt/btr_backup/beta
subvolume rootfs subvolume rootfs
subvolume dbdata subvolume dbdata
This will pull backups from alpha/beta.example.org and locally This will pull backups from alpha/beta.mydomain.com and locally
create: create:
* `/mnt/btr_backup/alpha/rootfs.YYYYMMDD` * `/mnt/btr_backup/alpha/rootfs.YYYYMMDD`
@ -385,7 +381,7 @@ running btrbk. Something like:
rsync -az --delete \ rsync -az --delete \
--inplace --numeric-ids --acls --xattrs \ --inplace --numeric-ids --acls --xattrs \
-e 'ssh -i /etc/btrbk/ssh/id_rsa' \ -e 'ssh -i /etc/btrbk/ssh/id_rsa' \
myhost.example.org:/data/ \ myhost.mydomain.com:/data/ \
/mnt/btr_backup/myhost_sync/ /mnt/btr_backup/myhost_sync/
exec /usr/bin/btrbk -q run exec /usr/bin/btrbk -q run
@ -412,7 +408,7 @@ compressed and piped through GnuPG.
raw_target_compress xz raw_target_compress xz
raw_target_encrypt gpg raw_target_encrypt gpg
gpg_keyring /etc/btrbk/gpg/pubring.gpg gpg_keyring /etc/btrbk/gpg/pubring.gpg
gpg_recipient btrbk@example.org gpg_recipient btrbk@mydomain.com
volume /mnt/btr_pool volume /mnt/btr_pool
subvolume home subvolume home
@ -428,7 +424,7 @@ host. For each backup, two files are created:
* `/backup/home.YYYYMMDD.btrfs.xz.gpg.info`: sidecar file containing * `/backup/home.YYYYMMDD.btrfs.xz.gpg.info`: sidecar file containing
metadata used by btrbk. metadata used by btrbk.
If you are using raw _incremental_ backups, please make sure you I you are using raw _incremental_ backups, please make sure you
understand the implications (see [btrbk.conf(5)], TARGET TYPES). understand the implications (see [btrbk.conf(5)], TARGET TYPES).
@ -449,7 +445,7 @@ will need the `btrfs` executable from the [btrfs-progs] package.
On the client side, create a ssh key dedicated to btrbk, without On the client side, create a ssh key dedicated to btrbk, without
password protection: password protection:
# ssh-keygen -t rsa -b 4096 -f /etc/btrbk/ssh/id_rsa -C btrbk@example.org -N "" # ssh-keygen -t rsa -b 4096 -f /etc/btrbk/ssh/id_rsa -C btrbk@mydomain.com -N ""
The content of the public key (/etc/btrbk/ssh/id_rsa.pub) is used for The content of the public key (/etc/btrbk/ssh/id_rsa.pub) is used for
authentication in "authorized_keys" on the server side (see [sshd(8)] authentication in "authorized_keys" on the server side (see [sshd(8)]
@ -555,14 +551,14 @@ to run it whenever the key is used for authentication. Example
"/root/.ssh/authorized_keys": "/root/.ssh/authorized_keys":
# example backup source (also allowing deletion of old snapshots) # example backup source (also allowing deletion of old snapshots)
command="/backup/scripts/ssh_filter_btrbk.sh -l --source --delete",restrict <pubkey>... command="/backup/scripts/ssh_filter_btrbk.sh -l --source --delete" <pubkey>...
# example backup target (also allowing deletion of old snapshots) # example backup target (also allowing deletion of old snapshots)
command="/backup/scripts/ssh_filter_btrbk.sh -l --target --delete",restrict <pubkey>... command="/backup/scripts/ssh_filter_btrbk.sh -l --target --delete" <pubkey>...
# example fetch-only backup source (snapshot_preserve_min=all, snapshot_create=no), # example fetch-only backup source (snapshot_preserve_min=all, snapshot_create=no),
# restricted to subvolumes within /home or /data # restricted to subvolumes within /home or /data
command="/backup/scripts/ssh_filter_btrbk.sh -l --send -p /home -p /data",restrict <pubkey>... command="/backup/scripts/ssh_filter_btrbk.sh -l --send -p /home -p /data" <pubkey>...
[ssh_filter_btrbk(1)]: https://digint.ch/btrbk/doc/ssh_filter_btrbk.1.html [ssh_filter_btrbk(1)]: https://digint.ch/btrbk/doc/ssh_filter_btrbk.1.html

1200
btrbk

File diff suppressed because it is too large Load Diff

View File

@ -17,13 +17,6 @@
# Enable transaction log # Enable transaction log
transaction_log /var/log/btrbk.log transaction_log /var/log/btrbk.log
# Specify SSH private key for remote connections
ssh_identity /etc/btrbk/ssh/id_ed25519
ssh_user root
# Use sudo if btrbk or lsbtr is run by regular user
backend_local_user btrfs-progs-sudo
# Enable stream buffer. Adding a buffer between the sending and # Enable stream buffer. Adding a buffer between the sending and
# receiving side is generally a good idea. # receiving side is generally a good idea.
# NOTE: If enabled, make sure to install the "mbuffer" package! # NOTE: If enabled, make sure to install the "mbuffer" package!
@ -36,7 +29,7 @@ stream_buffer 256m
# If you want to set a custom name for the snapshot (and backups), # If you want to set a custom name for the snapshot (and backups),
# use the "snapshot_name" option within the subvolume section. # use the "snapshot_name" option within the subvolume section.
# #
# NOTE: btrbk does not automatically create this directory, and the # NOTE: btrbk does not autmatically create this directory, and the
# snapshot creation will fail if it is not present. # snapshot creation will fail if it is not present.
# #
snapshot_dir _btrbk_snap snapshot_dir _btrbk_snap
@ -74,6 +67,12 @@ snapshot_dir _btrbk_snap
#archive_preserve_min no #archive_preserve_min no
#archive_preserve <NN>h <NN>d <NN>w <NN>m <NN>y #archive_preserve <NN>h <NN>d <NN>w <NN>m <NN>y
# Specify SSH private key for "ssh://" volumes / targets:
#ssh_identity /etc/btrbk/ssh/id_ed25519
#ssh_user root
#ssh_compression no
#ssh_cipher_spec default
# Enable compression for remote btrfs send/receive operations: # Enable compression for remote btrfs send/receive operations:
#stream_compress no #stream_compress no
#stream_compress_level default #stream_compress_level default
@ -83,9 +82,9 @@ snapshot_dir _btrbk_snap
# can be run at a time. # can be run at a time.
#lockfile /var/lock/btrbk.lock #lockfile /var/lock/btrbk.lock
# Don't wait for transaction commit on deletion. Enable this to make # Don't wait for transaction commit on deletion. Set this to "after"
# sure the deletion of subvolumes is committed to disk when btrbk # or "each" to make sure the deletion of subvolumes is committed to
# terminates. # disk when btrbk terminates.
#btrfs_commit_delete no #btrfs_commit_delete no
@ -179,21 +178,6 @@ volume ssh://my-remote-host.com/mnt/btr_pool
target /mnt/btr_backup/my-remote-host.com target /mnt/btr_backup/my-remote-host.com
# Backup on demand (noauto) to remote host running busybox, login as
# regular user using ssh-agent with current user name (ssh_user no)
# and default credentials (ssh_identity no).
volume /home
noauto yes
compat busybox
backend_remote btrfs-progs-sudo
ssh_user no
ssh_identity no
target ssh://my-user-host.com/mnt/btr_backup/home
subvolume alice
subvolume bob
# Resume backups from remote host which runs its own btrbk instance # Resume backups from remote host which runs its own btrbk instance
# creating snapshots for "home" in "/mnt/btr_pool/btrbk_snapshots". # creating snapshots for "home" in "/mnt/btr_pool/btrbk_snapshots".
volume ssh://my-remote-host.com/mnt/btr_pool volume ssh://my-remote-host.com/mnt/btr_pool

View File

@ -4,9 +4,6 @@
now=$(date +%Y%m%d) now=$(date +%Y%m%d)
declare -A rsync_src rsync_dst rsync_log rsync_rsh rsync_opt
declare -A sync_fs_onchange
##### start config section ##### ##### start config section #####
# Email recipients, separated by whitespace: # Email recipients, separated by whitespace:
@ -33,9 +30,7 @@ rsync_opt[example_data]="-az --delete --inplace --numeric-ids --acls --xattrs"
# If set, add "rsync_dst" to "sync_fs" (see below) if rsync reports files transferred # If set, add "rsync_dst" to "sync_fs" (see below) if rsync reports files transferred
#sync_fs_onchange[example_data]=yes #sync_fs_onchange[example_data]=yes
# Enable all rsync declarations (all indices of rsync_src array) # Enabled rsync declarations (whitespace-separated list)
#rsync_enable=${!rsync_src[@]}
# Explicitly enable rsync declarations (whitespace-separated list)
#rsync_enable="example_data" #rsync_enable="example_data"
rsync_enable= rsync_enable=
@ -60,8 +55,6 @@ btrbk_opts="-c /etc/btrbk/btrbk.conf"
#mail_cmd_block_prefix='\\u200B' # zero-width whitespace #mail_cmd_block_prefix='\\u200B' # zero-width whitespace
#mail_cmd_block_prefix=". " #mail_cmd_block_prefix=". "
# Newline character
BR=$'\n'
##### end config section ##### ##### end config section #####
@ -90,47 +83,47 @@ send_mail()
body+="$info" body+="$info"
fi fi
if [[ -n "$detail" ]] && [[ -n "$has_errors" ]] || [[ "${mail_detail:-no}" = "yes" ]]; then if [[ -n "$detail" ]] && [[ -n "$has_errors" ]] || [[ "${mail_detail:-no}" = "yes" ]]; then
[[ -n "$body" ]] && body+="${BR}${BR}DETAIL:${BR}" [[ -n "$body" ]] && body+="\n\nDETAIL:\n"
body+="$detail" body+="$detail"
fi fi
# skip sending mail on empty body # skip sending mail on empty body
if [[ -z "$body" ]] && [[ -n "$has_errors" ]]; then if [[ -z "$body" ]] && [[ -n "$has_errors" ]]; then
body+="FATAL: something went wrong (errors present but empty mail body)${BR}" body+="FATAL: something went wrong (errors present but empty mail body)\n"
fi fi
[[ -z "$body" ]] && exit 0 [[ -z "$body" ]] && exit 0
# send mail # send mail
echo "$body" | mail -s "$subject" $mailto echo -e "$body" | mail -s "$subject" $mailto
if [[ $? -ne 0 ]]; then if [[ $? -ne 0 ]]; then
echo "$0: Failed to send btrbk mail to \"$mailto\", dumping mail:${BR}" 1>&2 echo -e "$0: Failed to send btrbk mail to \"$mailto\", dumping mail:\n" 1>&2
echo "<mail_subject>$subject</mail_subject>${BR}<mail_body>${BR}$body</mail_body>" 1>&2 echo -e "<mail_subject>$subject</mail_subject>\n<mail_body>\n$body</mail_body>" 1>&2
fi fi
} }
einfo() einfo()
{ {
info+="$1${BR}" info+="$1\n"
} }
ebegin() ebegin()
{ {
ebtext=$1 ebtext=$1
detail+="${BR}### $1${BR}" detail+="\n### $1\n"
} }
eend() eend()
{ {
if [[ $1 -eq 0 ]]; then if [[ $1 -eq 0 ]]; then
eetext=${3-success} eetext=${3-success}
detail+="${BR}" detail+="\n"
else else
has_errors=1 has_errors=1
eetext="ERROR (code=$1)" eetext="ERROR (code=$1)"
[[ -n "$2" ]] && eetext+=": $2" [[ -n "$2" ]] && eetext+=": $2"
detail+="${BR}### $eetext${BR}" detail+="\n### $eetext\n"
fi fi
info+="$ebtext: $eetext${BR}" info+="$ebtext: $eetext\n"
return $1 return $1
} }
@ -146,10 +139,10 @@ run_cmd()
{ {
cmd_out=$("$@" 2>&1) cmd_out=$("$@" 2>&1)
local ret=$? local ret=$?
detail+="++ ${@@Q}${BR}" detail+="++ ${@@Q}\n"
if [[ -n "${mail_cmd_block_prefix:-}" ]] && [[ -n "$cmd_out" ]]; then if [[ -n "${mail_cmd_block_prefix:-}" ]] && [[ -n "$cmd_out" ]]; then
detail+=$(echo -n "$cmd_out" | sed "s/^/${mail_cmd_block_prefix}/") detail+=$(echo -n "$cmd_out" | sed "s/^/${mail_cmd_block_prefix}/")
detail+="${BR}" detail+="\n"
else else
detail+=$cmd_out detail+=$cmd_out
fi fi
@ -166,7 +159,7 @@ mount_all()
if [[ $? -eq 0 ]]; then if [[ $? -eq 0 ]]; then
eend -1 "already mounted" eend -1 "already mounted"
else else
detail+="${BR}" detail+="\n"
run_cmd mount --target $mountpoint run_cmd mount --target $mountpoint
eend $? && mounted+=" $mountpoint" eend $? && mounted+=" $mountpoint"
fi fi

View File

@ -31,7 +31,7 @@
# NOTE: Depending on your setup (hardware, btrfs mount options), # NOTE: Depending on your setup (hardware, btrfs mount options),
# btrbk-verify may eat all your CPU power and use high bandwidth! # btrbk-verify may eat all your CPU power and use high bandwidth!
# Consider nice(1), ionice(1). # Consider nice(1), ionice(1).
#
# Incomplete resource eater list: # Incomplete resource eater list:
# - rsync: checksums, heavy disk I/O # - rsync: checksums, heavy disk I/O
# - btrfs: decompression, encryption # - btrfs: decompression, encryption
@ -185,8 +185,6 @@ while [[ "$#" -ge 1 ]]; do
shift shift
done done
BR=$'\n'
log_line() log_line()
{ {
echo "$@" 1>&2 echo "$@" 1>&2
@ -209,13 +207,13 @@ tlog()
[[ -n "$dryrun" ]] && [[ "$status" == "starting" ]] && status="dryrun_starting" [[ -n "$dryrun" ]] && [[ "$status" == "starting" ]] && status="dryrun_starting"
local line="$(date --iso-8601=seconds) verify-rsync ${status} ${target} ${source} - -" local line="$(date --iso-8601=seconds) verify-rsync ${status} ${target} ${source} - -"
[[ -n "$comment" ]] && line="$line # $comment"; [[ -n "$comment" ]] && line="$line # $comment";
tlog_text+="$line${BR}" tlog_text+="$line\n"
log_debug "$line" log_debug "$line"
} }
tlog_print() tlog_print()
{ {
# tlog goes to stdout # tlog goes to stdout
echo "${BR}TRANSACTION LOG${BR}---------------${BR}${tlog_text:-}" echo -e "\nTRANSACTION LOG\n---------------\n${tlog_text:-}"
} }
# parse "rsync -i,--itemize-changes" output. # parse "rsync -i,--itemize-changes" output.

View File

@ -54,7 +54,7 @@ salt_hex = "".join(["{:02x}".format(x) for x in salt])
dk_hex = "".join(["{:02x}".format(x) for x in dk]) dk_hex = "".join(["{:02x}".format(x) for x in dk])
print("KEY=" + dk_hex); print("KEY=" + dk_hex);
print("algorithm=pbkdf2_hmac"); print("algoritm=pbkdf2_hmac");
print("hash_name=" + hash_name); print("hash_name=" + hash_name);
print("salt=" + salt_hex); print("salt=" + salt_hex);
print("iterations=" + str(iterations)); print("iterations=" + str(iterations));

View File

@ -42,7 +42,7 @@ my $compress_format_alt = 'gz|bz2|xz|lzo|lz4';
my $file_match = qr/[0-9a-zA-Z_@\+\-\.\/]+/; # note: ubuntu uses '@' in the subvolume layout: <https://help.ubuntu.com/community/btrfs> my $file_match = qr/[0-9a-zA-Z_@\+\-\.\/]+/; # note: ubuntu uses '@' in the subvolume layout: <https://help.ubuntu.com/community/btrfs>
my $uuid_match = qr/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/; my $uuid_match = qr/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/;
my $timestamp_postfix_match = qr/\.(?<YYYY>[0-9]{4})(?<MM>[0-9]{2})(?<DD>[0-9]{2})(T(?<hh>[0-9]{2})(?<mm>[0-9]{2})((?<ss>[0-9]{2})(?<zz>(Z|[+-][0-9]{4})))?)?(_(?<NN>[0-9]+))?/; # matches "YYYYMMDD[Thhmm[ss+0000]][_NN]" my $timestamp_postfix_match = qr/\.(?<YYYY>[0-9]{4})(?<MM>[0-9]{2})(?<DD>[0-9]{2})(T(?<hh>[0-9]{2})(?<mm>[0-9]{2})((?<ss>[0-9]{2})(?<zz>(Z|[+-][0-9]{4})))?)?(_(?<NN>[0-9]+))?/; # matches "YYYYMMDD[Thhmm[ss+0000]][_NN]"
my $raw_postfix_match = qr/--(?<received_uuid>$uuid_match)(\@(?<parent_uuid>$uuid_match))?\.btrfs?(\.(?<compress>($compress_format_alt)))?(\.(?<encrypt>gpg))?(\.(?<split>split_aa))?(\.(?<incomplete>part))?/; # matches ".btrfs_<received_uuid>[@<parent_uuid>][.gz|.bz2|.xz|...][.gpg][.split_aa][.part]" my $raw_postfix_match = qr/--(?<received_uuid>$uuid_match)(\@(?<parent_uuid>$uuid_match))?\.btrfs?(\.(?<compress>($compress_format_alt)))?(\.(?<encrypt>gpg))?(\.(?<split>split_aa))?(\.(?<incomplete>part))?/; # matches ".btrfs_<received_uuid>[@<parent_uuid>][.gz|bz2|xz][.gpg][.split_aa][.part]"
my $dryrun; my $dryrun;

View File

@ -196,7 +196,7 @@ def main():
parser.add_argument('restore_dir', help="target directory for restored subvolumes" parser.add_argument('restore_dir', help="target directory for restored subvolumes"
" (path argument for \"btrfs receive\")") " (path argument for \"btrfs receive\")")
parser.add_argument('-n', '--dry-run', action='store_true', parser.add_argument('-n', '--dry-run', action='store_true',
help="print commands that would be executed") help="print commmands that would be executed")
parser.add_argument('--ignore-missing', action='store_true', parser.add_argument('--ignore-missing', action='store_true',
help="do not fail on missing parent snapshots") help="do not fail on missing parent snapshots")

View File

@ -1,4 +1,5 @@
DOCS = FAQ.md DOCS = FAQ.md \
upgrade_to_v0.23.0.md
MAN_MAN1 = btrbk.1 \ MAN_MAN1 = btrbk.1 \
lsbtr.1 \ lsbtr.1 \
ssh_filter_btrbk.1 ssh_filter_btrbk.1
@ -17,19 +18,8 @@ ifeq ($(COMPRESS), yes)
endif endif
# convert using "asciidoctor": <https://asciidoctor.org> # convert using "asciidoctor": <https://asciidoctor.org>
# fallback to "a2x" from asciidoc package: <http://asciidoc.org> ASCIIDOCTOR_MANPAGE = asciidoctor -d manpage -b manpage
ifneq (, $(shell command -v asciidoctor 2> /dev/null)) ASCIIDOCTOR_HTML = asciidoctor -b html5 -d article
ASCIIDOC_MANPAGE = asciidoctor -d manpage -b manpage
ASCIIDOC_HTML = asciidoctor -b html5 -d article
else ifneq (, $(shell command -v a2x 2> /dev/null))
# NOTE: using -L (--no-xmllint), as xmllint is a separate package on many distros.
ASCIIDOC_MANPAGE = a2x -L -d manpage -f manpage
ASCIIDOC_HTML = asciidoc -b html -d article
else
ASCIIDOC_ERR = $(error "please install either asciidoc or asciidoctor")
ASCIIDOC_MANPAGE = $(ASCIIDOC_ERR)
ASCIIDOC_HTML = $(ASCIIDOC_ERR)
endif
# reproducible builds: reference date is ":date:" attribute from asciidoc source # reproducible builds: reference date is ":date:" attribute from asciidoc source
date_attr = $(shell sed -rn 's/:date:\s*//p' $(1)) date_attr = $(shell sed -rn 's/:date:\s*//p' $(1))
@ -60,10 +50,10 @@ clean:
gzip -9 -n -c $< > $@ gzip -9 -n -c $< > $@
%.1 : %.1.asciidoc %.1 : %.1.asciidoc
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_MANPAGE) $< SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_MANPAGE) -o $@ $<
%.5 : %.5.asciidoc %.5 : %.5.asciidoc
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_MANPAGE) $< SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_MANPAGE) -o $@ $<
%.html : %.asciidoc %.html : %.asciidoc
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_HTML) -o $@ $< SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_HTML) -o $@ $<

View File

@ -1,7 +1,7 @@
btrbk(1) btrbk(1)
======== ========
:date: 2023-03-25 :date: 2022-06-25
:release-version: 0.32.6 :release-version: 0.32.2
:man manual: Btrbk Manual :man manual: Btrbk Manual
:man source: Btrbk {release-version} :man source: Btrbk {release-version}
@ -88,15 +88,8 @@ OPTIONS
commands that would be executed. commands that would be executed.
--exclude <filter>:: --exclude <filter>::
Exclude configured sections matching '<filter>' (see Exclude configured sections matching '<filter>'. See
<<_filter_statements,FILTER STATEMENTS>> below), or any specific <<_filter_statements,FILTER STATEMENTS>> below.
snapshot from being backuped or deleted, or any specific backup
from being deleted.
+
Note that excluding specific snapshots from being backuped has impact
on scheduling: e.g. if the "first snapshot of the day" is excluded,
the "second snapshot of the day" shifts to "first", creating a backup
as "first backup of the day".
-p, --preserve:: -p, --preserve::
Preserve all snapshots and backups. Skips deletion of any Preserve all snapshots and backups. Skips deletion of any
@ -143,7 +136,7 @@ as "first backup of the day".
space-separated, quoted key=value pairs (machine readable). space-separated, quoted key=value pairs (machine readable).
+ +
If set to "col:", prints only the <columns> specified (comma-separated If set to "col:", prints only the <columns> specified (comma-separated
list). Header lines are omitted if the "h:" modifier is present. list). Header lines are ommitted if the "h:" modifier is present.
Columns prefixed with "-" are collapsed if empty. Columns postfixed Columns prefixed with "-" are collapsed if empty. Columns postfixed
with ":RALIGN" are right-aligned. with ":RALIGN" are right-aligned.
@ -161,9 +154,9 @@ with ":RALIGN" are right-aligned.
command (version >= 20180505) installed on the host running btrbk. command (version >= 20180505) installed on the host running btrbk.
--lockfile <file>:: --lockfile <file>::
Place an exclusive lock on <file> during program execution, using Create lockfile <file> on startup; checks lockfile before running
flock(2). If the lock is held by another process, exit before any btrfs commands (using perl "flock"), and exits if the lock is
running any actions. Overrides configuration option held by another btrbk instance. Overrides configuration option
"lockfile". Ignored on dryrun ('-n', '--dry-run'). "lockfile". Ignored on dryrun ('-n', '--dry-run').
--override <config_option>=<value>:: --override <config_option>=<value>::

View File

@ -1,7 +1,7 @@
btrbk.conf(5) btrbk.conf(5)
============= =============
:date: 2023-03-25 :date: 2022-06-25
:release-version: 0.32.6 :release-version: 0.32.2
:man manual: Btrbk Manual :man manual: Btrbk Manual
:man source: Btrbk {release-version} :man source: Btrbk {release-version}
@ -242,18 +242,15 @@ set to ``all'' (the default).
=== SSH Options === SSH Options
*ssh_identity* <file>|no:: *ssh_identity* <file>::
Absolute path to a ssh identity file (private key). If not set, Absolute path to a ssh identity file (private key). Note that if
the ssh default is used (see ssh(1), "-i identity_file"). Note the private key is password protected, btrbk will prompt for user
that if the identity key is password protected and no input, which is usually not desired.
authentication agent is used, btrbk will prompt for user input on
every connection attempt.
*ssh_user* <username>|no:: *ssh_user* <username>::
Remote username for ssh. Defaults to ``root''. Make sure the Remote username for ssh. Defaults to ``root''. Make sure the
remote user is able to run "btrfs" with root privileges (see remote user is able to run "btrfs" with root privileges (see
option 'backend' for details). If set to ``no'', the ssh default option 'backend' for details).
is used.
*ssh_compression* yes|no:: *ssh_compression* yes|no::
Enables or disables the compression of ssh connections. Defaults Enables or disables the compression of ssh connections. Defaults
@ -264,7 +261,13 @@ set to ``all'' (the default).
Selects the cipher specification for encrypting the session Selects the cipher specification for encrypting the session
(comma-separated list of ciphers in order of preference). See the (comma-separated list of ciphers in order of preference). See the
"-c cipher_spec" option in ssh(1) for more information. Defaults "-c cipher_spec" option in ssh(1) for more information. Defaults
to ``default'' (the ciphers specified in ssh_config(5)). to ``default'' (the ciphers specified in 'ssh_config').
Previous versions btrbk allowed you to set a *ssh_port* option, this
has been dropped in favor of the `ssh://hostname:port` notation in the
'volume' and 'target' <<_sections,sections>>. If you want to set a
global port for all SSH connections to remote hosts, set the ``Port''
option in ssh_config(5).
=== Data Stream Options === Data Stream Options
@ -274,7 +277,7 @@ set to ``all'' (the default).
remote locations. Defaults to ``no''. If enabled, make sure that remote locations. Defaults to ``no''. If enabled, make sure that
'<compress_command>' is available on the source and target '<compress_command>' is available on the source and target
hosts. Supported '<compress_command>': gzip, pigz, bzip2, pbzip2, hosts. Supported '<compress_command>': gzip, pigz, bzip2, pbzip2,
bzip3, xz, lzo, lz4, zstd. xz, lzo, lz4, zstd.
*stream_compress_level* default|<number>:: *stream_compress_level* default|<number>::
Compression level for the specified '<compress_command>'. Refer to Compression level for the specified '<compress_command>'. Refer to
@ -289,11 +292,11 @@ set to ``all'' (the default).
*stream_compress_threads* default|<number>:: *stream_compress_threads* default|<number>::
Number of threads to use for <compress_command>. Only supported Number of threads to use for <compress_command>. Only supported
for "pigz", "pbzip2", "bzip3", "zstd" and recent versions of "xz". for "pigz", "pbzip2", "zstd" and recent versions of "xz".
*stream_compress_adapt* yes|no:: *stream_compress_adapt* default|<number>::
Enable adaptive compression for <compress_command>. Only supported Enable adaptive compression for <compress_command>. Only supported
for "zstd" (version >= 1.3.6). Defaults to ``no''. for "zstd" (version >= 1.3.6).
*stream_buffer* <size>|no:: *stream_buffer* <size>|no::
Add a buffer to the btrfs send stream (locally, on uncompressed Add a buffer to the btrfs send stream (locally, on uncompressed
@ -359,10 +362,10 @@ constraints.
daemon, auth, lpr, news, cron, authpriv, local0..local7. daemon, auth, lpr, news, cron, authpriv, local0..local7.
*lockfile* <file>|no:: *lockfile* <file>|no::
Place an exclusive lock on <file> during program execution, using Create lockfile <file> on startup; checks lockfile before running
flock(2). If the lock is held by another process, exit before any btrfs commands (using perl "flock"), and exits if the lock is
running any actions. Ignored on dryrun ('-n', '--dry-run'). See held by another btrbk instance. Ignored on dryrun ('-n',
also '--lockfile' command-line option. '--dry-run'). See also '--lockfile' command-line option.
*backend* <backend>:: *backend* <backend>::
Backend filesystem utilities to be used for btrfs specific Backend filesystem utilities to be used for btrfs specific
@ -427,6 +430,12 @@ If you want to set this option for local or remote hosts only, set
=== Btrfs Specific Options === Btrfs Specific Options
*btrfs_commit_delete* after|each|no::
If set, make sure the deletion of snapshot and backup subvolumes
are committed to disk when btrbk terminates (sets '--commit-after'
or '--commit-each' options when running "btrfs subvolume
delete"). Defaults to ``no''.
*incremental_prefs* <list-spec>[:<amount>]...:: *incremental_prefs* <list-spec>[:<amount>]...::
Specify the preferences to determine the best common (correlated) Specify the preferences to determine the best common (correlated)
parent and clone sources for incremental backups, by choosing from parent and clone sources for incremental backups, by choosing from
@ -479,23 +488,6 @@ sources for all (!) known candidates on the filesystem.
access problems (when not running btrbk as root). Defaults to access problems (when not running btrbk as root). Defaults to
``mountpoint''. ``mountpoint''.
*btrfs_commit_delete* yes|no::
If set, wait for the transaction commit at the end of each
snapshot or backup deletion (sets '--commit-each' option for
"btrfs subvolume delete"). Defaults to ``no''.
*send_protocol* <number>|no _*experimental*_::
Use btrfs send protocol version N. If enabled on 'target', btrbk
adds "--proto <number>" to the btrfs-send(8) command. Defaults to
``no'' (btrfs default).
*send_compressed_data* yes|no _*experimental*_::
Send data that is compressed on the filesystem directly without
decompressing it. This requires protocol version 2 or higher
(btrfs-progs >= 5.19), and implies "send_protocol 2". If enabled
on 'target', btrbk adds "--compressed-data" to the btrfs-send(8)
command. Defaults to ``no'' (btrfs default).
*snapshot_qgroup_destroy* yes|no _*experimental*_:: {blank} *snapshot_qgroup_destroy* yes|no _*experimental*_:: {blank}
*target_qgroup_destroy* yes|no _*experimental*_:: {blank} *target_qgroup_destroy* yes|no _*experimental*_:: {blank}
*archive_qgroup_destroy* yes|no _*experimental*_:: *archive_qgroup_destroy* yes|no _*experimental*_::
@ -605,11 +597,15 @@ TARGET TYPES
btrfs-send(8), with optional compression and encryption. btrfs-send(8), with optional compression and encryption.
+ +
-- --
Note that the target preserve mechanism is currently disabled for
incremental raw backups (btrbk does not delete any incremental raw
files)!
Raw backups consist of two files: the main data file containing the Raw backups consist of two files: the main data file containing the
btrfs send stream, and a sidecar file ".info" containing metadata: btrfs send stream, and a sidecar file ".info" containing metadata:
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg] <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg]
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg].info <snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg].info
For 'incremental' backups ("incremental yes"), please note that: For 'incremental' backups ("incremental yes"), please note that:
@ -620,11 +616,6 @@ For 'incremental' backups ("incremental yes"), please note that:
make sure that a non-incremental backup is triggered from time to make sure that a non-incremental backup is triggered from time to
time. time.
* The scheduler will never delete dependent parents of backups
preserved by the retention policy (run btrbk with the '-S',
'--print-schedule' option to get a comprehensive output of the
scheduler results).
* There is currently no support for rotation of incremental backups: * There is currently no support for rotation of incremental backups:
if 'incremental' is set, a full backup must be triggered manually if 'incremental' is set, a full backup must be triggered manually
from time to time in order to be able to delete old backups. from time to time in order to be able to delete old backups.
@ -634,8 +625,8 @@ Additional options for raw targets:
*raw_target_compress* <compress_command>|no:: *raw_target_compress* <compress_command>|no::
Compression algorithm to use for raw backup target. Supported Compression algorithm to use for raw backup target. Supported
'<compress_command>': gzip, pigz, bzip2, pbzip2, bzip3, xz, lzo, '<compress_command>': gzip, pigz, bzip2, pbzip2, xz, lzo, lz4,
lz4, zstd. zstd.
*raw_target_compress_level* default|<number>:: *raw_target_compress_level* default|<number>::
Compression level for the specified <compress_command>. Compression level for the specified <compress_command>.
*raw_target_compress_long* default|<number>:: *raw_target_compress_long* default|<number>::

View File

@ -9,18 +9,16 @@ file, choose one of the following methods:
### Generic Linux System ### Generic Linux System
Install [asciidoctor] or [asciidoc] if you want to build the
documentation.
Download and unpack the latest [btrbk source tarball] and type: Download and unpack the latest [btrbk source tarball] and type:
sudo make install sudo make install
#### Try latest master from Github:
wget https://raw.githubusercontent.com/digint/btrbk/master/btrbk ### Gentoo Linux
chmod +x btrbk
sudo ./btrbk ls / btrbk is in portage:
emerge app-backup/btrbk
### Debian Based Distros ### Debian Based Distros
@ -32,7 +30,7 @@ Packages are also available via NeuroDebian: http://neuro.debian.net/pkgs/btrbk.
### Fedora Linux ### Fedora Linux
btrbk is in the official Fedora repos: https://src.fedoraproject.org/rpms/btrbk btrbk is in the official Fedora repos: https://apps.fedoraproject.org/packages/btrbk
sudo dnf install btrbk sudo dnf install btrbk
@ -49,13 +47,6 @@ btrbk is in the community repository
apk add btrbk apk add btrbk
### Gentoo Linux
btrbk is in portage:
emerge app-backup/btrbk
### Void Linux ### Void Linux
btrbk is in Void's `current` repository btrbk is in Void's `current` repository
@ -64,5 +55,3 @@ btrbk is in Void's `current` repository
[btrbk source tarball]: https://digint.ch/download/btrbk/releases/ [btrbk source tarball]: https://digint.ch/download/btrbk/releases/
[asciidoctor]: https://asciidoctor.org
[asciidoc]: https://asciidoc.org

View File

@ -1,7 +1,7 @@
lsbtr(1) lsbtr(1)
======== ========
:date: 2023-03-25 :date: 2022-06-25
:release-version: 0.32.6 :release-version: 0.32.2
:man manual: Btrbk Manual :man manual: Btrbk Manual
:man source: Btrbk {release-version} :man source: Btrbk {release-version}
@ -67,7 +67,7 @@ OPTIONS
space-separated key="value" pairs (machine readable). space-separated key="value" pairs (machine readable).
+ +
If set to "col:", prints only the <columns> specified (comma-separated If set to "col:", prints only the <columns> specified (comma-separated
list). Header lines are omitted if the "h:" modifier is present. list). Header lines are ommitted if the "h:" modifier is present.
Columns prefixed with "-" are collapsed if empty. Columns postfixed Columns prefixed with "-" are collapsed if empty. Columns postfixed
with ":RALIGN" are right-aligned. with ":RALIGN" are right-aligned.

View File

@ -1,7 +1,7 @@
ssh_filter_btrbk(1) ssh_filter_btrbk(1)
=================== ===================
:date: 2023-03-25 :date: 2022-06-25
:release-version: 0.32.6 :release-version: 0.32.2
:man manual: Btrbk Manual :man manual: Btrbk Manual
:man source: Btrbk {release-version} :man source: Btrbk {release-version}
@ -39,13 +39,13 @@ The following commands are always allowed:
- "readlink" - "readlink"
- "test -d" (only if "compat busybox" configuration option is set) - "test -d" (only if "compat busybox" configuration option is set)
- "cat /proc/self/mountinfo" - "cat /proc/self/mountinfo"
- pipes through "gzip", "pigz", "bzip2", "pbzip2", "bzip3", "xz", - pipes through "gzip", "pigz", "bzip2", "pbzip2", "xz", "lzop",
"lzop", "lz4", "zstd" (stream_compress) "lz4", "zstd" (stream_compress)
- pipes through "mbuffer" (stream_buffer, rate_limit) - pipes through "mbuffer" (stream_buffer, rate_limit)
Example line in /root/.ssh/authorized_keys on a backup target host: Example line in /root/.ssh/authorized_keys on a backup target host:
command="ssh_filter_btrbk.sh --target --delete --restrict-path /mnt/btr_backup",restrict ssh-rsa AAAAB3NzaC1...hwumXFRQBL btrbk@example.org command="ssh_filter_btrbk.sh --target --delete --restrict-path /mnt/btr_backup" ssh-rsa AAAAB3NzaC1...hwumXFRQBL btrbk@mydomain.com
OPTIONS OPTIONS
@ -82,11 +82,6 @@ OPTIONS
-p, --restrict-path <path>:: -p, --restrict-path <path>::
Restrict commands to <path>. Note that "btrfs subvolume show", Restrict commands to <path>. Note that "btrfs subvolume show",
"btrfs subvolume list" are NOT affected by this option. "btrfs subvolume list" are NOT affected by this option.
+
It is not possible to restrict commands to exact subvolume names, as
btrfs-receive(8) takes a <path> as argument (directory, not including
the subvolume file name to be created, this is encoded in the
send-stream).
-l, --log:: -l, --log::
Log ACCEPT and REJECT messages to the system log. Log ACCEPT and REJECT messages to the system log.

92
doc/upgrade_to_v0.23.0.md Normal file
View File

@ -0,0 +1,92 @@
Upgrading to btrbk-v0.23.0
==========================
In order to keep btrbk simple and intuitive while adding new features,
it became inevitable to change the semantics of the "retention policy"
related configuration options.
What has changed?
-----------------
### Preserve *first* instead of *last* snapshot/backup
btrbk used to *always* transfer the latest snapshot to the target
location, while considering the *last* snapshot/backup of a day as a
daily backup (and also the last weekly as a monthly). This made it
very cumbersome when running btrbk in a cron job as well as manually,
because the last manually created snapshot was immediately transferred
on every run, and used as the daily backup (instead of the one created
periodically by the cron job).
The new semantics are to consider the *first* (instead of *last*)
snapshot of a hour/day/week/month as the one to be preserved, while
only transferring the snapshots needed to satisfy the target retention
policy.
### Preserve snapshots for a minimum amount of time
In order to specify a minimum amount of time in which *all* snapshots
should be preserved, the new "snapshot_preserve_min" and
"target_preserve_min" configuration options were introduced. This was
previously covered by "snapshot_preserve_daily", which caused a lot of
confusion among users.
Upgrading the configuration file: /etc/btrbk/btrbk.conf
-------------------------------------------------------
Please read the description of the "run" command in [btrbk(1)], as
well as the "RETENTION POLICY" section in [btrbk.conf(5)] for a
detailed description. Make sure to understand the new concept, and run
`btrbk --print-schedule dryrun` after updating the configuration.
### Upgrade retention policy
If you want the same behaviour as before:
# replace this:
snapshot_preserve_daily <daily>
snapshot_preserve_weekly <weekly>
snapshot_preserve_monthly <monthly>
# with:
snapshot_preserve_min <daily>d
snapshot_preserve <weekly>w <monthly>m
# ... do the same with "target_preserve_*" options
But what you probably want is something like:
snapshot_preserve_min 5d
snapshot_preserve <daily>d <weekly>w <monthly>m
target_preserve_min no
target_preserve <daily>d <weekly>w <monthly>m *y
This states:
* Keep all snapshots for five days (no matter how many there are)
* Transfer only the first snapshot of a day to the target
* Keep all "first snapshots of a day" for `<daily>` days, etc.
### Upgrade "resume_missing"
If you have a line: "resume_missing yes" somwhere in your config,
simply remove it. btrbk always resumes missing backups.
If you have "resume_missing no", you can imitate this behaviour by
setting:
target_preserve_min latest
target_preserve no
This states: "always transfer the latest snapshot to the target".
[btrbk(1)]: https://digint.ch/btrbk/doc/btrbk.1.html
[btrbk.conf(5)]: https://digint.ch/btrbk/doc/btrbk.conf.5.html

View File

@ -1,11 +1,9 @@
#!/bin/sh #!/bin/bash
# initialise and sanitise the shell execution environment set -e
unset -v IFS set -u
export LC_ALL=C
export PATH='/sbin:/bin:/usr/sbin:/usr/bin'
set -e -u export PATH=/sbin:/bin:/usr/sbin:/usr/bin
enable_log= enable_log=
restrict_path_list= restrict_path_list=
@ -14,57 +12,48 @@ allow_exact_list=
allow_rate_limit=1 allow_rate_limit=1
allow_stream_buffer=1 allow_stream_buffer=1
allow_compress=1 allow_compress=1
compress_list='gzip|pigz|bzip2|pbzip2|bzip3|xz|lzop|lz4|zstd' compress_list="gzip|pigz|bzip2|pbzip2|xz|lzop|lz4|zstd"
# note that the backslash is NOT a metacharacter in a POSIX bracket expression! # note that the backslash is NOT a metacharacter in a POSIX bracket expression!
option_match='-[a-zA-Z0-9=-]+' # matches short as well as long options option_match='-[a-zA-Z0-9=-]+' # matches short as well as long options
file_match_sane='/[0-9a-zA-Z_@+./-]*' # matches file path (equal to ${file_match} in btrbk < 0.32.0) file_match_sane='/[0-9a-zA-Z_@+./-]*' # matches file path (equal to $file_match in btrbk < 0.32.0)
file_match="/[^']*" # btrbk >= 0.32.0 quotes file arguments: match all but single quote file_match="/[^']*" # btrbk >= 0.32.0 quotes file arguments: match all but single quote
file_arg_match="('${file_match}'|${file_match_sane})" # support btrbk < 0.32.0 file_arg_match="('${file_match}'|${file_match_sane})" # support btrbk < 0.32.0
log_cmd() log_cmd()
{ {
local priority="$1" if [[ -n "$enable_log" ]]; then
local authorisation_decision="$2" logger -p $1 -t ssh_filter_btrbk.sh "$2 (Name: ${LOGNAME:-<unknown>}; Remote: ${SSH_CLIENT:-<unknown>})${3:+: $3}: $SSH_ORIGINAL_COMMAND"
local reason="${3-}"
if [ -n "${enable_log}" ]; then
logger -p "${priority}" -t ssh_filter_btrbk.sh "${authorisation_decision} (Name: ${LOGNAME:-<unknown>}; Connection: ${SSH_CONNECTION:-<unknown>})${reason:+: ${reason}}: ${SSH_ORIGINAL_COMMAND}"
fi fi
} }
allow_cmd() allow_cmd()
{ {
local cmd="$1" allow_list="${allow_list}|$1"
allow_list="${allow_list}|${cmd}"
} }
allow_exact_cmd() allow_exact_cmd()
{ {
local cmd="$1" allow_exact_list="${allow_exact_list}|$1"
allow_exact_list="${allow_exact_list}|${cmd}"
} }
reject_and_die() reject_and_die()
{ {
local reason="$1" local reason=$1
log_cmd "auth.err" "btrbk REJECT" "$reason"
log_cmd 'auth.err' 'btrbk REJECT' "${reason}" echo "ERROR: ssh_filter_btrbk.sh: ssh command rejected: $reason: $SSH_ORIGINAL_COMMAND" 1>&2
printf 'ERROR: ssh_filter_btrbk.sh: ssh command rejected: %s: %s\n' "${reason}" "${SSH_ORIGINAL_COMMAND}" >&2
exit 255 exit 255
} }
run_cmd() run_cmd()
{ {
log_cmd 'auth.info' 'btrbk ACCEPT' log_cmd "auth.info" "btrbk ACCEPT"
eval " ${SSH_ORIGINAL_COMMAND}" eval " $SSH_ORIGINAL_COMMAND"
} }
reject_filtered_cmd() reject_filtered_cmd()
{ {
if [ -n "${restrict_path_list}" ]; then if [[ -n "$restrict_path_list" ]]; then
# match any of restrict_path_list, # match any of restrict_path_list,
# or any file/directory (matching file_match) below restrict_path # or any file/directory (matching file_match) below restrict_path
path_match="'(${restrict_path_list})(${file_match})?'" path_match="'(${restrict_path_list})(${file_match})?'"
@ -77,7 +66,7 @@ reject_filtered_cmd()
# btrbk >= 0.32.0 quotes files, allow both (legacy) # btrbk >= 0.32.0 quotes files, allow both (legacy)
path_match="(${path_match}|${path_match_legacy})" path_match="(${path_match}|${path_match_legacy})"
if [ -n "${allow_compress}" ]; then if [[ -n "$allow_compress" ]]; then
decompress_match="(${compress_list}) -d -c( -[pT][0-9]+)?" decompress_match="(${compress_list}) -d -c( -[pT][0-9]+)?"
compress_match="(${compress_list}) -c( -[0-9])?( -[pT][0-9]+)?" compress_match="(${compress_list}) -c( -[0-9])?( -[pT][0-9]+)?"
else else
@ -87,8 +76,8 @@ reject_filtered_cmd()
# rate_limit_remote and stream_buffer_remote use combined # rate_limit_remote and stream_buffer_remote use combined
# "mbuffer" as of btrbk-0.29.0 # "mbuffer" as of btrbk-0.29.0
if [ -n "${allow_stream_buffer}" ] || [ -n "${allow_rate_limit}" ]; then if [[ -n "$allow_stream_buffer" ]] || [[ -n "$allow_rate_limit" ]]; then
mbuffer_match='mbuffer -v 1 -q( -s [0-9]+[kmgKMG]?)?( -m [0-9]+[kmgKMG]?)?( -[rR] [0-9]+[kmgtKMGT]?)?' mbuffer_match="mbuffer -v 1 -q( -s [0-9]+[kmgKMG]?)?( -m [0-9]+[kmgKMG]?)?( -[rR] [0-9]+[kmgtKMGT]?)?"
else else
mbuffer_match= mbuffer_match=
fi fi
@ -98,39 +87,31 @@ reject_filtered_cmd()
stream_in_match="(${decompress_match} \| )?(${mbuffer_match} \| )?" stream_in_match="(${decompress_match} \| )?(${mbuffer_match} \| )?"
stream_out_match="( \| ${mbuffer_match})?( \| ${compress_match}$)?" stream_out_match="( \| ${mbuffer_match})?( \| ${compress_match}$)?"
# `grep`s `-q`-option is not used as it may cause an exit status of `0` even
# when an error occurred.
allow_stream_match="^${stream_in_match}${allow_cmd_match}${stream_out_match}" allow_stream_match="^${stream_in_match}${allow_cmd_match}${stream_out_match}"
if printf '%s' "${SSH_ORIGINAL_COMMAND}" | grep -E "${allow_stream_match}" >/dev/null 2>/dev/null; then if [[ $SSH_ORIGINAL_COMMAND =~ $allow_stream_match ]] ; then
return 0 return 0
fi fi
exact_cmd_match="^(${allow_exact_list})$"; exact_cmd_match="^(${allow_exact_list})$";
if printf '%s' "${SSH_ORIGINAL_COMMAND}" | grep -E "${exact_cmd_match}" >/dev/null 2>/dev/null; then if [[ $SSH_ORIGINAL_COMMAND =~ $exact_cmd_match ]] ; then
return 0 return 0
fi fi
local formatted_restrict_path_list="$(printf '%s' "${restrict_path_list}" | sed 's/|/", "/g')" reject_and_die "disallowed command${restrict_path_list:+ (restrict-path: \"${restrict_path_list//|/\", \"}\")}"
reject_and_die "disallowed command${restrict_path_list:+ (restrict-path: \"${formatted_restrict_path_list}\")}"
} }
# check for "--sudo" option before processing other options # check for "--sudo" option before processing other options
sudo_prefix= sudo_prefix=
for key in "$@"; do for key; do
if [ "${key}" = '--sudo' ]; then [[ "$key" == "--sudo" ]] && sudo_prefix="sudo -n "
sudo_prefix='sudo -n ' [[ "$key" == "--doas" ]] && sudo_prefix="doas -n "
fi
if [ "${key}" = '--doas' ]; then
sudo_prefix='doas -n '
fi
done done
while [ "$#" -ge 1 ]; do while [[ "$#" -ge 1 ]]; do
key="$1" key="$1"
case "${key}" in case $key in
-l|--log) -l|--log)
enable_log=1 enable_log=1
;; ;;
@ -180,7 +161,7 @@ while [ "$#" -ge 1 ]; do
;; ;;
*) *)
printf 'ERROR: ssh_filter_btrbk.sh: failed to parse command line option: %s\n' "${key}" >&2 echo "ERROR: ssh_filter_btrbk.sh: failed to parse command line option: $key" 1>&2
exit 255 exit 255
;; ;;
esac esac
@ -192,18 +173,16 @@ done
allow_exact_cmd "${sudo_prefix}btrfs subvolume (show|list)( ${option_match})* ${file_arg_match}"; allow_exact_cmd "${sudo_prefix}btrfs subvolume (show|list)( ${option_match})* ${file_arg_match}";
allow_cmd "${sudo_prefix}readlink" # resolve symlink allow_cmd "${sudo_prefix}readlink" # resolve symlink
allow_exact_cmd "${sudo_prefix}test -d ${file_arg_match}" # check directory (only for compat=busybox) allow_exact_cmd "${sudo_prefix}test -d ${file_arg_match}" # check directory (only for compat=busybox)
allow_exact_cmd 'cat /proc/self/mountinfo' # resolve mountpoints allow_exact_cmd "cat /proc/self/mountinfo" # resolve mountpoints
allow_exact_cmd 'cat /proc/self/mounts' # legacy, for btrbk < 0.27.0 allow_exact_cmd "cat /proc/self/mounts" # legacy, for btrbk < 0.27.0
# remove leading "|" on alternation lists # remove leading "|" on alternation lists
allow_list="${allow_list#\|}" allow_list=${allow_list#\|}
allow_exact_list="${allow_exact_list#\|}" allow_exact_list=${allow_exact_list#\|}
restrict_path_list="${restrict_path_list#\|}" restrict_path_list=${restrict_path_list#\|}
case "${SSH_ORIGINAL_COMMAND}" in case "$SSH_ORIGINAL_COMMAND" in
*\.\./*) reject_and_die 'directory traversal' ;; *\.\./*) reject_and_die 'directory traversal' ;;
*'
'*) reject_and_die 'unsafe character LF' ;;
*\$*) reject_and_die 'unsafe character "$"' ;; *\$*) reject_and_die 'unsafe character "$"' ;;
*\&*) reject_and_die 'unsafe character "&"' ;; *\&*) reject_and_die 'unsafe character "&"' ;;
*\(*) reject_and_die 'unsafe character "("' ;; *\(*) reject_and_die 'unsafe character "("' ;;
@ -212,7 +191,7 @@ case "${SSH_ORIGINAL_COMMAND}" in
*\<*) reject_and_die 'unsafe character "<"' ;; *\<*) reject_and_die 'unsafe character "<"' ;;
*\>*) reject_and_die 'unsafe character ">"' ;; *\>*) reject_and_die 'unsafe character ">"' ;;
*\`*) reject_and_die 'unsafe character "`"' ;; *\`*) reject_and_die 'unsafe character "`"' ;;
*\|*) [ -n "${allow_compress}" ] || [ -n "${allow_rate_limit}" ] || [ -n "${allow_stream_buffer}" ] || reject_and_die 'unsafe character "|"' ;; *\|*) [[ -n "$allow_compress" ]] || [[ -n "$allow_rate_limit" ]] || [[ -n "$allow_stream_buffer" ]] || reject_and_die 'unsafe character "|"' ;;
esac esac
reject_filtered_cmd reject_filtered_cmd