mirror of https://github.com/digint/btrbk
Compare commits
No commits in common. "master" and "v0.31.2" have entirely different histories.
|
@ -1,3 +0,0 @@
|
||||||
# These are supported funding model platforms
|
|
||||||
|
|
||||||
custom: ['https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=WFQSSCD9GNM4S', 'https://btc.com/19DYtoEepxBmn9ZPspJGZrhCtySKCxPcP1']
|
|
|
@ -1,26 +0,0 @@
|
||||||
---
|
|
||||||
name: Codespell
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [master]
|
|
||||||
pull_request:
|
|
||||||
branches: [master]
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
codespell:
|
|
||||||
name: Check for spelling errors
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout
|
|
||||||
uses: actions/checkout@v3
|
|
||||||
- name: Codespell
|
|
||||||
uses: codespell-project/actions-codespell@v2
|
|
||||||
with:
|
|
||||||
check_filenames: true
|
|
||||||
skip: ".git,*.pdf,*.svg"
|
|
||||||
ignore_words_list: uptodate
|
|
84
ChangeLog
84
ChangeLog
|
@ -1,87 +1,9 @@
|
||||||
btrbk-0.32.6
|
|
||||||
|
|
||||||
* Fix backup of unrelated (by parent_uuid) snapshots (close #339).
|
|
||||||
* Remove echo -e for portability (close #506).
|
|
||||||
* Support btrfs send protocol v2 (send_protocol and
|
|
||||||
send_compressed_data config options).
|
|
||||||
* Add bzip3 support.
|
|
||||||
* Convert ssh_filter_btrbk.sh to POSIX sh, and harden it.
|
|
||||||
* Slight change in ssh_filter_btrbk.sh logging output.
|
|
||||||
* Minor bugfixes, stability and documentation improvements.
|
|
||||||
|
|
||||||
btrbk-0.32.5
|
|
||||||
|
|
||||||
* Correct handling of zero-size raw info file (close #491).
|
|
||||||
|
|
||||||
btrbk-0.32.4
|
|
||||||
|
|
||||||
* Fix regression: wrong deprecation warnings in some cases.
|
|
||||||
|
|
||||||
btrbk-0.32.3
|
|
||||||
|
|
||||||
* Fix deletion of many subvolumes at once (close #476).
|
|
||||||
* Allow disabling ssh_identity and ssh_user options.
|
|
||||||
* Minor bugfixes and documentation improvements.
|
|
||||||
|
|
||||||
btrbk-0.32.2
|
|
||||||
|
|
||||||
* Fix regression: archive on missing target directories.
|
|
||||||
* Fix action "config print".
|
|
||||||
* Print version and help message to stdout (close #452).
|
|
||||||
* Check results of filesystem usage (close #469).
|
|
||||||
* Add "compat ignore_receive_errors" option (close #383).
|
|
||||||
* Support multiple gpg recipients (close #471).
|
|
||||||
* Fix changelog: remove "compat missing_otime" item in v0.32.1.
|
|
||||||
* Fix changelog: correct "warn_unknown_targets" item in v0.31.3.
|
|
||||||
* Minor bugfixes and framework improvements.
|
|
||||||
|
|
||||||
btrbk-0.32.1
|
|
||||||
|
|
||||||
* Fix regression: correctly read mountinfo (close #445).
|
|
||||||
* Fix regression: filter paths for "lsbtr" (action "ls").
|
|
||||||
* Add backend btrfs-progs-doas (close #444).
|
|
||||||
* Allow directory traversal for local command line arguments.
|
|
||||||
* Display all source subvolumes in "usage" action.
|
|
||||||
* Handle errors from btrfs filesystem usage.
|
|
||||||
|
|
||||||
btrbk-0.32.0
|
|
||||||
|
|
||||||
* MIGRATION
|
|
||||||
- If timestamp_format is not configured, explicitly set
|
|
||||||
"timestamp_format short" to revert old behavior.
|
|
||||||
- Update ssh_filter_btrbk.sh on remote hosts.
|
|
||||||
* Change default for timestamp_format to "long".
|
|
||||||
* Optional "volume" config section.
|
|
||||||
* Use "volume" section only for grouping and relative paths.
|
|
||||||
* Allow absolute path for "subvolume" and "snapshot_dir"
|
|
||||||
(close #407, #121).
|
|
||||||
* Support subvolume names with UTF-8 characters (close #392, #213,
|
|
||||||
#120). Add "safe_commands" option for paranoid people.
|
|
||||||
* Add "incremental_prefs" configuration option (close #387).
|
|
||||||
* Change incremental prefs policy, giving snapshots created by btrbk
|
|
||||||
higher preference than the global ones resolved by parent-uuid.
|
|
||||||
* Change "incremental_clones" option to boolean.
|
|
||||||
* Drop support of deprecated raw file format (btrbk < 0.26.0).
|
|
||||||
* Drop support of deprecated "-r, --resume-only" cmdline option.
|
|
||||||
* Fix ambiguous naming in --format=raw output of "list" actions.
|
|
||||||
* Accept quoted values in config.
|
|
||||||
* Use single quotes for raw table output.
|
|
||||||
|
|
||||||
btrbk-0.31.3
|
|
||||||
|
|
||||||
* Add "warn_unknown_targets" configuration option (close #393).
|
|
||||||
* Add -1,--single-column command-line option for listing actions.
|
|
||||||
* Allow relative path command line argument for all actions.
|
|
||||||
* Add support for zstd adaptive compression (close #397).
|
|
||||||
* Add contrib/tools/btrbk_restore_raw.py (close #401).
|
|
||||||
* Minor bugfixes and framework improvements.
|
|
||||||
|
|
||||||
btrbk-0.31.2
|
btrbk-0.31.2
|
||||||
|
|
||||||
* MIGRATION
|
* MIGRATION
|
||||||
- Update ssh_filter_btrbk.sh on remote hosts.
|
- Update ssh_filter_btrbk.sh on remote hosts.
|
||||||
* ssh_filter_btrbk.sh: Fix security vulnerability.
|
* ssh_filter_btrbk.sh: Fix security vulnerability.
|
||||||
Specially crafted commands may be executed without being properly
|
Specialy crafted commands may be executed without being propely
|
||||||
checked. Applies to remote hosts filtering ssh commands using
|
checked. Applies to remote hosts filtering ssh commands using
|
||||||
ssh_filter_btrbk.sh in authorized_keys.
|
ssh_filter_btrbk.sh in authorized_keys.
|
||||||
* Warn if no subvolume defined in config (close #378).
|
* Warn if no subvolume defined in config (close #378).
|
||||||
|
@ -248,7 +170,7 @@ btrbk-0.27.0
|
||||||
- Allow snapshot_dir to be a mountpoint.
|
- Allow snapshot_dir to be a mountpoint.
|
||||||
- Search complete target tree for correlated subvolumes.
|
- Search complete target tree for correlated subvolumes.
|
||||||
- Include snapshots from all mountpoints as candidates (disabled
|
- Include snapshots from all mountpoints as candidates (disabled
|
||||||
due to upstream bug: github.com/kdave/btrfs-progs/issues/96).
|
due to uptream bug: github.com/kdave/btrfs-progs/issues/96).
|
||||||
- Read /proc/self/mountinfo instead of /proc/self/mounts.
|
- Read /proc/self/mountinfo instead of /proc/self/mounts.
|
||||||
- Always read /proc/self/mountinfo.
|
- Always read /proc/self/mountinfo.
|
||||||
- Resolve realpath using readlink(1).
|
- Resolve realpath using readlink(1).
|
||||||
|
@ -319,7 +241,7 @@ btrbk-0.25.0
|
||||||
* Allow trailing comments in btrbk.conf (close #129).
|
* Allow trailing comments in btrbk.conf (close #129).
|
||||||
* Bugfix: rate limiting must be done after compression (close #134).
|
* Bugfix: rate limiting must be done after compression (close #134).
|
||||||
* raw_target_encrypt: Always set "gpg --no-random-seed-file":
|
* raw_target_encrypt: Always set "gpg --no-random-seed-file":
|
||||||
prevents creation of "~/.gnupg/random_seed" with slight performance
|
prevents creation of "~/.gnupg/random_seed" with slight perfomance
|
||||||
penalty.
|
penalty.
|
||||||
|
|
||||||
btrbk-0.24.0
|
btrbk-0.24.0
|
||||||
|
|
3
Makefile
3
Makefile
|
@ -17,8 +17,7 @@ SCRIPTS = ssh_filter_btrbk.sh \
|
||||||
contrib/cron/btrbk-mail \
|
contrib/cron/btrbk-mail \
|
||||||
contrib/cron/btrbk-verify \
|
contrib/cron/btrbk-verify \
|
||||||
contrib/migration/raw_suffix2sidecar \
|
contrib/migration/raw_suffix2sidecar \
|
||||||
contrib/crypt/kdf_pbkdf2.py \
|
contrib/crypt/kdf_pbkdf2.py
|
||||||
contrib/tools/btrbk_restore_raw.py
|
|
||||||
|
|
||||||
PN = btrbk
|
PN = btrbk
|
||||||
PREFIX ?= /usr
|
PREFIX ?= /usr
|
||||||
|
|
114
README.md
114
README.md
|
@ -9,7 +9,7 @@ The source and target locations are specified in a config file, which
|
||||||
allows to easily configure simple scenarios like "laptop with locally
|
allows to easily configure simple scenarios like "laptop with locally
|
||||||
attached backup disks", as well as more complex ones, e.g. "server
|
attached backup disks", as well as more complex ones, e.g. "server
|
||||||
receiving backups from several hosts via ssh, with different retention
|
receiving backups from several hosts via ssh, with different retention
|
||||||
policies".
|
policy".
|
||||||
|
|
||||||
Key Features:
|
Key Features:
|
||||||
|
|
||||||
|
@ -18,8 +18,7 @@ Key Features:
|
||||||
* Flexible retention policy
|
* Flexible retention policy
|
||||||
* Backups to multiple destinations
|
* Backups to multiple destinations
|
||||||
* Transfer via ssh
|
* Transfer via ssh
|
||||||
* Robust recovery from interrupted backups (for removable and mobile
|
* Resume backups (for removable and mobile devices)
|
||||||
devices)
|
|
||||||
* Archive to offline storage
|
* Archive to offline storage
|
||||||
* Encrypted backups to non-btrfs storage
|
* Encrypted backups to non-btrfs storage
|
||||||
* Wildcard subvolumes (useful for docker and lxc containers)
|
* Wildcard subvolumes (useful for docker and lxc containers)
|
||||||
|
@ -60,7 +59,7 @@ For more information, read the [installation documentation].
|
||||||
|
|
||||||
[btrfs-progs]: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
|
[btrfs-progs]: https://www.kernel.org/pub/linux/kernel/people/kdave/btrfs-progs/
|
||||||
[Perl interpreter]: https://www.perl.org
|
[Perl interpreter]: https://www.perl.org
|
||||||
[OpenSSH]: https://www.openssh.com
|
[OpenSSH]: https://www.openssh.org
|
||||||
[mbuffer]: https://www.maier-komor.de/mbuffer.html
|
[mbuffer]: https://www.maier-komor.de/mbuffer.html
|
||||||
|
|
||||||
|
|
||||||
|
@ -73,8 +72,8 @@ a full description of the command line options.
|
||||||
[btrbk(1)]: https://digint.ch/btrbk/doc/btrbk.1.html
|
[btrbk(1)]: https://digint.ch/btrbk/doc/btrbk.1.html
|
||||||
|
|
||||||
|
|
||||||
Configuration
|
Configuration File
|
||||||
=============
|
==================
|
||||||
|
|
||||||
Before running `btrbk`, you will need to create a configuration
|
Before running `btrbk`, you will need to create a configuration
|
||||||
file. You might want to take a look at `btrbk.conf.example` provided
|
file. You might want to take a look at `btrbk.conf.example` provided
|
||||||
|
@ -82,37 +81,21 @@ with this package. For a detailed description, please consult the
|
||||||
[btrbk.conf(5)] man-page.
|
[btrbk.conf(5)] man-page.
|
||||||
|
|
||||||
After a configuration change, it is highly recommended to check it by
|
After a configuration change, it is highly recommended to check it by
|
||||||
running btrbk with the `-n,--dry-run` option:
|
running btrbk with the `-n,--dryrun` option:
|
||||||
|
|
||||||
# btrbk -c /path/to/myconfig -v -n run
|
btrbk -c /path/to/myconfig -v -n run
|
||||||
|
|
||||||
This will read all btrfs information on the source/target filesystems
|
This will read all btrfs information on the source/target filesystems
|
||||||
and show what actions would be performed (without writing anything to
|
and show what actions would be performed (without writing anything to
|
||||||
the disks).
|
the disks).
|
||||||
|
|
||||||
The examples below assume that the btrfs subvolume containing `home`
|
|
||||||
and `rootfs` is mounted at `/mnt/btr_pool`. This is usually the btrfs
|
|
||||||
root subvolume, which always has `subvolid=5`.
|
|
||||||
|
|
||||||
Mounting `subvolid=5` is *recommended* (mandatory for btrbk < v0.32.0)
|
|
||||||
if you want to backup your root filesystem `/`.
|
|
||||||
|
|
||||||
/etc/fstab:
|
|
||||||
|
|
||||||
/dev/sda1 /mnt/btr_pool btrfs subvolid=5,noatime 0 0
|
|
||||||
|
|
||||||
Note that some default btrfs installations (e.g. Ubuntu) use subvolume
|
|
||||||
names `@` for rootfs (mounted at `/`) and `@home` for `/home`, as a
|
|
||||||
naming convention. If this is the case on your file system, replace
|
|
||||||
the `subvolume` declarations in the examples accordingly.
|
|
||||||
|
|
||||||
[btrbk.conf(5)]: https://digint.ch/btrbk/doc/btrbk.conf.5.html
|
[btrbk.conf(5)]: https://digint.ch/btrbk/doc/btrbk.conf.5.html
|
||||||
|
|
||||||
|
|
||||||
Example: Local Regular Snapshots (time-machine)
|
Example: Local Regular Snapshots (time-machine)
|
||||||
-----------------------------------------------
|
-----------------------------------------------
|
||||||
|
|
||||||
The simplest use case is to only create snapshots of your data. This
|
The simpliest use case is to only create snapshots of your data. This
|
||||||
will obviously not protect it against hardware failure, but can be
|
will obviously not protect it against hardware failure, but can be
|
||||||
useful for:
|
useful for:
|
||||||
|
|
||||||
|
@ -138,32 +121,15 @@ manage snapshots located on the same volume in `snapshot_dir`. Btrbk
|
||||||
does not create subdirs by default, the snapshot directory must first
|
does not create subdirs by default, the snapshot directory must first
|
||||||
be created manually:
|
be created manually:
|
||||||
|
|
||||||
# mkdir /mnt/btr_pool/btrbk_snapshots
|
sudo mkdir /mnt/btr_pool/btrbk_snapshots
|
||||||
|
|
||||||
The "volume" section is merely used as a specifier for a base
|
Start a dry run:
|
||||||
directory, and can be skipped if you prefer to configure everything
|
|
||||||
using absolute paths. The above configuration can also be written as:
|
|
||||||
|
|
||||||
snapshot_dir /mnt/btr_pool/btrbk_snapshots
|
sudo btrbk run -n
|
||||||
subvolume /mnt/btr_pool/home
|
|
||||||
|
|
||||||
If you don't want to mount the btrfs root filesystem to
|
|
||||||
`/mnt/btr_pool`, you might as well configure it like this:
|
|
||||||
|
|
||||||
snapshot_dir /btrbk_snapshots
|
|
||||||
subvolume /home
|
|
||||||
|
|
||||||
Start a dry run (-n, --dry-run):
|
|
||||||
|
|
||||||
# btrbk run -n
|
|
||||||
|
|
||||||
Create the first snapshot:
|
Create the first snapshot:
|
||||||
|
|
||||||
# btrbk run
|
sudo btrbk run
|
||||||
|
|
||||||
Print schedule (-S, --print-schedule):
|
|
||||||
|
|
||||||
# btrbk run -n -S
|
|
||||||
|
|
||||||
If it works as expected, configure a cron job to run btrbk hourly:
|
If it works as expected, configure a cron job to run btrbk hourly:
|
||||||
|
|
||||||
|
@ -172,10 +138,13 @@ If it works as expected, configure a cron job to run btrbk hourly:
|
||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
exec /usr/bin/btrbk -q run
|
exec /usr/bin/btrbk -q run
|
||||||
|
|
||||||
Snapshots will now be created every hour. All snapshots are preserved for at
|
Snapshots will now be created every hour, kept for 48h
|
||||||
least 18 hours (`snapshot_preserve_min`), whether they are created by the cron
|
(`snapshot_preserve`), then automatically removed.
|
||||||
job or manually by calling `sudo btrbk run` on the command line. Additionally,
|
|
||||||
48 hourly snapshots are preserved (`snapshot_preserve`).
|
With this setup, the snapshots will be kept at least for 18 hours
|
||||||
|
(`snapshot_preserve_min`). This can be useful to create manual
|
||||||
|
snapshots by calling `sudo btrbk run` on the command line and keep
|
||||||
|
them around for a while, in addition to the regular snapshots.
|
||||||
|
|
||||||
|
|
||||||
Example: Backups to USB Disk
|
Example: Backups to USB Disk
|
||||||
|
@ -208,9 +177,6 @@ Retention policy:
|
||||||
snapshot_preserve_min 2d
|
snapshot_preserve_min 2d
|
||||||
snapshot_preserve 14d
|
snapshot_preserve 14d
|
||||||
|
|
||||||
# Create snapshots only if the backup disk is attached
|
|
||||||
#snapshot_create ondemand
|
|
||||||
|
|
||||||
target_preserve_min no
|
target_preserve_min no
|
||||||
target_preserve 20d 10w *m
|
target_preserve 20d 10w *m
|
||||||
|
|
||||||
|
@ -236,11 +202,10 @@ Retention policy:
|
||||||
* `/mnt/btr_backup/mylaptop/rootfs.YYYYMMDD`
|
* `/mnt/btr_backup/mylaptop/rootfs.YYYYMMDD`
|
||||||
* `/mnt/btr_backup/mylaptop/home.YYYYMMDD`
|
* `/mnt/btr_backup/mylaptop/home.YYYYMMDD`
|
||||||
|
|
||||||
If you prefer triggering the backups manually, change the cron command
|
If you want the snapshots to be created only if the backup disk is
|
||||||
to run the `snapshot` action instead of `run`. Start the backups
|
attached, simply add the following line to the config:
|
||||||
manually by running:
|
|
||||||
|
|
||||||
# btrbk resume
|
snapshot_create ondemand
|
||||||
|
|
||||||
For a quick additional snapshot of your home, run:
|
For a quick additional snapshot of your home, run:
|
||||||
|
|
||||||
|
@ -250,8 +215,8 @@ For a quick additional snapshot of your home, run:
|
||||||
Example: Host-initiated Backup on Fileserver
|
Example: Host-initiated Backup on Fileserver
|
||||||
--------------------------------------------
|
--------------------------------------------
|
||||||
|
|
||||||
Let's say you have a fileserver at "myserver.example.org" where you
|
Let's say you have a fileserver at "myserver.mydomain.com" where you
|
||||||
want to create backups of your laptop disk. The config could look like
|
want to create backups of your laptop disk, the config would look like
|
||||||
this:
|
this:
|
||||||
|
|
||||||
ssh_identity /etc/btrbk/ssh/id_rsa
|
ssh_identity /etc/btrbk/ssh/id_rsa
|
||||||
|
@ -259,11 +224,11 @@ this:
|
||||||
volume /mnt/btr_pool
|
volume /mnt/btr_pool
|
||||||
subvolume rootfs
|
subvolume rootfs
|
||||||
target /mnt/btr_backup/mylaptop
|
target /mnt/btr_backup/mylaptop
|
||||||
target ssh://myserver.example.org/mnt/btr_backup/mylaptop
|
target ssh://myserver.mydomain.com/mnt/btr_backup/mylaptop
|
||||||
|
|
||||||
In addition to the backups on your local usb-disk mounted at
|
In addition to the backups on your local usb-disk mounted at
|
||||||
`/mnt/btr_backup/mylaptop`, incremental backups would also be pushed
|
`/mnt/btr_backup/mylaptop`, incremental backups would also be pushed
|
||||||
to `myserver.example.org`.
|
to `myserver.mydomain.com`.
|
||||||
|
|
||||||
|
|
||||||
Example: Fileserver-initiated Backups from Several Hosts
|
Example: Fileserver-initiated Backups from Several Hosts
|
||||||
|
@ -274,17 +239,17 @@ fileserver, the config would be something like:
|
||||||
|
|
||||||
ssh_identity /etc/btrbk/ssh/id_rsa
|
ssh_identity /etc/btrbk/ssh/id_rsa
|
||||||
|
|
||||||
volume ssh://alpha.example.org/mnt/btr_pool
|
volume ssh://alpha.mydomain.com/mnt/btr_pool
|
||||||
target /mnt/btr_backup/alpha
|
target /mnt/btr_backup/alpha
|
||||||
subvolume rootfs
|
subvolume rootfs
|
||||||
subvolume home
|
subvolume home
|
||||||
|
|
||||||
volume ssh://beta.example.org/mnt/btr_pool
|
volume ssh://beta.mydomain.com/mnt/btr_pool
|
||||||
target /mnt/btr_backup/beta
|
target /mnt/btr_backup/beta
|
||||||
subvolume rootfs
|
subvolume rootfs
|
||||||
subvolume dbdata
|
subvolume dbdata
|
||||||
|
|
||||||
This will pull backups from alpha/beta.example.org and locally
|
This will pull backups from alpha/beta.mydomain.com and locally
|
||||||
create:
|
create:
|
||||||
|
|
||||||
* `/mnt/btr_backup/alpha/rootfs.YYYYMMDD`
|
* `/mnt/btr_backup/alpha/rootfs.YYYYMMDD`
|
||||||
|
@ -385,7 +350,7 @@ running btrbk. Something like:
|
||||||
rsync -az --delete \
|
rsync -az --delete \
|
||||||
--inplace --numeric-ids --acls --xattrs \
|
--inplace --numeric-ids --acls --xattrs \
|
||||||
-e 'ssh -i /etc/btrbk/ssh/id_rsa' \
|
-e 'ssh -i /etc/btrbk/ssh/id_rsa' \
|
||||||
myhost.example.org:/data/ \
|
myhost.mydomain.com:/data/ \
|
||||||
/mnt/btr_backup/myhost_sync/
|
/mnt/btr_backup/myhost_sync/
|
||||||
|
|
||||||
exec /usr/bin/btrbk -q run
|
exec /usr/bin/btrbk -q run
|
||||||
|
@ -412,7 +377,7 @@ compressed and piped through GnuPG.
|
||||||
raw_target_compress xz
|
raw_target_compress xz
|
||||||
raw_target_encrypt gpg
|
raw_target_encrypt gpg
|
||||||
gpg_keyring /etc/btrbk/gpg/pubring.gpg
|
gpg_keyring /etc/btrbk/gpg/pubring.gpg
|
||||||
gpg_recipient btrbk@example.org
|
gpg_recipient btrbk@mydomain.com
|
||||||
|
|
||||||
volume /mnt/btr_pool
|
volume /mnt/btr_pool
|
||||||
subvolume home
|
subvolume home
|
||||||
|
@ -428,7 +393,7 @@ host. For each backup, two files are created:
|
||||||
* `/backup/home.YYYYMMDD.btrfs.xz.gpg.info`: sidecar file containing
|
* `/backup/home.YYYYMMDD.btrfs.xz.gpg.info`: sidecar file containing
|
||||||
metadata used by btrbk.
|
metadata used by btrbk.
|
||||||
|
|
||||||
If you are using raw _incremental_ backups, please make sure you
|
I you are using raw _incremental_ backups, please make sure you
|
||||||
understand the implications (see [btrbk.conf(5)], TARGET TYPES).
|
understand the implications (see [btrbk.conf(5)], TARGET TYPES).
|
||||||
|
|
||||||
|
|
||||||
|
@ -449,7 +414,7 @@ will need the `btrfs` executable from the [btrfs-progs] package.
|
||||||
On the client side, create a ssh key dedicated to btrbk, without
|
On the client side, create a ssh key dedicated to btrbk, without
|
||||||
password protection:
|
password protection:
|
||||||
|
|
||||||
# ssh-keygen -t rsa -b 4096 -f /etc/btrbk/ssh/id_rsa -C btrbk@example.org -N ""
|
ssh-keygen -t rsa -b 4096 -f /etc/btrbk/ssh/id_rsa -C btrbk@mydomain.com -N ""
|
||||||
|
|
||||||
The content of the public key (/etc/btrbk/ssh/id_rsa.pub) is used for
|
The content of the public key (/etc/btrbk/ssh/id_rsa.pub) is used for
|
||||||
authentication in "authorized_keys" on the server side (see [sshd(8)]
|
authentication in "authorized_keys" on the server side (see [sshd(8)]
|
||||||
|
@ -555,14 +520,14 @@ to run it whenever the key is used for authentication. Example
|
||||||
"/root/.ssh/authorized_keys":
|
"/root/.ssh/authorized_keys":
|
||||||
|
|
||||||
# example backup source (also allowing deletion of old snapshots)
|
# example backup source (also allowing deletion of old snapshots)
|
||||||
command="/backup/scripts/ssh_filter_btrbk.sh -l --source --delete",restrict <pubkey>...
|
command="/backup/scripts/ssh_filter_btrbk.sh -l --source --delete" <pubkey>...
|
||||||
|
|
||||||
# example backup target (also allowing deletion of old snapshots)
|
# example backup target (also allowing deletion of old snapshots)
|
||||||
command="/backup/scripts/ssh_filter_btrbk.sh -l --target --delete",restrict <pubkey>...
|
command="/backup/scripts/ssh_filter_btrbk.sh -l --target --delete" <pubkey>...
|
||||||
|
|
||||||
# example fetch-only backup source (snapshot_preserve_min=all, snapshot_create=no),
|
# example fetch-only backup source (snapshot_preserve_min=all, snapshot_create=no),
|
||||||
# restricted to subvolumes within /home or /data
|
# restricted to subvolumes within /home or /data
|
||||||
command="/backup/scripts/ssh_filter_btrbk.sh -l --send -p /home -p /data",restrict <pubkey>...
|
command="/backup/scripts/ssh_filter_btrbk.sh -l --send -p /home -p /data" <pubkey>...
|
||||||
|
|
||||||
|
|
||||||
[ssh_filter_btrbk(1)]: https://digint.ch/btrbk/doc/ssh_filter_btrbk.1.html
|
[ssh_filter_btrbk(1)]: https://digint.ch/btrbk/doc/ssh_filter_btrbk.1.html
|
||||||
|
@ -680,8 +645,8 @@ Donate
|
||||||
|
|
||||||
So btrbk saved your day?
|
So btrbk saved your day?
|
||||||
|
|
||||||
I will definitively continue to develop btrbk for free. If you want to
|
I will definitively continue developing btrbk for free, but if you
|
||||||
support my hard work with a donation, you are welcome to do so!
|
want to support me you can do so:
|
||||||
|
|
||||||
[](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=WFQSSCD9GNM4S)
|
[](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=WFQSSCD9GNM4S)
|
||||||
|
|
||||||
|
@ -712,7 +677,7 @@ If you would like to contribute or have found bugs:
|
||||||
|
|
||||||
* Visit the [btrbk project page on GitHub] and use the
|
* Visit the [btrbk project page on GitHub] and use the
|
||||||
[issues tracker] there.
|
[issues tracker] there.
|
||||||
* Talk to us on [Libera.Chat] in `#btrbk`.
|
* Talk to us on Freenode in `#btrbk`.
|
||||||
* Contact the author via email (the email address can be found in
|
* Contact the author via email (the email address can be found in
|
||||||
the sources).
|
the sources).
|
||||||
|
|
||||||
|
@ -720,7 +685,6 @@ Any feedback is appreciated!
|
||||||
|
|
||||||
[btrbk project page on GitHub]: https://github.com/digint/btrbk
|
[btrbk project page on GitHub]: https://github.com/digint/btrbk
|
||||||
[issues tracker]: https://github.com/digint/btrbk/issues
|
[issues tracker]: https://github.com/digint/btrbk/issues
|
||||||
[Libera.Chat]: https://libera.chat
|
|
||||||
|
|
||||||
|
|
||||||
License
|
License
|
||||||
|
|
|
@ -17,13 +17,6 @@
|
||||||
# Enable transaction log
|
# Enable transaction log
|
||||||
transaction_log /var/log/btrbk.log
|
transaction_log /var/log/btrbk.log
|
||||||
|
|
||||||
# Specify SSH private key for remote connections
|
|
||||||
ssh_identity /etc/btrbk/ssh/id_ed25519
|
|
||||||
ssh_user root
|
|
||||||
|
|
||||||
# Use sudo if btrbk or lsbtr is run by regular user
|
|
||||||
backend_local_user btrfs-progs-sudo
|
|
||||||
|
|
||||||
# Enable stream buffer. Adding a buffer between the sending and
|
# Enable stream buffer. Adding a buffer between the sending and
|
||||||
# receiving side is generally a good idea.
|
# receiving side is generally a good idea.
|
||||||
# NOTE: If enabled, make sure to install the "mbuffer" package!
|
# NOTE: If enabled, make sure to install the "mbuffer" package!
|
||||||
|
@ -36,7 +29,7 @@ stream_buffer 256m
|
||||||
# If you want to set a custom name for the snapshot (and backups),
|
# If you want to set a custom name for the snapshot (and backups),
|
||||||
# use the "snapshot_name" option within the subvolume section.
|
# use the "snapshot_name" option within the subvolume section.
|
||||||
#
|
#
|
||||||
# NOTE: btrbk does not automatically create this directory, and the
|
# NOTE: btrbk does not autmatically create this directory, and the
|
||||||
# snapshot creation will fail if it is not present.
|
# snapshot creation will fail if it is not present.
|
||||||
#
|
#
|
||||||
snapshot_dir _btrbk_snap
|
snapshot_dir _btrbk_snap
|
||||||
|
@ -74,6 +67,12 @@ snapshot_dir _btrbk_snap
|
||||||
#archive_preserve_min no
|
#archive_preserve_min no
|
||||||
#archive_preserve <NN>h <NN>d <NN>w <NN>m <NN>y
|
#archive_preserve <NN>h <NN>d <NN>w <NN>m <NN>y
|
||||||
|
|
||||||
|
# Specify SSH private key for "ssh://" volumes / targets:
|
||||||
|
#ssh_identity /etc/btrbk/ssh/id_ed25519
|
||||||
|
#ssh_user root
|
||||||
|
#ssh_compression no
|
||||||
|
#ssh_cipher_spec default
|
||||||
|
|
||||||
# Enable compression for remote btrfs send/receive operations:
|
# Enable compression for remote btrfs send/receive operations:
|
||||||
#stream_compress no
|
#stream_compress no
|
||||||
#stream_compress_level default
|
#stream_compress_level default
|
||||||
|
@ -83,19 +82,19 @@ snapshot_dir _btrbk_snap
|
||||||
# can be run at a time.
|
# can be run at a time.
|
||||||
#lockfile /var/lock/btrbk.lock
|
#lockfile /var/lock/btrbk.lock
|
||||||
|
|
||||||
# Don't wait for transaction commit on deletion. Enable this to make
|
# Don't wait for transaction commit on deletion. Set this to "after"
|
||||||
# sure the deletion of subvolumes is committed to disk when btrbk
|
# or "each" to make sure the deletion of subvolumes is committed to
|
||||||
# terminates.
|
# disk when btrbk terminates.
|
||||||
#btrfs_commit_delete no
|
#btrfs_commit_delete no
|
||||||
|
|
||||||
|
|
||||||
#
|
#
|
||||||
# Volume section (optional): "volume <volume-directory>"
|
# Volume section: "volume <volume-directory>"
|
||||||
#
|
#
|
||||||
# <volume-directory> Base path within a btrfs filesystem
|
# <volume-directory> Directory of a btrfs volume (or subvolume)
|
||||||
# containing the subvolumes to be backuped
|
# containing the subvolume to be backuped
|
||||||
# (usually the mount-point of a btrfs filesystem
|
# (usually the mount-point of a btrfs filesystem
|
||||||
# mounted with subvolid=5 option).
|
# mounted with subvolid=5 option)
|
||||||
#
|
#
|
||||||
# Subvolume section: "subvolume <subvolume-name>"
|
# Subvolume section: "subvolume <subvolume-name>"
|
||||||
#
|
#
|
||||||
|
@ -105,19 +104,19 @@ snapshot_dir _btrbk_snap
|
||||||
# Target section: "target <type> <volume-directory>"
|
# Target section: "target <type> <volume-directory>"
|
||||||
#
|
#
|
||||||
# <type> (optional) type, defaults to "send-receive".
|
# <type> (optional) type, defaults to "send-receive".
|
||||||
# <volume-directory> Directory within a btrfs filesystem
|
# <volume-directory> Directory of a btrfs volume (or subvolume)
|
||||||
# receiving the backups.
|
# receiving the backups.
|
||||||
#
|
#
|
||||||
# NOTE: The parser does not care about indentation, this is only for
|
# NOTE: The parser does not care about indentation, this is only for
|
||||||
# human readability. All options apply to the last section
|
# human readability. The options always apply to the last section
|
||||||
# encountered, overriding the corresponding option of the upper
|
# encountered, overriding the corresponding option of the upper
|
||||||
# section. This means that the global options must be set on top,
|
# section. This means that the global options must be set before any
|
||||||
# before any "volume", "subvolume" or "target section.
|
# "volume" section.
|
||||||
#
|
#
|
||||||
|
|
||||||
|
|
||||||
#
|
#
|
||||||
# Example retention policy:
|
# Example configuration:
|
||||||
#
|
#
|
||||||
snapshot_preserve_min 2d
|
snapshot_preserve_min 2d
|
||||||
snapshot_preserve 14d
|
snapshot_preserve 14d
|
||||||
|
@ -125,49 +124,29 @@ snapshot_preserve 14d
|
||||||
target_preserve_min no
|
target_preserve_min no
|
||||||
target_preserve 20d 10w *m
|
target_preserve 20d 10w *m
|
||||||
|
|
||||||
|
archive_preserve_min latest
|
||||||
|
archive_preserve 12m 10y
|
||||||
|
|
||||||
#
|
|
||||||
# Simple setup: Backup root and home to external disk
|
|
||||||
#
|
|
||||||
snapshot_dir /btrbk_snapshots
|
|
||||||
target /mnt/btr_backup
|
|
||||||
subvolume /
|
|
||||||
subvolume /home
|
|
||||||
|
|
||||||
|
|
||||||
#
|
|
||||||
# Complex setup
|
|
||||||
#
|
|
||||||
# In order to keep things organized, it is recommended to use "volume"
|
|
||||||
# sections and mount the top-level subvolume (subvolid=5):
|
|
||||||
#
|
|
||||||
# $ mount -o subvolid=5 /dev/sda1 /mnt/btr_pool
|
|
||||||
#
|
|
||||||
# Backup to external disk mounted on /mnt/btr_backup
|
# Backup to external disk mounted on /mnt/btr_backup
|
||||||
volume /mnt/btr_pool
|
volume /mnt/btr_pool
|
||||||
# Create snapshots in /mnt/btr_pool/btrbk_snapshots
|
# no action if external disk is not attached
|
||||||
snapshot_dir btrbk_snapshots
|
snapshot_create ondemand
|
||||||
|
|
||||||
# Target for all subvolume sections:
|
# propagates to all subvolume sections:
|
||||||
target /mnt/btr_backup
|
target /mnt/btr_backup/_btrbk
|
||||||
|
|
||||||
# Some default btrfs installations (e.g. Ubuntu) use "@" for rootfs
|
subvolume root_gentoo
|
||||||
# (mounted at "/") and "@home" (mounted at "/home"). Note that this
|
subvolume kvm
|
||||||
# is only a naming convention.
|
# use different retention policy for kvm backups
|
||||||
#subvolume @
|
target_preserve 7d 4w
|
||||||
subvolume root
|
|
||||||
subvolume home
|
|
||||||
subvolume kvm
|
|
||||||
# Use different retention policy for kvm backups:
|
|
||||||
target_preserve 7d 4w
|
|
||||||
|
|
||||||
|
|
||||||
# Backup data to external disk as well as remote host
|
# Backup to external disk as well as some remote host
|
||||||
volume /mnt/btr_data
|
volume /mnt/btr_data
|
||||||
subvolume data
|
subvolume home
|
||||||
# Always create snapshot, even if targets are unreachable
|
# always create snapshot, even if targets are unreachable
|
||||||
snapshot_create always
|
snapshot_create always
|
||||||
target /mnt/btr_backup
|
target /mnt/btr_backup/_btrbk
|
||||||
target ssh://backup.my-remote-host.com/mnt/btr_backup
|
target ssh://backup.my-remote-host.com/mnt/btr_backup
|
||||||
|
|
||||||
|
|
||||||
|
@ -176,22 +155,7 @@ volume ssh://my-remote-host.com/mnt/btr_pool
|
||||||
subvolume data_0
|
subvolume data_0
|
||||||
snapshot_dir snapshots/btrbk
|
snapshot_dir snapshots/btrbk
|
||||||
snapshot_name data_main
|
snapshot_name data_main
|
||||||
target /mnt/btr_backup/my-remote-host.com
|
target /mnt/btr_backup/_btrbk/my-remote-host.com
|
||||||
|
|
||||||
|
|
||||||
# Backup on demand (noauto) to remote host running busybox, login as
|
|
||||||
# regular user using ssh-agent with current user name (ssh_user no)
|
|
||||||
# and default credentials (ssh_identity no).
|
|
||||||
volume /home
|
|
||||||
noauto yes
|
|
||||||
compat busybox
|
|
||||||
backend_remote btrfs-progs-sudo
|
|
||||||
ssh_user no
|
|
||||||
ssh_identity no
|
|
||||||
|
|
||||||
target ssh://my-user-host.com/mnt/btr_backup/home
|
|
||||||
subvolume alice
|
|
||||||
subvolume bob
|
|
||||||
|
|
||||||
|
|
||||||
# Resume backups from remote host which runs its own btrbk instance
|
# Resume backups from remote host which runs its own btrbk instance
|
||||||
|
@ -201,4 +165,4 @@ volume ssh://my-remote-host.com/mnt/btr_pool
|
||||||
snapshot_create no
|
snapshot_create no
|
||||||
snapshot_preserve_min all
|
snapshot_preserve_min all
|
||||||
subvolume home
|
subvolume home
|
||||||
target /mnt/btr_backup/my-remote-host.com
|
target /mnt/btr_backup/_btrbk/my-remote-host.com
|
||||||
|
|
|
@ -4,7 +4,7 @@ _btrbk_init_cmds()
|
||||||
#
|
#
|
||||||
# for example, for this command:
|
# for example, for this command:
|
||||||
#
|
#
|
||||||
# btrbk -v --override warn_unknown_targets=yes list config --long
|
# btrbk -v list config --long
|
||||||
#
|
#
|
||||||
# then $cmds is:
|
# then $cmds is:
|
||||||
#
|
#
|
||||||
|
@ -14,11 +14,6 @@ _btrbk_init_cmds()
|
||||||
|
|
||||||
local i
|
local i
|
||||||
for ((i = 1; i < cword; i++)); do
|
for ((i = 1; i < cword; i++)); do
|
||||||
case "${words[i-1]}" in
|
|
||||||
'-c' | '--config' | '--exclude' | '-l' | '--loglevel' | '--format' | '--lockfile' | '--override')
|
|
||||||
continue
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
[[ ${words[i]} != -* ]] && cmds+=(${words[i]})
|
[[ ${words[i]} != -* ]] && cmds+=(${words[i]})
|
||||||
done
|
done
|
||||||
|
|
||||||
|
@ -34,28 +29,21 @@ _btrbk()
|
||||||
case "$prev" in
|
case "$prev" in
|
||||||
'-c' | '--config')
|
'-c' | '--config')
|
||||||
_filedir
|
_filedir
|
||||||
return
|
|
||||||
;;
|
;;
|
||||||
'--exclude')
|
'--exclude')
|
||||||
return
|
|
||||||
;;
|
;;
|
||||||
'-l' | '--loglevel')
|
'-l' | '--loglevel')
|
||||||
COMPREPLY=($(compgen -W 'error warn info debug trace' -- "$cur"))
|
COMPREPLY=($(compgen -W 'error warn info debug trace' -- "$cur"))
|
||||||
return
|
|
||||||
;;
|
;;
|
||||||
'--format')
|
'--format')
|
||||||
COMPREPLY=($(compgen -W 'table long raw' -- "$cur"))
|
COMPREPLY=($(compgen -W 'table long raw' -- "$cur"))
|
||||||
return
|
|
||||||
;;
|
;;
|
||||||
'--lockfile')
|
'--lockfile')
|
||||||
_filedir
|
_filedir
|
||||||
return
|
|
||||||
;;
|
;;
|
||||||
'--override')
|
'--override')
|
||||||
return
|
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
$split && return
|
$split && return
|
||||||
|
|
||||||
if [[ $cur == -* ]]; then
|
if [[ $cur == -* ]]; then
|
||||||
|
|
|
@ -4,9 +4,6 @@
|
||||||
|
|
||||||
now=$(date +%Y%m%d)
|
now=$(date +%Y%m%d)
|
||||||
|
|
||||||
declare -A rsync_src rsync_dst rsync_log rsync_rsh rsync_opt
|
|
||||||
declare -A sync_fs_onchange
|
|
||||||
|
|
||||||
##### start config section #####
|
##### start config section #####
|
||||||
|
|
||||||
# Email recipients, separated by whitespace:
|
# Email recipients, separated by whitespace:
|
||||||
|
@ -33,9 +30,7 @@ rsync_opt[example_data]="-az --delete --inplace --numeric-ids --acls --xattrs"
|
||||||
# If set, add "rsync_dst" to "sync_fs" (see below) if rsync reports files transferred
|
# If set, add "rsync_dst" to "sync_fs" (see below) if rsync reports files transferred
|
||||||
#sync_fs_onchange[example_data]=yes
|
#sync_fs_onchange[example_data]=yes
|
||||||
|
|
||||||
# Enable all rsync declarations (all indices of rsync_src array)
|
# Enabled rsync declarations (whitespace-separated list)
|
||||||
#rsync_enable=${!rsync_src[@]}
|
|
||||||
# Explicitly enable rsync declarations (whitespace-separated list)
|
|
||||||
#rsync_enable="example_data"
|
#rsync_enable="example_data"
|
||||||
rsync_enable=
|
rsync_enable=
|
||||||
|
|
||||||
|
@ -60,8 +55,6 @@ btrbk_opts="-c /etc/btrbk/btrbk.conf"
|
||||||
#mail_cmd_block_prefix='\\u200B' # zero-width whitespace
|
#mail_cmd_block_prefix='\\u200B' # zero-width whitespace
|
||||||
#mail_cmd_block_prefix=". "
|
#mail_cmd_block_prefix=". "
|
||||||
|
|
||||||
# Newline character
|
|
||||||
BR=$'\n'
|
|
||||||
|
|
||||||
##### end config section #####
|
##### end config section #####
|
||||||
|
|
||||||
|
@ -90,47 +83,47 @@ send_mail()
|
||||||
body+="$info"
|
body+="$info"
|
||||||
fi
|
fi
|
||||||
if [[ -n "$detail" ]] && [[ -n "$has_errors" ]] || [[ "${mail_detail:-no}" = "yes" ]]; then
|
if [[ -n "$detail" ]] && [[ -n "$has_errors" ]] || [[ "${mail_detail:-no}" = "yes" ]]; then
|
||||||
[[ -n "$body" ]] && body+="${BR}${BR}DETAIL:${BR}"
|
[[ -n "$body" ]] && body+="\n\nDETAIL:\n"
|
||||||
body+="$detail"
|
body+="$detail"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# skip sending mail on empty body
|
# skip sending mail on empty body
|
||||||
if [[ -z "$body" ]] && [[ -n "$has_errors" ]]; then
|
if [[ -z "$body" ]] && [[ -n "$has_errors" ]]; then
|
||||||
body+="FATAL: something went wrong (errors present but empty mail body)${BR}"
|
body+="FATAL: something went wrong (errors present but empty mail body)\n"
|
||||||
fi
|
fi
|
||||||
[[ -z "$body" ]] && exit 0
|
[[ -z "$body" ]] && exit 0
|
||||||
|
|
||||||
# send mail
|
# send mail
|
||||||
echo "$body" | mail -s "$subject" $mailto
|
echo -e "$body" | mail -s "$subject" $mailto
|
||||||
if [[ $? -ne 0 ]]; then
|
if [[ $? -ne 0 ]]; then
|
||||||
echo "$0: Failed to send btrbk mail to \"$mailto\", dumping mail:${BR}" 1>&2
|
echo -e "$0: Failed to send btrbk mail to \"$mailto\", dumping mail:\n" 1>&2
|
||||||
echo "<mail_subject>$subject</mail_subject>${BR}<mail_body>${BR}$body</mail_body>" 1>&2
|
echo -e "<mail_subject>$subject</mail_subject>\n<mail_body>\n$body</mail_body>" 1>&2
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
einfo()
|
einfo()
|
||||||
{
|
{
|
||||||
info+="$1${BR}"
|
info+="$1\n"
|
||||||
}
|
}
|
||||||
|
|
||||||
ebegin()
|
ebegin()
|
||||||
{
|
{
|
||||||
ebtext=$1
|
ebtext=$1
|
||||||
detail+="${BR}### $1${BR}"
|
detail+="\n### $1\n"
|
||||||
}
|
}
|
||||||
|
|
||||||
eend()
|
eend()
|
||||||
{
|
{
|
||||||
if [[ $1 -eq 0 ]]; then
|
if [[ $1 -eq 0 ]]; then
|
||||||
eetext=${3-success}
|
eetext=${3-success}
|
||||||
detail+="${BR}"
|
detail+="\n"
|
||||||
else
|
else
|
||||||
has_errors=1
|
has_errors=1
|
||||||
eetext="ERROR (code=$1)"
|
eetext="ERROR (code=$1)"
|
||||||
[[ -n "$2" ]] && eetext+=": $2"
|
[[ -n "$2" ]] && eetext+=": $2"
|
||||||
detail+="${BR}### $eetext${BR}"
|
detail+="\n### $eetext\n"
|
||||||
fi
|
fi
|
||||||
info+="$ebtext: $eetext${BR}"
|
info+="$ebtext: $eetext\n"
|
||||||
return $1
|
return $1
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -146,10 +139,10 @@ run_cmd()
|
||||||
{
|
{
|
||||||
cmd_out=$("$@" 2>&1)
|
cmd_out=$("$@" 2>&1)
|
||||||
local ret=$?
|
local ret=$?
|
||||||
detail+="++ ${@@Q}${BR}"
|
detail+="++ ${@@Q}\n"
|
||||||
if [[ -n "${mail_cmd_block_prefix:-}" ]] && [[ -n "$cmd_out" ]]; then
|
if [[ -n "${mail_cmd_block_prefix:-}" ]] && [[ -n "$cmd_out" ]]; then
|
||||||
detail+=$(echo -n "$cmd_out" | sed "s/^/${mail_cmd_block_prefix}/")
|
detail+=$(echo -n "$cmd_out" | sed "s/^/${mail_cmd_block_prefix}/")
|
||||||
detail+="${BR}"
|
detail+="\n"
|
||||||
else
|
else
|
||||||
detail+=$cmd_out
|
detail+=$cmd_out
|
||||||
fi
|
fi
|
||||||
|
@ -166,7 +159,7 @@ mount_all()
|
||||||
if [[ $? -eq 0 ]]; then
|
if [[ $? -eq 0 ]]; then
|
||||||
eend -1 "already mounted"
|
eend -1 "already mounted"
|
||||||
else
|
else
|
||||||
detail+="${BR}"
|
detail+="\n"
|
||||||
run_cmd mount --target $mountpoint
|
run_cmd mount --target $mountpoint
|
||||||
eend $? && mounted+=" $mountpoint"
|
eend $? && mounted+=" $mountpoint"
|
||||||
fi
|
fi
|
||||||
|
|
|
@ -31,7 +31,7 @@
|
||||||
# NOTE: Depending on your setup (hardware, btrfs mount options),
|
# NOTE: Depending on your setup (hardware, btrfs mount options),
|
||||||
# btrbk-verify may eat all your CPU power and use high bandwidth!
|
# btrbk-verify may eat all your CPU power and use high bandwidth!
|
||||||
# Consider nice(1), ionice(1).
|
# Consider nice(1), ionice(1).
|
||||||
#
|
|
||||||
# Incomplete resource eater list:
|
# Incomplete resource eater list:
|
||||||
# - rsync: checksums, heavy disk I/O
|
# - rsync: checksums, heavy disk I/O
|
||||||
# - btrfs: decompression, encryption
|
# - btrfs: decompression, encryption
|
||||||
|
@ -77,7 +77,7 @@ set -u
|
||||||
set -e
|
set -e
|
||||||
set -o pipefail
|
set -o pipefail
|
||||||
|
|
||||||
btrbk_version_min='0.32.0'
|
btrbk_version_min='0.28.0'
|
||||||
|
|
||||||
# defaults: ignore subvol dirs and root folder timestamp change
|
# defaults: ignore subvol dirs and root folder timestamp change
|
||||||
ignore_nested_subvolume_dir=1
|
ignore_nested_subvolume_dir=1
|
||||||
|
@ -107,8 +107,8 @@ options:
|
||||||
--ignore-xattrs ignore xattrs when verifying subvolumes
|
--ignore-xattrs ignore xattrs when verifying subvolumes
|
||||||
--ssh-identity FILE override ssh_identity from btrbk.conf(5) with FILE,
|
--ssh-identity FILE override ssh_identity from btrbk.conf(5) with FILE,
|
||||||
and clear all other ssh_* options (use with --ssh-user)
|
and clear all other ssh_* options (use with --ssh-user)
|
||||||
--ssh-user USER override ssh_user from btrbk.conf(5) with USER, and
|
--ssh-user USER override ssh_user from btrbk.conf(5) with USER
|
||||||
clear all other ssh_* options(use with --ssh-identity)
|
(only in conjunction with --ssh-identity)
|
||||||
--ssh-agent start ssh-agent(1) and add identity
|
--ssh-agent start ssh-agent(1) and add identity
|
||||||
|
|
||||||
commands:
|
commands:
|
||||||
|
@ -157,6 +157,7 @@ while [[ "$#" -ge 1 ]]; do
|
||||||
;;
|
;;
|
||||||
--ssh-identity)
|
--ssh-identity)
|
||||||
# use different ssh identity (-i option) for rsync rsh.
|
# use different ssh identity (-i option) for rsync rsh.
|
||||||
|
# if set, ssh_user defaults to root.
|
||||||
# NOTE: this overrides all btrbk ssh_* options
|
# NOTE: this overrides all btrbk ssh_* options
|
||||||
ssh_identity="$2"
|
ssh_identity="$2"
|
||||||
shift
|
shift
|
||||||
|
@ -185,8 +186,6 @@ while [[ "$#" -ge 1 ]]; do
|
||||||
shift
|
shift
|
||||||
done
|
done
|
||||||
|
|
||||||
BR=$'\n'
|
|
||||||
|
|
||||||
log_line()
|
log_line()
|
||||||
{
|
{
|
||||||
echo "$@" 1>&2
|
echo "$@" 1>&2
|
||||||
|
@ -209,13 +208,13 @@ tlog()
|
||||||
[[ -n "$dryrun" ]] && [[ "$status" == "starting" ]] && status="dryrun_starting"
|
[[ -n "$dryrun" ]] && [[ "$status" == "starting" ]] && status="dryrun_starting"
|
||||||
local line="$(date --iso-8601=seconds) verify-rsync ${status} ${target} ${source} - -"
|
local line="$(date --iso-8601=seconds) verify-rsync ${status} ${target} ${source} - -"
|
||||||
[[ -n "$comment" ]] && line="$line # $comment";
|
[[ -n "$comment" ]] && line="$line # $comment";
|
||||||
tlog_text+="$line${BR}"
|
tlog_text+="$line\n"
|
||||||
log_debug "$line"
|
log_debug "$line"
|
||||||
}
|
}
|
||||||
tlog_print()
|
tlog_print()
|
||||||
{
|
{
|
||||||
# tlog goes to stdout
|
# tlog goes to stdout
|
||||||
echo "${BR}TRANSACTION LOG${BR}---------------${BR}${tlog_text:-}"
|
echo -e "\nTRANSACTION LOG\n---------------\n${tlog_text:-}"
|
||||||
}
|
}
|
||||||
|
|
||||||
# parse "rsync -i,--itemize-changes" output.
|
# parse "rsync -i,--itemize-changes" output.
|
||||||
|
@ -268,14 +267,11 @@ rsync_rsh()
|
||||||
local rsh_match="(.*) ([a-z0-9_-]+)@([a-zA-Z0-9.-]+)$"
|
local rsh_match="(.*) ([a-z0-9_-]+)@([a-zA-Z0-9.-]+)$"
|
||||||
|
|
||||||
if [[ -z "$rsh" ]]; then
|
if [[ -z "$rsh" ]]; then
|
||||||
return
|
echo
|
||||||
elif [[ -n "$ssh_user" ]] || [[ -n "$ssh_identity" ]]; then
|
elif [[ -n "$ssh_identity" ]]; then
|
||||||
# override btrbk.conf from command line arguments
|
# override btrbk.conf from command line arguments
|
||||||
log_debug "Overriding all ssh_* options from btrbk.conf"
|
log_debug "Overriding all ssh_* options from btrbk.conf"
|
||||||
local cmd="ssh -q"
|
echo "ssh -q -i $ssh_identity -l $ssh_user"
|
||||||
[[ -n "$ssh_identity" ]] && cmd="$cmd -i '$ssh_identity'"
|
|
||||||
[[ -n "$ssh_user" ]] && cmd="$cmd -l '$ssh_user'"
|
|
||||||
echo "$cmd"
|
|
||||||
elif [[ $rsh =~ $rsh_match ]]; then
|
elif [[ $rsh =~ $rsh_match ]]; then
|
||||||
echo "${BASH_REMATCH[1]} -l ${BASH_REMATCH[2]}"
|
echo "${BASH_REMATCH[1]} -l ${BASH_REMATCH[2]}"
|
||||||
else
|
else
|
||||||
|
@ -306,7 +302,7 @@ start_ssh_agent()
|
||||||
|
|
||||||
eval_btrbk_resolved_line()
|
eval_btrbk_resolved_line()
|
||||||
{
|
{
|
||||||
local line=" $1"
|
local line=$1
|
||||||
local prefix=$2
|
local prefix=$2
|
||||||
local required_keys=$3
|
local required_keys=$3
|
||||||
# reset all variables first
|
# reset all variables first
|
||||||
|
@ -315,13 +311,25 @@ eval_btrbk_resolved_line()
|
||||||
done
|
done
|
||||||
for vv in $required_keys; do
|
for vv in $required_keys; do
|
||||||
# basic input validation, set prefixed variable (eval)
|
# basic input validation, set prefixed variable (eval)
|
||||||
local match=" ${vv}='([^']*('\\\\''[^']*)*)'"
|
local var_assignment=$(echo "$line" | grep -Eo "${vv}"'="[^"]*"')
|
||||||
if [[ $line =~ $match ]] ; then
|
if [[ $? -ne 0 ]] || [[ -z "$var_assignment" ]]; then
|
||||||
eval "${prefix}${vv}='${BASH_REMATCH[1]}'" || return 1
|
if [[ $vv == "target_type" ]]; then
|
||||||
else
|
# hack for btrbk <= 0.27.2 bug: missing "target_type" in $table_formats{resolved}
|
||||||
log_line "btrbk-verify: ERROR: Missing variable \"${vv}\""
|
# hardcode send-receive type, this results in rsync failing on raw targets with:
|
||||||
|
# rsync: ERROR: cannot stat destination "xxx.btrfs.xz.gpg/": Not a directory (20)
|
||||||
|
eval "${prefix}target_type=\"send-receive\"" || return 1
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
if [[ $vv == "target_rsh" ]] && [[ -z "$R_target_host" ]]; then
|
||||||
|
# hack for btrbk <= 0.27.2 bug: missing "target_rsh" in $table_formats{resolved}
|
||||||
|
# note that target_rsh is not needed if target_host is not set.
|
||||||
|
# variable has already been reset above, continue
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
log_line "btrbk-verify: missing required variable \"${vv}\" in btrbk --format=raw line"
|
||||||
return 1
|
return 1
|
||||||
fi
|
fi
|
||||||
|
eval "${prefix}${var_assignment}" || return 1
|
||||||
done
|
done
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -331,6 +339,10 @@ exit_trap_action()
|
||||||
[[ $verbose -gt 0 ]] && tlog_print
|
[[ $verbose -gt 0 ]] && tlog_print
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# restrictions from rsync_rsh():
|
||||||
|
[[ -z "$ssh_identity" ]] && [[ -n "$ssh_user" ]] && print_usage 2
|
||||||
|
[[ -n "$ssh_identity" ]] && [[ -z "$ssh_user" ]] && print_usage 2
|
||||||
|
|
||||||
# start ssh-agent(1)
|
# start ssh-agent(1)
|
||||||
[[ -n "$ssh_start_agent" ]] && start_ssh_agent
|
[[ -n "$ssh_start_agent" ]] && start_ssh_agent
|
||||||
|
|
||||||
|
@ -359,22 +371,20 @@ while read -r btrbk_list_line; do
|
||||||
log_debug "Evaluating [btrbk list] line: $btrbk_list_line"
|
log_debug "Evaluating [btrbk list] line: $btrbk_list_line"
|
||||||
[[ -z "$btrbk_list_line" ]] && continue
|
[[ -z "$btrbk_list_line" ]] && continue
|
||||||
if ! eval_btrbk_resolved_line "$btrbk_list_line" \
|
if ! eval_btrbk_resolved_line "$btrbk_list_line" \
|
||||||
"R_" "snapshot_subvolume target_subvolume source_host target_host target_type source_rsh target_rsh"
|
"R_" "snapshot_path target_path source_host target_host target_type source_rsh target_rsh"
|
||||||
then
|
then
|
||||||
log_line "btrbk-verify: ERROR: Parse error of command output: ${btrbk_cmd[@]}"
|
log_line "btrbk-verify: WARNING: Skipping task (parse error). Make sure to have >=btrbk-${btrbk_version_min} installed!"
|
||||||
log_line "Make sure to have >=btrbk-${btrbk_version_min} installed!"
|
continue
|
||||||
exitstatus=1
|
|
||||||
break
|
|
||||||
fi
|
fi
|
||||||
|
|
||||||
source="${R_snapshot_subvolume}/"
|
source="${R_snapshot_path}"
|
||||||
target="${R_target_subvolume}/"
|
target="${R_target_path}"
|
||||||
[[ -n "$R_source_host" ]] && source="${R_source_host}:${source}"
|
[[ -n "$R_source_host" ]] && source="${R_source_host}:${source}"
|
||||||
[[ -n "$R_target_host" ]] && target="${R_target_host}:${target}"
|
[[ -n "$R_target_host" ]] && target="${R_target_host}:${target}"
|
||||||
|
|
||||||
if [[ -z "$R_snapshot_subvolume" ]]; then
|
if [[ -z "$R_snapshot_path" ]]; then
|
||||||
log_line "WARNING: Skipping task (missing snapshot): target=$target"
|
log_line "WARNING: Skipping task (missing snapshot): target=$target"
|
||||||
elif [[ -z "$R_target_subvolume" ]]; then
|
elif [[ -z "$R_target_path" ]]; then
|
||||||
log_line "Skipping task (no target): source=$source"
|
log_line "Skipping task (no target): source=$source"
|
||||||
elif [[ "$R_target_type" != "send-receive" ]]; then
|
elif [[ "$R_target_type" != "send-receive" ]]; then
|
||||||
log_line "Skipping task (target_type=$R_target_type): source=$source, target=$target"
|
log_line "Skipping task (target_type=$R_target_type): source=$source, target=$target"
|
||||||
|
@ -386,11 +396,10 @@ while read -r btrbk_list_line; do
|
||||||
# rsync rsh is either source_rsh or target_rsh or empty
|
# rsync rsh is either source_rsh or target_rsh or empty
|
||||||
eff_rsh="$R_source_rsh"
|
eff_rsh="$R_source_rsh"
|
||||||
[[ -z "$eff_rsh" ]] && eff_rsh="$R_target_rsh"
|
[[ -z "$eff_rsh" ]] && eff_rsh="$R_target_rsh"
|
||||||
eff_rsh=$(rsync_rsh "$eff_rsh")
|
|
||||||
|
|
||||||
rsync_cmd=("rsync" "${rsync_args[@]}")
|
rsync_cmd=("rsync" "${rsync_args[@]}")
|
||||||
[[ -n "$eff_rsh" ]] && rsync_cmd+=("-e" "$eff_rsh")
|
[[ -n "$eff_rsh" ]] && rsync_cmd+=(-e "$(rsync_rsh "$eff_rsh")")
|
||||||
rsync_cmd+=("${source}" "${target}")
|
rsync_cmd+=("${source}/" "${target}/")
|
||||||
log_cmd "${rsync_cmd[@]}"
|
log_cmd "${rsync_cmd[@]}"
|
||||||
[[ -n "$dryrun" ]] && rsync_cmd=("cat" "/dev/null")
|
[[ -n "$dryrun" ]] && rsync_cmd=("cat" "/dev/null")
|
||||||
|
|
||||||
|
|
|
@ -54,7 +54,7 @@ salt_hex = "".join(["{:02x}".format(x) for x in salt])
|
||||||
dk_hex = "".join(["{:02x}".format(x) for x in dk])
|
dk_hex = "".join(["{:02x}".format(x) for x in dk])
|
||||||
|
|
||||||
print("KEY=" + dk_hex);
|
print("KEY=" + dk_hex);
|
||||||
print("algorithm=pbkdf2_hmac");
|
print("algoritm=pbkdf2_hmac");
|
||||||
print("hash_name=" + hash_name);
|
print("hash_name=" + hash_name);
|
||||||
print("salt=" + salt_hex);
|
print("salt=" + salt_hex);
|
||||||
print("iterations=" + str(iterations));
|
print("iterations=" + str(iterations));
|
||||||
|
|
|
@ -42,7 +42,7 @@ my $compress_format_alt = 'gz|bz2|xz|lzo|lz4';
|
||||||
my $file_match = qr/[0-9a-zA-Z_@\+\-\.\/]+/; # note: ubuntu uses '@' in the subvolume layout: <https://help.ubuntu.com/community/btrfs>
|
my $file_match = qr/[0-9a-zA-Z_@\+\-\.\/]+/; # note: ubuntu uses '@' in the subvolume layout: <https://help.ubuntu.com/community/btrfs>
|
||||||
my $uuid_match = qr/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/;
|
my $uuid_match = qr/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/;
|
||||||
my $timestamp_postfix_match = qr/\.(?<YYYY>[0-9]{4})(?<MM>[0-9]{2})(?<DD>[0-9]{2})(T(?<hh>[0-9]{2})(?<mm>[0-9]{2})((?<ss>[0-9]{2})(?<zz>(Z|[+-][0-9]{4})))?)?(_(?<NN>[0-9]+))?/; # matches "YYYYMMDD[Thhmm[ss+0000]][_NN]"
|
my $timestamp_postfix_match = qr/\.(?<YYYY>[0-9]{4})(?<MM>[0-9]{2})(?<DD>[0-9]{2})(T(?<hh>[0-9]{2})(?<mm>[0-9]{2})((?<ss>[0-9]{2})(?<zz>(Z|[+-][0-9]{4})))?)?(_(?<NN>[0-9]+))?/; # matches "YYYYMMDD[Thhmm[ss+0000]][_NN]"
|
||||||
my $raw_postfix_match = qr/--(?<received_uuid>$uuid_match)(\@(?<parent_uuid>$uuid_match))?\.btrfs?(\.(?<compress>($compress_format_alt)))?(\.(?<encrypt>gpg))?(\.(?<split>split_aa))?(\.(?<incomplete>part))?/; # matches ".btrfs_<received_uuid>[@<parent_uuid>][.gz|.bz2|.xz|...][.gpg][.split_aa][.part]"
|
my $raw_postfix_match = qr/--(?<received_uuid>$uuid_match)(\@(?<parent_uuid>$uuid_match))?\.btrfs?(\.(?<compress>($compress_format_alt)))?(\.(?<encrypt>gpg))?(\.(?<split>split_aa))?(\.(?<incomplete>part))?/; # matches ".btrfs_<received_uuid>[@<parent_uuid>][.gz|bz2|xz][.gpg][.split_aa][.part]"
|
||||||
|
|
||||||
my $dryrun;
|
my $dryrun;
|
||||||
|
|
||||||
|
|
|
@ -1,217 +0,0 @@
|
||||||
#!/usr/bin/env python3
|
|
||||||
|
|
||||||
import os
|
|
||||||
import logging
|
|
||||||
import subprocess
|
|
||||||
import argparse
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class TransformProcess:
|
|
||||||
def run(self, bfile, options, **kw):
|
|
||||||
return subprocess.Popen(self.get_cmd(bfile, options), **kw)
|
|
||||||
|
|
||||||
def get_cmd(self, bfile, options):
|
|
||||||
raise NotImplementedError()
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def add_parser_options(cls, parser):
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
class TransformOpensslDecrypt(TransformProcess):
|
|
||||||
@staticmethod
|
|
||||||
def get_cmd(bfile, options):
|
|
||||||
return [
|
|
||||||
'openssl', 'enc', '-d', '-' + bfile.info['cipher'], '-K',
|
|
||||||
open(options.openssl_keyfile, 'r').read(), '-iv', bfile.info['iv']
|
|
||||||
]
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def add_parser_options(parser):
|
|
||||||
parser.add_argument('--openssl-keyfile', help="path to private encryption key file")
|
|
||||||
|
|
||||||
|
|
||||||
class TransformDecompress(TransformProcess):
|
|
||||||
def __init__(self, program):
|
|
||||||
self.p = program
|
|
||||||
|
|
||||||
def get_cmd(self, bfile, options):
|
|
||||||
return [self.p, '-d']
|
|
||||||
|
|
||||||
|
|
||||||
class TransformBtrfsReceive(TransformProcess):
|
|
||||||
@classmethod
|
|
||||||
def run(cls, bfile, options, **kw):
|
|
||||||
return subprocess.Popen(cls.get_cmd(bfile, options), **kw)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def get_cmd(bfile, options):
|
|
||||||
return ['btrfs', 'receive', options.restore_dir]
|
|
||||||
|
|
||||||
|
|
||||||
TRANSFORMERS = (
|
|
||||||
TransformOpensslDecrypt, TransformDecompress, TransformBtrfsReceive
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class BtrfsPipeline:
|
|
||||||
def __init__(self, bfile):
|
|
||||||
self.bfile = bfile
|
|
||||||
self.processors = []
|
|
||||||
|
|
||||||
def append(self, transformer):
|
|
||||||
self.processors.append(transformer)
|
|
||||||
|
|
||||||
def run(self, options):
|
|
||||||
processes = []
|
|
||||||
with open(self.bfile.data_file, 'rb') as next_input:
|
|
||||||
for transformer in self.processors:
|
|
||||||
process = transformer.run(
|
|
||||||
self.bfile, options,
|
|
||||||
stdin=next_input, stdout=subprocess.PIPE,
|
|
||||||
stderr=subprocess.PIPE)
|
|
||||||
next_input = process.stdout
|
|
||||||
processes.append(process)
|
|
||||||
btrfs_process = TransformBtrfsReceive.run(
|
|
||||||
self.bfile, options, stdin=next_input,
|
|
||||||
stderr=subprocess.PIPE, stdout=subprocess.DEVNULL)
|
|
||||||
processes.append(btrfs_process)
|
|
||||||
# warning: the code below is pretty ugly and hacky
|
|
||||||
terminated = 0
|
|
||||||
while terminated < len(processes):
|
|
||||||
for p in processes:
|
|
||||||
if p.returncode is not None:
|
|
||||||
continue
|
|
||||||
msg = None
|
|
||||||
try:
|
|
||||||
p.wait(timeout=1)
|
|
||||||
except subprocess.TimeoutExpired as e:
|
|
||||||
pass
|
|
||||||
except Exception as e:
|
|
||||||
msg = e
|
|
||||||
else:
|
|
||||||
msg = p.stderr.read().decode('utf-8').strip()
|
|
||||||
finally:
|
|
||||||
if p.returncode is not None:
|
|
||||||
terminated += 1
|
|
||||||
if p.returncode != 0:
|
|
||||||
for p_other in processes:
|
|
||||||
p_other.terminate()
|
|
||||||
terminated += 1
|
|
||||||
if msg:
|
|
||||||
logger.error(f"error running {p.args}: {msg}")
|
|
||||||
|
|
||||||
def get_cmd(self, options):
|
|
||||||
command_pipe = [['cat', self.bfile.data_file]]
|
|
||||||
for transformer in self.processors:
|
|
||||||
command_pipe.append(transformer.get_cmd(self.bfile, options))
|
|
||||||
command_pipe.append(TransformBtrfsReceive.get_cmd(self.bfile, options))
|
|
||||||
return ' | '.join(' '.join(x) for x in command_pipe)
|
|
||||||
|
|
||||||
|
|
||||||
class BackupFile:
|
|
||||||
def __init__(self, path):
|
|
||||||
assert path.endswith('.info')
|
|
||||||
self.info_file = path
|
|
||||||
self.info = self._parse_info()
|
|
||||||
self.uuid = self.info['RECEIVED_UUID']
|
|
||||||
self.data_file = os.path.join(os.path.dirname(path), self.info['FILE'])
|
|
||||||
self.parent = self.info.get('RECEIVED_PARENT_UUID')
|
|
||||||
self.is_restored = False
|
|
||||||
|
|
||||||
def _parse_info(self):
|
|
||||||
config = {}
|
|
||||||
with open(self.info_file, 'r') as fh:
|
|
||||||
# skip command option line
|
|
||||||
for line in fh.readlines():
|
|
||||||
if '=' not in line:
|
|
||||||
continue
|
|
||||||
key, val = line.strip().split('=', maxsplit=1)
|
|
||||||
config[key] = val
|
|
||||||
return config
|
|
||||||
|
|
||||||
def get_transformers(self):
|
|
||||||
if 'encrypt' in self.info:
|
|
||||||
if self.info['encrypt'] == 'gpg':
|
|
||||||
raise NotImplementedError('gpg encryption')
|
|
||||||
elif self.info['encrypt'] == 'openssl_enc':
|
|
||||||
yield TransformOpensslDecrypt()
|
|
||||||
else:
|
|
||||||
raise Exception(f'unknown encryption type: "{self.info["encrypt"]}"')
|
|
||||||
if 'compress' in self.info:
|
|
||||||
yield TransformDecompress(self.info['compress'])
|
|
||||||
|
|
||||||
def restore_file(self, options):
|
|
||||||
assert self.info.get('TYPE') == 'raw'
|
|
||||||
assert not self.info.get('INCOMPLETE')
|
|
||||||
pipeline = BtrfsPipeline(self)
|
|
||||||
for transformer in self.get_transformers():
|
|
||||||
pipeline.append(transformer)
|
|
||||||
if options.dry_run:
|
|
||||||
print(pipeline.get_cmd(options))
|
|
||||||
else:
|
|
||||||
logger.info(f"restoring backup {os.path.basename(self.data_file)}")
|
|
||||||
pipeline.run(options)
|
|
||||||
self.is_restored = True
|
|
||||||
|
|
||||||
|
|
||||||
def restore_from_path(backup, options):
|
|
||||||
path = os.path.dirname(backup)
|
|
||||||
info_files = {}
|
|
||||||
backup_file = BackupFile(backup + '.info')
|
|
||||||
restored_files = set()
|
|
||||||
for entry in os.scandir(path):
|
|
||||||
if entry.is_file() and entry.name.endswith('.info'):
|
|
||||||
info = BackupFile(entry.path)
|
|
||||||
info_files[info.uuid] = info
|
|
||||||
restored_files.update(restore_backup(backup_file, info_files, options))
|
|
||||||
logger.info(f"finished; restored {len(restored_files)} backup files")
|
|
||||||
|
|
||||||
|
|
||||||
def restore_backup(bfile, parents, options):
|
|
||||||
if bfile.is_restored:
|
|
||||||
return
|
|
||||||
if bfile.parent:
|
|
||||||
parent = parents.get(bfile.parent)
|
|
||||||
if not parent:
|
|
||||||
msg = (f"missing parent {bfile.parent} for"
|
|
||||||
f"'{os.path.basename(bfile.info_file)}'")
|
|
||||||
if options.ignore_missing:
|
|
||||||
logger.warning(msg)
|
|
||||||
else:
|
|
||||||
raise Exception(msg)
|
|
||||||
else:
|
|
||||||
yield from restore_backup(parent, parents, options)
|
|
||||||
bfile.restore_file(options)
|
|
||||||
yield bfile.uuid
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description="restore btrbk raw backup")
|
|
||||||
parser.add_argument('backup', help="backup file to restore; for incremental"
|
|
||||||
" backups the parent files must be in the same directory")
|
|
||||||
parser.add_argument('restore_dir', help="target directory for restored subvolumes"
|
|
||||||
" (path argument for \"btrfs receive\")")
|
|
||||||
parser.add_argument('-n', '--dry-run', action='store_true',
|
|
||||||
help="print commands that would be executed")
|
|
||||||
parser.add_argument('--ignore-missing', action='store_true',
|
|
||||||
help="do not fail on missing parent snapshots")
|
|
||||||
|
|
||||||
for transformer in TRANSFORMERS:
|
|
||||||
transformer.add_parser_options(parser)
|
|
||||||
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
if args.dry_run:
|
|
||||||
logger.setLevel('ERROR')
|
|
||||||
|
|
||||||
restore_from_path(args.backup, args)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
logger.setLevel('INFO')
|
|
||||||
logging.basicConfig(format='%(asctime)s %(levelname)s - %(message)s')
|
|
||||||
main()
|
|
|
@ -110,8 +110,8 @@ have to create a run-time (rw) snapshot before booting into it:
|
||||||
How do I convert '/' (subvolid=5) into a subvolume?
|
How do I convert '/' (subvolid=5) into a subvolume?
|
||||||
---------------------------------------------------
|
---------------------------------------------------
|
||||||
|
|
||||||
There's several ways to achieve this, the solution described below
|
There's several ways to achieve this, the solution described below is
|
||||||
guarantees not to create new files (extents) on disk.
|
that it guarantees not to create new files (extents) on disk.
|
||||||
|
|
||||||
### Step 1: make a snapshot of your root filesystem
|
### Step 1: make a snapshot of your root filesystem
|
||||||
|
|
||||||
|
@ -191,7 +191,7 @@ What is the most efficient way to clone btrfs storage?
|
||||||
------------------------------------------------------
|
------------------------------------------------------
|
||||||
|
|
||||||
It is very common (and avisable!) to keep backups on a separate
|
It is very common (and avisable!) to keep backups on a separate
|
||||||
location. In some situations, it is also required to transport the
|
location. In some situations, is is also required to transport the
|
||||||
data physically, either to the datacenter or to your safe in the
|
data physically, either to the datacenter or to your safe in the
|
||||||
basement.
|
basement.
|
||||||
|
|
||||||
|
|
24
doc/Makefile
24
doc/Makefile
|
@ -1,4 +1,5 @@
|
||||||
DOCS = FAQ.md
|
DOCS = FAQ.md \
|
||||||
|
upgrade_to_v0.23.0.md
|
||||||
MAN_MAN1 = btrbk.1 \
|
MAN_MAN1 = btrbk.1 \
|
||||||
lsbtr.1 \
|
lsbtr.1 \
|
||||||
ssh_filter_btrbk.1
|
ssh_filter_btrbk.1
|
||||||
|
@ -17,19 +18,8 @@ ifeq ($(COMPRESS), yes)
|
||||||
endif
|
endif
|
||||||
|
|
||||||
# convert using "asciidoctor": <https://asciidoctor.org>
|
# convert using "asciidoctor": <https://asciidoctor.org>
|
||||||
# fallback to "a2x" from asciidoc package: <http://asciidoc.org>
|
ASCIIDOCTOR_MANPAGE = asciidoctor -d manpage -b manpage
|
||||||
ifneq (, $(shell command -v asciidoctor 2> /dev/null))
|
ASCIIDOCTOR_HTML = asciidoctor -b html5 -d article
|
||||||
ASCIIDOC_MANPAGE = asciidoctor -d manpage -b manpage
|
|
||||||
ASCIIDOC_HTML = asciidoctor -b html5 -d article
|
|
||||||
else ifneq (, $(shell command -v a2x 2> /dev/null))
|
|
||||||
# NOTE: using -L (--no-xmllint), as xmllint is a separate package on many distros.
|
|
||||||
ASCIIDOC_MANPAGE = a2x -L -d manpage -f manpage
|
|
||||||
ASCIIDOC_HTML = asciidoc -b html -d article
|
|
||||||
else
|
|
||||||
ASCIIDOC_ERR = $(error "please install either asciidoc or asciidoctor")
|
|
||||||
ASCIIDOC_MANPAGE = $(ASCIIDOC_ERR)
|
|
||||||
ASCIIDOC_HTML = $(ASCIIDOC_ERR)
|
|
||||||
endif
|
|
||||||
|
|
||||||
# reproducible builds: reference date is ":date:" attribute from asciidoc source
|
# reproducible builds: reference date is ":date:" attribute from asciidoc source
|
||||||
date_attr = $(shell sed -rn 's/:date:\s*//p' $(1))
|
date_attr = $(shell sed -rn 's/:date:\s*//p' $(1))
|
||||||
|
@ -60,10 +50,10 @@ clean:
|
||||||
gzip -9 -n -c $< > $@
|
gzip -9 -n -c $< > $@
|
||||||
|
|
||||||
%.1 : %.1.asciidoc
|
%.1 : %.1.asciidoc
|
||||||
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_MANPAGE) $<
|
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_MANPAGE) -o $@ $<
|
||||||
|
|
||||||
%.5 : %.5.asciidoc
|
%.5 : %.5.asciidoc
|
||||||
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_MANPAGE) $<
|
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_MANPAGE) -o $@ $<
|
||||||
|
|
||||||
%.html : %.asciidoc
|
%.html : %.asciidoc
|
||||||
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOC_HTML) -o $@ $<
|
SOURCE_DATE_EPOCH=$(call source_date_epoch,$<) $(ASCIIDOCTOR_HTML) -o $@ $<
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
btrbk(1)
|
btrbk(1)
|
||||||
========
|
========
|
||||||
:date: 2023-03-25
|
:date: 2021-03-21
|
||||||
:release-version: 0.32.6
|
:release-version: 0.31.2
|
||||||
:man manual: Btrbk Manual
|
:man manual: Btrbk Manual
|
||||||
:man source: Btrbk {release-version}
|
:man source: Btrbk {release-version}
|
||||||
|
|
||||||
|
@ -20,12 +20,11 @@ btrbk [-h|--help] [--version]
|
||||||
[-c|--config <file>] [-n|--dry-run] [--exclude <filter>]
|
[-c|--config <file>] [-n|--dry-run] [--exclude <filter>]
|
||||||
[-p|--preserve] [--preserve-snapshots] [--preserve-backups]
|
[-p|--preserve] [--preserve-snapshots] [--preserve-backups]
|
||||||
[-v|--verbose] [-q|--quiet] [-l|--loglevel <level>]
|
[-v|--verbose] [-q|--quiet] [-l|--loglevel <level>]
|
||||||
[-t|--table] [-L|--long] [-1|--single-column]
|
[-t|--table] [-L|--long] [--format <output-format>] [--pretty]
|
||||||
[--format <output-format>] [--pretty]
|
|
||||||
[-S|--print-schedule] [--progress]
|
[-S|--print-schedule] [--progress]
|
||||||
[--lockfile <file>]
|
[--lockfile <file>]
|
||||||
[--override <config_option>=<value>]
|
[--override <config_option>=<value>]
|
||||||
<command> [[--] <filter>...]
|
<command> [<args>]
|
||||||
|
|
||||||
|
|
||||||
DESCRIPTION
|
DESCRIPTION
|
||||||
|
@ -45,27 +44,26 @@ btrbk.conf(5) for more details.
|
||||||
|
|
||||||
=== Snapshots and Backups
|
=== Snapshots and Backups
|
||||||
|
|
||||||
Snapshots as well as backup subvolumes are created in the form:
|
Snapshots as well as backup subvolumes are created in form:
|
||||||
|
|
||||||
<snapshot-name>.<timestamp>[_N]
|
<snapshot-name>.<timestamp>[_N]
|
||||||
|
|
||||||
Where '<snapshot-name>' is identical to the source subvolume name,
|
Where '<snapshot-name>' is identical to the source subvolume name,
|
||||||
unless the configuration option 'snapshot_name' is set. '<timestamp>'
|
unless the configuration option 'snapshot_name' is set. The
|
||||||
is a timestamp describing the creation time (local time of the host
|
<timestamp> is either "YYYYMMDD" or "YYYYMMDDThhmm" (dependent of the
|
||||||
running btrbk) of the snapshot/backup. The format can be configured
|
'timestamp_format' configuration option), where "YYYY" is the year,
|
||||||
using the 'timestamp_format' option, refer to btrbk.conf(5) for
|
"MM" is the month, "DD" is the day, "hh" is the hour and "mm" is the
|
||||||
details. If multiple snapshots/backups are created on the same
|
minute of the creation time (local time of the host running btrbk). If
|
||||||
date/time, 'N' will be incremented on each snapshot, starting at 1.
|
multiple snapshots/backups are created on the same date/time, N will
|
||||||
|
be incremented on each snapshot, starting at 1.
|
||||||
|
|
||||||
If a snapshot or backup does not match the naming scheme above
|
If a snapshot or backup does not match the naming scheme above
|
||||||
(e.g. if it has been renamed manually), btrbk will leave it untouched.
|
(i.e. if it has been renamed manually), btrbk will leave it untouched.
|
||||||
|
|
||||||
Note that in btrfs terminology, a 'snapshot' is a ``subvolume with
|
Note that 'snapshot' is a btrfs terminology for a ``read-only
|
||||||
a given initial content of the original subvolume'' (showing a
|
subvolume'' (showing a parent-uuid, see btrfs-subvolume(8)); 'backup'
|
||||||
parent-uuid, see btrfs-subvolume(8)), and they can be read-write
|
is a btrbk terminology for a ``read-only subvolume created with
|
||||||
(default) or read-only. In btrbk terminology, 'snapshot' means
|
send/receive'' (showing a received-uuid).
|
||||||
``read-only btrfs snapshot'', and 'backup' means ``read-only subvolume
|
|
||||||
created with send/receive'' (showing a received-uuid).
|
|
||||||
|
|
||||||
|
|
||||||
OPTIONS
|
OPTIONS
|
||||||
|
@ -88,15 +86,8 @@ OPTIONS
|
||||||
commands that would be executed.
|
commands that would be executed.
|
||||||
|
|
||||||
--exclude <filter>::
|
--exclude <filter>::
|
||||||
Exclude configured sections matching '<filter>' (see
|
Exclude configured sections matching '<filter>'. See
|
||||||
<<_filter_statements,FILTER STATEMENTS>> below), or any specific
|
<<_filter_statements,FILTER STATEMENTS>> below.
|
||||||
snapshot from being backuped or deleted, or any specific backup
|
|
||||||
from being deleted.
|
|
||||||
+
|
|
||||||
Note that excluding specific snapshots from being backuped has impact
|
|
||||||
on scheduling: e.g. if the "first snapshot of the day" is excluded,
|
|
||||||
the "second snapshot of the day" shifts to "first", creating a backup
|
|
||||||
as "first backup of the day".
|
|
||||||
|
|
||||||
-p, --preserve::
|
-p, --preserve::
|
||||||
Preserve all snapshots and backups. Skips deletion of any
|
Preserve all snapshots and backups. Skips deletion of any
|
||||||
|
@ -112,7 +103,7 @@ as "first backup of the day".
|
||||||
specified in the configuration file.
|
specified in the configuration file.
|
||||||
|
|
||||||
--wipe::
|
--wipe::
|
||||||
Ignore configured snapshot retention policy, delete all but the latest
|
Ignore configured snapshot retention policy, delete all but latest
|
||||||
snapshots instead. All snapshots needed for incremental backup
|
snapshots instead. All snapshots needed for incremental backup
|
||||||
(latest common) are also preserved. Useful if you are getting low
|
(latest common) are also preserved. Useful if you are getting low
|
||||||
on disk space (ENOSPC).
|
on disk space (ENOSPC).
|
||||||
|
@ -135,15 +126,12 @@ as "first backup of the day".
|
||||||
-L, --long::
|
-L, --long::
|
||||||
Print output in long table format (shortcut for "--format=long").
|
Print output in long table format (shortcut for "--format=long").
|
||||||
|
|
||||||
-1, --single-column::
|
|
||||||
Print output as single column (not available for all commands).
|
|
||||||
|
|
||||||
--format table|long|raw|col:[h:]<columns>::
|
--format table|long|raw|col:[h:]<columns>::
|
||||||
Print output in specified format. If set to "raw", prints
|
Print output in specified format. If set to "raw", prints
|
||||||
space-separated, quoted key=value pairs (machine readable).
|
space-separated key="value" pairs (machine readable).
|
||||||
+
|
+
|
||||||
If set to "col:", prints only the <columns> specified (comma-separated
|
If set to "col:", prints only the <columns> specified (comma-separated
|
||||||
list). Header lines are omitted if the "h:" modifier is present.
|
list). Header lines are ommitted if the "h:" modifier is present.
|
||||||
Columns prefixed with "-" are collapsed if empty. Columns postfixed
|
Columns prefixed with "-" are collapsed if empty. Columns postfixed
|
||||||
with ":RALIGN" are right-aligned.
|
with ":RALIGN" are right-aligned.
|
||||||
|
|
||||||
|
@ -161,9 +149,9 @@ with ":RALIGN" are right-aligned.
|
||||||
command (version >= 20180505) installed on the host running btrbk.
|
command (version >= 20180505) installed on the host running btrbk.
|
||||||
|
|
||||||
--lockfile <file>::
|
--lockfile <file>::
|
||||||
Place an exclusive lock on <file> during program execution, using
|
Create lockfile <file> on startup; checks lockfile before running
|
||||||
flock(2). If the lock is held by another process, exit before
|
any btrfs commands (using perl "flock"), and exits if the lock is
|
||||||
running any actions. Overrides configuration option
|
held by another btrbk instance. Overrides configuration option
|
||||||
"lockfile". Ignored on dryrun ('-n', '--dry-run').
|
"lockfile". Ignored on dryrun ('-n', '--dry-run').
|
||||||
|
|
||||||
--override <config_option>=<value>::
|
--override <config_option>=<value>::
|
||||||
|
@ -222,9 +210,8 @@ configuring the retention policy.
|
||||||
snapshots that are not preserved by their configured retention
|
snapshots that are not preserved by their configured retention
|
||||||
policy will be deleted. Note that the latest snapshot (the one
|
policy will be deleted. Note that the latest snapshot (the one
|
||||||
created in step 1) as well as the latest snapshot/backup pair are
|
created in step 1) as well as the latest snapshot/backup pair are
|
||||||
always preserved, regardless of the retention policy. If any
|
always preserved, regardless of the retention policy.
|
||||||
target is unreachable or has errors, all snapshots are preserved
|
|
||||||
in order not to break the incremental chain.
|
|
||||||
|
|
||||||
*dryrun* [filter...]::
|
*dryrun* [filter...]::
|
||||||
Don't run any btrfs commands that would alter the filesystem, just
|
Don't run any btrfs commands that would alter the filesystem, just
|
||||||
|
@ -429,7 +416,8 @@ execution times and high ram usage. Consider setting 'cache_dir'.
|
||||||
lsbtr(1).
|
lsbtr(1).
|
||||||
|
|
||||||
*config* print|print-all::
|
*config* print|print-all::
|
||||||
Prints the parsed configuration file.
|
Prints the parsed configuration file. Use the '--format' command
|
||||||
|
line option to switch between different output formats.
|
||||||
|
|
||||||
|
|
||||||
FILTER STATEMENTS
|
FILTER STATEMENTS
|
||||||
|
@ -480,9 +468,6 @@ Note that for *run* and *snapshot* commands, a filter matching a
|
||||||
surrounding 'subvolume' section. If this is not desired, consider
|
surrounding 'subvolume' section. If this is not desired, consider
|
||||||
running *snapshot* and *resume* commands separately.
|
running *snapshot* and *resume* commands separately.
|
||||||
|
|
||||||
Filter statements can match multiple times (e.g. on group as well as
|
|
||||||
host name). In such a case, all matches are processed.
|
|
||||||
|
|
||||||
|
|
||||||
FILES
|
FILES
|
||||||
-----
|
-----
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
btrbk.conf(5)
|
btrbk.conf(5)
|
||||||
=============
|
=============
|
||||||
:date: 2023-03-25
|
:date: 2021-03-21
|
||||||
:release-version: 0.32.6
|
:release-version: 0.31.2
|
||||||
:man manual: Btrbk Manual
|
:man manual: Btrbk Manual
|
||||||
:man source: Btrbk {release-version}
|
:man source: Btrbk {release-version}
|
||||||
|
|
||||||
|
@ -36,21 +36,34 @@ global options must be set before any sections are defined.
|
||||||
Blank lines are ignored. A hash character (#) starts a comment
|
Blank lines are ignored. A hash character (#) starts a comment
|
||||||
extending until end of line.
|
extending until end of line.
|
||||||
|
|
||||||
|
Whitespace or unicode characters are not allowed for file
|
||||||
|
names. Allowed characters are:
|
||||||
|
|
||||||
|
[0-9] [a-z] [A-Z] and "._+-@"
|
||||||
|
|
||||||
|
This is for sanity/safety/security reasons, we apologize for the
|
||||||
|
inconvenience.
|
||||||
|
|
||||||
|
|
||||||
SECTIONS
|
SECTIONS
|
||||||
--------
|
--------
|
||||||
|
|
||||||
*volume* <volume-directory>|<url> (optional)::
|
*volume* <volume-directory>|<url>::
|
||||||
Absolute path pointing to a btrfs file system containing the
|
Directory of a btrfs volume containing the source subvolume(s) to
|
||||||
source subvolume(s) to be backed up. Usually the mount point of a
|
be backed up. '<volume-directory>' must be an absolute path and
|
||||||
btrfs filesystem mounted with the 'subvolid=5' option.
|
point to a btrfs volume (or subvolume). Usually the mount point of
|
||||||
|
a btrfs filesystem mounted with the 'subvolid=5' option.
|
||||||
|
|
||||||
*subvolume* <subvolume-name>::
|
*subvolume* <subvolume-name>::
|
||||||
Subvolume to be backed up, relative to the '<volume-directory>' of
|
Subvolume to be backed up, relative to the '<volume-directory>'
|
||||||
the 'volume' section, or absolute if the 'volume' section is
|
specified in the 'volume' section. Multiple 'subvolume' sections
|
||||||
omitted. Accepts wildcard character "*".
|
are allowed within 'volume' sections. Accepts wildcard character
|
||||||
|
"*".
|
||||||
+
|
+
|
||||||
--
|
--
|
||||||
|
If set to ".", the subvolume at '<volume-directory>' is used as backup
|
||||||
|
source, and the snapshots will be created within the source subvolume
|
||||||
|
itself (see 'snapshot_dir' option below), which is not recommended.
|
||||||
Note that if this subvolume is btrfs root (id=5), it needs to have a
|
Note that if this subvolume is btrfs root (id=5), it needs to have a
|
||||||
valid UUID, which is not the case for file systems created with
|
valid UUID, which is not the case for file systems created with
|
||||||
btrfs-progs < 4.16.
|
btrfs-progs < 4.16.
|
||||||
|
@ -96,7 +109,7 @@ otherwise.
|
||||||
|
|
||||||
*timestamp_format* short|long|long-iso::
|
*timestamp_format* short|long|long-iso::
|
||||||
Timestamp format used as postfix for new snapshot subvolume
|
Timestamp format used as postfix for new snapshot subvolume
|
||||||
names. Defaults to ``long''.
|
names. Defaults to ``short''.
|
||||||
+
|
+
|
||||||
--
|
--
|
||||||
ifndef::backend-docbook,backend-manpage[]
|
ifndef::backend-docbook,backend-manpage[]
|
||||||
|
@ -116,13 +129,12 @@ snapshots are created during a daylight saving time clock
|
||||||
change).
|
change).
|
||||||
+
|
+
|
||||||
Note that using ``long-iso'' has implications on the scheduling, see
|
Note that using ``long-iso'' has implications on the scheduling, see
|
||||||
<<_reference_time,Reference Time>> below.
|
<<_retention_policy,RETENTION POLICY>> (caveats) below.
|
||||||
|
|
||||||
*snapshot_dir* <directory>::
|
*snapshot_dir* <directory>::
|
||||||
Directory in which the btrfs snapshots are created, relative to
|
Directory in which the btrfs snapshots are created, relative to
|
||||||
'<volume-directory>' of the 'volume' section, or absolute if the
|
'<volume-directory>' of the 'volume' section. Note that btrbk does
|
||||||
'volume' section is omitted. Note that btrbk does not
|
not automatically create this directory, and the snapshot creation
|
||||||
automatically create this directory, and the snapshot creation
|
|
||||||
will fail if it is not present.
|
will fail if it is not present.
|
||||||
|
|
||||||
*snapshot_name* <basename>::
|
*snapshot_name* <basename>::
|
||||||
|
@ -185,9 +197,10 @@ not be certain about this, such operations are disallowed in
|
||||||
Defines after what time (in full hours since midnight) a
|
Defines after what time (in full hours since midnight) a
|
||||||
snapshot/backup is considered to be a "daily" backup. Daily,
|
snapshot/backup is considered to be a "daily" backup. Daily,
|
||||||
weekly, monthly and yearly backups are preserved on this hour (see
|
weekly, monthly and yearly backups are preserved on this hour (see
|
||||||
<<_retention_policy,RETENTION POLICY>> below). Ignored on
|
<<_retention_policy,RETENTION POLICY>> below). If you set this
|
||||||
snapshots or backups without time information ('timestamp_format
|
option, make sure to also set 'timestamp_format' to ``long'' or
|
||||||
short'). Defaults to ``0''.
|
``long-iso'' (backups and snapshots having no time information
|
||||||
|
will ignore this option). Defaults to ``0''.
|
||||||
|
|
||||||
*snapshot_preserve* no|<retention_policy>::
|
*snapshot_preserve* no|<retention_policy>::
|
||||||
Set retention policy for snapshots (see
|
Set retention policy for snapshots (see
|
||||||
|
@ -242,29 +255,32 @@ set to ``all'' (the default).
|
||||||
|
|
||||||
=== SSH Options
|
=== SSH Options
|
||||||
|
|
||||||
*ssh_identity* <file>|no::
|
*ssh_identity* <file>::
|
||||||
Absolute path to a ssh identity file (private key). If not set,
|
Absolute path to a ssh identity file (private key). Note that if
|
||||||
the ssh default is used (see ssh(1), "-i identity_file"). Note
|
the private key is password protected, btrbk will prompt for user
|
||||||
that if the identity key is password protected and no
|
input, which is usually not desired.
|
||||||
authentication agent is used, btrbk will prompt for user input on
|
|
||||||
every connection attempt.
|
|
||||||
|
|
||||||
*ssh_user* <username>|no::
|
*ssh_user* <username>::
|
||||||
Remote username for ssh. Defaults to ``root''. Make sure the
|
Remote username for ssh. Defaults to ``root''. Make sure the
|
||||||
remote user is able to run "btrfs" with root privileges (see
|
remote user is able to run "btrfs" with root privileges (see
|
||||||
option 'backend' for details). If set to ``no'', the ssh default
|
option 'backend' for details).
|
||||||
is used.
|
|
||||||
|
|
||||||
*ssh_compression* yes|no::
|
*ssh_compression* yes|no::
|
||||||
Enables or disables the compression of ssh connections. Defaults
|
Enables or disables the compression of ssh connections. Defaults
|
||||||
to ``no''. Note that if *stream_compress* is enabled, ssh
|
to ``no''. Note that if *stream_compress* is enabled, ssh
|
||||||
compression will always be disabled for send/receive operations.
|
compression will always be disabled for send/receive operations.
|
||||||
|
|
||||||
*ssh_cipher_spec* default|<cipher_spec>::
|
*ssh_cipher_spec* <cipher_spec>::
|
||||||
Selects the cipher specification for encrypting the session
|
Selects the cipher specification for encrypting the session
|
||||||
(comma-separated list of ciphers in order of preference). See the
|
(comma-separated list of ciphers in order of preference). See the
|
||||||
"-c cipher_spec" option in ssh(1) for more information. Defaults
|
"-c cipher_spec" option in ssh(1) for more information. Defaults
|
||||||
to ``default'' (the ciphers specified in ssh_config(5)).
|
to ``default'' (the ciphers specified in 'ssh_config').
|
||||||
|
|
||||||
|
Previous versions btrbk allowed you to set a *ssh_port* option, this
|
||||||
|
has been dropped in favor of the `ssh://hostname:port` notation in the
|
||||||
|
'volume' and 'target' <<_sections,sections>>. If you want to set a
|
||||||
|
global port for all SSH connections to remote hosts, set the ``Port''
|
||||||
|
option in ssh_config(5).
|
||||||
|
|
||||||
|
|
||||||
=== Data Stream Options
|
=== Data Stream Options
|
||||||
|
@ -274,7 +290,7 @@ set to ``all'' (the default).
|
||||||
remote locations. Defaults to ``no''. If enabled, make sure that
|
remote locations. Defaults to ``no''. If enabled, make sure that
|
||||||
'<compress_command>' is available on the source and target
|
'<compress_command>' is available on the source and target
|
||||||
hosts. Supported '<compress_command>': gzip, pigz, bzip2, pbzip2,
|
hosts. Supported '<compress_command>': gzip, pigz, bzip2, pbzip2,
|
||||||
bzip3, xz, lzo, lz4, zstd.
|
xz, lzo, lz4, zstd.
|
||||||
|
|
||||||
*stream_compress_level* default|<number>::
|
*stream_compress_level* default|<number>::
|
||||||
Compression level for the specified '<compress_command>'. Refer to
|
Compression level for the specified '<compress_command>'. Refer to
|
||||||
|
@ -289,11 +305,7 @@ set to ``all'' (the default).
|
||||||
|
|
||||||
*stream_compress_threads* default|<number>::
|
*stream_compress_threads* default|<number>::
|
||||||
Number of threads to use for <compress_command>. Only supported
|
Number of threads to use for <compress_command>. Only supported
|
||||||
for "pigz", "pbzip2", "bzip3", "zstd" and recent versions of "xz".
|
for "pigz", "pbzip2", "zstd" and recent versions of "xz".
|
||||||
|
|
||||||
*stream_compress_adapt* yes|no::
|
|
||||||
Enable adaptive compression for <compress_command>. Only supported
|
|
||||||
for "zstd" (version >= 1.3.6). Defaults to ``no''.
|
|
||||||
|
|
||||||
*stream_buffer* <size>|no::
|
*stream_buffer* <size>|no::
|
||||||
Add a buffer to the btrfs send stream (locally, on uncompressed
|
Add a buffer to the btrfs send stream (locally, on uncompressed
|
||||||
|
@ -352,21 +364,21 @@ constraints.
|
||||||
in a space-separated table format: "localtime type status
|
in a space-separated table format: "localtime type status
|
||||||
target_url source_url parent_url message".
|
target_url source_url parent_url message".
|
||||||
|
|
||||||
*transaction_syslog* <facility>|no::
|
*transaction_syslog* <facility>|no::
|
||||||
If set, all transactions (as described in 'transaction_log' above)
|
If set, all transactions (as described in 'transaction_log' above)
|
||||||
are logged to syslog. The program name used in the messages is
|
are logged to syslog. The program name used in the messages is
|
||||||
"btrbk". Accepted parameters for '<facility>': user, mail,
|
"btrbk". Accepted parameters for '<facility>': user, mail,
|
||||||
daemon, auth, lpr, news, cron, authpriv, local0..local7.
|
daemon, auth, lpr, news, cron, authpriv, local0..local7.
|
||||||
|
|
||||||
*lockfile* <file>|no::
|
*lockfile* <file>|no::
|
||||||
Place an exclusive lock on <file> during program execution, using
|
Create lockfile <file> on startup; checks lockfile before running
|
||||||
flock(2). If the lock is held by another process, exit before
|
any btrfs commands (using perl "flock"), and exits if the lock is
|
||||||
running any actions. Ignored on dryrun ('-n', '--dry-run'). See
|
held by another btrbk instance. Ignored on dryrun ('-n',
|
||||||
also '--lockfile' command-line option.
|
'--dry-run'). See also '--lockfile' command-line option.
|
||||||
|
|
||||||
*backend* <backend>::
|
*backend* btrfs-progs|btrfs-progs-btrbk|btrfs-progs-sudo::
|
||||||
Backend filesystem utilities to be used for btrfs specific
|
Backend filesystem utilities to be used for btrfs specific
|
||||||
operations. Available backends:
|
operations. Defaults to ``btrfs-progs''.
|
||||||
+
|
+
|
||||||
--
|
--
|
||||||
*btrfs-progs*::
|
*btrfs-progs*::
|
||||||
|
@ -386,9 +398,6 @@ constraints.
|
||||||
have appropriate (root) permissions for the "btrfs" command groups
|
have appropriate (root) permissions for the "btrfs" command groups
|
||||||
as well as the "readlink" and "test" commands in /etc/sudoers.
|
as well as the "readlink" and "test" commands in /etc/sudoers.
|
||||||
|
|
||||||
*btrfs-progs-doas*::
|
|
||||||
Similar to btrfs-progs-sudo, using prefix "doas -n".
|
|
||||||
|
|
||||||
If you want to set this option for local or remote hosts only, set
|
If you want to set this option for local or remote hosts only, set
|
||||||
*backend_local* or *backend_remote* (e.g. "backend_remote
|
*backend_local* or *backend_remote* (e.g. "backend_remote
|
||||||
btrfs-progs-btrbk").
|
btrfs-progs-btrbk").
|
||||||
|
@ -397,29 +406,12 @@ If you want to set this option for regular (non-root) user only, set
|
||||||
*backend_local_user*.
|
*backend_local_user*.
|
||||||
--
|
--
|
||||||
|
|
||||||
*compat* <compat-option>...::
|
*compat* busybox|no::
|
||||||
Enable compatibility options. Available 'compat-option':
|
If set to ``busybox'', use busybox compatible commands. Defaults
|
||||||
+
|
to ``no''.
|
||||||
--
|
If you want to set this option for local or remote hosts only, you
|
||||||
*busybox*::
|
can set *compat_local* or *compat_remote*
|
||||||
Use busybox compatible commands, at the expense of slight overhead
|
(e.g. "compat_remote busybox").
|
||||||
while reading filesystem information.
|
|
||||||
|
|
||||||
*ignore_receive_errors* _*experimental*_::
|
|
||||||
Tell btrfs-receive(8) to not terminate on errors by setting
|
|
||||||
"--max-errors=0" option. Print warnings instead.
|
|
||||||
+
|
|
||||||
A known use case for this are target hosts lacking xattr support
|
|
||||||
(e.g. some Synology NAS), while the send-stream contains "lsetxattr"
|
|
||||||
commands. Another case is targets failing to set otime, complaining
|
|
||||||
with "ERROR: attribute 12 requested but not present".
|
|
||||||
+
|
|
||||||
Note that there is *no guarantee that backups created with this
|
|
||||||
option enabled can be restored at all*.
|
|
||||||
|
|
||||||
If you want to set this option for local or remote hosts only, set
|
|
||||||
*compat_local* or *compat_remote* (e.g. "compat_remote busybox").
|
|
||||||
--
|
|
||||||
|
|
||||||
*cache_dir* <directory>::
|
*cache_dir* <directory>::
|
||||||
If set, cache extent maps for the "btrbk extents" command.
|
If set, cache extent maps for the "btrbk extents" command.
|
||||||
|
@ -427,74 +419,28 @@ If you want to set this option for local or remote hosts only, set
|
||||||
|
|
||||||
=== Btrfs Specific Options
|
=== Btrfs Specific Options
|
||||||
|
|
||||||
*incremental_prefs* <list-spec>[:<amount>]...::
|
*btrfs_commit_delete* after|each|no::
|
||||||
Specify the preferences to determine the best common (correlated)
|
If set, make sure the deletion of snapshot and backup subvolumes
|
||||||
parent and clone sources for incremental backups, by choosing from
|
are committed to disk when btrbk terminates. Defaults to ``no''.
|
||||||
predefined candidate lists.
|
|
||||||
+
|
|
||||||
--
|
|
||||||
The 'list-spec' defines from what candidate list the next
|
|
||||||
parent/clone-src should be appended to a result list; 'amount' defines
|
|
||||||
how many (e.g. "sro:1 sro:1" is identical to "sro:2"), or all if
|
|
||||||
omitted. Any candidate which is already in the results is dropped.
|
|
||||||
|
|
||||||
The resulting list of subvolumes is then used as parameters for the
|
*incremental_clones* <number>::
|
||||||
btrfs-send(8) command: the first for "-p <parent>", all others for
|
Maximum number of clone sources allowed for incremental send. If
|
||||||
"-c <clone-src>".
|
set, btrbk adds "-c <clone-src>" to the btrfs-send(8) command for
|
||||||
|
all present snapshot/backup pairs (correlated subvolumes matching
|
||||||
Available 'list-spec' (candidate lists = filtered subsets of
|
matching 'received_uuid', printed by "btrbk stats"). Set this to a
|
||||||
correlated subvolumes):
|
high number if you want to make sure that no common data is missed
|
||||||
|
on incremental backups, in expense of btrfs-send
|
||||||
*sro*,*srn*:: All from 'snapshot_dir' matching 'snapshot_name', with
|
performance. Defaults to 0.
|
||||||
parent_uuid relationship, sorted by btrbk timestamp (o=older
|
|
||||||
n=newer).
|
|
||||||
|
|
||||||
*sao*,*san*:: All from 'snapshot_dir' matching 'snapshot_name', sorted
|
|
||||||
by btrbk timestamp (o=older n=newer).
|
|
||||||
|
|
||||||
*aro*,*arn*:: All from 'incremental_resolve', with parent_uuid
|
|
||||||
relationship, sorted by cgen (o=older n=newer).
|
|
||||||
|
|
||||||
Defaults to "sro:1 srn:1 sao:1 san:1 aro:1 arn:1". Note that for
|
|
||||||
most operations the default resolves a single parent, as there usually
|
|
||||||
are no newer snapshots, and all "sro:1 sao:1 aro:1" resolve to the
|
|
||||||
same snapshot.
|
|
||||||
|
|
||||||
Example: "defaults,sao,san,aro,arn" takes the defaults, and adds clone
|
|
||||||
sources for all (!) known candidates on the filesystem.
|
|
||||||
--
|
|
||||||
|
|
||||||
*incremental_clones* yes|no::
|
|
||||||
If enabled, btrbk adds "-c <clone-src>" to the btrfs-send(8)
|
|
||||||
command for all correlated subvolumes resolved by
|
|
||||||
'incremental_prefs'. If disabled, only "-p <parent>" is
|
|
||||||
used. Defaults to ``yes''.
|
|
||||||
|
|
||||||
*incremental_resolve* mountpoint|directory::
|
*incremental_resolve* mountpoint|directory::
|
||||||
Specifies where to search for the best common parent for
|
Specifies where to search for the best common parent for
|
||||||
incremental backups. If set to ``mountpoint'', use parents in the
|
incremental backups. If set to ``mountpoint'', use parents in the
|
||||||
filesystem tree below the mount point of the snapshot and target
|
filesystem tree below mount points of source
|
||||||
directory. If set to ``directory'', use parents strictly below
|
"<volume-directory>/<snapshot-dir>" and target
|
||||||
snapshot/target directories. Set this to ``directory'' if you get
|
"<target-directory>". If set to ``directory'', use parents
|
||||||
access problems (when not running btrbk as root). Defaults to
|
strictly below source/target directories. Set this to
|
||||||
``mountpoint''.
|
``directory'' if you get access problems (when not running btrbk
|
||||||
|
as root). Defaults to ``mountpoint''.
|
||||||
*btrfs_commit_delete* yes|no::
|
|
||||||
If set, wait for the transaction commit at the end of each
|
|
||||||
snapshot or backup deletion (sets '--commit-each' option for
|
|
||||||
"btrfs subvolume delete"). Defaults to ``no''.
|
|
||||||
|
|
||||||
*send_protocol* <number>|no _*experimental*_::
|
|
||||||
Use btrfs send protocol version N. If enabled on 'target', btrbk
|
|
||||||
adds "--proto <number>" to the btrfs-send(8) command. Defaults to
|
|
||||||
``no'' (btrfs default).
|
|
||||||
|
|
||||||
*send_compressed_data* yes|no _*experimental*_::
|
|
||||||
Send data that is compressed on the filesystem directly without
|
|
||||||
decompressing it. This requires protocol version 2 or higher
|
|
||||||
(btrfs-progs >= 5.19), and implies "send_protocol 2". If enabled
|
|
||||||
on 'target', btrbk adds "--compressed-data" to the btrfs-send(8)
|
|
||||||
command. Defaults to ``no'' (btrfs default).
|
|
||||||
|
|
||||||
*snapshot_qgroup_destroy* yes|no _*experimental*_:: {blank}
|
*snapshot_qgroup_destroy* yes|no _*experimental*_:: {blank}
|
||||||
*target_qgroup_destroy* yes|no _*experimental*_:: {blank}
|
*target_qgroup_destroy* yes|no _*experimental*_:: {blank}
|
||||||
|
@ -505,39 +451,30 @@ sources for all (!) known candidates on the filesystem.
|
||||||
<https://bugzilla.kernel.org/show_bug.cgi?id=91751>
|
<https://bugzilla.kernel.org/show_bug.cgi?id=91751>
|
||||||
|
|
||||||
|
|
||||||
=== Informative Options
|
|
||||||
|
|
||||||
*warn_unknown_targets* yes|no::
|
|
||||||
If set, prints a warning if btrbk encounters a target subvolume at
|
|
||||||
a unknown location (i.e. not following btrbk naming scheme, or
|
|
||||||
outside the target directory). Defaults to ``no''.
|
|
||||||
|
|
||||||
|
|
||||||
RETENTION POLICY
|
RETENTION POLICY
|
||||||
----------------
|
----------------
|
||||||
|
|
||||||
Retention policies are defined individually for snapshots, backups and
|
btrbk uses separate retention policies for snapshots and backups,
|
||||||
archives (summarized as "backups" in the following text), using a
|
which are defined by the 'snapshot_preserve_min', 'snapshot_preserve',
|
||||||
combination of:
|
'target_preserve_min', 'target_preserve', 'preserve_day_of_week' and
|
||||||
|
'preserve_hour_of_day' configuration options.
|
||||||
|
|
||||||
**_preserve_min* all|latest|no|<number>{h,d,w,m,y}::
|
Within this section, any statement about "backups" is always valid for
|
||||||
Amount of time (duration) in which all backups are preserved.
|
backups as well as snapshots, referring to 'target_preserve' or
|
||||||
|
'snapshot_preserve' respectively.
|
||||||
**_preserve* no|<retention_policy>::
|
|
||||||
Schedule (single points in time) for which individual backups are
|
|
||||||
preserved.
|
|
||||||
|
|
||||||
Note that if "preserve_min" is set to ``all'' (the default), any
|
|
||||||
setting of "preserve" obviously has no effect.
|
|
||||||
|
|
||||||
The format for '<retention_policy>' is:
|
The format for '<retention_policy>' is:
|
||||||
|
|
||||||
[<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]
|
[<hourly>h] [<daily>d] [<weekly>w] [<monthly>m] [<yearly>y]
|
||||||
|
|
||||||
|
With the following semantics:
|
||||||
|
|
||||||
*hourly*::
|
*hourly*::
|
||||||
Defines how many hours back hourly backups should be
|
Defines how many hours back hourly backups should be
|
||||||
preserved. The first backup of an hour is considered an hourly
|
preserved. The first backup of an hour is considered an hourly
|
||||||
backup.
|
backup. Note that if you use <hourly> scheduling, make sure to
|
||||||
|
also set 'timestamp_format' to ``long'' or ``long-iso'', or the
|
||||||
|
scheduler will interpret the time as "00:00" (midnight).
|
||||||
|
|
||||||
*daily*::
|
*daily*::
|
||||||
Defines how many days back daily backups should be preserved. The
|
Defines how many days back daily backups should be preserved. The
|
||||||
|
@ -564,31 +501,29 @@ Use an asterisk for ``all'' (e.g. "target_preserve 60d *m"
|
||||||
states: "preserve daily backups for 60 days back, and all monthly
|
states: "preserve daily backups for 60 days back, and all monthly
|
||||||
backups").
|
backups").
|
||||||
|
|
||||||
|
The reference time (which defines the beginning of a day, week, month
|
||||||
|
or year) for all date/time calculations is the local time of the host
|
||||||
|
running btrbk.
|
||||||
|
|
||||||
Hint: Run btrbk with the '-S', '--print-schedule' option to get a
|
Hint: Run btrbk with the '-S', '--print-schedule' option to get a
|
||||||
comprehensive output of the scheduler results.
|
comprehensive output of the scheduler results.
|
||||||
|
|
||||||
|
Caveats:
|
||||||
|
|
||||||
=== Reference Time
|
* If you run a setup with several btrbk instances (e.g. one
|
||||||
|
snapshot-only instance per remote client, and a separate fetch-only
|
||||||
|
instance on the backup server), it makes perfectly sense to run
|
||||||
|
btrbk with different local time on the clients, in order to make
|
||||||
|
sure the backups from all the remote hosts are preserved for
|
||||||
|
"midnight", and not at "00:00 UTC" (which would be "14:00" in
|
||||||
|
Honolulu). If you want this behavior, do NOT use "timestamp_format
|
||||||
|
long-iso".
|
||||||
|
|
||||||
The local time on the host running btrbk defines the reference time
|
* If "timestamp_format long-iso" is set, running btrbk from different
|
||||||
for all date/time calculations, especially for "beginning of a day",
|
time zones leads to different interpretation of "first in day, week,
|
||||||
and as a consequence for the first daily, weekly, monthly or yearly
|
month, or year". Make sure to run btrbk with the same time zone on
|
||||||
backups. The local time on remote hosts (ssh source/target) is never
|
every host, e.g. by setting the TZ environment variable (see
|
||||||
used.
|
tzset(3)).
|
||||||
|
|
||||||
Unless "timestamp_format long-iso" is set, daily backups are preserved
|
|
||||||
at "preserve_hour_of_day" (defaults to midnight) of the respective
|
|
||||||
time zone (and not for "00:00 UTC", which would be "14:00" in
|
|
||||||
Honolulu). This becomes relevant for setups with multiple btrbk
|
|
||||||
instances, e.g. many snapshot-only instances (spread around the
|
|
||||||
world), and a fetch-only instance on the backup server.
|
|
||||||
|
|
||||||
Caveat:
|
|
||||||
|
|
||||||
* If "timestamp_format long-iso" is set, each btrbk instance on has a
|
|
||||||
different interpretation of "first in day". Make sure to run btrbk
|
|
||||||
with the same time zone on every host, e.g. by setting the TZ
|
|
||||||
environment variable (see tzset(3)).
|
|
||||||
|
|
||||||
|
|
||||||
TARGET TYPES
|
TARGET TYPES
|
||||||
|
@ -597,19 +532,24 @@ TARGET TYPES
|
||||||
*send-receive*::
|
*send-receive*::
|
||||||
Backup to a btrfs filesystem, using "btrfs send/receive". This is
|
Backup to a btrfs filesystem, using "btrfs send/receive". This is
|
||||||
the recommended (standard) target type. The '<target-directory>'
|
the recommended (standard) target type. The '<target-directory>'
|
||||||
must be an absolute path and point to a subvolume or directory
|
must be an absolute path and point to a btrfs volume (or
|
||||||
within a btrfs file system. See btrfs-send(8), btrfs-receive(8).
|
subvolume), or to a directory within a subvolume. See
|
||||||
|
btrfs-send(8), btrfs-receive(8).
|
||||||
|
|
||||||
*raw* _*experimental*_::
|
*raw* _*experimental*_::
|
||||||
Backup to a raw (filesystem independent) file from the output of
|
Backup to a raw (filesystem independent) file from the output of
|
||||||
btrfs-send(8), with optional compression and encryption.
|
btrfs-send(8), with optional compression and encryption.
|
||||||
+
|
+
|
||||||
--
|
--
|
||||||
|
Note that the target preserve mechanism is currently disabled for
|
||||||
|
incremental raw backups (btrbk does not delete any incremental raw
|
||||||
|
files)!
|
||||||
|
|
||||||
Raw backups consist of two files: the main data file containing the
|
Raw backups consist of two files: the main data file containing the
|
||||||
btrfs send stream, and a sidecar file ".info" containing metadata:
|
btrfs send stream, and a sidecar file ".info" containing metadata:
|
||||||
|
|
||||||
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg]
|
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg]
|
||||||
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|...][.gpg].info
|
<snapshot-name>.<timestamp>[_N].btrfs[.gz|.bz2|.xz][.gpg].info
|
||||||
|
|
||||||
For 'incremental' backups ("incremental yes"), please note that:
|
For 'incremental' backups ("incremental yes"), please note that:
|
||||||
|
|
||||||
|
@ -620,11 +560,6 @@ For 'incremental' backups ("incremental yes"), please note that:
|
||||||
make sure that a non-incremental backup is triggered from time to
|
make sure that a non-incremental backup is triggered from time to
|
||||||
time.
|
time.
|
||||||
|
|
||||||
* The scheduler will never delete dependent parents of backups
|
|
||||||
preserved by the retention policy (run btrbk with the '-S',
|
|
||||||
'--print-schedule' option to get a comprehensive output of the
|
|
||||||
scheduler results).
|
|
||||||
|
|
||||||
* There is currently no support for rotation of incremental backups:
|
* There is currently no support for rotation of incremental backups:
|
||||||
if 'incremental' is set, a full backup must be triggered manually
|
if 'incremental' is set, a full backup must be triggered manually
|
||||||
from time to time in order to be able to delete old backups.
|
from time to time in order to be able to delete old backups.
|
||||||
|
@ -634,8 +569,8 @@ Additional options for raw targets:
|
||||||
|
|
||||||
*raw_target_compress* <compress_command>|no::
|
*raw_target_compress* <compress_command>|no::
|
||||||
Compression algorithm to use for raw backup target. Supported
|
Compression algorithm to use for raw backup target. Supported
|
||||||
'<compress_command>': gzip, pigz, bzip2, pbzip2, bzip3, xz, lzo,
|
'<compress_command>': gzip, pigz, bzip2, pbzip2, xz, lzo, lz4,
|
||||||
lz4, zstd.
|
zstd.
|
||||||
*raw_target_compress_level* default|<number>::
|
*raw_target_compress_level* default|<number>::
|
||||||
Compression level for the specified <compress_command>.
|
Compression level for the specified <compress_command>.
|
||||||
*raw_target_compress_long* default|<number>::
|
*raw_target_compress_long* default|<number>::
|
||||||
|
@ -656,7 +591,7 @@ Additional options for "raw_target_encrypt gpg":
|
||||||
|
|
||||||
*gpg_keyring* <file>::
|
*gpg_keyring* <file>::
|
||||||
Keyring to use for gpg, e.g. "`/etc/btrbk/gpg/pubring.kbx`".
|
Keyring to use for gpg, e.g. "`/etc/btrbk/gpg/pubring.kbx`".
|
||||||
*gpg_recipient* <name>...::
|
*gpg_recipient* <name>::
|
||||||
Encrypt for user id '<name>' (email address).
|
Encrypt for user id '<name>' (email address).
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -9,18 +9,16 @@ file, choose one of the following methods:
|
||||||
|
|
||||||
### Generic Linux System
|
### Generic Linux System
|
||||||
|
|
||||||
Install [asciidoctor] or [asciidoc] if you want to build the
|
|
||||||
documentation.
|
|
||||||
|
|
||||||
Download and unpack the latest [btrbk source tarball] and type:
|
Download and unpack the latest [btrbk source tarball] and type:
|
||||||
|
|
||||||
sudo make install
|
sudo make install
|
||||||
|
|
||||||
#### Try latest master from Github:
|
|
||||||
|
|
||||||
wget https://raw.githubusercontent.com/digint/btrbk/master/btrbk
|
### Gentoo Linux
|
||||||
chmod +x btrbk
|
|
||||||
sudo ./btrbk ls /
|
btrbk is in portage:
|
||||||
|
|
||||||
|
emerge app-backup/btrbk
|
||||||
|
|
||||||
|
|
||||||
### Debian Based Distros
|
### Debian Based Distros
|
||||||
|
@ -32,7 +30,7 @@ Packages are also available via NeuroDebian: http://neuro.debian.net/pkgs/btrbk.
|
||||||
|
|
||||||
### Fedora Linux
|
### Fedora Linux
|
||||||
|
|
||||||
btrbk is in the official Fedora repos: https://src.fedoraproject.org/rpms/btrbk
|
btrbk is in the official Fedora repos: https://apps.fedoraproject.org/packages/btrbk
|
||||||
|
|
||||||
sudo dnf install btrbk
|
sudo dnf install btrbk
|
||||||
|
|
||||||
|
@ -49,13 +47,6 @@ btrbk is in the community repository
|
||||||
apk add btrbk
|
apk add btrbk
|
||||||
|
|
||||||
|
|
||||||
### Gentoo Linux
|
|
||||||
|
|
||||||
btrbk is in portage:
|
|
||||||
|
|
||||||
emerge app-backup/btrbk
|
|
||||||
|
|
||||||
|
|
||||||
### Void Linux
|
### Void Linux
|
||||||
|
|
||||||
btrbk is in Void's `current` repository
|
btrbk is in Void's `current` repository
|
||||||
|
@ -64,5 +55,3 @@ btrbk is in Void's `current` repository
|
||||||
|
|
||||||
|
|
||||||
[btrbk source tarball]: https://digint.ch/download/btrbk/releases/
|
[btrbk source tarball]: https://digint.ch/download/btrbk/releases/
|
||||||
[asciidoctor]: https://asciidoctor.org
|
|
||||||
[asciidoc]: https://asciidoc.org
|
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
lsbtr(1)
|
lsbtr(1)
|
||||||
========
|
========
|
||||||
:date: 2023-03-25
|
:date: 2021-03-21
|
||||||
:release-version: 0.32.6
|
:release-version: 0.31.2
|
||||||
:man manual: Btrbk Manual
|
:man manual: Btrbk Manual
|
||||||
:man source: Btrbk {release-version}
|
:man source: Btrbk {release-version}
|
||||||
|
|
||||||
|
@ -17,12 +17,11 @@ SYNOPSIS
|
||||||
|
|
||||||
[verse]
|
[verse]
|
||||||
lsbtr [-h|--help] [--version]
|
lsbtr [-h|--help] [--version]
|
||||||
[-l|--long] [-u|--uuid] [-1|--single-column] [--raw]
|
[-l|--long] [-u|--uuid] [--raw]
|
||||||
[--format <output-format>]
|
|
||||||
[-v|--verbose]
|
[-v|--verbose]
|
||||||
[-c|--config <file>]
|
[-c|--config <file>]
|
||||||
[--override <config_option>=<value>]
|
[--override <config_option>=<value>]
|
||||||
[[--] <path>|<url>...]
|
[<path>|<url>...]
|
||||||
|
|
||||||
|
|
||||||
DESCRIPTION
|
DESCRIPTION
|
||||||
|
@ -56,21 +55,9 @@ OPTIONS
|
||||||
-u, --uuid::
|
-u, --uuid::
|
||||||
Print UUID table (parent/received relations).
|
Print UUID table (parent/received relations).
|
||||||
|
|
||||||
-1, --single-column::
|
|
||||||
Print path column only (delimited by newline).
|
|
||||||
|
|
||||||
--raw::
|
--raw::
|
||||||
Print space-separated key="value" pairs (machine readable).
|
Print space-separated key="value" pairs (machine readable).
|
||||||
|
|
||||||
--format table|long|raw|col:[h:]<columns>::
|
|
||||||
Print output in specified format. If set to "raw", prints
|
|
||||||
space-separated key="value" pairs (machine readable).
|
|
||||||
+
|
|
||||||
If set to "col:", prints only the <columns> specified (comma-separated
|
|
||||||
list). Header lines are omitted if the "h:" modifier is present.
|
|
||||||
Columns prefixed with "-" are collapsed if empty. Columns postfixed
|
|
||||||
with ":RALIGN" are right-aligned.
|
|
||||||
|
|
||||||
-v, --verbose::
|
-v, --verbose::
|
||||||
Increase the level of verbosity.
|
Increase the level of verbosity.
|
||||||
|
|
||||||
|
|
|
@ -1,7 +1,7 @@
|
||||||
ssh_filter_btrbk(1)
|
ssh_filter_btrbk(1)
|
||||||
===================
|
===================
|
||||||
:date: 2023-03-25
|
:date: 2021-03-21
|
||||||
:release-version: 0.32.6
|
:release-version: 0.31.2
|
||||||
:man manual: Btrbk Manual
|
:man manual: Btrbk Manual
|
||||||
:man source: Btrbk {release-version}
|
:man source: Btrbk {release-version}
|
||||||
|
|
||||||
|
@ -39,13 +39,13 @@ The following commands are always allowed:
|
||||||
- "readlink"
|
- "readlink"
|
||||||
- "test -d" (only if "compat busybox" configuration option is set)
|
- "test -d" (only if "compat busybox" configuration option is set)
|
||||||
- "cat /proc/self/mountinfo"
|
- "cat /proc/self/mountinfo"
|
||||||
- pipes through "gzip", "pigz", "bzip2", "pbzip2", "bzip3", "xz",
|
- pipes through "gzip", "pigz", "bzip2", "pbzip2", "xz", "lzop",
|
||||||
"lzop", "lz4", "zstd" (stream_compress)
|
"lz4", "zstd" (stream_compress)
|
||||||
- pipes through "mbuffer" (stream_buffer, rate_limit)
|
- pipes through "mbuffer" (stream_buffer, rate_limit)
|
||||||
|
|
||||||
Example line in /root/.ssh/authorized_keys on a backup target host:
|
Example line in /root/.ssh/authorized_keys on a backup target host:
|
||||||
|
|
||||||
command="ssh_filter_btrbk.sh --target --delete --restrict-path /mnt/btr_backup",restrict ssh-rsa AAAAB3NzaC1...hwumXFRQBL btrbk@example.org
|
command="ssh_filter_btrbk.sh --target --delete --restrict-path /mnt/btr_backup" ssh-rsa AAAAB3NzaC1...hwumXFRQBL btrbk@mydomain.com
|
||||||
|
|
||||||
|
|
||||||
OPTIONS
|
OPTIONS
|
||||||
|
@ -82,11 +82,6 @@ OPTIONS
|
||||||
-p, --restrict-path <path>::
|
-p, --restrict-path <path>::
|
||||||
Restrict commands to <path>. Note that "btrfs subvolume show",
|
Restrict commands to <path>. Note that "btrfs subvolume show",
|
||||||
"btrfs subvolume list" are NOT affected by this option.
|
"btrfs subvolume list" are NOT affected by this option.
|
||||||
+
|
|
||||||
It is not possible to restrict commands to exact subvolume names, as
|
|
||||||
btrfs-receive(8) takes a <path> as argument (directory, not including
|
|
||||||
the subvolume file name to be created, this is encoded in the
|
|
||||||
send-stream).
|
|
||||||
|
|
||||||
-l, --log::
|
-l, --log::
|
||||||
Log ACCEPT and REJECT messages to the system log.
|
Log ACCEPT and REJECT messages to the system log.
|
||||||
|
|
|
@ -0,0 +1,92 @@
|
||||||
|
Upgrading to btrbk-v0.23.0
|
||||||
|
==========================
|
||||||
|
|
||||||
|
In order to keep btrbk simple and intuitive while adding new features,
|
||||||
|
it became inevitable to change the semantics of the "retention policy"
|
||||||
|
related configuration options.
|
||||||
|
|
||||||
|
|
||||||
|
What has changed?
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
### Preserve *first* instead of *last* snapshot/backup
|
||||||
|
|
||||||
|
btrbk used to *always* transfer the latest snapshot to the target
|
||||||
|
location, while considering the *last* snapshot/backup of a day as a
|
||||||
|
daily backup (and also the last weekly as a monthly). This made it
|
||||||
|
very cumbersome when running btrbk in a cron job as well as manually,
|
||||||
|
because the last manually created snapshot was immediately transferred
|
||||||
|
on every run, and used as the daily backup (instead of the one created
|
||||||
|
periodically by the cron job).
|
||||||
|
|
||||||
|
The new semantics are to consider the *first* (instead of *last*)
|
||||||
|
snapshot of a hour/day/week/month as the one to be preserved, while
|
||||||
|
only transferring the snapshots needed to satisfy the target retention
|
||||||
|
policy.
|
||||||
|
|
||||||
|
|
||||||
|
### Preserve snapshots for a minimum amount of time
|
||||||
|
|
||||||
|
In order to specify a minimum amount of time in which *all* snapshots
|
||||||
|
should be preserved, the new "snapshot_preserve_min" and
|
||||||
|
"target_preserve_min" configuration options were introduced. This was
|
||||||
|
previously covered by "snapshot_preserve_daily", which caused a lot of
|
||||||
|
confusion among users.
|
||||||
|
|
||||||
|
|
||||||
|
Upgrading the configuration file: /etc/btrbk/btrbk.conf
|
||||||
|
-------------------------------------------------------
|
||||||
|
|
||||||
|
Please read the description of the "run" command in [btrbk(1)], as
|
||||||
|
well as the "RETENTION POLICY" section in [btrbk.conf(5)] for a
|
||||||
|
detailed description. Make sure to understand the new concept, and run
|
||||||
|
`btrbk --print-schedule dryrun` after updating the configuration.
|
||||||
|
|
||||||
|
|
||||||
|
### Upgrade retention policy
|
||||||
|
|
||||||
|
If you want the same behaviour as before:
|
||||||
|
|
||||||
|
# replace this:
|
||||||
|
snapshot_preserve_daily <daily>
|
||||||
|
snapshot_preserve_weekly <weekly>
|
||||||
|
snapshot_preserve_monthly <monthly>
|
||||||
|
|
||||||
|
# with:
|
||||||
|
snapshot_preserve_min <daily>d
|
||||||
|
snapshot_preserve <weekly>w <monthly>m
|
||||||
|
|
||||||
|
# ... do the same with "target_preserve_*" options
|
||||||
|
|
||||||
|
|
||||||
|
But what you probably want is something like:
|
||||||
|
|
||||||
|
snapshot_preserve_min 5d
|
||||||
|
snapshot_preserve <daily>d <weekly>w <monthly>m
|
||||||
|
|
||||||
|
target_preserve_min no
|
||||||
|
target_preserve <daily>d <weekly>w <monthly>m *y
|
||||||
|
|
||||||
|
This states:
|
||||||
|
|
||||||
|
* Keep all snapshots for five days (no matter how many there are)
|
||||||
|
* Transfer only the first snapshot of a day to the target
|
||||||
|
* Keep all "first snapshots of a day" for `<daily>` days, etc.
|
||||||
|
|
||||||
|
|
||||||
|
### Upgrade "resume_missing"
|
||||||
|
|
||||||
|
If you have a line: "resume_missing yes" somwhere in your config,
|
||||||
|
simply remove it. btrbk always resumes missing backups.
|
||||||
|
|
||||||
|
If you have "resume_missing no", you can imitate this behaviour by
|
||||||
|
setting:
|
||||||
|
|
||||||
|
target_preserve_min latest
|
||||||
|
target_preserve no
|
||||||
|
|
||||||
|
This states: "always transfer the latest snapshot to the target".
|
||||||
|
|
||||||
|
|
||||||
|
[btrbk(1)]: https://digint.ch/btrbk/doc/btrbk.1.html
|
||||||
|
[btrbk.conf(5)]: https://digint.ch/btrbk/doc/btrbk.conf.5.html
|
|
@ -1,11 +1,9 @@
|
||||||
#!/bin/sh
|
#!/bin/bash
|
||||||
|
|
||||||
# initialise and sanitise the shell execution environment
|
set -e
|
||||||
unset -v IFS
|
set -u
|
||||||
export LC_ALL=C
|
|
||||||
export PATH='/sbin:/bin:/usr/sbin:/usr/bin'
|
|
||||||
|
|
||||||
set -e -u
|
export PATH=/sbin:/bin:/usr/sbin:/usr/bin
|
||||||
|
|
||||||
enable_log=
|
enable_log=
|
||||||
restrict_path_list=
|
restrict_path_list=
|
||||||
|
@ -14,70 +12,55 @@ allow_exact_list=
|
||||||
allow_rate_limit=1
|
allow_rate_limit=1
|
||||||
allow_stream_buffer=1
|
allow_stream_buffer=1
|
||||||
allow_compress=1
|
allow_compress=1
|
||||||
compress_list='gzip|pigz|bzip2|pbzip2|bzip3|xz|lzop|lz4|zstd'
|
compress_list="gzip|pigz|bzip2|pbzip2|xz|lzop|lz4|zstd"
|
||||||
|
|
||||||
# note that the backslash is NOT a metacharacter in a POSIX bracket expression!
|
# note that the backslash is NOT a metacharacter in a POSIX bracket expression!
|
||||||
option_match='-[a-zA-Z0-9=-]+' # matches short as well as long options
|
option_match='-[a-zA-Z0-9=-]+' # matches short as well as long options
|
||||||
file_match_sane='/[0-9a-zA-Z_@+./-]*' # matches file path (equal to ${file_match} in btrbk < 0.32.0)
|
file_match='[0-9a-zA-Z_@+./-]*' # matches file path (equal to $file_match in btrbk)
|
||||||
file_match="/[^']*" # btrbk >= 0.32.0 quotes file arguments: match all but single quote
|
|
||||||
file_arg_match="('${file_match}'|${file_match_sane})" # support btrbk < 0.32.0
|
|
||||||
|
|
||||||
log_cmd()
|
log_cmd()
|
||||||
{
|
{
|
||||||
local priority="$1"
|
if [[ -n "$enable_log" ]]; then
|
||||||
local authorisation_decision="$2"
|
logger -p $1 -t ssh_filter_btrbk.sh "$2 (Name: ${LOGNAME:-<unknown>}; Remote: ${SSH_CLIENT:-<unknown>})${3:+: $3}: $SSH_ORIGINAL_COMMAND"
|
||||||
local reason="${3-}"
|
|
||||||
|
|
||||||
if [ -n "${enable_log}" ]; then
|
|
||||||
logger -p "${priority}" -t ssh_filter_btrbk.sh "${authorisation_decision} (Name: ${LOGNAME:-<unknown>}; Connection: ${SSH_CONNECTION:-<unknown>})${reason:+: ${reason}}: ${SSH_ORIGINAL_COMMAND}"
|
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
allow_cmd()
|
allow_cmd()
|
||||||
{
|
{
|
||||||
local cmd="$1"
|
allow_list="${allow_list}|$1"
|
||||||
|
|
||||||
allow_list="${allow_list}|${cmd}"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
allow_exact_cmd()
|
allow_exact_cmd()
|
||||||
{
|
{
|
||||||
local cmd="$1"
|
allow_exact_list="${allow_exact_list}|$1"
|
||||||
|
|
||||||
allow_exact_list="${allow_exact_list}|${cmd}"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
reject_and_die()
|
reject_and_die()
|
||||||
{
|
{
|
||||||
local reason="$1"
|
local reason=$1
|
||||||
|
log_cmd "auth.err" "btrbk REJECT" "$reason"
|
||||||
log_cmd 'auth.err' 'btrbk REJECT' "${reason}"
|
echo "ERROR: ssh_filter_btrbk.sh: ssh command rejected: $reason: $SSH_ORIGINAL_COMMAND" 1>&2
|
||||||
printf 'ERROR: ssh_filter_btrbk.sh: ssh command rejected: %s: %s\n' "${reason}" "${SSH_ORIGINAL_COMMAND}" >&2
|
|
||||||
exit 255
|
exit 255
|
||||||
}
|
}
|
||||||
|
|
||||||
run_cmd()
|
run_cmd()
|
||||||
{
|
{
|
||||||
log_cmd 'auth.info' 'btrbk ACCEPT'
|
log_cmd "auth.info" "btrbk ACCEPT"
|
||||||
eval " ${SSH_ORIGINAL_COMMAND}"
|
eval " $SSH_ORIGINAL_COMMAND"
|
||||||
}
|
}
|
||||||
|
|
||||||
reject_filtered_cmd()
|
reject_filtered_cmd()
|
||||||
{
|
{
|
||||||
if [ -n "${restrict_path_list}" ]; then
|
if [[ -n "$restrict_path_list" ]]; then
|
||||||
# match any of restrict_path_list,
|
# match any of restrict_path_list with or without trailing slash,
|
||||||
# or any file/directory (matching file_match) below restrict_path
|
# or any file/directory (matching file_match) below restrict_path
|
||||||
path_match="'(${restrict_path_list})(${file_match})?'"
|
path_match="(${restrict_path_list})(/${file_match})?"
|
||||||
path_match_legacy="(${restrict_path_list})(${file_match_sane})?"
|
|
||||||
else
|
else
|
||||||
# match any absolute file/directory (matching file_match)
|
# match any absolute file/directory (matching file_match)
|
||||||
path_match="'${file_match}'"
|
path_match="/${file_match}"
|
||||||
path_match_legacy="${file_match_sane}"
|
|
||||||
fi
|
fi
|
||||||
# btrbk >= 0.32.0 quotes files, allow both (legacy)
|
|
||||||
path_match="(${path_match}|${path_match_legacy})"
|
|
||||||
|
|
||||||
if [ -n "${allow_compress}" ]; then
|
if [[ -n "$allow_compress" ]]; then
|
||||||
decompress_match="(${compress_list}) -d -c( -[pT][0-9]+)?"
|
decompress_match="(${compress_list}) -d -c( -[pT][0-9]+)?"
|
||||||
compress_match="(${compress_list}) -c( -[0-9])?( -[pT][0-9]+)?"
|
compress_match="(${compress_list}) -c( -[0-9])?( -[pT][0-9]+)?"
|
||||||
else
|
else
|
||||||
|
@ -87,8 +70,8 @@ reject_filtered_cmd()
|
||||||
|
|
||||||
# rate_limit_remote and stream_buffer_remote use combined
|
# rate_limit_remote and stream_buffer_remote use combined
|
||||||
# "mbuffer" as of btrbk-0.29.0
|
# "mbuffer" as of btrbk-0.29.0
|
||||||
if [ -n "${allow_stream_buffer}" ] || [ -n "${allow_rate_limit}" ]; then
|
if [[ -n "$allow_stream_buffer" ]] || [[ -n "$allow_rate_limit" ]]; then
|
||||||
mbuffer_match='mbuffer -v 1 -q( -s [0-9]+[kmgKMG]?)?( -m [0-9]+[kmgKMG]?)?( -[rR] [0-9]+[kmgtKMGT]?)?'
|
mbuffer_match="mbuffer -v 1 -q( -s [0-9]+[kmgKMG]?)?( -m [0-9]+[kmgKMG]?)?( -[rR] [0-9]+[kmgtKMGT]?)?"
|
||||||
else
|
else
|
||||||
mbuffer_match=
|
mbuffer_match=
|
||||||
fi
|
fi
|
||||||
|
@ -98,44 +81,35 @@ reject_filtered_cmd()
|
||||||
stream_in_match="(${decompress_match} \| )?(${mbuffer_match} \| )?"
|
stream_in_match="(${decompress_match} \| )?(${mbuffer_match} \| )?"
|
||||||
stream_out_match="( \| ${mbuffer_match})?( \| ${compress_match}$)?"
|
stream_out_match="( \| ${mbuffer_match})?( \| ${compress_match}$)?"
|
||||||
|
|
||||||
# `grep`’s `-q`-option is not used as it may cause an exit status of `0` even
|
|
||||||
# when an error occurred.
|
|
||||||
|
|
||||||
allow_stream_match="^${stream_in_match}${allow_cmd_match}${stream_out_match}"
|
allow_stream_match="^${stream_in_match}${allow_cmd_match}${stream_out_match}"
|
||||||
if printf '%s' "${SSH_ORIGINAL_COMMAND}" | grep -E "${allow_stream_match}" >/dev/null 2>/dev/null; then
|
if [[ $SSH_ORIGINAL_COMMAND =~ $allow_stream_match ]] ; then
|
||||||
return 0
|
return 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
exact_cmd_match="^(${allow_exact_list})$";
|
exact_cmd_match="^(${allow_exact_list})$";
|
||||||
if printf '%s' "${SSH_ORIGINAL_COMMAND}" | grep -E "${exact_cmd_match}" >/dev/null 2>/dev/null; then
|
if [[ $SSH_ORIGINAL_COMMAND =~ $exact_cmd_match ]] ; then
|
||||||
return 0
|
return 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
local formatted_restrict_path_list="$(printf '%s' "${restrict_path_list}" | sed 's/|/", "/g')"
|
reject_and_die "disallowed command${restrict_path_list:+ (restrict-path: \"${restrict_path_list//|/\", \"}\")}"
|
||||||
reject_and_die "disallowed command${restrict_path_list:+ (restrict-path: \"${formatted_restrict_path_list}\")}"
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
# check for "--sudo" option before processing other options
|
# check for "--sudo" option before processing other options
|
||||||
sudo_prefix=
|
sudo_prefix=
|
||||||
for key in "$@"; do
|
for key; do
|
||||||
if [ "${key}" = '--sudo' ]; then
|
[[ "$key" == "--sudo" ]] && sudo_prefix="sudo -n "
|
||||||
sudo_prefix='sudo -n '
|
|
||||||
fi
|
|
||||||
if [ "${key}" = '--doas' ]; then
|
|
||||||
sudo_prefix='doas -n '
|
|
||||||
fi
|
|
||||||
done
|
done
|
||||||
|
|
||||||
while [ "$#" -ge 1 ]; do
|
while [[ "$#" -ge 1 ]]; do
|
||||||
key="$1"
|
key="$1"
|
||||||
|
|
||||||
case "${key}" in
|
case $key in
|
||||||
-l|--log)
|
-l|--log)
|
||||||
enable_log=1
|
enable_log=1
|
||||||
;;
|
;;
|
||||||
|
|
||||||
--sudo|--doas)
|
--sudo)
|
||||||
# already processed above
|
# already processed above
|
||||||
;;
|
;;
|
||||||
|
|
||||||
|
@ -180,7 +154,7 @@ while [ "$#" -ge 1 ]; do
|
||||||
;;
|
;;
|
||||||
|
|
||||||
*)
|
*)
|
||||||
printf 'ERROR: ssh_filter_btrbk.sh: failed to parse command line option: %s\n' "${key}" >&2
|
echo "ERROR: ssh_filter_btrbk.sh: failed to parse command line option: $key" 1>&2
|
||||||
exit 255
|
exit 255
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
@ -189,21 +163,19 @@ done
|
||||||
|
|
||||||
# NOTE: subvolume queries are NOT affected by "--restrict-path":
|
# NOTE: subvolume queries are NOT affected by "--restrict-path":
|
||||||
# btrbk also calls show/list on the mount point of the subvolume
|
# btrbk also calls show/list on the mount point of the subvolume
|
||||||
allow_exact_cmd "${sudo_prefix}btrfs subvolume (show|list)( ${option_match})* ${file_arg_match}";
|
allow_exact_cmd "${sudo_prefix}btrfs subvolume (show|list)( ${option_match})* ${file_match}";
|
||||||
allow_cmd "${sudo_prefix}readlink" # resolve symlink
|
allow_cmd "${sudo_prefix}readlink" # resolve symlink
|
||||||
allow_exact_cmd "${sudo_prefix}test -d ${file_arg_match}" # check directory (only for compat=busybox)
|
allow_exact_cmd "${sudo_prefix}test -d ${file_match}" # check directory (only for compat=busybox)
|
||||||
allow_exact_cmd 'cat /proc/self/mountinfo' # resolve mountpoints
|
allow_exact_cmd "cat /proc/self/mountinfo" # resolve mountpoints
|
||||||
allow_exact_cmd 'cat /proc/self/mounts' # legacy, for btrbk < 0.27.0
|
allow_exact_cmd "cat /proc/self/mounts" # legacy, for btrbk < 0.27.0
|
||||||
|
|
||||||
# remove leading "|" on alternation lists
|
# remove leading "|" on alternation lists
|
||||||
allow_list="${allow_list#\|}"
|
allow_list=${allow_list#\|}
|
||||||
allow_exact_list="${allow_exact_list#\|}"
|
allow_exact_list=${allow_exact_list#\|}
|
||||||
restrict_path_list="${restrict_path_list#\|}"
|
restrict_path_list=${restrict_path_list#\|}
|
||||||
|
|
||||||
case "${SSH_ORIGINAL_COMMAND}" in
|
case "$SSH_ORIGINAL_COMMAND" in
|
||||||
*\.\./*) reject_and_die 'directory traversal' ;;
|
*\.\./*) reject_and_die 'directory traversal' ;;
|
||||||
*'
|
|
||||||
'*) reject_and_die 'unsafe character LF' ;;
|
|
||||||
*\$*) reject_and_die 'unsafe character "$"' ;;
|
*\$*) reject_and_die 'unsafe character "$"' ;;
|
||||||
*\&*) reject_and_die 'unsafe character "&"' ;;
|
*\&*) reject_and_die 'unsafe character "&"' ;;
|
||||||
*\(*) reject_and_die 'unsafe character "("' ;;
|
*\(*) reject_and_die 'unsafe character "("' ;;
|
||||||
|
@ -212,7 +184,7 @@ case "${SSH_ORIGINAL_COMMAND}" in
|
||||||
*\<*) reject_and_die 'unsafe character "<"' ;;
|
*\<*) reject_and_die 'unsafe character "<"' ;;
|
||||||
*\>*) reject_and_die 'unsafe character ">"' ;;
|
*\>*) reject_and_die 'unsafe character ">"' ;;
|
||||||
*\`*) reject_and_die 'unsafe character "`"' ;;
|
*\`*) reject_and_die 'unsafe character "`"' ;;
|
||||||
*\|*) [ -n "${allow_compress}" ] || [ -n "${allow_rate_limit}" ] || [ -n "${allow_stream_buffer}" ] || reject_and_die 'unsafe character "|"' ;;
|
*\|*) [[ -n "$allow_compress" ]] || [[ -n "$allow_rate_limit" ]] || [[ -n "$allow_stream_buffer" ]] || reject_and_die 'unsafe character "|"' ;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
reject_filtered_cmd
|
reject_filtered_cmd
|
||||||
|
|
Loading…
Reference in New Issue