ambiser
/
btrbk
mirror of https://github.com/digint/btrbk
9
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'digint:master' into master

pull/547/head
Korbinian Pöppel 2023-10-04 09:22:00 +02:00 committed by GitHub
parent aeffe002fe 18ddc65979
commit c0f11cb96b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 45 additions and 162 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
.github/workflows/codespell.yml vendored Normal file
View File

@ -0,0 +1,26 @@
---
name: Codespell
on:
push:
branches: [master]
pull_request:
branches: [master]
permissions:
contents: read
jobs:
codespell:
name: Check for spelling errors
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Codespell
uses: codespell-project/actions-codespell@v2
with:
check_filenames: true
skip: ".git,*.pdf,*.svg"
ignore_words_list: uptodate

8
ChangeLog
View File

@ -47,7 +47,7 @@ btrbk-0.32.1
btrbk-0.32.0
* MIGRATION
- If timestamp_format is not configured, explicitely set
- If timestamp_format is not configured, explicitly set
"timestamp_format short" to revert old behavior.
- Update ssh_filter_btrbk.sh on remote hosts.
* Change default for timestamp_format to "long".
@ -81,7 +81,7 @@ btrbk-0.31.2
* MIGRATION
- Update ssh_filter_btrbk.sh on remote hosts.
* ssh_filter_btrbk.sh: Fix security vulnerability.
Specialy crafted commands may be executed without being propely
Specially crafted commands may be executed without being properly
checked. Applies to remote hosts filtering ssh commands using
ssh_filter_btrbk.sh in authorized_keys.
* Warn if no subvolume defined in config (close #378).
@ -248,7 +248,7 @@ btrbk-0.27.0
- Allow snapshot_dir to be a mountpoint.
- Search complete target tree for correlated subvolumes.
- Include snapshots from all mountpoints as candidates (disabled
due to uptream bug: github.com/kdave/btrfs-progs/issues/96).
due to upstream bug: github.com/kdave/btrfs-progs/issues/96).
- Read /proc/self/mountinfo instead of /proc/self/mounts.
- Always read /proc/self/mountinfo.
- Resolve realpath using readlink(1).
@ -319,7 +319,7 @@ btrbk-0.25.0
* Allow trailing comments in btrbk.conf (close #129).
* Bugfix: rate limiting must be done after compression (close #134).
* raw_target_encrypt: Always set "gpg --no-random-seed-file":
prevents creation of "~/.gnupg/random_seed" with slight perfomance
prevents creation of "~/.gnupg/random_seed" with slight performance
penalty.
btrbk-0.24.0

64
btrbk
View File

@ -166,39 +166,6 @@ my %config_options = (
# deprecated options
ssh_port => { default => "default", accept => [qw( default ), qr/[0-9]+/ ],
deprecated => { DEFAULT => { warn => 'Please use "ssh://hostname[:port]" notation in the "volume" and "target" configuration lines.' } } },
btrfs_progs_compat => { default => undef, accept => [qw( yes no )],
deprecated => { DEFAULT => { ABORT => 1, warn => 'This feature has been dropped in btrbk-v0.23.0. Please update to newest btrfs-progs, AT LEAST >= $BTRFS_PROGS_MIN' } } },
snapshot_preserve_daily => { default => 'all', accept => [qw( all ), qr/[0-9]+/ ], context => [qw( global volume subvolume )],
deprecated => { DEFAULT => { FAILSAFE_PRESERVE => 1, warn => 'Please use "snapshot_preserve" and/or "snapshot_preserve_min"' } } },
snapshot_preserve_weekly => { default => 0, accept => [qw( all ), qr/[0-9]+/ ], context => [qw( global volume subvolume )],
deprecated => { DEFAULT => { FAILSAFE_PRESERVE => 1, warn => 'Please use "snapshot_preserve" and/or "snapshot_preserve_min"' } } },
snapshot_preserve_monthly => { default => 'all', accept => [qw( all ), qr/[0-9]+/ ], context => [qw( global volume subvolume )],
deprecated => { DEFAULT => { FAILSAFE_PRESERVE => 1, warn => 'Please use "snapshot_preserve" and/or "snapshot_preserve_min"' } } },
target_preserve_daily => { default => 'all', accept => [qw( all ), qr/[0-9]+/ ],
deprecated => { DEFAULT => { FAILSAFE_PRESERVE => 1, warn => 'Please use "target_preserve" and/or "target_preserve_min"' } } },
target_preserve_weekly => { default => 0, accept => [qw( all ), qr/[0-9]+/ ],
deprecated => { DEFAULT => { FAILSAFE_PRESERVE => 1, warn => 'Please use "target_preserve" and/or "target_preserve_min"' } } },
target_preserve_monthly => { default => 'all', accept => [qw( all ), qr/[0-9]+/ ],
deprecated => { DEFAULT => { FAILSAFE_PRESERVE => 1, warn => 'Please use "target_preserve" and/or "target_preserve_min"' } } },
resume_missing => { default => "yes", accept => [qw( yes no )],
deprecated => { yes => { warn => 'ignoring (missing backups are always resumed since btrbk v0.23.0)' },
no => { FAILSAFE_PRESERVE => 1, warn => 'Please use "target_preserve_min latest" and "target_preserve no" if you want to keep only the latest backup', },
DEFAULT => {} } },
snapshot_create_always => { default => undef, accept => [qw( yes no )],
deprecated => { yes => { warn => "Please use \"snapshot_create always\"",
replace_key => "snapshot_create",
replace_value => "always",
},
no => { warn => "Please use \"snapshot_create no\" or \"snapshot_create ondemand\"",
replace_key => "snapshot_create",
replace_value => "ondemand",
},
DEFAULT => {},
},
},
receive_log => { default => undef, accept => [qw( sidecar no )], accept_file => { absolute => 1 },
deprecated => { DEFAULT => { warn => "ignoring" } },
}
);
my @config_target_types = qw(send-receive raw); # first in list is default
@ -707,7 +674,7 @@ sub stream_buffer_cmd_text($)
# NOTE: mbuffer takes defaults from /etc/mbuffer.rc
my @cmd = ( "mbuffer" );
push @cmd, ( "-v", "1" ); # disable warnings (they arrive asynchronously and cant be caught)
push @cmd, ( "-v", "1" ); # disable warnings (they arrive asynchronously and can't be caught)
push @cmd, "-q" unless($progress);
push @cmd, ( "-s", $blocksize ) if($blocksize);
push @cmd, ( "-m", lc($bufsize) ) if($bufsize);
@ -1833,7 +1800,7 @@ sub btrfs_send_to_file($$$;$$)
};
}
else {
die "Usupported encryption type (raw_target_encrypt)";
die "Unsupported encryption type (raw_target_encrypt)";
}
}
@ -2196,7 +2163,7 @@ sub read_extentmap_cache($)
read($fh, $buf, 24 + 8 * 2); # read header
my ($v, $gen, $time) = unpack('a24Q<Q<', $buf);
unless(($v =~ /^btrbk_extentmap_v1/) && $gen && $time) {
ERROR "Ambigous cache file: $file";
ERROR "Ambiguous cache file: $file";
next;
}
if($gen != $vol->{node}{gen}) {
@ -2326,7 +2293,7 @@ sub aio_extentmap($)
if($_->[3] & IO::AIO::FIEMAP_EXTENT_DATA_INLINE()) {
$inline_count++;
next if($ignore_inline);
WARN_ONCE "Ambigous inline region [$_->[1] .. $_->[1] + $_->[2] - 1] for $file" if((($_->[1] != 0) || ($_->[2] != 4096)));
WARN_ONCE "Ambiguous inline region [$_->[1] .. $_->[1] + $_->[2] - 1] for $file" if((($_->[1] != 0) || ($_->[2] != 4096)));
}
push @range, [ $_->[1], $_->[1] + $_->[2] - 1 ];
}
@ -2376,7 +2343,7 @@ sub extentmap_merge(@) {
my $end = -2;
foreach (@range) {
if($_->[0] <= $end + 1) {
# range overlaps the preceeding one, or is adjacent to it
# range overlaps the preceding one, or is adjacent to it
$end = $_->[1] if($_->[1] > $end);
}
else {
@ -3142,7 +3109,7 @@ sub _vinfo_subtree_list
# if $vinfo_parent->{NODE_SUBDIR} is set, vinfo_parent->{PATH} does
# not point to a subvolume directly, but to "<path_to_subvolume>/NODE_SUBDIR".
# skip nodes wich are not in NODE_SUBDIR, or strip NODE_SUBDIR from from rel_path.
# skip nodes which are not in NODE_SUBDIR, or strip NODE_SUBDIR from from rel_path.
my $node_subdir_filter = ($depth == 0) ? $vinfo_parent->{NODE_SUBDIR} : undef;
foreach my $node (@{$tree->{SUBTREE}}) {
my $rel_path = $node->{REL_PATH};
@ -4170,17 +4137,6 @@ sub append_config_option($$$$;@)
push @wmsg, "Using \"$key $value\"";
}
WARN @wmsg if(@wmsg);
if($dh->{FAILSAFE_PRESERVE}) {
unless($config_override{FAILSAFE_PRESERVE}) { # warn only once
WARN "Entering failsafe mode:";
WARN " - preserving ALL snapshots for ALL subvolumes";
WARN " - ignoring ALL targets (skipping backup creation)";
WARN " - please read \"doc/upgrade_to_v0.23.0.md\"";
$config_override{FAILSAFE_PRESERVE} = "Failsafe mode active (deprecated configuration)";
}
$config_override{snapshot_preserve_min} = 'all';
return $config;
}
}
if($opt->{allow_multiple}) {
@ -4306,7 +4262,7 @@ sub parse_config_line($$$;@)
target_type => $target_type,
url => $url_prefix . $path,
};
# NOTE: target sections are propagated to the apropriate SUBSECTION in _config_propagate_target()
# NOTE: target sections are propagated to the appropriate SUBSECTION in _config_propagate_target()
$cur->{TARGET} //= [];
push(@{$cur->{TARGET}}, $target);
$cur = $target;
@ -6429,12 +6385,6 @@ MAIN:
DEBUG "Initializing target section: $droot->{PRINT}";
my $target_type = $droot->{CONFIG}->{target_type} || die;
if($config_override{FAILSAFE_PRESERVE}) {
ABORTED($droot, $config_override{FAILSAFE_PRESERVE});
WARN "Skipping target \"$droot->{PRINT}\": " . ABORTED_TEXT($droot);
next;
}
if(config_key($droot, "target_create_dir")) {
unless(my $ret = vinfo_mkdir($droot)) {
ABORTED($droot, "Failed to create directory: $droot->{PRINT}/");

2
contrib/cron/btrbk-mail
View File

@ -35,7 +35,7 @@ rsync_opt[example_data]="-az --delete --inplace --numeric-ids --acls --xattrs"
# Enable all rsync declarations (all indices of rsync_src array)
#rsync_enable=${!rsync_src[@]}
# Explicitely enable rsync declarations (whitespace-separated list)
# Explicitly enable rsync declarations (whitespace-separated list)
#rsync_enable="example_data"
rsync_enable=

2
contrib/crypt/kdf_pbkdf2.py
View File

@ -54,7 +54,7 @@ salt_hex = "".join(["{:02x}".format(x) for x in salt])
dk_hex = "".join(["{:02x}".format(x) for x in dk])
print("KEY=" + dk_hex);
print("algoritm=pbkdf2_hmac");
print("algorithm=pbkdf2_hmac");
print("hash_name=" + hash_name);
print("salt=" + salt_hex);
print("iterations=" + str(iterations));

2
contrib/tools/btrbk_restore_raw.py
View File

@ -196,7 +196,7 @@ def main():
parser.add_argument('restore_dir', help="target directory for restored subvolumes"
" (path argument for \"btrfs receive\")")
parser.add_argument('-n', '--dry-run', action='store_true',
help="print commmands that would be executed")
help="print commands that would be executed")
parser.add_argument('--ignore-missing', action='store_true',
help="do not fail on missing parent snapshots")

3
doc/Makefile
View File

@ -1,5 +1,4 @@
DOCS = FAQ.md \
upgrade_to_v0.23.0.md
DOCS = FAQ.md
MAN_MAN1 = btrbk.1 \
lsbtr.1 \
ssh_filter_btrbk.1

2
doc/btrbk.1.asciidoc
View File

@ -143,7 +143,7 @@ as "first backup of the day".
space-separated, quoted key=value pairs (machine readable).
+
If set to "col:", prints only the <columns> specified (comma-separated
list). Header lines are ommitted if the "h:" modifier is present.
list). Header lines are omitted if the "h:" modifier is present.
Columns prefixed with "-" are collapsed if empty. Columns postfixed
with ":RALIGN" are right-aligned.

4
doc/btrbk.conf.5.asciidoc
View File

@ -291,9 +291,9 @@ set to ``all'' (the default).
Number of threads to use for <compress_command>. Only supported
for "pigz", "pbzip2", "bzip3", "zstd" and recent versions of "xz".
*stream_compress_adapt* default|<number>::
*stream_compress_adapt* yes|no::
Enable adaptive compression for <compress_command>. Only supported
for "zstd" (version >= 1.3.6).
for "zstd" (version >= 1.3.6). Defaults to ``no''.
*stream_buffer* <size>|no::
Add a buffer to the btrfs send stream (locally, on uncompressed

2
doc/lsbtr.1.asciidoc
View File

@ -67,7 +67,7 @@ OPTIONS
space-separated key="value" pairs (machine readable).
+
If set to "col:", prints only the <columns> specified (comma-separated
list). Header lines are ommitted if the "h:" modifier is present.
list). Header lines are omitted if the "h:" modifier is present.
Columns prefixed with "-" are collapsed if empty. Columns postfixed
with ":RALIGN" are right-aligned.

92
doc/upgrade_to_v0.23.0.md
View File

@ -1,92 +0,0 @@
Upgrading to btrbk-v0.23.0
==========================
In order to keep btrbk simple and intuitive while adding new features,
it became inevitable to change the semantics of the "retention policy"
related configuration options.
What has changed?
-----------------
### Preserve *first* instead of *last* snapshot/backup
btrbk used to *always* transfer the latest snapshot to the target
location, while considering the *last* snapshot/backup of a day as a
daily backup (and also the last weekly as a monthly). This made it
very cumbersome when running btrbk in a cron job as well as manually,
because the last manually created snapshot was immediately transferred
on every run, and used as the daily backup (instead of the one created
periodically by the cron job).
The new semantics are to consider the *first* (instead of *last*)
snapshot of a hour/day/week/month as the one to be preserved, while
only transferring the snapshots needed to satisfy the target retention
policy.
### Preserve snapshots for a minimum amount of time
In order to specify a minimum amount of time in which *all* snapshots
should be preserved, the new "snapshot_preserve_min" and
"target_preserve_min" configuration options were introduced. This was
previously covered by "snapshot_preserve_daily", which caused a lot of
confusion among users.
Upgrading the configuration file: /etc/btrbk/btrbk.conf
-------------------------------------------------------
Please read the description of the "run" command in [btrbk(1)], as
well as the "RETENTION POLICY" section in [btrbk.conf(5)] for a
detailed description. Make sure to understand the new concept, and run
`btrbk --print-schedule dryrun` after updating the configuration.
### Upgrade retention policy
If you want the same behaviour as before:
# replace this:
snapshot_preserve_daily <daily>
snapshot_preserve_weekly <weekly>
snapshot_preserve_monthly <monthly>
# with:
snapshot_preserve_min <daily>d
snapshot_preserve <weekly>w <monthly>m
# ... do the same with "target_preserve_*" options
But what you probably want is something like:
snapshot_preserve_min 5d
snapshot_preserve <daily>d <weekly>w <monthly>m
target_preserve_min no
target_preserve <daily>d <weekly>w <monthly>m *y
This states:
* Keep all snapshots for five days (no matter how many there are)
* Transfer only the first snapshot of a day to the target
* Keep all "first snapshots of a day" for `<daily>` days, etc.
### Upgrade "resume_missing"
If you have a line: "resume_missing yes" somwhere in your config,
simply remove it. btrbk always resumes missing backups.
If you have "resume_missing no", you can imitate this behaviour by
setting:
target_preserve_min latest
target_preserve no
This states: "always transfer the latest snapshot to the target".
[btrbk(1)]: https://digint.ch/btrbk/doc/btrbk.1.html
[btrbk.conf(5)]: https://digint.ch/btrbk/doc/btrbk.conf.5.html