ambiser
/
btrbk
mirror of https://github.com/digint/btrbk
9
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,285 Commits
12 Branches
50 Tags
5 MiB
Commit Graph

918 Commits (53ea4efe5bc9beeff5726cef06226a96cd209a72)

Author SHA1 Message Date
Christoph Anton Mitterer 914f9286c7 btrbk: add bzip3 compression 
This adds support for bzip3 [1].

 [1] https://github.com/kspalaiologos/bzip3

Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>

Cosmetics: swap order pbzip2 / bzip3

Signed-off-by: Axel Burri <axel@tty0.ch>
 2022-11-16 00:45:39 +01:00
Axel Burri af86dc8c52 btrbk: allow compressors not to have a compression level 
Not all compressors support compression level (option `-#`): print a
warning if compress_level is set, and ignore its value for such
compressors.
 2022-11-16 00:44:53 +01:00
Axel Burri cc08cef27c btrbk: add send_protocol and send_compressed_data config options 
support btrfs send protocol v2
 2022-10-30 17:08:06 +01:00
Axel Burri 1f7aa7e247 btrbk: unify btrfs send options 2022-10-30 17:08:06 +01:00
Axel Burri af4681319c btrbk: bump version to 0.32.6-dev 2022-10-30 17:08:05 +01:00
Axel Burri 58fffab405 change version to 0.32.5; update Changelog 2022-10-23 12:37:28 +02:00
Axel Burri dcee6222be btrbk: append to info file instead of rewriting 
A redirection (e.g. `echo foo > bar.info`) can cause empty (zero-size)
files in some circumstances.

We still write INCOMPLETE=1 to the info file before send/receive, but
instead of re-creating it without the INCOMPLETE flag, we append
INCOMPLETE=0 (keeping up compatibility with old versions of btrbk).

Ref: 4e5ae975d8 btrbk: ignore zero-size info files
 2022-10-19 11:38:11 +02:00
Axel Burri 51b2a5a9f5 btrbk: tidy comments 2022-10-19 11:38:11 +02:00
Axel Burri 25c8eb1705 btrbk: add timestamp to info file on write 2022-10-19 11:38:11 +02:00
Axel Burri 08cb900a9e btrbk: add append option to system_write_raw_info 2022-10-19 11:38:11 +02:00
Axel Burri 89c0aa31d7 btrbk: tidy raw info file parsing 2022-10-19 11:37:45 +02:00
Axel Burri d6910e43ea btrbk: add append_to_file option in run_cmd 2022-10-19 11:33:40 +02:00
Matthieu Patou 4e5ae975d8 btrbk: ignore zero-size info files 
When backuping from devices that have configured to use raw backup and
that might disconnect from the network (ie. laptops) you end up once in
a while with 0 size info file (and backup file).
btrbk don't know how to handle 0 file and stop backing up until the zero
size file is removed.
With this change 0 size info file will be ignored, and hence the backup
for the given backup will be redone.

Signed-off-by: Matthieu Patou <mat@matws.net>
 2022-10-18 11:46:32 +02:00
Axel Burri f77a33b340 btrbk: bump version to 0.32.5-dev 2022-10-18 11:08:31 +02:00
Axel Burri 6877825708 change version to 0.32.4; update Changelog 2022-08-20 14:26:36 +02:00
Axel Burri 116aface3c btrbk: fix regression: wrong deprecation warnings 
Warning for btrfs_commit_delete is always printed, regardless of the
(possibly valid) values.

regression in btrbk-0.32.3

   687e0508b7 btrbk: tidy deprecation warnings
 2022-08-20 14:24:17 +02:00
Axel Burri 411034e93f change version to 0.32.3; update Changelog 2022-08-07 11:04:39 +02:00
Axel Burri 79b6a662f2 btrbk: fix missing error in action "ls" 2022-07-28 13:44:42 +02:00
Axel Burri cf2065df30 btrbk: hint missing ssh_identity on errors 
It is perfectly ok to run btrbk without ssh_identity (using ssh
defaults), printing a warning if the option is not set is wrong.
Instead, hackily check for ssh_identity on ssh errors, and give a hint
in the error message.
 2022-07-28 13:44:37 +02:00
Axel Burri 2621fb38ba btrbk: allow ssh_user=no 2022-07-28 12:10:06 +02:00
Axel Burri dd638ec9f4 btrbk: allow ssh_identity=no 2022-07-28 12:06:22 +02:00
Axel Burri 6b465bf06b btrbk: never delete multiple subvolumes at once 
Deleting multiple subvolumes at once always caused the problem that we
need to parse stderr of "rm" and "btrfs subvolume delete" in order to
know which subvolume actually failed, which is problematic (version
dependent, language dependent). Also, we would need to restrict the
number of subvolumes based on the maximum allowed length for shell
commands, which is system-dependent (check `getconf ARG_MAX`).

Deleting subvolumes sequentially has slightly negative impact on
execution time (multiple rsh commands), with the benefit of being more
robust and reducing the codesize.
 2022-07-27 20:41:25 +02:00
Axel Burri b7df717611 btrbk: hide only fully deprecated keys when dumping 2022-07-27 18:36:20 +02:00
Axel Burri 687e0508b7 btrbk: tidy deprecation warnings 2022-07-27 18:20:33 +02:00
Axel Burri c5cb919f6c btrbk: tidy log functions 2022-07-27 18:19:46 +02:00
Axel Burri 805ab93fef btrbk: fix function argument 
Does not trigger a bug in current program logic.

Ref: 3ea7746700 btrbk: cosmetics: separate get_btrbk_date function
 2022-07-27 14:35:24 +02:00
Axel Burri 43a125d4e1 btrbk: bump version to 0.32.3-dev 2022-07-27 14:35:15 +02:00
Axel Burri bdc1f5caba change version to 0.32.2; update Changelog 2022-06-25 18:52:11 +02:00
Axel Burri 054146ac00 btrbk: keep quotes intact when removing comments in config 2022-06-06 22:00:46 +02:00
Axel Burri 9d21785778 btrbk: fix parsing of quoted "target" config line 
Quotes were removed before passing value to parse_config_line,
resulting in failuer of parsing target url if quoted.
 2022-06-06 21:59:11 +02:00
Axel Burri fb13848faf btrbk: tidy function prototype 2022-06-06 21:02:13 +02:00
Axel Burri f169456613 btrbk: tidy logging 2022-06-06 18:31:45 +02:00
Axel Burri a70248376f btrbk: use no-default-recipient on gpg encrypt 
In case the user has set a default recipient in his gpg config file,
ignore it.
 2022-05-29 21:34:43 +02:00
Steven Brudenell b824d62449 btrbk: support multiple gpg recipients 2022-05-29 21:34:43 +02:00
Axel Burri eadeb89232 btrbk: check results of btrfs filesystem usage 2022-05-29 14:32:27 +02:00
Axel Burri b8464119f6 btrbk: add compat=ignore_receive_errors option 2022-05-29 12:29:19 +02:00
Axel Burri 527d1bf74f btrbk: improve action "config print" 
- honor overrides
 - print unset lines as comments
 - skip lines on forbidden context
 - fix values "no" and "<unset>"
 2022-05-29 12:26:37 +02:00
Axel Burri f187d6f6ee btrbk: tidy formatting 2022-05-29 12:26:37 +02:00
Axel Burri 321f3783d5 btrbk: use split in ssh_cipher_spec config definition 2022-05-29 12:21:39 +02:00
Axel Burri ce2afb427d btrbk: split all config values with same regex 2022-05-28 21:28:42 +02:00
Axel Burri ba3ee70d38 btrbk: tidy config options definition 2022-05-28 21:28:39 +02:00
Axel Burri 823419d510 btrbk: tidy config options parser 
- handle regex within accept list
 - handle numeric as regex
 - split before checking accept
 2022-05-28 21:27:39 +02:00
Axel Burri 04e0695429 btrbk: tidy logging 2022-05-27 19:31:48 +02:00
Axel Burri 12f608d828 btrbk: fix regression: clear realpath cache on mkdir 
On btrbk archive, after creating a directory without dry-run, the
archive target is skipped with "Failed to fetch subvolume detail" due
to caching of realpath.

Regression in btrbk-0.32.0:

    eb69bc883e btrbk: refactor mountinfo
 2022-05-27 19:31:48 +02:00
Axel Burri 4fcbbad802 btrbk: print version and help to stdout 2022-03-25 15:20:20 +01:00
Axel Burri be07f1908a btrbk: print shorter error help message 2022-03-25 15:20:14 +01:00
Axel Burri fc49abbbf8 btrbk: bump version to 0.32.2-dev 2022-03-25 15:19:59 +01:00
Axel Burri 593fa6b254 change version to 0.32.1; update Changelog 2022-02-26 18:35:09 +01:00
Axel Burri d9b9f20899 bump copyright year 2022-02-26 18:35:02 +01:00
Axel Burri 047d3a1a65 btrbk: allow directory traversal for subvol_args 2022-02-26 18:11:09 +01:00
Axel Burri a25486336b btrbk: fix regression: filter paths for action "ls" 
regression from: eb69bc883e btrbk: refactor mountinfo
 2022-02-26 18:11:09 +01:00
Axel Burri ff0a887c32 btrbk: call filesystem usage from mount point 2022-02-22 20:28:23 +01:00
Axel Burri e9fe2cb5f5 btrbk: display all source subvolumes in btrbk usage 2022-02-22 20:28:17 +01:00
Axel Burri 31ce14e4c2 btrbk: handle errors from btrfs filesystem usage 
Fix `btrbk usage` unhandled error case if usage data cannot be
retrieved, e.g.:

   ERROR: Failed to fetch btrfs filesystem usage for: example.com:/mnt/btr_pool
   ERROR: ... sh: ssh -i 'XXX' btrbk@example.com 'btrfs-filesystem-usage '\''/mnt/btr_pool'\'''
   ERROR: ... cannot access '/mnt/btr_pool': Permission denied
 2022-02-22 20:28:01 +01:00
Axel Burri 177f8547e3 btrbk: add backend btrfs-progs-doas 2022-02-08 01:06:04 +01:00
Axel Burri d3b7540b87 btrbk: tidy backend command map 2022-02-08 01:05:59 +01:00
Axel Burri db7082b4bc btrbk: add config_key_lru comfort function 2022-02-06 17:44:30 +01:00
Axel Burri 68d5fe3f55 btrbk: add match argument in config_key 2022-02-06 17:44:30 +01:00
Axel Burri e6ed21343c btrbk: remove dead code 2022-02-06 16:05:12 +01:00
Axel Burri 0172e4a240 btrbk: fix mountinfo tree root detection 
Set node as tree root if mount_id == parent_id. For some reasons this
is never the case on my (mostly gentoo) systems.

Regression from: eb69bc88 btrbk: refactor mountinfo
 2022-02-06 12:35:57 +01:00
Axel Burri ed8142ed9d btrbk: bump version to 0.32.1-dev 2022-02-06 12:35:53 +01:00
Axel Burri 8676191d45 change version to 0.32.0; update Changelog 2022-02-05 19:34:54 +01:00
Axel Burri 8cc74fecc1 btrbk: default timestamp_format=long 
This is a relict of early days of btrbk, and I have already hesitated
for too long to change the default from legacy "short" to sane "long"
format.

Tests show that the scheduling behaves in a sane/expected way if this
change is applied unattended. I suppose everybody who has
preserve_hour_of_day set is already using timestamp_format=long.
 2022-02-05 18:47:49 +01:00
Axel Burri ca166d47b6 btrbk: add safe_commands config option 
For the paranoid. For convenience, filename checking was removed in
[1], and quoting was (hopefully) implemented correctly in [2].

Allowing special characters as well as UTF8 leave behind a bad
feeling, as there are many special cases that needs to be taken care
of (e.g. newlines in file names, right-to-left encoding, etc.). In
order to mitigate attacks expoiting these error classes, leave an
option to power users which do only allow "sane" characters in their
filename hierarchy.

  [1] 6a29b08f00 btrbk: remove filename restrictions
  [2] acc7f9fc83 btrbk: quote unsafe characters in shell commands
 2021-11-06 16:40:47 +01:00
Axel Burri 4f72ad123f btrbk: tidy error message 2021-11-06 16:09:51 +01:00
Axel Burri cb38b7efa4 btrbk: parse additional "btrfs subvolume show" lines from btrfs-progs-5.14.2 2021-10-24 10:39:16 +02:00
Axel Burri c538702d8c btrbk: abort on unexpected only if receive target is not present 2021-10-04 19:17:36 +02:00
Axel Burri bef13275d3 btrbk: fix regression: print quoted rsh command in table output 
regression from: 27388c7589 btrbk: treat all filenames as unsafe
 2021-09-04 15:46:57 +02:00
Axel Burri 1bda5fd978 btrbk: tidy safe_cmd; die if quoting leading dash for command 
This should never happen, as all our filenames are checked to be
absolute.
 2021-09-04 15:46:09 +02:00
Axel Burri d1247359f8 btrbk: always return abort key 2021-08-28 14:49:12 +02:00
Axel Burri faec607d02 btrbk: drop deprecated -r,--resume-only argument 
Deprecated as of btrbk v0.26.0 (more than 3 years ago).
 2021-08-28 14:06:10 +02:00
Axel Burri 3f84345bed btrbk: tidy logging 2021-08-28 14:06:10 +02:00
Axel Burri 5dc2375a75 btrbk: fix raw file deletion 2021-08-28 14:06:10 +02:00
Axel Burri 424127441d btrbk: fix unsafe path for btrfs send to file 2021-08-28 14:06:10 +02:00
Axel Burri fe53198661 btrbk: drop support for deprecated raw format 
Raw backups created by btrbk < 0.26.0 are now ignored:

   *.btrfs_<received_uuid>[@<parent_uuid>][.gz|bz2|xz][.gpg][.split][.part]
 2021-08-28 14:06:02 +02:00
Axel Burri 85c2d14287 btrbk: change incremental_prefs defaults 
New defaults gives the btrbk_direct_leaf snapshots higher preference
than the global ones resolved by parent-uuid (which are best-guess).

This way the parent has a higher chance of being a backup created by
btrbk, which results in "btrfs receive" to start work on a snapshot of
this (and preferably not on the "best-guess" ones).
 2021-08-27 17:20:47 +02:00
Axel Burri bde0c10a6c btrbk: add incremental_prefs config option 2021-08-27 17:20:47 +02:00
Axel Burri 6ca00feeb6 btrbk: tidy parent selection 
Preparatory for incremental_prefs feature, no functional changes.
 2021-08-27 17:20:47 +02:00
Axel Burri 10a6f51730 btrbk: remove deprecation on incremental_clones 
btrbk does not set config defaults to global scope for deprecated
keys.
 2021-08-27 17:20:47 +02:00
Axel Burri 9f877a4670 btrbk: remove extra clone source, deprecate numeric incremental_clones 2021-08-27 17:20:47 +02:00
Axel Burri 1cb958ee30 btrbk: add "match" target for deprecated options 2021-08-27 17:20:39 +02:00
Axel Burri ad6298bf37 btrbk: add global split_match (cosmetics) 2021-08-19 17:16:50 +02:00
Axel Burri eb69bc883e btrbk: refactor mountinfo 
- Create tree from /proc/self/mountinfo, and use it to find mount
  points.

- Populate realpath cache from mount points, possibly reducing calls
  to `realpath`.

- Replace btrfs_mountpoint with vinfo_mountpoint(fs_type => 'btrfs)

- Tidy action "ls".

- Move code
 2021-08-18 13:14:10 +02:00
Axel Burri 063c25ad24 btrbk: add sanity checks on mountinfo list 2021-08-18 13:14:10 +02:00
Axel Burri 6a29b08f00 btrbk: remove filename restrictions 2021-08-18 13:14:10 +02:00
Axel Burri b8370de9de documentation: terminate options using double dash "--" 2021-08-18 13:14:10 +02:00
Axel Burri 81a04e4287 btrbk: use single quotes for raw table format 2021-08-18 02:08:28 +02:00
Axel Burri 48bf4f05b9 btrbk: fix table output for utf8 characters (require Text::CharWidth) 
Use Text::CharWidth::mbswidth() if installed, fallback to
length(Encode::decode_utf8()), fallback to length().

 - Text::CharWidth handles wide chars (e.g. asian, taking up two
   columns on the terminal) correctly.

 - length(Encode::decode_utf8()) handles single-width chars only, and
   should be installed on most systems (perl >= v5.7.3).

 - langth() counts bytes, as we do not convert anything to UTF-8 in
   btrbk (NOT using `perl -CIOEioA` or binmode(STDOUT, ":utf8"))
 2021-08-18 02:08:28 +02:00
Axel Burri 94a415e420 btrbk: check/sanitize leading/trailing whitespace on files 2021-08-18 02:08:28 +02:00
Axel Burri 3e9b581a54 btrbk: add --unsafe-filenames option 2021-08-18 02:08:28 +02:00
Axel Burri 1a7bbff767 btrbk: tidy run_cmd calls 2021-08-18 02:08:28 +02:00
Axel Burri 27388c7589 btrbk: treat all filenames as unsafe 2021-08-18 02:08:28 +02:00
Axel Burri 2933e65cbe btrbk: adapt usage of unsafe arguments for run_cmd 2021-08-18 02:08:28 +02:00
Axel Burri acc7f9fc83 btrbk: quote unsafe characters in shell commands 
As filenames can contain meta characters like '$', we can't just put
ssh commands in double quotes. E.g. the following would trigger
variable expansion on local shell:

    ssh example.com "ls -l 'evil$x'"

Instead, we quote the ssh using single quotes (adding the need to
escape single quotes), while also quoting unsafe filenames using
single quotes. The above becomes:

    ssh example.com 'ls -l '\''evil$x'\'''

Or more complex, for a file named "file with'single quotes'":

    ssh example.com 'ls -l '\''file with'\''\'\'''\''single quotes'\''\'\'''\'''\'''

On the remote shell, this will expand to:

    ls -l 'file with'\''single quotes'\'''
 2021-08-18 02:08:28 +02:00
Axel Burri d7f6d5fecf btrbk: accept quoted values in config 2021-08-18 02:08:28 +02:00
Axel Burri 5d94de9142 btrbk: use File::Spec for relative file arguments 2021-08-18 02:08:28 +02:00
Axel Burri eccb24ecd7 btrbk: fix action "usage" 
Regression from:

   b658fba08c btrbk: print snapdir and targets for action "usage"
 2021-08-18 02:08:28 +02:00
Axel Burri 3cf69f3537 btrbk: add missing table ralign 2021-08-18 02:08:28 +02:00
Axel Burri 2f88d5ab4c btrbk: check for deleted subvol before panicking 2021-08-18 02:08:28 +02:00
Axel Burri 6e7c8c409b btrbk: cosmetics: fix trace log 2021-08-18 02:08:28 +02:00