Commit Graph

824 Commits (c9fb366d09c4923e408a2a14b9881039ea0f4e04)

Author SHA1 Message Date
Axel Burri c9fb366d09 btrbk.conf.example: suggest stream_buffer=256m; adapt text; add links 2019-08-04 23:08:11 +02:00
Axel Burri 06a7a53fee documentation: btrbk.conf.5: add rate_limit_remote, fix stream_buffer / rate_limit text 2019-08-04 23:08:11 +02:00
Axel Burri 0435f32619 btrbk: add stream_buffer_remote, rate_limit_remote; run stream_buffer only on local host
Change sematics of stream_buffer: If set, run on local host only. If a
remote stream buffer is required, use stream_buffer_remote /
rate_limit_remote.
2019-08-04 23:08:11 +02:00
Axel Burri 73108d2309 btrbk: fix: "readonly" is required key for "btrfs subvolume show" output
The "Flags: readonly|-" line is present as of btrfs-progs v4.6.1, and
is required in "target metadata check" of btrfs_send_receive().
2019-08-04 13:52:45 +02:00
Axel Burri 74be074e6f btrbk: remove warnings on superfluous "btrfs subvolume show" output
Silently accept compatible versions of btrfs-progs. Required keys are
being checked below, no need to warn here.

e.g. btrfs-progs >= v5.2.1 prints:

    Quota group:            0/258
      Limit referenced:     -
      Limit exclusive:      1.00GiB
      Usage referenced:     16.00KiB
      Usage exclusive:      16.00KiB
2019-08-04 13:29:39 +02:00
Axel Burri 7d656540d7 btrbk: fix btr_tree for subvolumes containing special (unsafe) characters
Note that any action (e.g. btrfs send -p) on a subvolume with unsafe
characters will still fail with:

    ERROR: Unsafe command [...] (offending string: "/tmp/btrfs_pool/svol with spaces")
2019-07-31 13:44:30 +02:00
Axel Burri 10b826b5f3 documentation: btrbk.1: add action "ls" 2019-07-28 19:03:42 +02:00
Axel Burri 7271732f9e btrbk: add action "ls": list all btrfs subvolumes below path
Features:

 - get mounted filesystems from /proc/self/mountinfo
 - fetch subvolumes using "btrfs subvolume list" (fast, needs root)
 - filter and print subvolumes below mount point
2019-07-28 19:03:19 +02:00
Axel Burri d88221e8bc btrbk: add fallback_default_config action option, use for action "diff"
Preparatory for action "ls"
2019-07-28 19:03:19 +02:00
Axel Burri 381a12241b btrbk: add dir_args: relative path arguments for actions reqiring directories
Preparatory for action "ls".
2019-07-28 19:03:19 +02:00
Axel Burri 217317b274 ChangeLog: add migration instructions; add mbuffer changes 2019-07-28 17:25:34 +02:00
Axel Burri dc14611911 documentation: adapt rate_limit and --progress text 2019-07-28 17:25:34 +02:00
Axel Burri 9dc717c701 btrbk: use mbuffer instead of pv; add stream_buffer_sink framework
Add run_cmd option stream_buffer_sink, which handles stream_buffer,
rate_limit as well as --progress.

For rate limiting, run "mbuffer" (on the target host) in combination
with stream_buffer and --progress, instead of running "pv" (on the
source host).

Reasons:

 - mbuffer limits the read rate: For remote targets, we want a stream
   buffer in front of the rsh command pipe, before decompression.
 - For local targets, this can be combined with --process.
 - Combined stream_buffer and rate_limit: less commands in pipe.

Further changes:

 - always set mbuffer -v1 option (never show warnings)
 - restrict raw_target_block_size to "kmgKMG": compatibility to
   stream_compress and rate_limit options, simplicity.
 - use mbuffer blocksize option where applicable
2019-07-28 17:25:34 +02:00
Axel Burri c7a8d0bb11 btrbk: fix naming: compress_cmd -> compress_cmd_text 2019-07-28 17:25:34 +02:00
Axel Burri cf2c314627 btrbk: change version to 0.29.0-dev 2019-07-28 17:25:34 +02:00
Axel Burri 23770f2e67 change version to 0.28.3; update ChangeLog 2019-07-28 15:43:14 +02:00
Axel Burri 8ffd7ac1e9 btrbk: fix mount point resolving
Searching for longest match in mountinfo is plain wrong, as it is
possible (while very uncommon) to have a later mount point shadowing a
longer mount.
2019-07-18 15:34:52 +02:00
Axel Burri fb3d4d96ff btrbk: action origin: print uuid of unknown subvolumes 2019-07-15 18:36:12 +02:00
Axel Burri 5afa270577 btrbk: action origin: correctly handle root subvolumes having uuid 2019-07-15 18:36:06 +02:00
Axel Burri 8de7f45e89 btrbk: remove dead code
setting info{ERROR} does not make sense if info hash is not pushed to
SUBVOL_RECEIVED.

as of: c06bca17
2019-07-15 18:31:07 +02:00
Axel Burri e7787e8a44 btrbk: replace "unrecoverable" hack by FIX_MANUALLY framework 2019-07-15 18:28:31 +02:00
Lubos Kolouch 836efa472b btrbk: return error code 10 if unrecoverable problem
https://github.com/digint/btrbk/issues/285
2019-07-15 18:03:02 +02:00
Axel Burri aaef1cd1e9 btrbk: change version to 0.28.3-dev 2019-07-15 17:50:50 +02:00
Axel Burri 31c25d0ee2 change version to 0.28.2; update ChangeLog 2019-05-23 14:11:33 +02:00
Axel Burri 0439e6079f btrbk: fix vinfo_resolved
Fix for rare cases:
- fix for paths named "0"
- return undef if not strictly below $vol (or allow_equal is set)
2019-05-23 13:14:07 +02:00
Axel Burri 461eaada66 btrbk: rename "get_related_readonly_nodes" to "get_related_nodes"; add readonly and omit_self options 2019-05-23 13:14:07 +02:00
Axel Burri f169ab4482 btrbk: cosmetics: test loglevel before dumping cmd return value (performance) 2019-05-23 13:12:45 +02:00
Axel Burri 45b487707f btrbk: honor incremental_resolve on targets (fix: defaults to "mountpoint" instead of "directory")
Use $resolve_droot instead of $droot for calls to get_best_correlated
(probably missed commit), same as $resolve_sroot.

Fixes possible regression of:

514e69243a btrbk: add "incremental_resolve" configuration option
2019-05-19 19:01:50 +02:00
Axel Burri 01230cde22 btrbk: bugfix: add missing (fake) mountpoint for raw targets
For raw targets, get_best_parent() dies as VINFO_MOUNTPOINT is not
defined on raw vinfo.

Fixes regression of:

d64e237e94 btrbk: get_best_parent: consider all parent/child relations
2019-05-19 15:27:27 +02:00
Axel Burri 538dab5396 btrbk: change version to 0.28.2-dev 2019-05-19 15:17:37 +02:00
Axel Burri f132c94c65 change version to 0.28.1; update ChangeLog 2019-05-16 17:16:52 +02:00
Axel Burri ef69f1ffee documentation: remove pre-generated man pages
Package maintainers like to build everything from scratch, removing
overhead. Again, we apologize for the inconvenience.

Note that reproducible builds are still guaranteed by setting
SOURCE_DATE_EPOCH in doc/Makefile.

Reverts: a6dbd60e5a documentation: add pre-generated man pages: from groff to asciidoc and back again
2019-05-16 17:06:15 +02:00
Axel Burri f54853dd18 btrbk: cosmetics (trace logs) 2019-05-16 17:01:51 +02:00
Axel Burri 10bde449c6 btrbk: remove obsolete "resolve_subdir" option of vinfo_init_root
Removed in: b37ef84e36 btrbk: always read mountpoints
2019-05-16 17:01:51 +02:00
Axel Burri b62bb3b1a8 btrbk: get_related_readonly_nodes: vastly increase abort_distance
It's not uncommon to have a large intact parent-chain on targets
(e.g. target_preserve_min=all).

If this is the case, performance drops a bit on "btrbk archive".
Note that we could limit the search depth in get_best_parent() for
some performance improvements, as this only affects extra clones.
2019-05-03 16:13:24 +02:00
Axel Burri 37b0bd3477 btrbk: bugfix: make get_related_readonly_nodes non-recursive
Perl hates recursions, and dies if recursion depth = 100:

    Deep recursion on subroutine "main::_push_related_children"

Unfortunately this happens before the implemented abort condition
(distance=256).

Fixed by re-implementing get_related_readonly_nodes() non-recursive.

Refs: https://github.com/digint/btrbk/issues/279
2019-05-03 16:10:41 +02:00
Axel Burri 206e706d85 btrbk: allow overriding options with context restrictions
This allows commands like:

    btrbk run mysubvol --override=snapshot_name=mysubvol-oneshot
2019-04-30 13:38:47 +02:00
Axel Burri 3d169d1df4 btrbk: change version to 0.28.1-dev 2019-04-30 13:38:33 +02:00
Axel Burri a6dbd60e5a documentation: add pre-generated man pages: from groff to asciidoc and back again
This reduces build-time dependencies to zero, helping package
maintainers and providing reproducible builds.

NOTE: generated man pages will only be updated on releases. In order
to make sure the docs are correctly rebuilt, run "make clean man". We
apologize for the inconvenience.

From Groff to Asciidoc and Back Again [1]
=========================================

| Comparison                                                               | Links                                                                                                          |
| -------------------------------------------------------------------------| -------------------------------------------------------------------------------------------------------------- |
| **Plain ROFF**                                                           |                                                                                                                |
| +++ Best result for `man` (our main goal!)                               |                                                                                                                |
| - Not supported by github                                                | [btrbk(1) v0.25.1 in plain groff](https://github.com/digint/btrbk/blob/v0.25.1/doc/btrbk.1)                    |
| - No decent converters: e.g. `groff -Txhtml -mandoc`                     | [btrbk(1) v0.25.1 at digint.ch (official site)](https://digint.ch/btrbk/doc/archive/btrbk-0.25.1/btrbk.1.html) |
|                                                                          |                                                                                                                |
| **asciidoc**                                                             |                                                                                                                |
| + Decent (scriptable!) html                                              | [btrbk(1) v0.26.1 at digint.ch (official site)](https://digint.ch/btrbk/doc/archive/btrbk-0.26.1/btrbk.1.html) |
| + Supported by github (helps contributors writing decent documentation)  | [btrbk.1.asciidoc v0.26.1 at github](https://github.com/digint/btrbk/blob/master/doc/btrbk.1.asciidoc)         |
|                                                                          |                                                                                                                |
| **asciidoc (`xmlto`, `atx`)**                                            | http://asciidoc.org/                                                                                           |
| + Good result for `man`                                                  |                                                                                                                |
| - EOL (why care that much? it works fine!)                               | [asciidoc EOL notice at github](https://github.com/asciidoc/asciidoc/releases/tag/8.6.10)                      |
|                                                                          |                                                                                                                |
| **asciidoc (`asciidoctor`)**                                             | https://asciidoctor.org                                                                                        |
| - Pulls in tons of ruby (build depends)                                  | https://bugs.gentoo.org/681056                                                                                 |
| -- Implies to commit a pre-converted `doc/btrbk.1`                       |                                                                                                                |
| + Seems more evolved than `xmlto`, still actively maintained             |                                                                                                                |
|                                                                          |                                                                                                                |
| **rst (`xrst2man.py` from docutils)**                                    |                                                                                                                |
| - Worse result for `man`                                                 |                                                                                                                |
| ++ Good html converters (after having a quick look at it)                |                                                                                                                |
| - Not so well supported by github                                        | [btrbk.1.rst v0.26.1 at github](https://github.com/digint/btrbk/blob/rst2man/doc/btrbk.1.rst)                  |

 [1] https://github.com/digint/btrbk/pull/219 (edited)
2019-04-24 19:45:54 +02:00
Axel Burri fc485565ce change version to 0.28.0; update ChangeLog 2019-04-24 19:45:09 +02:00
Axel Burri 5b34f620aa documentation: reproducible builds: set SOURCE_DATE_EPOCH to "📅" tag 2019-04-24 19:41:14 +02:00
Axel Burri e3176a0027 documentation: asciidoc: add "release-version" attribute, and append it to "man source" attribute 2019-04-24 19:41:14 +02:00
Axel Burri 818cc42d9b documentation: Makefile: dont compress documentation by default
Many (all?) distro builds scripts want to do compression by their own.
2019-04-24 19:41:14 +02:00
Axel Burri 173319e7e1 documentation: Makefile: always build using asciidoctor (remove a2x)
Remove support for man page generation using asciidoc "a2x": The
project is discontinued, and depends on Python 2.7.

As we will provide pre-generated man pages as of btrbk-0.28.0, this is
not needed any more.
2019-04-24 19:40:59 +02:00
Axel Burri d231a955ce documentation: btrbk.conf.5: cosmetics: replace tabs with whitespace 2019-04-24 19:22:30 +02:00
Axel Burri fe9ce255c5 remove experimental tag on "btrbk archive" 2019-04-24 19:22:30 +02:00
Axel Burri 725a284f16 documentation: README.md: adapt indentation in examples 2019-04-24 19:22:30 +02:00
Axel Burri f5c7c74108 documentation: simplify examples, remove optional "send-receive" on "target" declarations 2019-04-24 19:22:29 +02:00
Axel Burri cb1c12a6c1 documentation: btrbk.conf.5: more comprehensive text on "target" declaration 2019-04-24 13:52:05 +02:00
Axel Burri 6115cbdace btrbk: make target_type optional for "target" section 2019-04-24 13:51:52 +02:00