Commit Graph

908 Commits (lsbtr-related)

Author SHA1 Message Date
Axel Burri 6e7d649588 btrbk: add "noauto" configuration option 2019-04-18 15:09:15 +02:00
Axel Burri ae004e48fd btrbk: map empty value to "yes" in configuration 2019-04-18 15:09:15 +02:00
Axel Burri 1503a07ad1 btrbk: enhance ABORTED framework (distinguish skipped and aborted) 2019-04-18 15:09:15 +02:00
Axel Burri 4224577960 btrbk: distinguish filter_args and subvol_args
preparatory for noauto and exclude options
2019-04-18 15:09:11 +02:00
Ian Kelling ce15c19e44 Clarify license is gpl version 3 or later
https://www.gnu.org/licenses/identify-licenses-clearly.en.html
2019-04-16 11:04:16 -04:00
Axel Burri ff23e9605b btrbk: cosmetics: fix snapshot create debug log
Add additional log line, containing context for subsequent lines
2019-04-12 21:04:49 +02:00
Axel Burri 5697c38e4a btrbk: globally rename snapshot_children to related_snapshots
As a consequence, also rename $child -> $snapshot
2019-04-12 21:04:49 +02:00
Axel Burri 0a93f69135 btrbk: treat all related readonly subvolumes within snapdir as "snapshots"
With this, previous snapshots (far relations) are still listed when
restoring a snapshot.

Example (S = source subvolume, readwrite):

  After 3 snapshots:
  A->S, B->S, C->S

  Restore B: `btrfs subvol delete S; btrfs subvol snapshot B S'`
  A->S, B->S, C->S, S'->B

Previous implementation would show now snapshots for S', as no
snapshot has parent_uuid=S'.

New implementation shows A, B, C as snapshots for S', as orphaned
siblings (A, B, C pointing to deleted S) are also related.
2019-04-12 21:04:48 +02:00
Axel Burri adfdf925e2 btrbk: add btrbk_direct_leaf option for vinfo_resolved() 2019-04-12 21:04:48 +02:00
Axel Burri 318126b831 btrbk: disallow unrelated (by parent_uuid) parents for btrfs-send if incremental=strict
Makes sure that if, for whatever reason, a subvolume having correct
btrbk name scheme does NOT share any extents with previous snapshots
is never used as parent.

Note that if a related parent is found, the unrelated closest
older/newer (by btrbk timestamp) subvolumes are still added as clone
sources.
2019-04-12 21:03:40 +02:00
Axel Burri c407d41db2 btrbk: unconditionally add clone sources to btrfs-send if necessary
Preferences for parent (and required clone sources):

 1. closest older in snapdir (by btrbk timestamp), related
 2. closest older related (by cgen)
 3. closest newer related (by cgen)
 4. closest older in snapdir (by btrbk timestamp)
 5. closest newer in snapdir (by btrbk timestamp)

Note: prefering 1 over 2 helps keeping parent-chain within droot on
target (assuming that btrfs always uses correlated parent on
btrfs-receive).

This will e.g. add a clone source on "btrbk resume", if both older AND
newer snapshot/backup pairs exists.

Also makes sure that the closest older btrbk snapshot is always added
as clone source, even if another related subvolume has newer cgen.
2019-04-11 16:19:48 +02:00
Axel Burri 95e25eb2d1 btrbk: fix get_related_subvolume_nodes: add all parents and orphaned siblings
Old implementation was missing last readonly parent in chain, as well
as orphaned siblings.

Also sort all by cgen, not by distance, then cgen.

Also skip self.
2019-04-11 16:16:00 +02:00
Axel Burri 20c390893a btrbk: honor multiple -v command line options (increase loglevel) 2019-04-05 19:29:37 +02:00
Axel Burri 3a753c9140 ChangeLog: add missing changes on btrbk-master (since 0.27.2) 2019-04-05 12:29:47 +02:00
Axel Burri 44edc97aef btrbk: add "incremental_clones" configuration option (btrfs-send <clone-src>) 2019-04-05 12:06:41 +02:00
Axel Burri f38c69316f btrbk: get_best_parent(): add clone_src argument 2019-04-05 12:05:46 +02:00
Axel Burri 7cf1bfb354 btrbk: rename functions (use "correlated" where applicable)
_receive_target_nodes   -> _correlated_nodes
get_best_receive_target -> get_best_correlated_target
2019-04-05 12:05:46 +02:00
Axel Burri ce81350df0 btrbk: remove unused function: vinfo_subvol_list_all_accessible() 2019-04-05 12:05:46 +02:00
Axel Burri 514e69243a btrbk: add "incremental_resolve" configuration option
Allowed values for "incremental_resolve":

 - "mountpoint" (default): Use parents in the filesystem tree below
   mount points of source `<volume-directory>/<snapshot-dir>` and
   target `<target-directory>`.

 - "directory": Use parents strictly below source/target
   directories. Useful when restricting access, e.g. when using
   ssh_filter_btrbk.sh.

 - "_all_accessible" (experimental): Use parents from all mount points.

Note that using "_all_accessible" causes btrfs-progs to fail:

  - btrfs send -p: "ERROR: not on mount point: /path/to/mountpoint"
  - btrfs receive: "ERROR: parent subvol is not reachable from inside the root subvol"

see also: https://github.com/kdave/btrfs-progs/issues/96
2019-04-05 12:05:46 +02:00
Axel Burri d64e237e94 btrbk: get_best_parent: consider all parent/child relations 2019-04-05 12:05:46 +02:00
Axel Burri cb23c65eed btrbk: get_related_subvolumes: add direct children from parent chain 2019-04-05 12:05:46 +02:00
Axel Burri ad91b6f618 Makefile: add contrib/cron/btrbk-verify 2019-04-05 12:05:46 +02:00
Axel Burri c1268cdd66 btrbk: table_format "transaction": make "*host" and "*port" keys skip-row-if-empty (for consistency) 2019-04-04 17:09:59 +02:00
Axel Burri 2e51de1047 documentation: README.md: add example: virtual machine setup 2019-04-01 20:12:38 +02:00
Axel Burri cba4a93152 btrbk: table_formats: add "*port" keys (skip-row-if-empty) 2019-04-01 20:12:33 +02:00
Axel Burri ca3bd505ef btrbk: make check for duplicate snapshot/backup locations aware of shared btr_tree
Build check hash within btr_tree node instead per URL. This makes it
aware of shared btr_tree (different hostname:port pointing to same
btrfs filesystem).
2019-04-01 18:43:53 +02:00
Axel Burri 2f09a9a723 btrbk: virtual machine setups; use MACHINE_ID for caches; use port in URL
Common virtual machine setups have multiple volume sections with same
host, but distinct port numbers for each machine.

- make caches dependent on MACHINE_ID instead of HOST
- append port number to URL
- add MACHINE_ID to vinfo
- use MACHINE_ID where applicable

This even works if virtual machines share the same btrfs filesystems:
If a equal UUID is found on distinct machines, btr_tree() will return
the already present tree, in order to be consistent after node
injections.
2019-04-01 18:43:49 +02:00
Axel Burri d8b7988ffa btrbk: deprecate ssh_port option in favor of ssh://hostname[:port] notation
Setting the ssh port directly in the "volume" / "target" config lines
adds the possibility to have a create a unique "hostname:port"
identifier (preparatory for MACHINE_ID to distinguish virtual machines
on same host with different ports.)
2019-04-01 18:26:07 +02:00
Axel Burri 318e3af088 btrbk: table_formats: add skip-row-if-empty flag on all "*host" 2019-04-01 18:24:30 +02:00
Axel Burri 716d420a40 btrbk: print_formatted: add skip-row-if-empty functionality for table_formats
If the row key is prefixed with "-", and is all <undef>, omit output
of this row.
2019-04-01 18:22:30 +02:00
Axel Burri 805d7f4a0d btrbk-verify: add workaround for btrbk <= 0.27.2 bug: missing target_rsh, target_type
btrbk <= 0.27.2 does not print "target_rsh" and "target_type" when
called with --format=raw, see $table_formats{resolved}. This is fixed
in 0.28.0.

Hardcoding target_type=send-receive is not so bad, as for raw targets
btrbk-verify complains first with:

  btrbk-verify: missing required variable "target_rsh" in btrbk --format=raw line

So we should not run rsync (which is not really a problem, rsync just
fails with "not a directory").
2019-04-01 16:27:49 +02:00
Axel Burri 9ed41c8937 btrbk-verify: tool for automated backup integrity check based on rsync
Compare files and attributes by checksum, using rsync(1) in dry-run
mode with all preserve options enabled.

Resolves snapshot/backup pairs by evaluating the output of
"btrbk list latest [filter...]".

Restrictions:
 - ".d..t...... ./" lines are ignored by default:
   Root folder timestamp always differ.
 - "cd+++++++++ .*" lines are ignored by default:
   Nested subvolumes appear as new empty directories.
 - btrbk raw targets are skipped
 - rsync needs root in most cases (see --ssh-* options)
2019-04-01 16:27:45 +02:00
Axel Burri 0afc455c40 btrbk: fix table format "resolved": add target_type=send-receive|raw; add missing target_rsh
Required for "btrbk-check", which must be able do distinguish between
"send-receive" and "raw" targets.
2019-04-01 16:27:39 +02:00
Axel Burri 0699a860a9 btrbk: cosmetics: remove horizonal line in help message
This was added by mistake in:

96956959 btrbk: cosmetics: use heredocs for the help message
2019-04-01 15:31:58 +02:00
Axel Burri e3a81c8126 btrbk: do not print help message if -q,--quiet is set
When called from another script, we dont want the help message printed
on errors. E.g. when running something like:

    btrbk list snapshots -q filter_which_does_not_match
2019-04-01 15:31:58 +02:00
Axel Burri 6d465da6dc btrbk: change version to 0.28.0-dev; bump copyright year 2019-03-29 04:35:35 +01:00
Axel Burri bb2e9877b0 change version to 0.27.2 2019-03-26 15:08:55 +01:00
Axel Burri dfabd4e856 documentation: btrbk.conf.5: move info about allowed characters to top section 2019-03-26 15:08:55 +01:00
Axel Burri 4a23568c40 documentation: btrbk.conf.5: remove unnecessary "{zwsp}" (zero width space) 2019-03-26 15:08:55 +01:00
Axel Burri 86a930daf1 Makefile: add option to bypass compression (COMPRESS=no)
While on traditional UNIX the documentation (especially the man pages)
are gzip'ed, modern distros have helpers to compress it.

This patch adds an option to disable compression:

    make COMPRESS=no
2019-03-26 15:08:47 +01:00
Axel Burri a412908451 Makefile: add documentation on "all" build target 2019-03-25 14:41:55 +01:00
Axel Burri 0e63843195 doc/Makefile: prefer asciidoctor (fallback to asciidoc) for man page generation
Changed preference for man page generation, as "asciidoc" is
discontinued and relies on python-2.7.
2019-03-25 14:37:35 +01:00
Axel Burri 6f68b713ee btrbk: fix scheduler when overriding "target_preserve_min" in combination with global "target" section
When configuring "target" in a global (or "volume") context, and
overriding target_preserve_min in "subvolume" section, the scheduler
has undefined behavior (mixing up the "min" values).

Fixed by returning a copy of the preserve hash in
config_preserve_hash().
2019-03-04 16:05:38 +01:00
Axel Burri 17957a163a btrbk: trigger autofs mount while resolving mountpoints
If resolved mount point has fs_type=autofs, trigger automount by
calling "btrfs subvolume show" on the mount point, and check mountinfo
again.
2019-01-17 16:43:42 +01:00
Axel Burri f37a638f30 btrbk: bump version to 0.27.2-dev 2019-01-17 16:43:39 +01:00
Axel Burri 8abaed26c5 documentation: README.md: bump dependency >= btrfs-progs-4.12 (as of btrbk-0.27.0)
dependency introduced in:

0acbf74c btrbk: add btrfs_subvolume_list_complete: fetch all subvolumes with all flags
2019-01-04 15:49:10 +01:00
Axel Burri bd66f83663 btrbk: bump to forgotten dependency >= btrfs-progs-4.12 (as of btrbk-0.27.0)
dependency introduced in:

0acbf74c btrbk: add btrfs_subvolume_list_complete: fetch all subvolumes with all flags
2018-12-25 22:20:24 +01:00
Axel Burri e879620195 change version to 0.27.1 2018-12-05 22:18:19 +01:00
Axel Burri e52153e13c ChangeLog: add missing changelog for v0.27.1 2018-12-05 22:17:27 +01:00
Axel Burri 0e6c1f9025 btrbk: fix regression: call "sudo readlink" for backend=btrfs-progs-sudo
It is possible that the subvolume path is not accessible by the user
calling btrbk. When resolving mount points, "readlink" is used on the
path, which also needs to be wrapped with "sudo".
2018-12-05 22:01:07 +01:00