Commit Graph

734 Commits (cba4a93152255f2e3145ad540c1f6dfdd9b86d45)

Author SHA1 Message Date
Axel Burri cba4a93152 btrbk: table_formats: add "*port" keys (skip-row-if-empty) 2019-04-01 20:12:33 +02:00
Axel Burri ca3bd505ef btrbk: make check for duplicate snapshot/backup locations aware of shared btr_tree
Build check hash within btr_tree node instead per URL. This makes it
aware of shared btr_tree (different hostname:port pointing to same
btrfs filesystem).
2019-04-01 18:43:53 +02:00
Axel Burri 2f09a9a723 btrbk: virtual machine setups; use MACHINE_ID for caches; use port in URL
Common virtual machine setups have multiple volume sections with same
host, but distinct port numbers for each machine.

- make caches dependent on MACHINE_ID instead of HOST
- append port number to URL
- add MACHINE_ID to vinfo
- use MACHINE_ID where applicable

This even works if virtual machines share the same btrfs filesystems:
If a equal UUID is found on distinct machines, btr_tree() will return
the already present tree, in order to be consistent after node
injections.
2019-04-01 18:43:49 +02:00
Axel Burri d8b7988ffa btrbk: deprecate ssh_port option in favor of ssh://hostname[:port] notation
Setting the ssh port directly in the "volume" / "target" config lines
adds the possibility to have a create a unique "hostname:port"
identifier (preparatory for MACHINE_ID to distinguish virtual machines
on same host with different ports.)
2019-04-01 18:26:07 +02:00
Axel Burri 318e3af088 btrbk: table_formats: add skip-row-if-empty flag on all "*host" 2019-04-01 18:24:30 +02:00
Axel Burri 716d420a40 btrbk: print_formatted: add skip-row-if-empty functionality for table_formats
If the row key is prefixed with "-", and is all <undef>, omit output
of this row.
2019-04-01 18:22:30 +02:00
Axel Burri 805d7f4a0d btrbk-verify: add workaround for btrbk <= 0.27.2 bug: missing target_rsh, target_type
btrbk <= 0.27.2 does not print "target_rsh" and "target_type" when
called with --format=raw, see $table_formats{resolved}. This is fixed
in 0.28.0.

Hardcoding target_type=send-receive is not so bad, as for raw targets
btrbk-verify complains first with:

  btrbk-verify: missing required variable "target_rsh" in btrbk --format=raw line

So we should not run rsync (which is not really a problem, rsync just
fails with "not a directory").
2019-04-01 16:27:49 +02:00
Axel Burri 9ed41c8937 btrbk-verify: tool for automated backup integrity check based on rsync
Compare files and attributes by checksum, using rsync(1) in dry-run
mode with all preserve options enabled.

Resolves snapshot/backup pairs by evaluating the output of
"btrbk list latest [filter...]".

Restrictions:
 - ".d..t...... ./" lines are ignored by default:
   Root folder timestamp always differ.
 - "cd+++++++++ .*" lines are ignored by default:
   Nested subvolumes appear as new empty directories.
 - btrbk raw targets are skipped
 - rsync needs root in most cases (see --ssh-* options)
2019-04-01 16:27:45 +02:00
Axel Burri 0afc455c40 btrbk: fix table format "resolved": add target_type=send-receive|raw; add missing target_rsh
Required for "btrbk-check", which must be able do distinguish between
"send-receive" and "raw" targets.
2019-04-01 16:27:39 +02:00
Axel Burri 0699a860a9 btrbk: cosmetics: remove horizonal line in help message
This was added by mistake in:

96956959 btrbk: cosmetics: use heredocs for the help message
2019-04-01 15:31:58 +02:00
Axel Burri e3a81c8126 btrbk: do not print help message if -q,--quiet is set
When called from another script, we dont want the help message printed
on errors. E.g. when running something like:

    btrbk list snapshots -q filter_which_does_not_match
2019-04-01 15:31:58 +02:00
Axel Burri 6d465da6dc btrbk: change version to 0.28.0-dev; bump copyright year 2019-03-29 04:35:35 +01:00
Axel Burri bb2e9877b0 change version to 0.27.2 2019-03-26 15:08:55 +01:00
Axel Burri dfabd4e856 documentation: btrbk.conf.5: move info about allowed characters to top section 2019-03-26 15:08:55 +01:00
Axel Burri 4a23568c40 documentation: btrbk.conf.5: remove unnecessary "{zwsp}" (zero width space) 2019-03-26 15:08:55 +01:00
Axel Burri 86a930daf1 Makefile: add option to bypass compression (COMPRESS=no)
While on traditional UNIX the documentation (especially the man pages)
are gzip'ed, modern distros have helpers to compress it.

This patch adds an option to disable compression:

    make COMPRESS=no
2019-03-26 15:08:47 +01:00
Axel Burri a412908451 Makefile: add documentation on "all" build target 2019-03-25 14:41:55 +01:00
Axel Burri 0e63843195 doc/Makefile: prefer asciidoctor (fallback to asciidoc) for man page generation
Changed preference for man page generation, as "asciidoc" is
discontinued and relies on python-2.7.
2019-03-25 14:37:35 +01:00
Axel Burri 6f68b713ee btrbk: fix scheduler when overriding "target_preserve_min" in combination with global "target" section
When configuring "target" in a global (or "volume") context, and
overriding target_preserve_min in "subvolume" section, the scheduler
has undefined behavior (mixing up the "min" values).

Fixed by returning a copy of the preserve hash in
config_preserve_hash().
2019-03-04 16:05:38 +01:00
Axel Burri 17957a163a btrbk: trigger autofs mount while resolving mountpoints
If resolved mount point has fs_type=autofs, trigger automount by
calling "btrfs subvolume show" on the mount point, and check mountinfo
again.
2019-01-17 16:43:42 +01:00
Axel Burri f37a638f30 btrbk: bump version to 0.27.2-dev 2019-01-17 16:43:39 +01:00
Axel Burri 8abaed26c5 documentation: README.md: bump dependency >= btrfs-progs-4.12 (as of btrbk-0.27.0)
dependency introduced in:

0acbf74c btrbk: add btrfs_subvolume_list_complete: fetch all subvolumes with all flags
2019-01-04 15:49:10 +01:00
Axel Burri bd66f83663 btrbk: bump to forgotten dependency >= btrfs-progs-4.12 (as of btrbk-0.27.0)
dependency introduced in:

0acbf74c btrbk: add btrfs_subvolume_list_complete: fetch all subvolumes with all flags
2018-12-25 22:20:24 +01:00
Axel Burri e879620195 change version to 0.27.1 2018-12-05 22:18:19 +01:00
Axel Burri e52153e13c ChangeLog: add missing changelog for v0.27.1 2018-12-05 22:17:27 +01:00
Axel Burri 0e6c1f9025 btrbk: fix regression: call "sudo readlink" for backend=btrfs-progs-sudo
It is possible that the subvolume path is not accessible by the user
calling btrbk. When resolving mount points, "readlink" is used on the
path, which also needs to be wrapped with "sudo".
2018-12-05 22:01:07 +01:00
Axel Burri ff011d55af btrbk: fix "config print" command (undefined value in format_preserve_matrix())
regression: 5791d72171 btrbk: add "preserve_hour_of_day" configuration option
2018-12-05 21:43:54 +01:00
Axel Burri de851955b6 btrbk: print_formatted: no blank lines for format: raw,tlog,syslog 2018-10-31 14:33:07 +01:00
Axel Burri ebaaee2139 btrbk: print_formatted: omit title for format: raw,tlog,syslog 2018-10-31 14:06:14 +01:00
Axel Burri 74873081c3 btrbk: bump version to 0.27.1-dev 2018-10-31 13:57:06 +01:00
Axel Burri c59488f405 change version to 0.27.0 2018-10-16 12:19:44 +02:00
Axel Burri 97e4a1e5d3 ChangeLog: add migration documentation regarding scheduler bugfix 2018-10-16 12:04:19 +02:00
Axel Burri 56ee8acf3b btrbk: add "-S" command line option (shortcut for --print-schedule) 2018-10-15 16:25:07 +02:00
Axel Burri 7a8df85f2e btrbk: fix regression: add node to return value of vinfo_resolved()
The FORCE_PRESERVE information is set on the node, and was lost for
"latest common target" as get_receive_targets() returned vinfo without
node information.

fixes regression: 6c502cb btrbk: search complete target tree for correlated subvolumes
2018-10-15 16:19:52 +02:00
Axel Burri 49adbe6e28 documentation: README.md: explain send/receive using parent when restoring backups 2018-10-13 14:24:26 +02:00
Axel Burri 5c434bc197 documentation: cosmetics: delete trainling whitespace 2018-10-13 13:44:04 +02:00
Axel Burri ad20b9a360 documentation: btrbk.1: explain root privileges and snapshot/backup terminology 2018-10-13 13:29:14 +02:00
Axel Burri 37def3dc7e documentation: btrbk.conf.5: adaptions for backend and ssh_user option (cosmetics) 2018-10-13 13:29:14 +02:00
Axel Burri f554a36978 documentation: adaptions for asciidoctor (literal blocks) 2018-10-13 13:29:14 +02:00
Axel Burri d324f0d338 documentation: README.md: remove "upgrading from v0.22.2" section
This is more than two years old, everybody should have newer btrbk
release by now.

Note: "doc/upgrade_to_v0.23.0.md" is still available.
2018-10-13 13:29:14 +02:00
Axel Burri c2a344fed1 documentation: move installation instructions from README.md to doc/install.md 2018-10-13 13:29:08 +02:00
Axel Burri 756e1e9449 documentation: remove broken link to gmane (received_uuid patch in kernel 4.4)
gmane is dead, see kernel patch:

b96b1db039 btrfs: fix resending received snapshot with parent
2018-10-11 16:03:31 +02:00
Axel Burri 0a4fb02eeb documentation: change links to https 2018-10-10 23:11:44 +02:00
Axel Burri 526e332b02 ssh_filter_btrbk.sh: bugfix: ignore --restrict-path for "btrfs subvolume list"
btrbk now runs "btrfs subvolume list" from the mountpoint instead of
the volume path, which for some users is not below --restrict-path. As
the output of "btrfs subvolume list" is the same (complete btrfs tree
for the filesystem), it is ok to ignore the restrict-path here.
2018-10-10 22:51:13 +02:00
Axel Burri f7ef5139de documentation: rephrase ssh setup; cleanup 2018-10-10 00:58:38 +02:00
Axel Burri c087082543 documentation: btrbk.1.asciidoc: add notes on propagation of filter statements 2018-10-09 15:21:30 +02:00
Axel Burri 3528927ea6 btrbk: bugfix: do not keep latest common snapshot for "incremental no"
There is no reason to keep the latest common source snapshot if
"incremental no" is configured (see #252).
2018-10-02 17:28:46 +02:00
Axel Burri 1862bc16d3 documentation: btrbk.1.asciidoc: add note about "prune": source/target must be accessible
Related to: c65f937473
2018-08-27 15:50:36 +02:00
Axel Burri 069304f3bb btrbk: use system_list_mountinfo(), remove system_list_mounts(); rename fs_spec -> mount_source
Reasons for changing to `cat /proc/self/mountinfo`:
 - `/proc/self/mounts` (or `/proc/mounts`) lacks extra info
   (namespaces) and is not documented in kernel (deprecated?).
 - findmnt(8) also uses /proc/self/mountinfo

Refs:
 - https://www.kernel.org/doc/Documentation/filesystems/proc.txt
 - https://bugzilla.redhat.com/show_bug.cgi?id=491924
 - https://git.kernel.org/pub/scm/linux/kernel/git/bwh/linux-stable.git/commit/?id=2d4d4864ac08caff5c204a752bd004eed4f08760
2018-08-27 14:54:32 +02:00
Axel Burri e02c2cf249 btrbk: add system_list_mountinfo: parse /proc/self/mountinfo
preparatory patch for removing system_list_mounts().
2018-08-27 14:52:28 +02:00