Commit Graph

879 Commits (665e40b3d1ae09ff0acb4fa242721ab663472bf5)

Author SHA1 Message Date
Axel Burri 37ef87ddaf btrbk: add url to fs_list raw format 2021-07-15 13:50:11 +02:00
Axel Burri cd69d29705 btrbk: tidy variable name
Duplicate use (not a bug)
2021-07-15 13:50:04 +02:00
Axel Burri 043e270522 btrbk: change version to 0.31.3-dev 2021-05-27 23:26:38 +02:00
Axel Burri e8622fc6be change version to 0.31.2; update ChangeLog 2021-03-21 13:34:26 +01:00
Axel Burri b8a6aaf799 btrbk: warn if no subvolume is configured 2021-03-18 20:03:03 +01:00
Axel Burri a8341659be btrbk: change version to 0.31.2-dev 2021-03-18 19:56:43 +01:00
Axel Burri 8402195487 change version to 0.31.1; update ChangeLog 2021-03-07 17:49:26 +01:00
Axel Burri db8cc1cd14 btrbk: collapse empty column in total of action "stats" 2021-02-27 13:26:31 +01:00
Axel Burri 713fe50372 btrbk: add empty_cell_char argument to print_formatted 2021-02-27 13:26:03 +01:00
Axel Burri cee6553a49 btrbk: fix duplicate snapshot location check
If snapshot_create=no, no snapshot is created and thus should be
excluded from "duplicate snapshot locations" check.
2021-02-14 15:05:03 +01:00
Axel Burri 8d25852ce2 btrbk: fix action "list all" 2021-02-14 14:23:17 +01:00
Axel Burri 9c2e4e1ee2 btrbk: change version to 0.31.1-dev 2021-01-16 20:40:49 +01:00
Axel Burri 7dc827bdc3 change version to 0.31.0; update ChangeLog 2021-01-10 16:54:53 +01:00
Axel Burri 28e5e08305 bump copyright year 2021-01-10 16:54:50 +01:00
Axel Burri e0b8c0b34f btrbk: consistently return array
While returning an arrayref might be slightly faster, consistently
return an array in all node lookup functions.
2020-12-30 17:57:39 +01:00
Axel Burri 42dd296ce7 btrbk: make related_nodes and correlated_nodes functions operate on nodes only 2020-12-30 17:57:23 +01:00
Lukas Straub dd5991099a btrbk: add suport for zstd long distance matching 2020-12-26 15:46:47 +01:00
Lukas Straub 3562e75a70 btrbk: add support for zstandard (zstd) compression 2020-12-26 15:46:20 +01:00
Axel Burri 6cce65e1eb btrbk: remove big header in action "diff" 2020-12-21 00:30:06 +01:00
Axel Burri c7dd838a81 btrbk: simplify action "diff" 2020-12-21 00:20:15 +01:00
Axel Burri 2c4827ca9f btrbk: support custom table format 2020-12-20 20:12:20 +01:00
Axel Burri 792c3de6c0 btrbk: separate format for "ist snapshots"
Make sure target_subvolume is always printed for "resolved" format.
2020-12-20 18:32:47 +01:00
Axel Burri 28d4ec6a0b btrbk: print latest snapshot after latest correlated 2020-12-20 18:20:43 +01:00
Axel Burri 4fa6dfde25 btrbk: add target_type to "list config" table 2020-12-20 13:45:30 +01:00
Axel Burri 4f8274b28e btrbk: collapse all target columns for "resolved" format 2020-12-20 13:45:30 +01:00
Axel Burri 1bd902d955 btrbk: collapse host columns for most formats 2020-12-20 13:45:30 +01:00
Axel Burri bba7c06486 btrbk: refactor action list and stats; add "list all" 2020-12-20 13:30:50 +01:00
Gavin Yancey 31475a303f btrbk: look in the correct directory for btrbk list snapshots 2020-12-20 13:27:21 +01:00
Axel Burri 6669855936 btrbk: add source and target url to resolved raw format 2020-12-13 17:33:15 +01:00
Axel Burri 3ff42c60d1 btrbk: move snapshot_preserve column towards target_preserve 2020-12-13 17:29:48 +01:00
Axel Burri 4b7f4f6d81 btrbk: always collapse port and rsh columns 2020-12-13 17:29:05 +01:00
Axel Burri 16746caa16 btrbk: fix target_type column for "list backups" action 2020-12-13 15:07:29 +01:00
Axel Burri 557f2f5387 btrbk: add rsh column to list actions with format=long 2020-12-13 14:42:38 +01:00
Axel Burri bda59056e8 btrbk: more accurate wording on type values for action list
Terminology for "backup" is specified in btrbk(1), use it:

  Backup is a btrbk terminology for a "read-only subvolume created
  with send/receive" (showing a received-uuid).
2020-12-13 14:05:34 +01:00
Axel Burri 33e3dee046 btrbk: fix action origin and ls
Regression:

  a96c613ab8 btrbk: add subvol_args_init: map subvol_args cmdline arguments into vinfo
2020-12-12 21:43:23 +01:00
Axel Burri 53d5536929 btrbk: fix fs_path
Fixes action "origin" and logging.

Regression:

  cc3f6c95b9 btrbk: return array in _fs_path
2020-12-12 21:43:23 +01:00
Axel Burri db7e92b134 btrbk: extents-diff: add extents list command 2020-12-12 21:43:23 +01:00
Axel Burri b01a97c36f btrbk: extents-diff: more logging 2020-12-12 21:43:23 +01:00
Axel Burri f9cc5af7b9 btrbk: extents-diff: add ID to long format 2020-12-12 21:43:23 +01:00
Axel Burri 63def23feb btrbk: extents-diff: always display set-exclusive size 2020-12-12 21:43:23 +01:00
Axel Burri e3987866d3 btrbk: extents-diff: fix logging, documentation 2020-12-12 21:43:23 +01:00
Axel Burri 22b71bf7da btrbk: extents-diff: list all related (not only older) 2020-12-12 21:43:23 +01:00
Axel Burri 208fc36231 btrbk: extents-diff: add (hidden) option: ignore_extent_data_inline 2020-12-12 21:43:23 +01:00
Axel Burri 0719fc415d btrbk: cleanup aio extentmap 2020-12-12 21:43:23 +01:00
Axel Burri dc1b7f1b5c extents map: add alternative implementation using IO::AIO (slightly faster) 2020-12-12 21:43:23 +01:00
Axel Burri 70cdcb01c6 btrbk: cosmetics: fix spelling 2020-12-12 21:43:23 +01:00
Axel Burri 74a84f715b btrbk: extents-diff: remove blocksize from regions
`filefrag -b1` effectively calculates byte regions.
2020-12-12 21:43:23 +01:00
Axel Burri afd6f80739 btrbk: extents-diff: calculate exclusive size; add "exclusive" option 2020-12-12 21:43:23 +01:00
Axel Burri ea2ec1ceaa btrbk: print_formatted: allow custom format 2020-12-12 21:43:23 +01:00
Axel Burri 7facb44833 btrbk: add extentmap_cache (new cache_dir option) 2020-12-12 21:43:23 +01:00
Axel Burri c6375967b8 btrbk: add action "extents-diff" 2020-12-12 21:43:23 +01:00
Axel Burri 1a96216e7b btrbk: add subvol_args_init: map subvol_args cmdline arguments into vinfo
Preparatory for extents-diff command
2020-12-12 21:43:23 +01:00
Axel Burri a8e82a6b2c btrbk: add extentmap framework (filefrag)
Preparatory for extents-diff command
2020-12-12 21:43:23 +01:00
Axel Burri 3cbcc74bc3 btrbk: run_cmd: add large_output option (suppress stdout dump of command) 2020-12-12 21:43:23 +01:00
Axel Burri a99292aba8 btrbk: fix alignment if no header is printed 2020-10-30 19:09:50 +01:00
Axel Burri 528a152d6d btrbk: action "diff": use print framework; convert units 2020-10-30 19:09:26 +01:00
Axel Burri 4866ec5cb0 btrbk: add print_size, and command line option modifier 2020-10-30 18:35:52 +01:00
Axel Burri 81c92940c1 btrbk: fix unitialized value in warning
Regression from: 5c32ced7d6
2020-10-19 00:34:11 +02:00
Axel Burri eb852271b6 btrbk: change version to 0.30.1-dev 2020-10-19 00:33:20 +02:00
Axel Burri f40e3adce3 change version to 0.30.0; update ChangeLog 2020-09-23 15:48:44 +02:00
Axel Burri e605bc7b42 btrbk: add backend_local_user config option 2020-09-23 12:59:50 +02:00
Axel Burri 13708abd00 btrbk: consequently fence all trace logs (for performance) 2020-08-29 13:29:17 +02:00
Axel Burri 5d81632b5d btrbk: cosmetics on logging 2020-08-29 13:28:32 +02:00
Axel Burri 5c32ced7d6 btrbk: show paths from all mounts in fs_path; add _fs_info node resolver
Reference complete mountinfo in nodes, instead of { file, subvolid }.
2020-08-29 13:28:32 +02:00
Axel Burri cc3f6c95b9 btrbk: return array in _fs_path 2020-08-29 13:28:32 +02:00
Axel Burri fbfd1edc3d btrbk: move mount info to front in fs_list table format 2020-08-29 13:28:32 +02:00
Axel Burri 739645444e btrbk: dont show subvolume_rel_path in fs_list table format 2020-08-29 13:28:32 +02:00
Axel Burri 9415214bc0 btrbk: reword/fix verbose and loglevel options 2020-08-29 13:28:32 +02:00
Axel Burri 910274f814 btrbk: support lsbtr symlink (alias for "btrbk ls")
If called via "lsbtr" symlink (program name):

 - Provide user-friendly table options
 - Assume "./" as path if no path arguments are given
2020-08-29 13:07:57 +02:00
Axel Burri ec037952cf btrbk: add "compat" config option (busybox: add test -d command)
Add compat, compat_local, compat_remote configuration options.

Used for busybox: instead of running `readlink -e` (which is not
available on busybox), run `readlink -f` followed by `test -d`.
2020-08-28 21:19:21 +02:00
Axel Burri 52a823bb93 btrbk: rename context "root" to "global"
Use same naming as in documentation: btrbk.conf.5
2020-08-28 20:49:28 +02:00
Axel Burri c81e390415 btrbk: cosmetics: capitalize debug logs 2020-08-28 20:48:17 +02:00
Axel Burri a140cbd898 btrbk: case-insensitive match hostname/ipv6 in filter statement 2020-08-27 10:38:23 +02:00
Axel Burri 61419d420a btrbk: add support for ipv6 addresses 2020-08-27 10:38:23 +02:00
Axel Burri 12d80ee30b btrbk: fix check on host 2020-08-20 17:02:45 +02:00
Axel Burri b80d7e84de btrbk: cosmetics on logging 2020-08-20 17:02:45 +02:00
Axel Burri 952e41443a btrbk: always log target before source
Consistent for send/receive and send-to-raw.
2020-08-20 17:01:25 +02:00
Axel Burri c11ab1507d btrbk: sanitize url on PRINT
For consistency with PATH and URL, $vol->{PRINT} was not sanitized
(trailing "//" if parent is "/"). This is now dropped, as this is more
confusing than useful.
2020-05-24 02:48:12 +02:00
Axel Burri 936f88a68f btrbk: action ls: support url argument; fix duplicates / sorting 2020-05-24 02:48:00 +02:00
Axel Burri 30edf5b766 btrbk: action ls: fix match on child subvols of "/"
If PATH contains double slash (intentional behavior of vinfo_child),
the match on root_path fails. Fixed by removing "//" from path.
2020-03-07 01:11:52 +01:00
Axel Burri cb7d49d075 btrbk: change version to 0.29.2-dev 2020-02-29 17:40:55 +01:00
Axel Burri e920283434 change version to 0.29.1; update ChangeLog 2020-02-09 16:18:00 +01:00
Axel Burri 0d19b0243e btrbk: action ls: change default output to format=short 2020-01-02 16:58:07 +01:00
Axel Burri c5b3ebb808 btrbk: action ls: allow multiple path arguments; soft fail on errors 2020-01-02 16:58:07 +01:00
Axel Burri 631a4c30fe btrbk: fix @stderr propagation from vinfo_init_root
As we print @stderr in warnings if vinfo_init_root() function fails,
we need to make sure that @stderr contains sane values.

Clearing @stderr is required when calling caching functions!
2019-12-15 20:03:37 +01:00
Axel Burri 67d24695f1 btrbk: logging prints "..." on consecutive lines 2019-12-15 20:03:37 +01:00
Axel Burri 1f7773dddc btrbk: try readlink only once per url
Change semantics of %realpath_cache: if "" (empty string), do not try
to run readlink shell command again on same URL.
2019-12-15 19:12:14 +01:00
Axel Burri 8c11c2eb7d btrbk: run_cmd: fix regression: dont print ssh errors
Explicitely printing ssh errors (or even warnings) is
inconsistent. stderr is printed outside the run_cmd framework if
needed.

reverts a80e05984a btrbk: catch ssh errors

See ssh(1):

    ssh exits with the exit status of the remote command or with 255
    if an error occurred.

Note that this includes network errors as well as dns failures
2019-12-15 19:12:06 +01:00
Axel Burri 9e13dbd933 btrbk: init sroot as late as possible when expanding wildcards 2019-12-15 19:10:01 +01:00
Axel Burri 37b30caeb2 btrbk: bugfix: vinfo_match: skip match against invalid paths
Dont check url_regex if $vinfo->{URL} is not a valid path (e.g. when
checking against a "volume" or "subvolume" having wildcards):

    "Use of uninitialized value in join or string" at line 3030
2019-12-14 17:06:15 +01:00
Axel Burri c36890e1eb btrbk: change version to 0.29.1-dev 2019-12-14 17:05:54 +01:00
Axel Burri b64ffbacf5 change version to 0.29.0; update ChangeLog 2019-10-27 14:00:40 +01:00
Axel Burri 8551a9f52a btrbk: consistently use "subvolume" (instead of "subvol") for table headings 2019-09-08 18:23:30 +02:00
Axel Burri 5e87e8248b btrbk: add --pretty option: print lowercase column headings with separator line 2019-09-08 18:23:30 +02:00
Axel Burri 4629d5ae11 btrbk: change table output format: remove separator line, uppercase column headings
Print table output column headings single-line uppercase instead of
lowercase and underlined.

This is common ascii table format, is easy parseable and offers better
readability e.g. in pager.
2019-09-08 18:23:30 +02:00
Axel Burri 485bc3ab0c btrbk: use global stderr (replace err); refeactor filter_stderr, add fatal_stderr
if fatal_stderr is set and returns true, the command exit code is set
to -1, resulting in run_cmd() returning undef.
2019-09-08 18:23:30 +02:00
Axel Burri 360c8918bb btrbk: catch verbose errors from readlink
readlink -v enables "report error messages", printing useful errors
like:

    readlink: /some/dir/: Permission denied
    readlink: /some/dir/: No such file or directory
2019-09-08 18:23:30 +02:00
Axel Burri 6e4aeae323 btrbk: cosmetics: enhance trace log 2019-09-08 18:23:30 +02:00
Axel Burri d0f0c18f64 btrbk: logging functions take array as argument 2019-09-08 18:23:29 +02:00
Axel Burri fa3334cbb7 btrbk: btrfs_subvolume_list_readonly_flag: dont die on parse errors 2019-09-08 18:23:29 +02:00
Axel Burri 3798677893 btrbk: run_cmd: return array of stdout lines instead of scalar 2019-09-08 18:23:29 +02:00
Axel Burri 4e39ed0fca btrbk: print ssh errors (remove -q option) 2019-09-08 18:23:29 +02:00
Axel Burri 9f347a1067 btrbk: use open3 in run_cmd: fetch stderr separately 2019-09-08 18:23:29 +02:00
Axel Burri a80e05984a btrbk: catch ssh errors 2019-09-08 18:23:29 +02:00
Axel Burri df471c3692 btrbk: refactor filter_stderr; more flexible error handling in run_cmd; simplify btrfs_subvolume_delete parsing 2019-09-08 18:23:29 +02:00
Axel Burri 8434c6881c btrbk: system_list_mountinfo: also fetch mount_options (not only super_options)
This should have no impact on current code: "subvol" and "subvolid"
are used, which are only in super_options.
2019-09-08 18:23:29 +02:00
Axel Burri 8570ee585d btrbk: fix backup summary: dont display no_action if volume is skipped by --exclude or noauto
The backup summary does not print "<no_action>" if a subvolume is
skipped by --exclude or noauto. Alas, skipped volumes results in
no_action still being printed (see #291). Adding an additional
IS_ABORTED($sroot, "skip_") check fixes this issue.
2019-09-07 13:44:25 +02:00
Axel Burri 2c9f1389a0 btrbk: fix vinfo() prototype; remove dead code; fix scheduler text; fix action diff header text 2019-08-07 21:35:54 +02:00
Axel Burri 4845bc6691 btrbk: print_formatted: fix skip-row-if-empty for --print-schedule
Fix: dont modify "-" %table_formats hash key.

fixes 716d420a40 btrbk: print_formatted: add skip-row-if-empty functionality for table_formats
2019-08-06 17:29:45 +02:00
Axel Burri 3ea7746700 btrbk: cosmetics: separate get_btrbk_date function 2019-08-05 15:15:55 +02:00
Axel Burri ac175c0093 btrbk: cosmetics: rename timestamp_postfix_match -> btrbk_timestamp_match
Take separator match "\." out of statement:

    $timestamp_postfix_match == \.$btrbk_timestamp_match

Remove/Fix unneeded "timestamp" hash key (dead code).
2019-08-05 15:15:55 +02:00
Axel Burri 8b93ef7c82 btrbk: fix fallback_default_config
- add for action-archive
- exit if config is present but has parse errors
2019-08-05 15:15:55 +02:00
Axel Burri 3e40903720 btrbk: add btrbk archive --raw option 2019-08-05 15:15:30 +02:00
Axel Burri dcc57abac4 btrbk: archive: also apply exclude patterns on source subvolumes
Allows patterns like "svol.2019*".

Needs @exclude_vf to be a global variable, which is ok as this only
holds command-line args (and global context for archive_exclude).

Note that adding the same in "create backups" (action run) and
macro_delete() is not a good idea, as this has weird implications on
the "forced: last" snapshot/backup pair.
2019-08-05 14:55:38 +02:00
Axel Burri 6e55e08db7 btrbk: add archive_exclude_older option: skip resume of old backups 2019-08-05 14:55:38 +02:00
Axel Burri 97240de0c9 btrbk: add -L, --long options (shortcut for --format=long) 2019-08-05 14:54:04 +02:00
Axel Burri 66d15d1d64 btrbk: action "ls": add mount_subvol column to long formats 2019-08-05 14:54:04 +02:00
Axel Burri 0435f32619 btrbk: add stream_buffer_remote, rate_limit_remote; run stream_buffer only on local host
Change sematics of stream_buffer: If set, run on local host only. If a
remote stream buffer is required, use stream_buffer_remote /
rate_limit_remote.
2019-08-04 23:08:11 +02:00
Axel Burri 73108d2309 btrbk: fix: "readonly" is required key for "btrfs subvolume show" output
The "Flags: readonly|-" line is present as of btrfs-progs v4.6.1, and
is required in "target metadata check" of btrfs_send_receive().
2019-08-04 13:52:45 +02:00
Axel Burri 74be074e6f btrbk: remove warnings on superfluous "btrfs subvolume show" output
Silently accept compatible versions of btrfs-progs. Required keys are
being checked below, no need to warn here.

e.g. btrfs-progs >= v5.2.1 prints:

    Quota group:            0/258
      Limit referenced:     -
      Limit exclusive:      1.00GiB
      Usage referenced:     16.00KiB
      Usage exclusive:      16.00KiB
2019-08-04 13:29:39 +02:00
Axel Burri 7d656540d7 btrbk: fix btr_tree for subvolumes containing special (unsafe) characters
Note that any action (e.g. btrfs send -p) on a subvolume with unsafe
characters will still fail with:

    ERROR: Unsafe command [...] (offending string: "/tmp/btrfs_pool/svol with spaces")
2019-07-31 13:44:30 +02:00
Axel Burri 7271732f9e btrbk: add action "ls": list all btrfs subvolumes below path
Features:

 - get mounted filesystems from /proc/self/mountinfo
 - fetch subvolumes using "btrfs subvolume list" (fast, needs root)
 - filter and print subvolumes below mount point
2019-07-28 19:03:19 +02:00
Axel Burri d88221e8bc btrbk: add fallback_default_config action option, use for action "diff"
Preparatory for action "ls"
2019-07-28 19:03:19 +02:00
Axel Burri 381a12241b btrbk: add dir_args: relative path arguments for actions reqiring directories
Preparatory for action "ls".
2019-07-28 19:03:19 +02:00
Axel Burri 9dc717c701 btrbk: use mbuffer instead of pv; add stream_buffer_sink framework
Add run_cmd option stream_buffer_sink, which handles stream_buffer,
rate_limit as well as --progress.

For rate limiting, run "mbuffer" (on the target host) in combination
with stream_buffer and --progress, instead of running "pv" (on the
source host).

Reasons:

 - mbuffer limits the read rate: For remote targets, we want a stream
   buffer in front of the rsh command pipe, before decompression.
 - For local targets, this can be combined with --process.
 - Combined stream_buffer and rate_limit: less commands in pipe.

Further changes:

 - always set mbuffer -v1 option (never show warnings)
 - restrict raw_target_block_size to "kmgKMG": compatibility to
   stream_compress and rate_limit options, simplicity.
 - use mbuffer blocksize option where applicable
2019-07-28 17:25:34 +02:00
Axel Burri c7a8d0bb11 btrbk: fix naming: compress_cmd -> compress_cmd_text 2019-07-28 17:25:34 +02:00
Axel Burri cf2c314627 btrbk: change version to 0.29.0-dev 2019-07-28 17:25:34 +02:00
Axel Burri 23770f2e67 change version to 0.28.3; update ChangeLog 2019-07-28 15:43:14 +02:00
Axel Burri 8ffd7ac1e9 btrbk: fix mount point resolving
Searching for longest match in mountinfo is plain wrong, as it is
possible (while very uncommon) to have a later mount point shadowing a
longer mount.
2019-07-18 15:34:52 +02:00
Axel Burri fb3d4d96ff btrbk: action origin: print uuid of unknown subvolumes 2019-07-15 18:36:12 +02:00
Axel Burri 5afa270577 btrbk: action origin: correctly handle root subvolumes having uuid 2019-07-15 18:36:06 +02:00
Axel Burri 8de7f45e89 btrbk: remove dead code
setting info{ERROR} does not make sense if info hash is not pushed to
SUBVOL_RECEIVED.

as of: c06bca17
2019-07-15 18:31:07 +02:00
Axel Burri e7787e8a44 btrbk: replace "unrecoverable" hack by FIX_MANUALLY framework 2019-07-15 18:28:31 +02:00
Lubos Kolouch 836efa472b btrbk: return error code 10 if unrecoverable problem
https://github.com/digint/btrbk/issues/285
2019-07-15 18:03:02 +02:00
Axel Burri aaef1cd1e9 btrbk: change version to 0.28.3-dev 2019-07-15 17:50:50 +02:00
Axel Burri 31c25d0ee2 change version to 0.28.2; update ChangeLog 2019-05-23 14:11:33 +02:00
Axel Burri 0439e6079f btrbk: fix vinfo_resolved
Fix for rare cases:
- fix for paths named "0"
- return undef if not strictly below $vol (or allow_equal is set)
2019-05-23 13:14:07 +02:00
Axel Burri 461eaada66 btrbk: rename "get_related_readonly_nodes" to "get_related_nodes"; add readonly and omit_self options 2019-05-23 13:14:07 +02:00
Axel Burri f169ab4482 btrbk: cosmetics: test loglevel before dumping cmd return value (performance) 2019-05-23 13:12:45 +02:00
Axel Burri 45b487707f btrbk: honor incremental_resolve on targets (fix: defaults to "mountpoint" instead of "directory")
Use $resolve_droot instead of $droot for calls to get_best_correlated
(probably missed commit), same as $resolve_sroot.

Fixes possible regression of:

514e69243a btrbk: add "incremental_resolve" configuration option
2019-05-19 19:01:50 +02:00
Axel Burri 01230cde22 btrbk: bugfix: add missing (fake) mountpoint for raw targets
For raw targets, get_best_parent() dies as VINFO_MOUNTPOINT is not
defined on raw vinfo.

Fixes regression of:

d64e237e94 btrbk: get_best_parent: consider all parent/child relations
2019-05-19 15:27:27 +02:00
Axel Burri 538dab5396 btrbk: change version to 0.28.2-dev 2019-05-19 15:17:37 +02:00
Axel Burri f132c94c65 change version to 0.28.1; update ChangeLog 2019-05-16 17:16:52 +02:00
Axel Burri f54853dd18 btrbk: cosmetics (trace logs) 2019-05-16 17:01:51 +02:00
Axel Burri 10bde449c6 btrbk: remove obsolete "resolve_subdir" option of vinfo_init_root
Removed in: b37ef84e36 btrbk: always read mountpoints
2019-05-16 17:01:51 +02:00
Axel Burri b62bb3b1a8 btrbk: get_related_readonly_nodes: vastly increase abort_distance
It's not uncommon to have a large intact parent-chain on targets
(e.g. target_preserve_min=all).

If this is the case, performance drops a bit on "btrbk archive".
Note that we could limit the search depth in get_best_parent() for
some performance improvements, as this only affects extra clones.
2019-05-03 16:13:24 +02:00
Axel Burri 37b0bd3477 btrbk: bugfix: make get_related_readonly_nodes non-recursive
Perl hates recursions, and dies if recursion depth = 100:

    Deep recursion on subroutine "main::_push_related_children"

Unfortunately this happens before the implemented abort condition
(distance=256).

Fixed by re-implementing get_related_readonly_nodes() non-recursive.

Refs: https://github.com/digint/btrbk/issues/279
2019-05-03 16:10:41 +02:00
Axel Burri 206e706d85 btrbk: allow overriding options with context restrictions
This allows commands like:

    btrbk run mysubvol --override=snapshot_name=mysubvol-oneshot
2019-04-30 13:38:47 +02:00
Axel Burri 3d169d1df4 btrbk: change version to 0.28.1-dev 2019-04-30 13:38:33 +02:00
Axel Burri fc485565ce change version to 0.28.0; update ChangeLog 2019-04-24 19:45:09 +02:00
Axel Burri fe9ce255c5 remove experimental tag on "btrbk archive" 2019-04-24 19:22:30 +02:00
Axel Burri 6115cbdace btrbk: make target_type optional for "target" section 2019-04-24 13:51:52 +02:00
Axel Burri 32d92ff3b7 btrbk: allow space separated groups 2019-04-20 14:05:42 +02:00
Axel Burri 0a2c5dbfb3 btrbk: use transaction for mkdir (cosmetics)
Prints correct "dryrun_success" (instead of "DRYRUN") in transaction
log.
2019-04-18 17:28:15 +02:00
Axel Burri 3d2936d40c btrbk: fix regression for "btrbk archive -n"
resolve by mountpoint needs VINFO_MOUNTPOINT, which was not inserted
on fake mkdir:

514e69243a btrbk: add "incremental_resolve" configuration option
2019-04-18 17:24:38 +02:00
Axel Burri e36289a769 btrbk: use "archive_target" config context for archives
Distinguish normal "target" and (generated) "archive_target"
configuration context (printed in transaction log for
"abort_<context>" actions).
2019-04-18 16:40:25 +02:00
Axel Burri a7da44cf2f btrbk: bugfix: print aborted subvolumes to transaction log
Print all (deferred) actions to transaction log (file, syslog) in
init_transaction_log(), e.g. ABORTED() in "expand subvolume globs".
2019-04-18 16:28:53 +02:00
Axel Burri 25f0a76e28 btrbk: cosmetics: fix error message 2019-04-18 16:28:45 +02:00
Axel Burri 45b24d6477 documentation: btrbk.1: add "exclude" command line option 2019-04-18 15:09:15 +02:00
Axel Burri 95819f03ea documentation: btrbk.1: cosmetics: same order for options in all docs 2019-04-18 15:09:15 +02:00
Axel Burri b2f434b396 btrbk: add "exclude" command line option 2019-04-18 15:09:15 +02:00
Axel Burri f0cff5ee5a btrbk: add vinfo_filter_statement framework 2019-04-18 15:09:15 +02:00
Axel Burri 6e7d649588 btrbk: add "noauto" configuration option 2019-04-18 15:09:15 +02:00
Axel Burri ae004e48fd btrbk: map empty value to "yes" in configuration 2019-04-18 15:09:15 +02:00
Axel Burri 1503a07ad1 btrbk: enhance ABORTED framework (distinguish skipped and aborted) 2019-04-18 15:09:15 +02:00
Axel Burri 4224577960 btrbk: distinguish filter_args and subvol_args
preparatory for noauto and exclude options
2019-04-18 15:09:11 +02:00
Axel Burri ff23e9605b btrbk: cosmetics: fix snapshot create debug log
Add additional log line, containing context for subsequent lines
2019-04-12 21:04:49 +02:00
Axel Burri 5697c38e4a btrbk: globally rename snapshot_children to related_snapshots
As a consequence, also rename $child -> $snapshot
2019-04-12 21:04:49 +02:00
Axel Burri 0a93f69135 btrbk: treat all related readonly subvolumes within snapdir as "snapshots"
With this, previous snapshots (far relations) are still listed when
restoring a snapshot.

Example (S = source subvolume, readwrite):

  After 3 snapshots:
  A->S, B->S, C->S

  Restore B: `btrfs subvol delete S; btrfs subvol snapshot B S'`
  A->S, B->S, C->S, S'->B

Previous implementation would show now snapshots for S', as no
snapshot has parent_uuid=S'.

New implementation shows A, B, C as snapshots for S', as orphaned
siblings (A, B, C pointing to deleted S) are also related.
2019-04-12 21:04:48 +02:00
Axel Burri adfdf925e2 btrbk: add btrbk_direct_leaf option for vinfo_resolved() 2019-04-12 21:04:48 +02:00
Axel Burri 318126b831 btrbk: disallow unrelated (by parent_uuid) parents for btrfs-send if incremental=strict
Makes sure that if, for whatever reason, a subvolume having correct
btrbk name scheme does NOT share any extents with previous snapshots
is never used as parent.

Note that if a related parent is found, the unrelated closest
older/newer (by btrbk timestamp) subvolumes are still added as clone
sources.
2019-04-12 21:03:40 +02:00
Axel Burri c407d41db2 btrbk: unconditionally add clone sources to btrfs-send if necessary
Preferences for parent (and required clone sources):

 1. closest older in snapdir (by btrbk timestamp), related
 2. closest older related (by cgen)
 3. closest newer related (by cgen)
 4. closest older in snapdir (by btrbk timestamp)
 5. closest newer in snapdir (by btrbk timestamp)

Note: prefering 1 over 2 helps keeping parent-chain within droot on
target (assuming that btrfs always uses correlated parent on
btrfs-receive).

This will e.g. add a clone source on "btrbk resume", if both older AND
newer snapshot/backup pairs exists.

Also makes sure that the closest older btrbk snapshot is always added
as clone source, even if another related subvolume has newer cgen.
2019-04-11 16:19:48 +02:00
Axel Burri 95e25eb2d1 btrbk: fix get_related_subvolume_nodes: add all parents and orphaned siblings
Old implementation was missing last readonly parent in chain, as well
as orphaned siblings.

Also sort all by cgen, not by distance, then cgen.

Also skip self.
2019-04-11 16:16:00 +02:00
Axel Burri 20c390893a btrbk: honor multiple -v command line options (increase loglevel) 2019-04-05 19:29:37 +02:00
Axel Burri 44edc97aef btrbk: add "incremental_clones" configuration option (btrfs-send <clone-src>) 2019-04-05 12:06:41 +02:00
Axel Burri f38c69316f btrbk: get_best_parent(): add clone_src argument 2019-04-05 12:05:46 +02:00
Axel Burri 7cf1bfb354 btrbk: rename functions (use "correlated" where applicable)
_receive_target_nodes   -> _correlated_nodes
get_best_receive_target -> get_best_correlated_target
2019-04-05 12:05:46 +02:00
Axel Burri ce81350df0 btrbk: remove unused function: vinfo_subvol_list_all_accessible() 2019-04-05 12:05:46 +02:00
Axel Burri 514e69243a btrbk: add "incremental_resolve" configuration option
Allowed values for "incremental_resolve":

 - "mountpoint" (default): Use parents in the filesystem tree below
   mount points of source `<volume-directory>/<snapshot-dir>` and
   target `<target-directory>`.

 - "directory": Use parents strictly below source/target
   directories. Useful when restricting access, e.g. when using
   ssh_filter_btrbk.sh.

 - "_all_accessible" (experimental): Use parents from all mount points.

Note that using "_all_accessible" causes btrfs-progs to fail:

  - btrfs send -p: "ERROR: not on mount point: /path/to/mountpoint"
  - btrfs receive: "ERROR: parent subvol is not reachable from inside the root subvol"

see also: https://github.com/kdave/btrfs-progs/issues/96
2019-04-05 12:05:46 +02:00
Axel Burri d64e237e94 btrbk: get_best_parent: consider all parent/child relations 2019-04-05 12:05:46 +02:00
Axel Burri cb23c65eed btrbk: get_related_subvolumes: add direct children from parent chain 2019-04-05 12:05:46 +02:00
Axel Burri c1268cdd66 btrbk: table_format "transaction": make "*host" and "*port" keys skip-row-if-empty (for consistency) 2019-04-04 17:09:59 +02:00
Axel Burri cba4a93152 btrbk: table_formats: add "*port" keys (skip-row-if-empty) 2019-04-01 20:12:33 +02:00
Axel Burri ca3bd505ef btrbk: make check for duplicate snapshot/backup locations aware of shared btr_tree
Build check hash within btr_tree node instead per URL. This makes it
aware of shared btr_tree (different hostname:port pointing to same
btrfs filesystem).
2019-04-01 18:43:53 +02:00
Axel Burri 2f09a9a723 btrbk: virtual machine setups; use MACHINE_ID for caches; use port in URL
Common virtual machine setups have multiple volume sections with same
host, but distinct port numbers for each machine.

- make caches dependent on MACHINE_ID instead of HOST
- append port number to URL
- add MACHINE_ID to vinfo
- use MACHINE_ID where applicable

This even works if virtual machines share the same btrfs filesystems:
If a equal UUID is found on distinct machines, btr_tree() will return
the already present tree, in order to be consistent after node
injections.
2019-04-01 18:43:49 +02:00
Axel Burri d8b7988ffa btrbk: deprecate ssh_port option in favor of ssh://hostname[:port] notation
Setting the ssh port directly in the "volume" / "target" config lines
adds the possibility to have a create a unique "hostname:port"
identifier (preparatory for MACHINE_ID to distinguish virtual machines
on same host with different ports.)
2019-04-01 18:26:07 +02:00
Axel Burri 318e3af088 btrbk: table_formats: add skip-row-if-empty flag on all "*host" 2019-04-01 18:24:30 +02:00
Axel Burri 716d420a40 btrbk: print_formatted: add skip-row-if-empty functionality for table_formats
If the row key is prefixed with "-", and is all <undef>, omit output
of this row.
2019-04-01 18:22:30 +02:00
Axel Burri 0afc455c40 btrbk: fix table format "resolved": add target_type=send-receive|raw; add missing target_rsh
Required for "btrbk-check", which must be able do distinguish between
"send-receive" and "raw" targets.
2019-04-01 16:27:39 +02:00
Axel Burri 0699a860a9 btrbk: cosmetics: remove horizonal line in help message
This was added by mistake in:

96956959 btrbk: cosmetics: use heredocs for the help message
2019-04-01 15:31:58 +02:00
Axel Burri e3a81c8126 btrbk: do not print help message if -q,--quiet is set
When called from another script, we dont want the help message printed
on errors. E.g. when running something like:

    btrbk list snapshots -q filter_which_does_not_match
2019-04-01 15:31:58 +02:00
Axel Burri 6d465da6dc btrbk: change version to 0.28.0-dev; bump copyright year 2019-03-29 04:35:35 +01:00
Axel Burri bb2e9877b0 change version to 0.27.2 2019-03-26 15:08:55 +01:00
Axel Burri 6f68b713ee btrbk: fix scheduler when overriding "target_preserve_min" in combination with global "target" section
When configuring "target" in a global (or "volume") context, and
overriding target_preserve_min in "subvolume" section, the scheduler
has undefined behavior (mixing up the "min" values).

Fixed by returning a copy of the preserve hash in
config_preserve_hash().
2019-03-04 16:05:38 +01:00
Axel Burri 17957a163a btrbk: trigger autofs mount while resolving mountpoints
If resolved mount point has fs_type=autofs, trigger automount by
calling "btrfs subvolume show" on the mount point, and check mountinfo
again.
2019-01-17 16:43:42 +01:00
Axel Burri f37a638f30 btrbk: bump version to 0.27.2-dev 2019-01-17 16:43:39 +01:00
Axel Burri bd66f83663 btrbk: bump to forgotten dependency >= btrfs-progs-4.12 (as of btrbk-0.27.0)
dependency introduced in:

0acbf74c btrbk: add btrfs_subvolume_list_complete: fetch all subvolumes with all flags
2018-12-25 22:20:24 +01:00
Axel Burri e879620195 change version to 0.27.1 2018-12-05 22:18:19 +01:00
Axel Burri 0e6c1f9025 btrbk: fix regression: call "sudo readlink" for backend=btrfs-progs-sudo
It is possible that the subvolume path is not accessible by the user
calling btrbk. When resolving mount points, "readlink" is used on the
path, which also needs to be wrapped with "sudo".
2018-12-05 22:01:07 +01:00
Axel Burri ff011d55af btrbk: fix "config print" command (undefined value in format_preserve_matrix())
regression: 5791d72171 btrbk: add "preserve_hour_of_day" configuration option
2018-12-05 21:43:54 +01:00