5bdc3e527b
btrbk: add "prune" command
2017-09-28 14:17:00 +02:00
de7628ac7c
btrbk: add openssl_enc encryption for raw targets; add system_urandom()
...
Example:
Manually create a key:
# KEYFILE=/some/secure/place/btrbk.key
# dd if=/dev/urandom bs=1 count=32 | od -x -A n | tr -d "[:space:]" > $KEYFILE
btrbk.conf:
volume /mnt/btr_pool
incremental no
raw_target_encrypt openssl_enc
openssl_ciphername aes-256-cbc
openssl_iv_size 16 # NOTE: set to "no" if no IV is needed by the selected cipher
openssl_keyfile /some/secure/place/btrbk.key
subvolume home
target raw ssh://cloud.example.com/backup
2017-09-28 14:17:00 +02:00
e804930b5e
btrbk: allow deletion of non-incremental raw targets
...
implemented directly in btrfs_subvolume_delete()
2017-09-28 14:17:00 +02:00
dcb0c5aa28
contrib/migration/raw_suffix2sidecar: add migration tool for creating raw sidecar files from uuid-suffixed raw backup files
2017-09-28 14:17:00 +02:00
cd8d7e3a0a
btrbk: use sidecar file "*.info" instead of encoding uuids into filename for raw targets
...
pros:
- better forward compatibility, e.g. symmetrical encryption
- better readability of files
cons:
- two files per backup
2017-09-28 14:16:53 +02:00
571dae4428
btrbk: do not run in perl taint mode by default: remove "perl -T" in hashbang; hardcode $PATH only if taint mode is enabled
...
While taint mode [1] is a nice feature of perl, e.g. it disallows
using variables (such as filenames from the config file) which were
not validated in system() commands, it also treats $PATH as insecure
(which inherently is, as perl cannot know who messed around with it).
[1] perlsec(1): http://perldoc.perl.org/perlsec.html
[2] perlrun(1): http://perldoc.perl.org/perlrun.html
Note that btrbk still does all taint checks, and can be run in taint
mode:
- by executing `perl -T /usr/sbin/btrbk`,
- or by changing the hashbang to: `!#/usr/bin/perl -T`.
2017-09-25 17:53:35 +02:00
6c2d56d621
btrbk: action "clean": dont list failed deletes in summary
2017-09-21 12:39:48 +02:00
f113436626
btrbk: bugfix: check path when expanding wildcards (warn and ignore if check fails)
...
Before this patch, btrbk errored (unsafe command), and then died.
Fixes #181 .
2017-09-11 18:49:14 +02:00
88aa8c1fea
btrbk: remove "duration" column from transaction_log/transaction_syslog
...
The "duration" column in the transaction log has proven to be
confusing to some users, especially on errors (e.g. "send-receive
ERROR 27" in issue #177 ). As it's not really necessary (duration can
be computed from the corresponding "starting" log entry), it's now
being dropped.
2017-08-28 17:55:27 +02:00
6acea6a08d
btrbk: bugfix: print correct time in end_transaction()
2017-08-28 17:54:17 +02:00
ee17c1a2b4
ssh_filter_btrbk: accept mbuffer command (stream_compress)
2017-08-21 14:39:40 +02:00
5428e9cd93
btrbk: add "resume" and "snapshot" commands; add --preserve-snapshots and --preserve-backups options; deprecate "-r, --resume-only" command line option
2017-08-21 13:23:20 +02:00
a9f1b6b24a
change version to 0.25.1
2017-07-30 16:03:58 +02:00
6cf5d59644
btrbk: add support for btrfs-progs 4.12: do not fail or set realpath_cache for relative paths in btrfs_subvolume_show()
...
As of btrfs-progs-v4.12, the "btrfs subvolume show" command does not
print the full (absolute, resolved) path anymore [1]. Instead, it prints
the relative path to btrfs root (or "/" if it is the root).
The impact for btrbk is that we cannot fill our realpath_cache in
btrfs_subvolume_show() anymore. This is not fatal, but has the
following consequences:
- The "check for duplicate snapshot locations" may now miss
subvolumes specified by symlinks.
- If multiple "volume" sections point to the same subvolume (e.g. if
specified using symlinks) an additional "btrfs subvolume list" is
called. Note that the subvolume will still be recognized as
identical, and the btr_tree will not be rebuilt.
[1] btrfs-progs commit: b7df24aa5cddc4802b9938f56372b73869775cd9
2017-07-30 15:25:32 +02:00
356231bc2a
btrbk: abort if source subvolume has readonly or received_uuid flag
2017-07-29 19:03:23 +02:00
c0c546eb4e
btrbk: stats command: show "correlated" status instead of "orphaned" (inverse logic which caused a lot of confusion)
2017-06-25 21:16:08 +02:00
15bae5e051
btrbk: stats command: print up-to-date status on backups
2017-06-25 21:14:03 +02:00
b2cc99b0fb
btrbk: bugfix: accept "no" for "transaction_log", "transaction_syslog" and "lockfile" configuration options
2017-06-25 18:06:06 +02:00
315b3f24a1
btrbk: add config option "stream_buffer <size>". if set, pipes send stream through "mbuffer -m <size>"
2017-06-08 12:35:22 +02:00
7436d9432c
change version to 0.25.0
2017-03-18 16:53:07 +01:00
d31ffb8cb0
btrbk: allow trailing comments in btrbk.conf
2017-03-18 16:34:53 +01:00
28c65e4675
btrbk: add raw_target_split option, using "split" instead of "dd" to write raw files
2017-03-18 14:47:43 +01:00
da70158c24
ssh_filter_btrbk: correctly handle --sudo option
2017-03-18 12:19:07 +01:00
3b7ede773a
btrbk: bugfix: make sure rate limiting comes after compression in cmd_pipe
2017-03-18 12:18:47 +01:00
cb82bd5fa4
btrbk: add supoort for "backend btrfs-progs-sudo" option; adaptions in ssh_filter_btrbk.sh
2017-01-04 15:04:22 +01:00
d02f67a924
btrbk: get_receive_targets: also match "src.received_uuid == target.uuid"
...
This gets important when using an old backup disk as source.
In terms of btrfs send/receive, all subvolumes matching "uuid /
received_uuid" are valid backups.
Merged (amend) from pull request: #116
Verified by Axel Burri <axel@tty0.ch>
2017-01-04 12:41:05 +01:00
79637de5aa
btrbk: raw_target_encrypt: always set "gpg --no-random-seed-file": prevents creation of "~/.gnupg/random_seed" with slight perfomance penalty.
...
We set "--no-random-seed-file" because one of the btrbk
design principles is to not create any files unasked. Enabling
"--no-random-seed-file" creates ~/.gnupg/random_seed, and as
such depends on $HOME to be set correctly (think on running in
cron). From gpg2(1) man page:
--no-random-seed-file GnuPG uses a file to store its
internal random pool over invocations This makes random
generation faster; however sometimes write operations are not
desired. This option can be used to achieve that with the cost
of slower random generation.
2017-01-03 13:41:39 +01:00
b69e9ebf34
btrbk: show aggregate "size" and "used" for "usage" action
2016-12-11 15:36:48 +01:00
da849b37ae
change version to 0.24.0
2016-11-16 15:32:17 +01:00
ec63e9932b
btrbk: support for btrfs-progs v4.8.3
...
Fix parsing of "btrfs sub show" output, which has changed for toplevel
subvolume.
2016-11-16 15:02:49 +01:00
57be688f6b
btrbk: bugfix: fix "list latest" with no snapshots
2016-11-12 13:21:33 +01:00
a8a311c8c7
btrbk: add "backend" configuration option: support for distinct binaries from btrfs-progs-btrbk
...
btrfs-progs-btrbk sources (fork of btrfs-progs) can be found at:
https://github.com/digint/btrfs-progs-btrbk
2016-08-29 18:40:42 +02:00
961f96833a
btrbk: use "dd" for raw target output, with configurable block size (raw_target_block_size)
...
We use "dd" instead of shell redirections, as it is common to have
special filesystems (like NFS, SMB, FUSE) mounted on the raw target
path. By using "dd" we make sure to write in reasonably large blocks
(default=128K), which is not always the case when using redirections
(e.g. "gpg > outfile" writes in 8K blocks).
Another approach would be to always pipe through "cat", which uses
st_blksize from fstat(2) (with a minimum of 128K) to determine the
block size.
2016-08-25 13:14:01 +02:00
96f0d4c175
btrbk: fix and cleanup stream compression
2016-08-24 15:25:48 +02:00
13d27c8616
btrbk: perform extra metadata check on target subvolume after "btrfs receive"
...
Eliminates error cases where "btrfs receive" input is null, at the
cost of an additional call to "btrfs subvolume show".
2016-08-19 17:41:03 +02:00
9913e2785a
documentation: add stream_compress documentation, update Changelog
2016-08-19 17:36:33 +02:00
edffbd4af9
ssh_filter_btrbk: update ssh_filter_btrbk.sh to use readlink
...
bugfix for: 796b6bd9bf796b6bd
2016-08-19 01:05:02 +02:00
9a5e8311be
change version to 0.23.3
2016-07-14 01:41:45 +02:00
796b6bd9bf
btrbk: replace "realpath -e" with "readlink -e" for mountpoint discovery
2016-06-18 13:04:43 +02:00
91d02ca97b
change version to 0.23.2
2016-06-07 16:41:41 +02:00
e4a7ba4be5
btrbk: add lockfile support, using perl "flock": add configuration option "lockfile <file>", and command line option "--lockfile"'
2016-06-07 16:17:02 +02:00
7326b9816c
btrbk: bugfix: raw targets: correctly handle multiple backups in same target directory
2016-05-09 12:42:04 +02:00
eabdba482e
btrbk-mail: Use relative instead of absolute calls to system binaries
...
Make sure to have apropriate PATH set when calling btrbk-mail.
Usually specified in /etc/crontab, and should be correct by default on
all distros.
2016-05-09 12:13:22 +02:00
cb0851c8b6
change version to 0.23.1
2016-05-03 16:21:34 +02:00
12d435d42a
btrbk: explain "orphaned" status in "stats" command, and suppress it on "list backups" command
2016-04-30 13:01:12 +02:00
49ae46d51d
btrbk: do not print headers to transaction_log, these are documented in btrbk.conf(5)
2016-04-28 12:51:00 +02:00
7a469597ac
documentation: describe transaction log table format; cosmetics
2016-04-28 12:44:33 +02:00
59d96420c8
btrbk: add transaction logging to syslog
...
Add configuration option transaction_syslog, which can be set to a short
name of a syslog facility, like user or local5. Most of the ones besides
localX do not really make sense, but whatever, let the user decide.
The only logging that is relevant for logging to syslog is the logging
generated inside sub action, so it's easy to hijack all messages in
there and also send them to syslog if needed.
All output is done via print_formatted, which expects a file handle.
So, abuse a file handle to a string to be able to change as less code as
needed for this feature.
Since syslog already adds the timestamps for us, I added a syslog
formatting pattern, which is very similar to tlog, omitting the
timestap.
2016-04-25 17:32:08 +02:00
191284cd43
btrbk: bugfix: set correct parent section when propagating targets
2016-04-24 15:59:17 +02:00
4ffaf430c5
change version to 0.23.0
2016-04-23 15:06:57 +02:00
c13c99ada5
btrbk: add new time_format "long-iso", with seconds and timezone offset (iso8601 format); add function timestamp(): remove dependency to POSIX
2016-04-22 19:47:01 +02:00
e824c21f50
btrbk: use build-in Time::Local in combination with localtime() instead of Date::Calc; correctly calculate leap hours
2016-04-20 22:45:11 +02:00
a27f99af46
change version to 0.23.0-rc1
2016-04-17 21:46:56 +02:00
edfebb8193
btrbk: action "origin": use --format options; do not recurse into parent chain
2016-04-16 22:05:26 +02:00
689d3d1233
btrbk: action "archive": print created subdirectories on summary; cleanup
2016-04-16 17:31:14 +02:00
6e997674fc
btrbk: remove "btrfs_progs_compat" configuration option, from now on we REQUIRE btrfs-progs >= v3.18.2: we rely on received_uuid for most operations
2016-04-15 11:07:16 +02:00
73906caacd
documentation: update/fix ChangeLog, README.md, btrbk.conf.5; add "doc/upgrade_to_v0.23.0.md"
2016-04-13 22:05:20 +02:00
326edfcc29
btrbk: change preserve semantics (incompatible):
...
- add "{snapshot,target}_preserve_all" configuration option
- change semantics of "{snapshot,target}_preserve NNd" to "preserve
latest daily only"
- change default of "preserve daily,monthly" to 0 (was: all)
- add deprecated warning and enter compatibility mode: preserve
everything if deprecated {snapshot,target}_preserve_* options are
encountered
2016-04-12 11:47:28 +02:00
ccc2a35520
ChangeLog: update missing items
2016-04-07 15:53:50 +02:00
90a3537433
documentation: add documentation for target propagation and target directories
2016-03-31 17:24:24 +02:00
f9ca7504c8
btrbk: add configuration option "rate_limit" (using pv -L)
...
Original patch provided by @janpascal: "Jan-Pascal van Best"
2016-03-23 11:58:35 +01:00
7bb18050f8
btrbk: detect interrupted transfers of raw targets; delete incomplete raw targets on action "clean"
2016-03-22 19:28:05 +01:00
cd8ac043a8
ChangeLog: add missing items
2016-03-16 18:34:28 +01:00
247f023bab
btrbk: allow wildcards in subvolume section; add option "wildcards" in check_file()
2016-03-16 18:17:44 +01:00
4434bc17ef
changed version to 0.22.2
2016-02-02 11:26:09 +01:00
6f66f72c26
btrbk: bugfix: fix checks on "btrfs sub show" output, which resulted in breakage on btrfs-progs < 4.1
2016-02-01 17:05:12 +01:00
872779398e
changed version to 0.22.1
2016-01-20 18:47:29 +01:00
7a4aa4b650
btrbk: add support for btrfs-progs v4.4
2016-01-20 17:53:36 +01:00
9595adc2a5
changed version to 0.22.0
2016-01-19 21:08:19 +01:00
614438183f
btrbk: bugfix: run_cmd: do not redirect all stderr output, as this kills progress-viewer (pv) output
2016-01-19 17:52:27 +01:00
a1698ef4b0
btrbk: print "$hostname:$path" instead of "{$hostname}$path" in summary and logs
2016-01-17 19:44:59 +01:00
e7ff20114c
btrbk: add "stats" action (print snapshot/backup statistics)
2016-01-15 02:06:03 +01:00
a82c1f533a
btrbk: added configuration options "raw_target_compress_level", "raw_target_compress_threads"
2016-01-14 18:02:53 +01:00
9718ad1460
btrbk: add "-n, --dry-run" option
2016-01-14 16:11:33 +01:00
3624a8fba0
btrbk: add "clean" action (delete incomplete, garbled backups)
2016-01-14 16:11:30 +01:00
33c19e622a
btrbk: bugfix: fix monthly schedule if older than 10 weeks
2015-12-17 19:00:45 +01:00
3b740948fb
btrbk: bugfix: config option "timestamp_format long": correct number of arguments for sprintf (breaks when using perl v5.22.0)
2015-12-15 12:49:34 +01:00
cb01507bb1
changed version to 0.21.0
2015-10-25 12:29:52 +01:00
799665bac1
documentation: moved documentation on non-btrfs sources from FAQ to README; adapted key features and debian link
2015-10-25 12:20:59 +01:00
3f09d2c37f
btrbk/documentation: rename "resolve *" commands to "list *"; cleanup
2015-10-22 17:59:17 +02:00
f526d454f7
btrbk: parse stderr from send/receive commands (error on /^ERROR/), as "btrfs receive" returns 0 even if nothing is received.
2015-10-21 13:51:31 +02:00
05bfeaff1b
btrbk: replace "info" command with "usage", with tabular output; add support for right-aligned columns
2015-10-19 22:10:08 +02:00
46c7841e93
btrbk: add transaction log (configuration option "transaction_log <file>"
2015-10-13 20:08:10 +02:00
93249d1154
btrbk: show scheduling summary if -v is set on run/dryrun
2015-10-13 20:08:07 +02:00
01b7ab0ebf
btrbk: add "-t, --table" command line option (shortcut for "--format=table")
2015-10-13 20:08:04 +02:00
956b010143
documentation: add "--format=table|long|raw" command line option
2015-10-13 20:02:02 +02:00
e5c629e218
btrbk: use formatted output on run/dryrun actions; cleanup
2015-10-13 20:02:02 +02:00
e87373b043
btrbk: add "config print" action: prints internal representation of config
2015-10-13 20:02:01 +02:00
1905d80969
btrbk: return exit status 10 instead of 0 if one or more backup tasks aborted, and exit status 2 on parse errors
2015-09-30 14:00:39 +02:00
31e91e8bcd
btrbk: enforce preserve of all raw incremental images with new FORCE_PRESERVE tag; changed raw target names; added "timestamp_format short|long" configuration option (YYYYMMDD or YYYYMMDD<T>HHMM)
2015-09-29 21:48:47 +02:00
370f435297
btrbk: bugfix: correctly handle "incremental no" option
2015-09-29 14:10:13 +02:00
c06bca17bd
btrbk: new target_type "raw": send subvolume to file rather than receiving it, with compression and encryption support; added configuration options "raw_target_compress", "raw_target_encrypt", "gpg_keyring", "gpg_recipient"; skip deletion of raw targets for now;
2015-09-26 14:02:55 +02:00
d73e3f184b
btrbk: added "config dump" command (tagged as experimental): dump filtered btrbk config, useful for scripting
2015-09-24 16:00:35 +02:00
8059fa28db
documentation: added documentation for ---raw-output command line option
2015-09-24 01:02:42 +02:00
dd6104339e
btrbk: added "ssh_cipher_spec" configuration option, overriding default SSH ciphers (ssh -c option)
2015-09-20 18:32:19 +02:00
4115941478
ChangeLog: added changes since v0.20.0 to btrbk-current
2015-09-20 17:51:32 +02:00
5b0e618632
changed version to 0.20.0
2015-09-02 12:27:13 +02:00
22d3e95e3a
btrbk: bugfix: correct handling of empty "snapshot_dir"
2015-09-02 11:48:32 +02:00
68876f0562
btrbk: added "ssh_compression" configuration option for setting the SSH compression flag
2015-09-02 11:22:50 +02:00
d8e8df0ebd
btrbk: added command line option "--progress", which simply pipes btrfs send through `pv`
2015-08-15 18:43:03 +02:00
Axel Burri