mirror of https://github.com/digint/btrbk
Write to a file without rewriting constant parts
This little tool writes its standard input to a file. It only writes to parts that actually change. This is very useful for keeping incremental backups small when the file in question is mostly-constant. Example: Backing up database tables that are append-only or change seldom or never.pull/604/head
parent
18ddc65979
commit
0fa156e240
|
@ -0,0 +1,47 @@
|
|||
#!/usr/bin/python3
|
||||
|
||||
"""
|
||||
This simple tool reads data from stdin and writes them to a file,
|
||||
carefully *not* overwriting blocks that already have the desired content.
|
||||
|
||||
Usage example:
|
||||
|
||||
cd /mnt/backup/mysql
|
||||
mysql -Ne "show databases;" | grep -v '_schema$' | while read db ; do
|
||||
mysql -Ne "show tables;" "$db" | while read t ; do
|
||||
f="$db/$t.db"
|
||||
if ! test -f "$f" ; then
|
||||
mkdir -p $db
|
||||
touch "$f"
|
||||
fi
|
||||
echo "mysqldump '$db' '$t' | write_to '$f'"
|
||||
done
|
||||
done | parallel
|
||||
|
||||
The effect is that when your tables don't change/ are only appended to,
|
||||
your files are not overwritten and thus your incremental backups stay
|
||||
nice and small.
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
if len(sys.argv) != 2:
|
||||
raise RuntimeError(f"Usage: {sys.argv[0]} destfile")
|
||||
|
||||
fpos=0
|
||||
bs=4096
|
||||
|
||||
fi = sys.stdin.buffer
|
||||
with open(sys.argv[1], "rb+") as fo:
|
||||
fo.seek(0)
|
||||
while True:
|
||||
od=fi.read(bs)
|
||||
if not od:
|
||||
fo.truncate(fpos)
|
||||
break
|
||||
nd=fo.read(bs)
|
||||
if nd != od:
|
||||
fo.seek(fpos)
|
||||
fo.write(od)
|
||||
fpos += len(od)
|
||||
|
Loading…
Reference in New Issue