Borg: --read-special์„ ์‚ฌ์šฉํ•˜์—ฌ LVM์—์„œ KVM ๊ฐ€์ƒ ๋จธ์‹  ๋ฐฑ์—…

์— ๋งŒ๋“  2016๋…„ 11์›” 11์ผ  ยท  4์ฝ”๋ฉ˜ํŠธ  ยท  ์ถœ์ฒ˜: borgbackup/borg

๋ณ„๋„์˜ ๋…ผ๋ฆฌ ๋ณผ๋ฅจ(LV)์— ๋‘ ๊ฐœ์˜ ๋“œ๋ผ์ด๋ธŒ๊ฐ€ ์žˆ๋Š” LVM์— KVM ๊ฐ€์ƒ ๋จธ์‹ ์ด ์žˆ์Šต๋‹ˆ๋‹ค.

borg๋กœ ๋ฐฑ์—…์„ ์‹œ์ž‘ํ•  ๋•Œ ๋จผ์ € ์Šค๋ƒ…์ƒท์„ ์ƒ์„ฑํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

bash ์Šคํฌ๋ฆฝํŠธ์—์„œ ์–ด๋–ป๊ฒŒ ์ฒ˜๋ฆฌํ•ฉ๋‹ˆ๊นŒ?

๋จผ์ € ๋“œ๋ผ์ด๋ธŒ 1์˜ ์Šค๋ƒ…์ƒท์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค. ๋ณด๊ทธ ๋ฐฑ์—…๋ณด๋‹ค. ์Šค๋ƒ…์ƒท ๋“œ๋ผ์ด๋ธŒ 1์„ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.

๋‘ ๋ฒˆ์งธ ๋“œ๋ผ์ด๋ธŒ 2์˜ ์Šค๋ƒ…์ƒท์„ ๋งŒ๋“ญ๋‹ˆ๋‹ค. ๋ณด๊ทธ ๋ฐฑ์—…๋ณด๋‹ค. ์Šค๋ƒ…์ƒท ๋“œ๋ผ์ด๋ธŒ 2๋ฅผ ์‚ญ์ œํ•ฉ๋‹ˆ๋‹ค.

์Šคํฌ๋ฆฝํŠธ ๋.

๋‹ค์Œ ์˜ˆ์—์„œ ์Šค๋ƒ…์ƒท์€ ๋ชจ๋‘ borg๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฐฑ์—… ์ด์ „์— ์ƒ์„ฑ๋˜๊ณ  ์ดํ›„์— ์‚ญ์ œ๋ฉ๋‹ˆ๋‹ค.

๊ทธ๊ฒƒ์„ ์ฒ˜๋ฆฌํ•˜๋Š” ๊ฐ€์žฅ ์ข‹์€(๋˜๋Š” ์œ ์ผํ•œ ๋ฐฉ๋ฒ•) ๋ฐฉ๋ฒ•์€ ๋ฌด์—‡์ž…๋‹ˆ๊นŒ?

๊ฐ์‚ฌ ํ•ด์š”
๋‚จ์ž ์ด๋ฆ„

#!/bin/bash
echo "Backup `hostname`"

REPOSITORY=/media/vmbackup/server1/
LVPATH=/dev/vg_vmimages/
LVNAME=`lvs --noheadings -o lv_name | tr -d '  '`


#remove old snapshots
for VM in $LVNAME;

do
    lvremove -f  /dev/vg_vmimages/$VM-snapshot
    echo "old snapshots deleted"
done

#create snapshots
for VM in $LVNAME;

do
    lvcreate --size 5120M --snapshot --name "$VM-snapshot" $LVPATH$VM
    echo "Snapshots created"
done

#create lvdisplay.txt
    lvdisplay > lvdisplay.txt
    echo "lvdisplay created"

#create borg backup
borg create --progress --info --list --stats --compress lz4 --read-special \
  $REPOSITORY::`hostname`-`date +%Y-%m-%d` lvdisplay.txt $LVPATH*-snapshot

#remove lvdisplay.txt
rm -f lvdisplay.txt
echo "lvdisplay.txt deleted"

#remove snapshots
for VM in $LVNAME;

do
    lvremove -f  /dev/vg_vmimages/$VM-snapshot
    echo "Snapshots deleted"
done

#delete all files which do not meet "keep-conditions"
borg prune --info --list $REPOSITORY --prefix `hostname`- \
  --keep-daily=7 --keep-weekly=4 --keep-monthly=3
question

๊ฐ€์žฅ ์œ ์šฉํ•œ ๋Œ“๊ธ€

๊ธ€์Ž„, ๋‹น์‹ ์€ (ํ…Œ์ŠคํŠธ๋˜์ง€ ์•Š์€) ๋ผ์ธ์„ ๋”ฐ๋ผ ๋ญ”๊ฐ€๋ฅผ ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค

echo "Backup `hostname`"

REPOSITORY=/media/vmbackup/server1/
LVPATH=/dev/vg_vmimages/
LVNAME=`lvs --noheadings -o lv_name | tr -d '  '`

backup_LV() {
  echo "Backing up LV $1"
  echo " => Creating snapshot $1-snapshot"
  lvcreate --size 5120M --snapshot --name "$1-snapshot" $LVPATH$1
  echo " => Creating archive"
  SNAPSHOT=$LVPATH$1-snapshot
  borg create -svp --list -C lz4 --read-special \
    $REPOSITORY::`hostname`-`date +%Y-%m-%d` lvdisplay.txt $SNAPSHOT
  #              ^- can also use https://borgbackup.rtfd.io/en/latest/usage.html#borg-help-placeholders
  echo " => Deleting snapshot"
  lvremove -f $SNAPSHOT
  echo "Done with LV $1"
}

for VM in $LVNAME;
do
  backup_LV $VM
done


#delete all files which do not meet "keep-conditions"
borg prune --info --list $REPOSITORY --prefix `hostname`- \
  --keep-daily=7 --keep-weekly=4 --keep-monthly=3

๋ชจ๋“  4 ๋Œ“๊ธ€

๊ธ€์Ž„, ๋‹น์‹ ์€ (ํ…Œ์ŠคํŠธ๋˜์ง€ ์•Š์€) ๋ผ์ธ์„ ๋”ฐ๋ผ ๋ญ”๊ฐ€๋ฅผ ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค

echo "Backup `hostname`"

REPOSITORY=/media/vmbackup/server1/
LVPATH=/dev/vg_vmimages/
LVNAME=`lvs --noheadings -o lv_name | tr -d '  '`

backup_LV() {
  echo "Backing up LV $1"
  echo " => Creating snapshot $1-snapshot"
  lvcreate --size 5120M --snapshot --name "$1-snapshot" $LVPATH$1
  echo " => Creating archive"
  SNAPSHOT=$LVPATH$1-snapshot
  borg create -svp --list -C lz4 --read-special \
    $REPOSITORY::`hostname`-`date +%Y-%m-%d` lvdisplay.txt $SNAPSHOT
  #              ^- can also use https://borgbackup.rtfd.io/en/latest/usage.html#borg-help-placeholders
  echo " => Deleting snapshot"
  lvremove -f $SNAPSHOT
  echo "Done with LV $1"
}

for VM in $LVNAME;
do
  backup_LV $VM
done


#delete all files which do not meet "keep-conditions"
borg prune --info --list $REPOSITORY --prefix `hostname`- \
  --keep-daily=7 --keep-weekly=4 --keep-monthly=3

FWIW๋Š” KVM libvirt ๊ด€๋ฆฌ๋˜๋Š” VM์ด โ€‹โ€‹์‹คํ–‰๋˜๋Š” ๋™์•ˆ ๋ฐฑ์—…ํ•˜๋Š” ๋ฐฉ๋ฒ•์ž…๋‹ˆ๋‹ค. ์Šค๋ƒ…์ƒท์ด ํ•„์š”ํ•˜์ง€ ์•Š์Šต๋‹ˆ๋‹ค. QEMU์˜ copy-on-write ์ด๋ฏธ์ง€ ์œ ํ˜•์„ ์‚ฌ์šฉํ•˜์—ฌ ์‹คํ–‰ ์ค‘์ธ ์ด๋ฏธ์ง€๋ฅผ ํ•ซ์Šค์™‘ํ•ฉ๋‹ˆ๋‹ค.

https://gist.github.com/milkey-mouse/b8cb865910b4d574d9f63ffcd8db09f5

์œ„์˜ ๋‚ด์šฉ์ด ์ด๋ฏธ ์กด์žฌํ•˜๋Š” ํ•ญ๋ชฉ์— ์ถ”๊ฐ€๋˜์–ด ์žˆ๋Š”์ง€ ๋ชจ๋“  ์‚ฌ๋žŒ์ด ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

https://github.com/borgbackup/community/

๊ทธ๋ ‡๋‹ค๋ฉด ๋งํฌ๋ฅผ ์ถ”๊ฐ€ํ•˜๋ฉด ์ด ๋ฌธ์ œ๋ฅผ ์ข…๋ฃŒํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์ €๋Š” Python์—์„œ ํ•ด๋‹น ์Šคํฌ๋ฆฝํŠธ์˜ ๋ณด๋‹ค ๊ฐ•๋ ฅํ•œ ๋ฒ„์ „์„ ์ž‘์—… ์ค‘์ž…๋‹ˆ๋‹ค(libvirt ๋ฐ borg API๋ฅผ ์ง์ ‘ ์‚ฌ์šฉ). ์™„๋ฃŒ๋˜๋ฉด ์ปค๋ฎค๋‹ˆํ‹ฐ ๋ฆฌํฌ์ง€ํ† ๋ฆฌ์— PR์„ ์ œ์ถœํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค.

์ด ํŽ˜์ด์ง€๊ฐ€ ๋„์›€์ด ๋˜์—ˆ๋‚˜์š”?
0 / 5 - 0 ๋“ฑ๊ธ‰

๊ด€๋ จ ๋ฌธ์ œ

zatricky picture zatricky  ยท  3์ฝ”๋ฉ˜ํŠธ

rugk picture rugk  ยท  4์ฝ”๋ฉ˜ํŠธ

htho picture htho  ยท  5์ฝ”๋ฉ˜ํŠธ

anarcat picture anarcat  ยท  4์ฝ”๋ฉ˜ํŠธ

chebee7i picture chebee7i  ยท  5์ฝ”๋ฉ˜ํŠธ