*New 11.4 series Release:
2020-07-03: XigmaNAS 11.4.0.4.7633 - released!

*New 12.1 series Release:
2020-04-17: XigmaNAS 12.1.0.4.7542 - released


We really need "Your" help on XigmaNAS https://translations.launchpad.net/xigmanas translations. Please help today!

Producing and hosting XigmaNAS costs money. Please consider donating for our project so that we can continue to offer you the best.
We need your support! eg: PAYPAL

XigmaNAS RootOnZFS installation backup/restore

Forum rules
Set-Up GuideFAQsForum Rules
Post Reply
User avatar
Maurizio
Starter
Starter
Posts: 67
Joined: 05 Jul 2018 21:49
Location: Linate (MIlan)
Status: Offline

XigmaNAS RootOnZFS installation backup/restore

#1

Post by Maurizio »

I would like to know if someone has implemented this type of backup.
JoseMR, with his bemanager, give us a way for backup/restore the BE but not the full zroot pool.
My ideal type of backup is:
  • a script launched at midnight that make a full backup of the zroot pool in a safe place, under the /mnt directory of the HDD pool. Here can be used the zfs send/receive commands.
  • after 14 days the older backups are deleted.
My ideal type of restore is, if I lose the zroot pool:
  • Make a new installation of XigmaNAS RootOnZFS on a USB or SSD.
  • Boot in the new installation.
  • On the console :
  • import the pool on HDD
  • run a script on the HDD that make a full restore on USB/SSD
  • reboot in the restored environment.
Regards
XigmaNAS 12.1.0.4 on Dell R710 144GB RAM - RootOnZFS zroot on 2x 64GB 15k HDDs in mirror, zdata on 3x 1TB SSD in RAIDZ1.
2x XigmaNAS 11.2.0.4 - RootOnZFS on HPE Proliant Microserver gen10 X3216 - 3x 4TB WD RED. In mirror with zrep.

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1145
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS installation backup/restore

#2

Post by JoseMR »

Hi Maurizio, you can do this with a simple zfs send/revc script optimized to your needs indeed(gpt disk preparation and partitioning is required to skip initial OS install), however saving a whole zroot/ will include every boot environments followed by every children datasets etc., unlike saving just the active be@snap only in which just take a fraction of space.

To share my experience, I did somewhat similar what you proposed here with bemanager, I took a snapshot of the active BE(bemanager now can manually snapshot thru beadm) and saved the be@snap to my remote FreeBSD server, I pulled my Firewall appliance and replaced the unreliable and slow USB3.0 by an 2.5" HDD, then installed a fresh copy of FreeBSD(since the Firewall runs on FreeBSD) then restored the boot environment, activated/reboot and all went back exactly the same were I left in a matter of minutes.

Also I still haven't added the CLI mode to bemanager to run it silently with cron, but wen it gets more polished and well tested, I will include the CLI optional switches for convenience.

Regards
System: FreeBSD 12 RootOnZFS Mirror, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
XigmaNAS RootOnZFS
Addons at GitHub
BastilleBSD
Boot Environments Intro
Resources Home Page

User avatar
Maurizio
Starter
Starter
Posts: 67
Joined: 05 Jul 2018 21:49
Location: Linate (MIlan)
Status: Offline

Re: XigmaNAS RootOnZFS installation backup/restore

#3

Post by Maurizio »

Hi JoseMR, thank you very much for your suggestions.
I have already tested your bemanager for backup/restore my RootOnZFS installation but the zroot/tmp and the zroot/var datasets are not restored because they are not part of the BE.

But probably I can use bemanager, with a fresh snapshot, to save the active be and save/restore the missing datasets with zfs send/receive. I will test this idea.

I am very happy to test the bemanager CLI mode when it will be available.
Regards
XigmaNAS 12.1.0.4 on Dell R710 144GB RAM - RootOnZFS zroot on 2x 64GB 15k HDDs in mirror, zdata on 3x 1TB SSD in RAIDZ1.
2x XigmaNAS 11.2.0.4 - RootOnZFS on HPE Proliant Microserver gen10 X3216 - 3x 4TB WD RED. In mirror with zrep.

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1145
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS installation backup/restore

#4

Post by JoseMR »

Maurizio wrote:
23 Jul 2018 19:21
...
But probably I can use bemanager, with a fresh snapshot, to save the active be and save/restore the missing datasets with zfs send/receive. I will test this idea.
...

You are right, some datasets are not tied to the BE and are shared across them on purpose, also I would like to know about your testing results, maybe I could add option to save entire pool, even though the utility is towards boot environments, but lets see.

Regards
System: FreeBSD 12 RootOnZFS Mirror, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
XigmaNAS RootOnZFS
Addons at GitHub
BastilleBSD
Boot Environments Intro
Resources Home Page

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1145
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS installation backup/restore

#5

Post by JoseMR »

Maurizio wrote:
23 Jul 2018 19:21
...
I am very happy to test the bemanager CLI mode when it will be available.
Regards

Hi Maurizio, full CLI usage mode has been added as for bemanager v0.6.0, while most of this commands can be directly performed with "beadm", bemanager will use the existing configuration file for most of the task saving the user from typing long commands/parameters, especially mount/unmount and backup/restore commands.

Sample usage output:

Code: Select all

freebsd-server: ~# bemanager -h
Usage: bemanager -[option] [source] | [target]
Options:
      -a  Activate Boot Environment.
      -c  Create Boot Environment.
      -m  Mount Boot Environment.
      -u  Unmount Boot Environment.
      -n  Rename Boot Environment.
      -b  Backup Boot Environment.
      -r  Restore Boot Environment.
      -s  Snapshot Boot Environment.
      -d  Destroy Boot Environment.
      -h  Display this help message.
freebsd-server: ~#
The [source] can be the <BE>, <BE@Snap> or <be_backup_name.zfs> items, and the [target] can refer to <local> or <remote> locations.

Note: To restore BE on new system/disk, the base OS has to be installed first as expected.

Regards
System: FreeBSD 12 RootOnZFS Mirror, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
XigmaNAS RootOnZFS
Addons at GitHub
BastilleBSD
Boot Environments Intro
Resources Home Page

User avatar
Maurizio
Starter
Starter
Posts: 67
Joined: 05 Jul 2018 21:49
Location: Linate (MIlan)
Status: Offline

Re: XigmaNAS RootOnZFS installation backup/restore

#6

Post by Maurizio »

Hi JoseMR,
I am testing the backup option -b
With the command:

Code: Select all

# bemanager -b upgrade-2018-07-24-165635@2018-07-24-16:56:35 local
I can backup the snapshot in the default location; but I must specify local when it present in the config file:

Code: Select all

# grep ZFS_BACKUP_MODE /usr/local/etc/bemanager.conf
ZFS_BACKUP_MODE="local"
Why ?
IMHO the right approach is: if the user specify the "${PARAM3}" it is used regardless the config file content. If "${PARAM3}" is missing the config file ZFS_BACKUP_MODE is used. The idea is: use the config file if the user do not specify the parameter via command line.
Thanks again for this utility.

P.S. I am writing a backup/restore script that use bemanager. I hope to write somethings in these days about it.
XigmaNAS 12.1.0.4 on Dell R710 144GB RAM - RootOnZFS zroot on 2x 64GB 15k HDDs in mirror, zdata on 3x 1TB SSD in RAIDZ1.
2x XigmaNAS 11.2.0.4 - RootOnZFS on HPE Proliant Microserver gen10 X3216 - 3x 4TB WD RED. In mirror with zrep.

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1145
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS installation backup/restore

#7

Post by JoseMR »

Maurizio wrote:
26 Jul 2018 16:56
...
but I must specify local when it present in the config file:
...
P.S. I am writing a backup/restore script that use bemanager. I hope to write somethings in these days about it.

Hi, that's correct, I forgot to add ability to read backup mode from config file if $3 empty, then trow warning/usage if both config and $3 are empty or incorrect, I add for the next release. :oops:

EDIT: The missing bits has been added in bemanager v0.6.1. :)

P.S Looking forward for your backup/restore script results.

Thanks for the feedback.
System: FreeBSD 12 RootOnZFS Mirror, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
XigmaNAS RootOnZFS
Addons at GitHub
BastilleBSD
Boot Environments Intro
Resources Home Page

User avatar
Maurizio
Starter
Starter
Posts: 67
Joined: 05 Jul 2018 21:49
Location: Linate (MIlan)
Status: Offline

Restore

#8

Post by Maurizio »

Before starting : you should be comfortable with command line, you should know how to edit script and configuration files, install packages and understanding the script debug output; also you should have the XigmaNAS system email working, to receive warning emails if something goes wrong during the backup.
I suppose you have the XigmaNAS RooorOnZFS installation on a USB/SSD pool named zroot and the zdata pool is on a different media, eg. a multi-TB HDDs.
The boot environment (BE) backup is done in /mnt/zdata/DATA/backup_be and the rest of the OS and the restore script are saved in /mnt/zdata/DATA/backup_os.
The restore script is:

Code: Select all

#!/bin/sh
#
# SCRIPT:	/root/bin/restore_os.sh
# SCOPE:	OS restore
# NOTE:
# LICENSE:	BSD2CLAUSE (BSD 2-clause Simplified License).
# VERSION:
# 2018/07/27	alpha, just written.
#
pause()
{
    read -rp "Press enter to continue!" a
}

# debug
# set -x

## CHANGE THESE DIRS
DATA=/mnt/zdata/DATA
BCK_OS="$DATA/backup_os"
BCK_ROOT="$BCK_OS/root_dir"

if ! test -d "$BCK_ROOT"
then
    printf "Error: the data pool is not imported.\\nUse the 'zpool import' command to import it\\n"
    exit 1
fi

echo "Restoring $BCK_ROOT on / (root)"
pause
rsync -aIv "$BCK_ROOT/" /

echo "Please restore the last BE with bemanager"
pause
cp "$BCK_OS/bemanager" /usr/local/sbin/bemanager
cp "$BCK_OS/bemanager.conf" /usr/local/etc/bemanager.conf
/usr/local/sbin/bemanager

# EOF
Copy this script in /root/bin/restore_os.sh and set it executable

Code: Select all

# chmod +x /root/bin/restore_os.sh
The restore process is:
  • Install XigmaNAS on a new USB/SSD
  • boot in the new installation, 20) Console Keyboard Map (if needed) 6) Shell
  • Import the pool:

    Code: Select all

    # zpool import -f zdata
  • run the restore script:

    Code: Select all

    # /mnt/zdata/DATA/backup_os/restore_os.sh
  • follow the script instructions
  • reboot in the restored BE.
DONE!
XigmaNAS 12.1.0.4 on Dell R710 144GB RAM - RootOnZFS zroot on 2x 64GB 15k HDDs in mirror, zdata on 3x 1TB SSD in RAIDZ1.
2x XigmaNAS 11.2.0.4 - RootOnZFS on HPE Proliant Microserver gen10 X3216 - 3x 4TB WD RED. In mirror with zrep.

User avatar
Maurizio
Starter
Starter
Posts: 67
Joined: 05 Jul 2018 21:49
Location: Linate (MIlan)
Status: Offline

Backup

#9

Post by Maurizio »

The backup needs some packages: zfsnap, zrep, bemanager.
zfsnap can be installed with the command :

Code: Select all

# pkg install zfsnap
zrep is missing in the pkg repository

Code: Select all

# pkg install ksh93
to install dependencies, download the latest version directly from github

Code: Select all

# fetch -o /usr/local/bin https://raw.githubusercontent.com/bolthole/zrep/master/zrep
change the first line of the zrep script with:
#!/usr/local/bin/ksh93 -p
make it executable

Code: Select all

# chmod +x /usr/local/bin/zrep
# rehash
test if it works :

Code: Select all

# zrep version
zrep 1.7.3
http://www.bolthole.com/solaris/zrep
http://www.github.com/bolthole/zrep 
Create the 2 ZFS datasets: zdata/DATA/backup_os and zdata/DATA/backup_os/root_dir
and issue the 2 commands:

Code: Select all

# zrep init zroot/var  localhost zdata/DATA/backup_os/root_dir/var
# zrep init zroot/tmp  localhost zdata/DATA/backup_os/root_dir/tmp
Check the above commands with:

Code: Select all

# zrep list
zdata/DATA/backup_os/root_dir/tmp
zdata/DATA/backup_os/root_dir/var
zroot/tmp
zroot/var
bemanager has inspired these backup/restore scripts, it can be installed following the instructions in its post . Make your first BE backup in /mnt/zdata/DATA/backup_be to test if it works. Note a bemanager version 0.6.2+ in needed.
Finally the backup script:

Code: Select all

#!/bin/sh
#
# SCRIPT:	/root/bin/backup_os.sh
# SCOPE:	Backup a XigmaNAS RootOnZFS OS.
# NOTE:		TO DO: check if zfSnap, zrep and bemanager 0.6.1+ are installed
# LICENSE:	BSD2CLAUSE (BSD 2-clause Simplified License).
# VERSION:
# 2018/07/27	alpha, just written.
#
## CHANGE THESE DIRS
DATA=/mnt/zdata/DATA
BCK_OS="$DATA/backup_os"

## CHANGE THESE EMAIL ARRDESSES
FROM="root@example.com"
TO="maurizio@example.com"

# Debug
#set -x
#exec > $DATA/log/"$(basename "$0" .sh)".log 2>&1

# Set environment.
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

# Error message. Used to send a warning email.
err_msg=""

## Backup active BE
# Active be
active_be=$(beadm list -H | grep NR | awk '{print $1}')
result="$?"
if test "$result" -ne 0 || test -n "$active_be"
then
    # New snapshot with zfSnap
    zfSnap_out=$(zfSnap -v -a 14d "zroot/ROOT/$active_be")
    result="$?"
    test "$result" -ne 0 && err_msg="Error $result in zfSnap. $err_msg"
    snapshot=$(echo "$zfSnap_out" | awk '{print $3}')
    name_part=$(echo "$snapshot" | sed 's:.*/::')
    # backup the snapshot
    bemanager -b "$name_part" local
    result="$?"
    test "$result" -ne 0 && err_msg="Error $result in bemanager -b. $err_msg"
else
    err_msg="Error $result getting the active BE: \"$active_be\". $err_msg"
fi

## Backup zroot/tmp and zroot/var with zrep
# Initialized with the command :
#	# zrep init zroot/tmp  localhost zdata/DATA/backup_os/root_dir/tmp
zrep sync zroot/tmp
result="$?"
test "$result" -ne 0 && err_msg="Error $result in sync zroot/tmp. $err_msg"

# Initialized with the command :
#	# zrep init zroot/var  localhost zdata/DATA/backup_os/root_dir/var
zrep sync zroot/var
result="$?"
test "$result" -ne 0 &&  err_msg="Error $result in sync zroot/var. $err_msg"

## Backup bemanager, its configuration file and the restore_os.sh script
rsync -avH /usr/local/sbin/bemanager /usr/local/etc/bemanager.conf \
    /root/bin/restore_os.sh "$BCK_OS"
result="$?"
test "$result" -ne 0 &&  err_msg="Error $result rsync. $err_msg"

## Email if there is an error
SUBJECT="Error running $0 on $(/bin/hostname)"
MSMTPCONF=/var/etc/msmtp.conf
test -n "$err_msg" && BODY="$err_msg" && printf \
    "From:%s\\nTo:%s\\nSubject:%s\\n\\n%s" "$FROM" "$TO" "$SUBJECT" "$BODY" | \
    msmtp --file=$MSMTPCONF -t

#set -x
# EOF
copy this script in /root/bin/backup_os.sh and make it executable

Code: Select all

# chmod +x /root/bin/backup_os.sh
Test the backup script running

Code: Select all

# /root/bin/backup_os.sh
==> Trying to backup default-install@2018-07-29_22.03.48--14d Boot Environment...
  100 %       222.2 MiB / 879.2 MiB = 0.253    12 MiB/s       1:15             
Boot Environment default-install@2018-07-29_22.03.48--14d saved to /mnt/zdata/DATA/backup_be/default-install-2018-07-29-220349.zfs successfully!
sending zroot/tmp@zrep_000003 to localhost:zdata/DATA/backup_os/root_dir/tmp
Expiring zrep snaps on zroot/tmp
Also running expire on localhost:zdata/DATA/backup_os/root_dir/tmp now...
Expiring zrep snaps on zdata/DATA/backup_os/root_dir/tmp
sending zroot/var@zrep_000003 to localhost:zdata/DATA/backup_os/root_dir/var
Expiring zrep snaps on zroot/var
Also running expire on localhost:zdata/DATA/backup_os/root_dir/var now...
Expiring zrep snaps on zdata/DATA/backup_os/root_dir/var
sending incremental file list

sent 99 bytes  received 12 bytes  222.00 bytes/sec
total size is 24,458  speedup is 220.34
XigmaNAS 12.1.0.4 on Dell R710 144GB RAM - RootOnZFS zroot on 2x 64GB 15k HDDs in mirror, zdata on 3x 1TB SSD in RAIDZ1.
2x XigmaNAS 11.2.0.4 - RootOnZFS on HPE Proliant Microserver gen10 X3216 - 3x 4TB WD RED. In mirror with zrep.

User avatar
JoseMR
Hardware & Software Guru
Hardware & Software Guru
Posts: 1145
Joined: 16 Apr 2014 04:15
Location: PR
Contact:
Status: Offline

Re: XigmaNAS RootOnZFS installation backup/restore

#10

Post by JoseMR »

Hi Maurizio, very nice and detailed script work you have been done here plus well explanations(I lack in that explanation/manual parts -.-).

Maybe bemanager can save/restore whole [zroot] from snapshot with small edit, but might defeat the simplicity intended purpose, maybe a TUI wrapper more like for general ZFS send/recv will be quite nice tool to have, will play abit more into soon.

Looking forward to play with a simple Boot Environment export./import WebGUI feature as a backup alternative, maybe easier than develop a complete BE WebGUI manager, at least for me, lets see how this goes though.

Regards
System: FreeBSD 12 RootOnZFS Mirror, MB: Supermicro X8SI6-F, Xeon X3450, 16GB DDR3 ECC RDIMMs.
XigmaNAS RootOnZFS
Addons at GitHub
BastilleBSD
Boot Environments Intro
Resources Home Page

Post Reply

Return to “ZFS (only!)”