vSphere ESXi backup strategies for lowering power consumption

For my home development servers I use the vSphere ESXi 6.5 hypervisor.

At home I do not need all the virtual machines on that server to be online 24/24 365/365, so this server is not always powered on. I need strategies to power up this server on needs.

That was achived by using the Wake on LAN (WoL).

I enable WOL on BIOS of that PC, so I can start the server whenever I need using the poweroff.exe utility (http://users.telenet.be/jbosman/applications.html) and a small batch file that simply launch poweroff.

Let's suppose my server has the IP 192.168.1.10 and mac 010203040506 the command below should start the server:

poweroff.exe wol -ip 192.168.1.10 -subnet 255.255.255.0 -mac 010203040506

Then I need something to backup virtual machines on that hypervisor, I build a bounch of script to perform the backup strategy I need.

But at first, let's talk about the hypervisor and the hardware used.

I was using ESXi since 5.0, this year I switch from 5.5 to 6.5.

My main server it's a HP Compaq dx2400 microtower powered by a Xeon E5440 and 8Gb RAM, the hypervisor is intalled on a 32Gb USB flash drive and hosts on a 1Tb HDD. It uses almost 60W.

This PC comes out with a Core 2 Due processor, but I've upgraded it to a Xeon E5440, due to the simplicity of the operation. A note about this upgrade, the Xeon processor I've used comes from a chinese seller already modded for the LGA 775 socket, also the latest BIOS for the motherboard of this PC (5.37 Rev.) does support the Xeon E5440.
I just need to install the new Xeon CPU, reset the BIOS, and power it up. Then I've enabled VT on bios.

For further information on this mod, search for LGA 771 to 775 adapter mod. To be sure my motherboard supports this CPU I've used the intelmicrocodelist.exe tool to look at the CPU supported by my motherboard BIOS.

I would like to backup each virtual machine on a remote disk. I've done this mounting a NFS disk on the ESXi.
The NFS service and disk is provided by a Netgear Stora NAS.

On that NAS i enabled NFS by kernel mode, If you've fot a Netgear Stora you can find further information here: http://www.openstora.com/wiki/index.php?title=Kernel_mode_NFS.

I use a Samba shared folder as main NFS entry point, in order to simply the backups download and look at all the files. My NFS entry in /etc/exports map the NFS user to the main stora user, that way by the share I can eventually delete files. On NFS this is authorized by the all_squash option. Let's suppose my store username is yourusername, the NFS folder is /home/yourusername/folder and yourusername id and group id are 10 and 11 (use the folling commands to get a username and group id on linux $id -u yourusername and $id -g yourusername).

My /etc/export entry looks like this:

/home/yourusername/folder 192.168.1.0/255.255.255.0(rw,sync,all_squash,anonuid=10,anongid=11,no_subtree_check)
As a side note, that Netgear Stora NAS is also modded to mount two disk wihout raid. I use one of the disk for PCs backup and the other one for ESXi backups.

Now, let's came to the backups.
I've chosen ghettoVCB script (https://github.com/lamw/ghettoVCB) as my backup engine because it's simple, opensource, and well documented.
ghettoVBS is is installed using instructions from the ghettoVCB page. This are the basic steps:
Download the ghettoVCB offline bundle to tmp folder and run
$esxcli software vib install -d /vghetto-ghettoVCB-offline-bundle.zip -f
Mount a NFS datastore on the hypervisor (ex backups01)
Make a ghettoVCB folder in the backup datastore (or other one)
$mkdir /vmfs/volumes/backups01/ghettovcb
Copy ghettoVCB.conf to the folder created
$cp /etc/ghettovcb/ghettoVCB.conf /vmfs/volumes/backups01/ghettovcb/ghettoVCB.conf
Edit ghettoVCB.conf at least set the backup folder
  VM_BACKUP_VOLUME=/vmfs/volumes/backups01
and the backup rotation count
  VM_BACKUP_ROTATION_COUNT=2

The backup strategy I've implemented is simple.
I run ghettoVCB each week to perform a full backup of each virtual machine, ghettoVCB handles the backups rotation. In order to lowering power consumption (remember that this server is not always on) I've implement this simple strategy, running a script at the the boot stage the hypervisor, This script:

1. adds ghettoVCB crontab for day N of week time HH:MM+20min
2. check if the hypervistor is switched on "next to" day N of week time HH:MM, if so
• add a shutdown trigger file (the shutdown operation only runs if exists a shutdown trigger file)
• add crontab for shutdown the server after H hours

An external device sends a WOL packet at day N of week time HH:MM-10min, if the hypervisor it is switched on at that time nothing happens, if not it will power up, schedule the ghettoVCB backup and eventually power down after the backups end.

The shutdown trigger file it is usefull to interrupt the shutdown procedure. Imagine that one want the server to stay on, he just has to delete the shutdown trigger file, available on the NFS Samba shared folder of backups.
This is all packed in the backuphelper.sh file you can find here https://gist.github.com/davidegironi/73f7dc8c65625d51b9a45e9fbe2f73c2
To use the backuphelper, you have to make a backuphelper folded
$mkdir /vmfs/volumes/backups01/backuphelper
Copy backuphelper.sh and esxidown.sh to the folder created
$cp /tmp/backuphelper.sh /vmfs/volumes/backups01/backuphelper/backuphelper.sh
$cp /tmp/esxidown.sh /vmfs/volumes/backups01/backuphelper/esxidown.sh
Edit backuphelper.sh setting the running parameters at top of file
Run backuphelper.sh at startup, edit file /etc/rc.local.d/local.sh adding commands below
  /vmfs/volumes/backups01/backuphelper/backuphelper.sh S

This backuphelper uses the esxidown script (https://github.com/sixdimensionalarray/esxidown) to gently shutdown the hypervisor.

The external device I'm using to send the WOL packet is a D-Link DSL-2750B/DSL-2751 rev D1 with OpenWrt installed, which I'm using as the main DHCP/DNS server in my home network. On this OpenWrt the etherwake package is installed, the etherwake command for WOL is set as a Scheduled Task.

Let's suppose my server mac is 010203040506 and I want to switch the server on on day 6 at 2:10 using the eth0.1 port, the command will look like this:

10 2 * * 6 /usr/bin/etherwake -D -i "eth0.1" "01:02:03:04:05:06"
Also, if you would like to make a backup, let's say the first saturday of month, the crontab schedule will look like:

10 2 1-7 *   *  [ "$(date '+%a')" = "Sat" ] && /usr/bin/etherwake -D -i "eth0.1" "01:02:03:04:05:06"

Another thing to take in consideration is the timezone, and time accuracy, all my devices have the NTP client enabled, and all the timezone are UTC.

Notes

• read risk disclaimer
• excuse my bad english