0 downtime snapshot based XENServer 5.5 backup script.
Description
This script allows you to take snapshot based backups with xenserver 5.5. This script also comes with a helper script that allows you to purge the backup after a set number of hours.
Download
http://forums.citrix.com/servlet/JiveServlet/download/510-246630-1387688-27061/scripts.zipXEN-Backup1.1.zip
Configuration
backup.pl
$backupdir = "/mnt/offsite/"; The directory you wish to backup to. This should be an NFS share or something rather large #mail notification setup $mailNotification = true; Enable Mail Notification, set to false to disable. $MailTo = "helpdesk\@csinet.ca"; The email address you want to send notifcations to. $MailFrom = "helpdesk\@csinet.ca"; The email address that the mail will come from (some smtp servers require this to be a valid exisiting email address. $host = "CGL-XENServer"; The host name of the server #Setup Strings $Message = " \"XEN Backup Notification\n"; The fist line of the backup notifcation mail. $TopicHeader = "[".$host ." XEN Server Backup Script] "; The topic of the email. $Topic = ""; @skip = ('d9eac24b-887c-4e5a-a49a-2cbbf5490dd0'); This array needs to contain the uuid of your DOM0 machine and any other machines you want to skip.
purgeBackup.php
$parseDirs = array('/backup/XEN-Backup/Admin/',
'/backup/XEN-Backup/CGL/',
'/backup/mailBackup/'); Directory's to cleanse of old backups
$backupIdentifier = 'xva'; Only grab filenames that contain this string
$days = 5; Days of backup files to keep
sendEmail
"hostname" => 'backup.whatever.com', the hostname of the server this is being run on (not required) "server" =>'172.16.17.66', The ip address or hostname of your smtp server
crontab on nfs server
0 1 * * * /opt/purgeBackup.php >> /backup/XEN-Backup/purge.log
crontab on the XENserver
#this runs the backup script every morning at 1 am
01 01 * * * /root/scripts/backup.pl >> /mnt/offsite/backup.log 2>&1
To do:
- Web based interface for downloading / managing / restoring vm's from backups. This is more for simplicity's sake as you can just rename the backup file to xva and double click it in windows to restore it.
- Need to setup a XEN Server cluster capable of hot migrations to test functionality.
- Break configuration out of backup.pl script
- Figure out how to mount an SR in the DOM0 for remote sites. Backups can then be done to removable media (usb drives) or mirrored storage. Also create rsnapshot or rsynch script to move only the delta of the xva file to the remote backup storage.
Version 1.1:
- Added HA support, once again thanks Roberto, (I do not have a way of testing that this works, it does not seem to affect the backup of non HA machines so let me know if it blows your server up).
Version 1.0:
- Added Roberto Cespa's VMName patch. Can be seen at line 37
- Added space removal to VMName. Line 38
Disclaimer
These software applications are provided to you as is with no representations, warranties or conditions of any kind. You may use and distribute it at your own risk. CITRIX DISCLAIMS ALL WARRANTIES WHATSOEVER, EXPRESS, IMPLIED, WRITTEN, ORAL OR STATUTORY, INCLUDING WITHOUT LIMITATION WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NONINFRINGEMENT. Without limiting the generality of the foregoing, you acknowledge and agree that (a) the software application may exhibit errors, design flaws or other problems, possibly resulting in loss of data or damage to property; (b) it may not be possible to make the software application fully functional; and (c) Citrix may, without notice or liability to you, cease to make available the current version and/or any future versions of the software application. In no event should the code be used to support of ultra-hazardous activities, including but not limited to life support or blasting activities. NEITHER CITRIX NOR ITS AFFILIATES OR AGENTS WILL BE LIABLE, UNDER BREACH OF CONTRACT OR ANY OTHER THEORY OF LIABILITY, FOR ANY DAMAGES WHATSOEVER ARISING FROM USE OF THE SOFTWARE APPLICATION, INCLUDING WITHOUT LIMITATION DIRECT, SPECIAL, INCIDENTAL, PUNITIVE, CONSEQUENTIAL OR OTHER DAMAGES, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. You agree to indemnify and defend Citrix against any and all claims arising from your use, modification or distribution of the code.
Comments (44)
Jun 30
Anonymous says:
wow !!! I've looking for a script like this for a long time. Thank you !!!!!!!!!...wow !!! I've looking for a script like this for a long time. Thank you !!!!!!!!!!!
Jul 06
Anonymous says:
Working like a charm! I've made some minor changes to save the file with a ...Working like a charm!
I've made some minor changes to save the file with a human readable filename instead of VM uuid.
Thank you very much for sharing!
Roberto Cespa
Jul 09
Eric Bernath says:
Love the script! Works great. I had to teach myself a lot on how...Love the script! Works great. I had to teach myself a lot on how to do very simple linux commands like mounting a cifs share, copying the script to the server or how to even run a perl file but google showed me the way.
Roberto, where do I add this in the script? Anywhere?
Also can anyone point me int he direction of how to schedule the script?
Thanks
Jul 10
Anonymous says:
crontab -e <- used to edit your schedule file #this runs the b...crontab -e <- used to edit your schedule file
#this runs the backup script every morning at 1 am
01 01 * * * /root/scripts/backup.pl >> /mnt/offsite/backup.log 2>&1
crontab -l lists what is in your crontab. Do this on the root account. Google vi editor for help with the editor, its a great thing to learn how to use.
- Jerermy
Jul 14
Eric Bernath says:
Thanks Jeremy! This script has been my saviour. Do you know how to implement...Thanks Jeremy! This script has been my saviour.
Do you know how to implement Roberto Cespa's renaming script?
Where would I put this or does it replace code?
$VMName = `xe vm-list uuid=$guest | grep name-label | cut
b24`;$VMName=~ s/\n//g;
$VMName=~ s/\r//g;
$exportstring = $backupdir.$VMName."
xva".$fdate";Thanks again,
Eric
Jul 15
Anonymous says:
When I have a chance today or tomorrow I will add the naming code to the base sc...When I have a chance today or tomorrow I will add the naming code to the base script. Thanks Roberto, it was something I intended to do just never got around to.
Jul 21
Anonymous says:
Hey sorry to be a pain. I was wondering if you had a chance to monkey with...Hey sorry to be a pain. I was wondering if you had a chance to monkey with this?
Eric
Jul 25
Jeff Bourassa says:
Can you provide some details here? I am testing XenServer and backup is a big p...Can you provide some details here?
I am testing XenServer and backup is a big part of the exercise.... this looks like a good solution so would like to try this out.
Jeff
Jul 27
Anonymous says:
I really have not had time to modify this yet. Hopefully some time this week wor...I really have not had time to modify this yet. Hopefully some time this week work is pretty crazy right now.
-Jeremy
Jul 27
Anonymous says:
for using purgeBackup on xen host with no php installed: #!/usr/bin/per...for using purgeBackup on xen host with no php installed:
#!/usr/bin/perl
use POSIX;
use warnings;
$days = 3;
@parseDir = ('/var/run/sr-mount/2df1d096-253a-c8ed-a491-49f3c0f0321a/backup/');
foreach $dir(@parseDir){
print("purging of: ".$dir ."\n");
@files = ();
opendir(DIR, $dir) || die "can't opendir $dir: $!";
while( ($filename = readdir(DIR))){
if (index($filename,'xva') !=-1 )
}
closedir(DIR);
foreach $filename(@files){
$date = substr($filename,-6,6);
$datestamp = mktime (0,0,0,substr($date,0,2),substr($date,2,2)-1,substr($date,4,2)+100);
if ($datestamp < (time()-($days*86400)))
}
}
Jul 29
Anonymous says:
hi I'm trying to make this script work but I have no external shared storage.&nb...hi I'm trying to make this script work but I have no external shared storage. Is there a way to make it backup to another server instead of a mounted device? or can we simply back it up on his own hd? We don't have the money to buy a shared storage, so can someone suggest me another way please?
Jul 29
Anonymous says:
I would suggest a linux based NFS server, its quite easy to setup just google fo...I would suggest a linux based NFS server, its quite easy to setup just google for Debian NFS Server.
Cost is free for software and pretty cheap for hardware.
Jul 30
Anonymous says:
Hi! I have a question according to the backup-script. I want to migrate fro...Hi!
I have a question according to the backup-script. I want to migrate from VirtualIron to Xenserver, but I need the "Essentials" for HA.
In my HA Environment, I don't know, on which node, the VM is running at the moment. How can I start the backup in this case on the "right" node?
Its not clear for me, if this is possible.
Phil
Jul 30
Anonymous says:
I am positive it is possible. 100% absolutely sure it can be done. But I dont ru...I am positive it is possible. 100% absolutely sure it can be done. But I dont run xen in HA so I dont have a test environment to attempt to set this up on.. Sorry I wish I could help you out..
-Jeremy
Jul 30
Roberto Cespa says:
Sorry for the delay. The code: $VMName = `xe vm-list uuid=$guest | grep name-la...Sorry for the delay.
The code:
$VMName = `xe vm-list uuid=$guest | grep name-label | cut b24`;
$VMName=~ s/\n//g;
$VMName=~ s/\r//g;
$exportstring = $backupdir.$VMName."xva".$fdate";
should be inserted somewhere around line 45.
Please consider that using this method has a big drawback due to a feature/bug of XenServer: it does not free up space in LVM shared storage (iSCSI) after deleting snapshot and VM.
More on this can be found here: http://forums.citrix.com/thread.jspa?threadID=249121&tstart=75
I've 15 VMs with 20GB hard drive each for a total disk space of 300GB; XenServer reports more than 600GB used on the shared storage.
Ciao,
Roberto
Jul 31
Anonymous says:
This Code does not work if VMName has Spaces!!! you should add: $VM...This Code does not work if VMName has Spaces!!!
you should add: $VMName =~ s/ /_/g;
Jul 31
Roberto Cespa says:
Ooops! yes, you're more than right! I didn't notice the problem because I've go...Ooops!
yes, you're more than right!
I didn't notice the problem because I've got underscores everywhere to avoid problems with spaces...
Thank you for your correction,
Roberto
Aug 06
Anonymous says:
Roberto, Is it possible to post the entire script with your modifications? Tha...Roberto,
Is it possible to post the entire script with your modifications?
Thanks!!!
Aug 07
Roberto Cespa says:
You can see the full code here: http://www.robertocespa.com/2009/08/07/backup-d...You can see the full code here:
http://www.robertocespa.com/2009/08/07/backup-di-virtual-machines-xen-senza-interruzione/423/index.html
It's a post on my blog I've just made after one year of silence!
I'm sorry, but the post is in Italian only. As soon as I have some spare time I'll translate it in English.
Consider that my version is "tailor made" for my setup.
While backing up I use the .xvatmp extension so that my backup software does not try to back it up.
At the end, the file is renamed from .xvatmp to .xva.
Ciao,
Roberto
Aug 07
Roberto Cespa says:
I reply to my self just to tell you that I translated my blog post. Without some...I reply to my self just to tell you that I translated my blog post.
Without some considerations (written in the post) my code could be useless.
English post is available here:
http://www.robertocespa.com/en/2009/08/07/backup-di-virtual-machines-xen-senza-interruzione/423/index.html
Aug 10
Anonymous says:
Roberto, thanks for post your entire script. It worked like a charm to me! ...Roberto, thanks for post your entire script.
It worked like a charm to me!
Carlos Eduardo
Aug 06
Anonymous says:
Roberto, I too would like to see the entire script posted with your modificatio...Roberto,
I too would like to see the entire script posted with your modifications.
I also saw this beta that claims to have Data Deduplication, I've downloaded it, but haven't tried it yet.
http://www.quorumsoft.com/beta/signup.php\\
Thanks,
Peter
Aug 07
Roberto Cespa says:
I use Jeremy's script with some minor modifications to suit my needs and setup. ...I use Jeremy's script with some minor modifications to suit my needs and setup.
I've some got some knowledge now with this configuration: Xen + Staging Server for data + CrashPlan Pro backup software and I'm happy with this.
The backup process of 16 VMs starts at 17.30 everyday and lasts until 2.00AM (about 30 mins per VM).
CrashPlan Pro needs from 2.5 to 3.5 hours to backup the whole thing (about 260GB) because it uses a rsync-like approach:
"Completed backup in 3 hours: 16 files (269.09 GB) backed up, 21.64 GB encrypted and sent @ 1666.41 KB/s (Effective rate: 32699.14 KB/s)"
As you can see only 21.64GB of data has been transfered over the net (1Gbps to the secondary server three floors under).
Ciao,
Roberto
P.S.: FYI recovery test shows that I need about 1 hour to import the xva file into the Xen pool
Aug 07
Roberto Cespa says:
As Paolo Nacci is writing on my blog post comment (http://www.robertocespa.com/2...As Paolo Nacci is writing on my blog post comment (http://www.robertocespa.com/2009/08/07/backup-di-virtual-machines-xen-senza-interruzione/423/index.html#comment-39), consider that if the VM you're backing up with this script is protected by HA, XenServer will restart it as soon as the script sets the value "is-a-template" to false.
This could lead to an unusable and unmanageable clone: no start/stop operations could be completed.
As a solution to this is Paolo suggests to change the line that reads:
$status= `xe template-param-set is-a-template=false uuid=$snapshotUUID`;
to
$status= `xe template-param-set is-a-template=false ha-always-run=false uuid=$snapshotUUID`;
Thanks to Paolo for this suggestion.
Ciao,
Roberto
Aug 11
Anonymous says:
This is Great!!! Roberto, your script worked perfectly. The HA pie...This is Great!!!
Roberto, your script worked perfectly.
The HA piece helped a lot!
I only have one question, is there a way to preserve the network settings ?
When importing it defaults to dhcp and all the previous settings are gone.
Any Ideas?
Aug 11
Roberto Cespa says:
Well, Jeremy's script works perfectly, not mine! I've only changed a few things ...Well, Jeremy's script works perfectly, not mine!
I've only changed a few things to fit my needs...
And HA modification is not mine, too!
Let's go with network config problem.
As far as I can understand the DHCP fallback after recovery is due to the fact that the imported VM has a different MAC address and Windows set up the "new" adapter in its standard way: DHCP+firewall.
I don't think it's possible to store MAC address info into the .xva file, so the only solution is to keep a database with VM to MAC associations and set the correct MAC during restore.
For me this is not a problem: I'm setting up all this stuff as a home made disaster recovery solution and resetting the IPs doesn't matter for me in case of disaster.
Consider one more thing: Xen snapshots are (or at least should be) crash consistent; this means that when you recover a VM is like turning on a PC after power loss.
What happens if the "PC" was running a database?
Crash consistent is not data consistent, so your db could get corrupted.
In this scenario you can choose between two alternatives:
Ciao,
Roberto
Aug 11
Anonymous says:
Add "preserve=true" when importing to keep the MAC, from the administrator's gui...Add "preserve=true" when importing to keep the MAC, from the administrator's guide:
Import a VM from a previously-exported file. If preserve is set to true, the MAC address of the original VM will be preserved.
I modified this useful script a bit, it will now try a snapshot with quiesce first. If that fails, a regular snapshot will be taken.
Aug 18
Anonymous says:
Many Thanks for this excellent script, Im trying it out now but how do I locate ...Many Thanks for this excellent script, Im trying it out now but how do I locate the NFS mount in xen console?
I have the NFS share mounted and disk added but I dont see it in /mnt/ ?
Aug 20
Anonymous says:
Hi - Does anyone know if this script allows you to export individual drives?&nbs...Hi - Does anyone know if this script allows you to export individual drives? For example, if we just wanted to suck out the C: drives. We currently do this is in our Virtual Iron environment, so if in DR we can simply re-instate the O/S and program drives and then use backup software to restore data from the other drives.
If we migrate to Xen, then this something we want to do rather than suck out all drives associated with the virtual machine.
thanks
Aug 27
Anonymous says:
Hello nice script and useful but watch out, your VM cant have any ( ) in the n...Hello
nice script and useful
but watch out, your VM cant have any ( ) in the name.
Sep 23
Tonya Harrison says:
We are not new to XenServer, but are new to Perl scripting. This script is tryin...We are not new to XenServer, but are new to Perl scripting. This script is trying to run successfully, but is having issues moving the snapshots to our Backup directory. I think I have it configured incorrectly. We're using a CIFS share on a Windows server and I have set the variable $backupdir = "/mnt/Backups/"; ("Backups" is the name of the storage repository attached to the Xen pool).
Does this variable configuration look correct?
Awesome script, thanks for your help!
Sep 27
Anonymous says:
/mnt/whatever Needs to be mounted to your file share. Not a storage repository.../mnt/whatever
Needs to be mounted to your file share. Not a storage repository, I am not sure you can mount a repository from the underlying linux dom0. Definitely try it out though and see what you can find out!
Sorry I have not had time to update this guys, my IT dept is down to 2 people now and we are running like mad.
-Jeremy
Oct 06
jeremy tirrell says:
I have updated the script today. Please let me know if you find any issues. Tha...I have updated the script today. Please let me know if you find any issues.
Thanks!
Oct 11
Anonymous says:
I have been using this basic approach for XenServer guest backup for a few month...I have been using this basic approach for XenServer guest backup for a few months. Works well with one exception. I have orphaned space on my SRs, it seems that XenServer has a bug that does not free up space used by snapshots after they are removed. So slowly (or fast) depending on your SR and VM size you will consume the disk space in your SR with this script.
More info at: http://forums.citrix.com/thread.jspa?threadID=249121&tstart=0\\
Jeff
Oct 12
gerd moser says:
hi togehter, we are very new into scripting xenserver. i read all the post abov...hi togehter,
we are very new into scripting xenserver. i read all the post above an could not find the right solution for my problem.
my question is about, we are doing backup our xenserver with bacula, it works like charm, very well. wie connect the backupstore as a nfs share with xencenter like a nfs virtual disk storage. this is to easily startup the xensnapshots, without restoring them timewasting from bacupspace.
does anybody know, how to show the xva files in xencenter? i can see the file at commandline, bit not in xencenter.
i know we can mount the xva snapshots from cli, but we have a few power user who are not able to use commandline.
gerd moser
http://datenstrom.at
Oct 15
Anonymous says:
To restore the file you need to copy it to your desktop first, using SCP or SFTP...To restore the file you need to copy it to your desktop first, using SCP or SFTP or something along those lines. Otherwise you way want to do some research on adding a web interface to the nfs share you are backing up to so users can just download the xva file and push it back onto the server.
Oct 29
Anonymous says:
I am getting an error when exporting. Exporting: f8a6fce8-b30a-bac4-6208&...I am getting an error when exporting.
Exporting: f8a6fce8-b30a-bac4-6208-1cdc41be71ba
The server failed to handle your request, due to an internal error. The given message may give details useful for debugging the problem.
message: End_of_file
Can someone assist?
Oct 29
Anonymous says:
Are you out of space on the mnt you are backing up to? type df -h to view availa...Are you out of space on the mnt you are backing up to? type df -h to view available disk space. Also try changing to the mnt/backup directory and typing du -h to see the free space in that directory.
Nov 11
Anonymous says:
Hi, I am wondering if some one could explain how the script is applied to the x...Hi,
I am wondering if some one could explain how the script is applied to the xen server.
Right now I have two XEN servers sharing one sSCSI storage which is where the VMs reside.
My plan is to use OpenFiler with NFS configured so that I can mount it to the pool and send my snapshot there.
I just don't know how to apply the script.
Thank you.
Nov 16
Anonymous says:
Everything you need should be above this comment.Everything you need should be above this comment.
Nov 16
Tom Jendruczak says:
Hi, I'm a little new to all this, especially perl scripting. I'm trying to run t...Hi,
I'm a little new to all this, especially perl scripting. I'm trying to run the script after modifying it for my own environment, but I receive an error as follows: "Setuid/gid script is writable by world." I have chmod the backup.pl file to 700 and 755 but get the same error. Any ideas?
Nov 16
Anonymous says:
That I have never heard of before. A quick google gets this though: http://omgi...That I have never heard of before.
A quick google gets this though:
http://omgili.com/mailinglist/linux-cifs-client/lists/samba/org/20071214075802v9lqhmffy8s4wc8gimp4freefr.html
It appears to be an issue with the way you are mounting your cifs share.
Nov 16
Tom Jendruczak says:
Newbie mistake on my part. I just copied the newest version of the script from R...Newbie mistake on my part. I just copied the newest version of the script from Roberto's website. I was foolishly trying to run the script like an executable ( ./backup.pl). I have it running now as a proper perl script (perl backup.pl). Just need to know how I should edit crontabs to schedule it. Thanks for any help.
Nov 16
Tom Jendruczak says:
Got it figured out on the crontab setup. Script is running beautifully but not r...Got it figured out on the crontab setup. Script is running beautifully but not receiving the email notification. I'm sure I don't have it setup correctly as I'm not sure that sendmail is configured at all. I changed the to and from email addresses in the script to my own settings but did not change anything else. Some help on this would be greatly appreciated. Also a few notes for those who might find this helpful. I'm using StarWind on a Win2k3 server as my iSCSI SAN for my SR. I setup a standard Windows share on the same Windows server and used XenCenter to add that share as a CIFS ISO Library. When you run the mount command from the console it will tell you the path to the CIFS share. This is the path I used in my perl script as the backup target. Also in the crontab, the synatx for running the job is as follows: 30 22 * * * perl /usr/bin/backup.pl <maybe some other newbie like myself will find this useful
>
Just noticed also that the script said sendEmail on the [$mailScript = "/usr/local/scripts/sendEmail";] line, and not sendmail. DOH!!
Here is the last few lines in the console after the backup.pl runs:
sh: command substitution: line 0: unexpected EOF while looking for matching `''
sh: command substitution: line 1: syntax error: unexpected end of file
sh: /usr/scripts/sendEmail: No such file or directory
I had created a file named sendEmail which contains the IP address of my email server in the path above, but obviously this is wrong. HELP!!
Add Comment