Using bpipe to stream dumps, VM clones and another data to your backup (MySQL, PostgreSQL, Firebird, LDAP, Xen etc.)
This is the fastest, easiest, most automated and economic way of doing hot backups of specific application such Postgresql, Mysql, Firebird, Oracle, LDAP databases; Xen and Kvm Virtual Machines export / snapshots; tar packages with several files; and more!
The bpipe plugins automatically creates, writes and reads from a FIFO, and is able to restore information directly to your application. E.g.: restore a database directly from Bacula, without the need to do any further operation. And you can do that with Bacula Community or Enterprise.
This is a simple intuitive example of what I'm talking about:
Plugin = "bpipe:/mnt/fifo/test.txt:/bin/echo fifo_teste:/usr/bin/tee /tmp/test.txt"
Another example to backup using tar (/etc) and unpack automatically during restore:
Plugin = "bpipe:/mnt/fifo/test.tar:/bin/tar -cf - /etc:/bin/tar -C /tmp -xvf -"
A MySQL backup and automatic restore:
Plugin = "bpipe:/mnt/fifo/heitorfaria.sql:mysqldump -u root -p123456 --databases heitorfaria:mysql -u root -p123456"
Any information that I send though the echo it goes but is not really stored on the FIFO (/mnt/fifo/test.txt); it is read directly by the Bacula and written into backup volume. You don't use disk space for that.
More information and practical examples are available at my Udemy Bacula bpipe training (here with discount), but you can also try the official documentation:
The bpipe plugin is provided in the directory src/plugins/fd/bpipe-fd.c of the Bacula source distribution. When the plugin is compiled and linking into the resulting dynamic shared object (DSO), it will have the name bpipe-fd.so. Please note that this is a very simple plugin that was written for demonstration and test purposes. It is and can be used in production, but that was never really intended.
The purpose of the plugin is to provide an interface to any system program for backup and restore. As specified above the bpipe plugin is specified in the Include section of your Job's FileSet resource. The full syntax of the plugin directive as interpreted by the bpipe plugin (each plugin is free to specify the sytax as it wishes) is:
Plugin = "field1:field2:field3:field4"
field1 is the name of the plugin with the trailing -fd.so stripped off, so in this case, we would put bpipe in this field.
field2 specifies the namespace, which for bpipe is the pseudo path and filename under which the backup will be saved. This pseudo path and filename will be seen by the user in the restore file tree. For example, if the value is /MYSQL/regress.sql, the data backed up by the plugin will be put under that "pseudo" path and filename. You must be careful to choose a naming convention that is unique to avoid a conflict with a path and filename that actually exists on your system.
field3 for the bpipe plugin specifies the "reader" program that is called by the plugin during backup to read the data. bpipe will call this program by doing a popen on it.
field4 for the bpipe plugin specifies the "writer" program that is called by the plugin during restore to write the data back to the filesystem.
Please note that for two items above describing the "reader" and "writer" fields, these programs are "executed" by Bacula, which means there is no shell interpretation of any command line arguments you might use. If you want to use shell characters (redirection of input or output, ...), then we recommend that you put your command or commands in a shell script and execute the script. In addition if you backup a file with the reader program, when running the writer program during the restore, Bacula will not automatically create the path to the file. Either the path must exist, or you must explicitly do so with your command or in a shell script.
The bpipe plugin is a generic pipe program, that simply transmits the data from a specified program to Bacula for backup, and then from Bacula to a specified program for restore.
By using different command lines to bpipe, you can backup any kind of data (ASCII or binary) depending on the program called.