Tuning: Better Performance and Bottleneck Solutions
- Remember some companies use Antivirus in Windows and more rarely Linux. Register Bacula daemons as verification exceptions.
- Divide FileSet when backing up a huge number of files (e.g.: more than 20 million).
Especially Windows file systems do not handle volumes with gigantic amounts of small files. In this case, it is best to create multiple FileSets and Jobs (eg one for each Volume or some directories) in order to parallelize copy operations. For example, a server with volumes C: and E :.
Job1, FileSet1. Include, File = "C: /"
Job2, FileSet2. Include, Plugin = "alldrives: exclude = C"
Job3, FileSet3. Include, Plugin = "vss:/@ SYSTEMSTATE/"
Using alldrives is important to back up all other drives except C :, which is already backupped by Job1. If someone creates a new Volume on this server, Job2 will automatically back up.
Job3 would be for Windows System State exclusive backup (if you want to separate as well). Vss: is a unique Enterprise plugin, but you can also use scripts to generate Windows System State on a separate volume.
You can group the jobs of the same client for better management and statistics in some graphical interfaces (such as Bweb).
- Lower GZIP compression level (e.g.: GZIP1) or use LZO. Do not use Bacula software compression for tapes.
- Run several simultaneous jobs (Maximum Concurrent Jobs)
Make sure you've enabled concurrency in 4 places:
a) Director stanza in bacula-dir.conf
b) Storage stanza in bacula-dir.conf
c) Storage stanza in bacula-sd.conf
d) Device stanza in bacula-sd.conf
- Write in several disks or Storages Daemons simultaneously.
- Tapes: enable SSD/NVME disk spooling. Traditional HDs might be slowers than the tape.
- Tapes: Increase Minimum (e.g.: 256K) and Maximum Block Size to 512K in bacula-sd.conf, Device resource (values recommended for LTO4. 1M its too big and may cause hardware wear). It is needed to recreate volumes once you change those values.
- Tapes: increase Maximum File Size to 10GB or 20GB (bacula-sd.conf, Device resource). Better writing speeds, slightly slower restores.
- Disable Clients and Jobs (create an Admin Job to Prune their information once a day at other time.
- Turn on Attribute Spooling for all Jobs.
- Make sure database and Bacula supports batch insert (normally default).
- Avoid the creation of additional indexes.
- Settings that bring lots of performance increase (postgresql.conf):
work_mem = 1MB
wal_buffer = 64k
shared_buffers = 1GB # up to 8GB
work_mem = 64MB
effective_cache_size = 2GB
checkpoint_segments = 64
checkpoint_timeout = 20min
checkpoint_completion_target = 0.9
maintenance_work_mem = 256MB
postgresql synchronous_commit = on
- Run a postgreSQL vacuumdb once in a while. 
- Some settings for MySQL:
sort_buffer_size = 2MB
innodb_buffer_pool_size = 128MB
innodb_flush_log_at_trx_commit = 0
innodb_flush_method = O_DIRECT
- Run mysqltuner (apt-get install mysqltuner) and verify some items to improve.
Network (SD and FD)
- Add more interfaces (bonding / NIC Teaming) and faster switches (you can use the Bacula status network command or the ethtool application to check the speed of your ethernet connection).
- Adjust Maximum Network Buffer Size = bytes. Default is 32,768 bytes., but some users report better performance at local networks using 65,536 bytes.
- Add more network interfaces and faster switches.
- Evitar tráfico por firewalls y enrutadores.
- Use Jumbo Frames.
- Custom Kernel (Ref.: https://fasterdata.es.net/host-tuning/linux/):
- RAM (> 8GB)
- vm.dirty_ratio = 2
- vm.dirty_background_ratio = 1
- vm.swappiness = 10
- vm.zone_reclaim_node = 0
- Use the XFS File-System since it excels in the execution of parallel input/output (I/O) operations due to its design, which is based on allocation groups (a type of subdivision of the physical volumes in which XFS is used-also shortened to AGs). Because of this, XFS enables extreme scalability of I/O threads, file system bandwidth, and size of files and of the file system itself when spanning multiple physical storage devices.
- Use the "deadline disk scheduler".
- Use RAID with a nice controller (e.g.: ARECA).