Commit Graph

523 Commits

Author SHA1 Message Date
Jerome Charaoui
86c6b19119 Merge branch 'bugfix-borg-handler-create-options' into 'master'
Quoting test of $create_options in borg handler

Closes #11304

See merge request liberate/backupninja!39
2021-01-06 19:35:59 -08:00
Jerome Charaoui
85f4ea886a rsync: improve initial directory creation
The code added in the previous commit is only relevant for the mirror
format. With the long format, directories are created correctly in the
rotation routine. For the short format, the directory is initially
created, but only for a local destination, so adding a short snippet to
the rotation routine does it.
2021-01-06 20:52:32 -05:00
Fabian Raab
a2646868b3 Create path hirarchy of source path automatically in destination
Before it was only possible to backup root directories (depth one, eg.
/etc, /var, /usr, …). For example consider the following truncated toy
configuration for rsync:

    [general]
    mountpoint = /media/
    backupdir = backup
    [source]
    include = /foo/bar
    include = /etc

We assume you have prepared the existent but empty base-directory
/media/backup/ at your destination, then against the expectation, the
following error rises:

    rsync: mkdir "/media/backup/foo/bar/" failed: No such file or directory (2)

The reason is that rsync will take the files from the folder /foo/bar/
and tries to put them into the ulitmate – and therefore existent –
destination /media/backup/foo/bar. This is not what you would expect,
since the prepared directory /media/backup exists and the application
should create the rest itself.

This Merge Request adds a little script which creates the required
directories in the destination before running rsync. It assumes that
the base-directory already exists, so that not too much directories
are created automatically (could be a mess if you accidentally type the
wrong path)
2021-01-06 20:49:44 -05:00
Jerome Charaoui
7c8ba1e71e rsync: improve log msgs, add debug for rsync success 2021-01-06 20:40:05 -05:00
Jerome Charaoui
e017810941 rdiff: add extra sanity check for keep parameter 2021-01-06 11:33:58 -05:00
Jerome Charaoui
db7daa3d59 rdiff: make version check more verbose in debug 2021-01-06 11:33:24 -05:00
Jerome Charaoui
f1d876be82 rdiff: fix typo 2021-01-06 11:33:05 -05:00
Jerome Charaoui
9f5952ce34 dup: streamline encryption and signing logic 2021-01-05 11:00:54 -05:00
Jerome Charaoui
d949454366 dup: bandwidthlimit is compatible with desturl
However it's not compatible with local filesystem desturl.
2021-01-05 10:59:49 -05:00
Jerome Charaoui
f007227e4c dup: execute ssh connection test in test mode
Also bring in line with the code used in other handlers.
2021-01-05 10:56:26 -05:00
Jerome Charaoui
e657d42aab dup: improve sanity checks
* also validate desthost and destuser when desturl undefined
 * move desturl vs destuser/host/dir warning in sanity checks section
 * ensure includes are specified
 * drop checks on local destination, duplicity autocreates directory
 * add some needed whitespace
2021-01-05 10:52:46 -05:00
Jerome Charaoui
02c28a7899 borg: improve destination checks 2021-01-03 14:31:12 -05:00
Jerome Charaoui
70f11dd701 borg: add envvar changes to debug log 2021-01-03 14:30:20 -05:00
Jerome Charaoui
45e4c5831c borg: raise fatal error for missing includes 2021-01-03 12:18:46 -05:00
Jerome Charaoui
beff3adc1e Remove remaining vserver bits 2021-01-03 12:07:56 -05:00
Jerome Charaoui
40a346ff13 rsync: change id_dsa to id_rsa in rsync handler, fixes #11315
Also changed in example config.
2021-01-02 15:56:54 -05:00
Jerome Charaoui
eabda5615c Remove vserver support (fixes #11289)
This is probably completely unused by now, as vservers have been
superseeded by alternative technology for many years already.
2021-01-02 15:27:32 -05:00
Jerome Charaoui
076ef20ead borg: fix typos in debug messages 2021-01-02 14:59:58 -05:00
Jerome Charaoui
fe30b5cce3 tar: add support from zstd compression filter 2021-01-02 13:49:41 -05:00
Jerome Charaoui
ff2376f56c tar: properly default to gzip in case of unknown compress filter 2021-01-02 13:49:22 -05:00
lavamind
ac8b41eb87 Merge branch 'fix_11316' into 'master'
Support LUKS v2 headers, fixes #11316

Closes #11316

See merge request riseuplabs/backupninja!47
2020-12-31 08:12:39 -08:00
Gabriel Filion
5c6c583181 Enable use of ionice for all handlers to alleviate IO load
On some workloads, a process creating backups should not become so
aggressive on resources that it prevents normal activity of a machine
from running. It is especially important when writing to disk.

One way to make backups have less impact on the main workload is to
force backups to a lower ionice level.

With the new ionicelevel configuration, it becomes possible for users to
activate the use of ionice and to define the level within the
best-effort class to better suit their needs.
2020-12-31 02:08:50 -05:00
Jerome Charaoui
79903cd689 Replace dd with cryptsetup luksHeaderBackup
Since the luks headers backup was introduced in the handler, cryptsetup
has gained the "luksHeaderBackup" command. We should use this instead
of dd, as it is likely more reliable. Plus, it supports backing up luks
version 2 headers, which the current code doesn't handle.
2020-12-30 15:25:30 -05:00
Jerome Charaoui
c655eceaa3 Reuse lsblk command to list devices for luksheaders
The previous sfdisk-based command would also list partitions, leading to
partition-based luks volumes being duplicated. Instead, reuse the same
command used in the partition backup component.
2020-12-30 15:20:02 -05:00
lavamind
0491432147 Merge branch '11293-partition_table_backup' into 'master'
Do not try to dump partition tables for a disk when non are detected

Closes #11293

See merge request riseuplabs/backupninja!22
2020-12-30 09:57:28 -08:00
Emil Breiner
582024f4bf Quoting test of $create_options in borg handler
The check for the string length in the condition for appending the $create_options fails because it isn't a string. Fixed with quoting the variable.

Change-Id: I6bdacb81f25cf924dc61e3aa6f4b8ebfbe09da65
Signed-Off-By: Emil Breiner <emil.breiner@krumedia.com>
2020-09-08 12:13:23 +02:00
Matthias Blümel
ab8d5a3a39 Merge branch 'fix_11272' into 'master'
Fix ninjahelper bug in the mysql helper

Closes #11272

See merge request riseuplabs/backupninja!23
2020-09-01 09:20:15 -07:00
Guillaume Subiron
10b8887b8e Merge branch 'borg_cache_dir_location' into 'master'
Borg cache dir location

See merge request riseuplabs/backupninja!29
2020-08-14 05:46:34 -07:00
LeLutin
e28da8c390 borg: add configuration to displace the cache directory
borg will by default create its cache directory in ~/.cache/borg. This
means that during backup runs, borg will read and write quite
extensively from/to this directory.

In some situations, it is rather undesirable to have this amount of IO
activity in this location and it would make sense to tell borg to place
its cache elsewhere. This can help for example with placing the cache on
a hard drive where the added IO load will not have as big of an impact
on other running activity for the system.

This change also makes sure that the cache directory environment
variable is cleared out when the configuration option is unset. This
should avoid unpleasant surprises when this environment variable is set
to some unknown value in the context where backupninja is called, which
could lead to borg reading and writing to random places on the system.
2020-08-14 05:46:34 -07:00
Guillaume Subiron
edcdf9dc52 Merge branch 'borg-port' into 'master'
handlers/borg: add port support

Closes #11296

See merge request riseuplabs/backupninja!31
2020-08-14 05:32:03 -07:00
tiredpixel
3d7494bfc2 handlers/borg: add port support
This enables the handlers to be used for hosts which use a non-standard
port, such as is offered by some commercial services. Default to port
22, so it's optional and backwards-compatible.
2020-07-16 08:38:46 +00:00
Julien Rabier
ac40f35542 Update sys.in to add missing "disk" flag.
It seems like only the short output of hwinfo --disk is provided to hardware.txt. I'd suggest we also add disk to the flags for the "complete" output.
That's especially useful when you need to indicate the serial number of a dead drive in order for it to be replaced. Once the disk is dead, using hardware.txt can be the only way to retrieve this info.
2019-02-04 09:12:12 -08:00
Aurélien Olivier
82c25f4fee Remove double quotes when parsing $result 2018-11-17 11:03:30 +01:00
Aurélien Olivier
e3c1a7a5e9 checkBox function: use on/off instead of yes/no
This function uses on/off to define default values.
2018-11-17 10:51:02 +01:00
Kienan Stewart
f449c7c656
Do not try to dump partition tables for a disk when non are detected
Refs #11293
2018-10-05 14:25:48 -04:00
lavamind
cc289f4c87 Merge branch 'master' into 'master'
Error out if cache directory does not exist when creating a fresh dup backup

See merge request riseuplabs/backupninja!15
2018-06-27 17:34:46 -07:00
Jerome Charaoui
1b1a857cbf Make sfdisk grep for luks headers more specific 2018-06-27 20:33:46 -04:00
lavamind
012dc70f60 Merge branch 'feature/support_luks_partitions' into 'master'
Support LUKS in disk partitions

See merge request riseuplabs/backupninja!13
2018-06-27 17:32:09 -07:00
lavamind
fb3b45343b Merge branch 'patch-1' into 'master'
Exclude zram devices, they don't have a partition table

See merge request riseuplabs/backupninja!12
2018-06-27 17:31:56 -07:00
lavamind
497ff73b65 Merge branch 'fix_11285' into 'master'
Attempt to sanitize the database identifiers, fix #11285

Closes #11285

See merge request riseuplabs/backupninja!11
2018-06-27 17:31:13 -07:00
Lyz
417394cf74
Changed /sed/@SED@/ 2018-06-25 16:31:16 +02:00
Jerome Charaoui
0d9b10890e [borg] use alternative repo ssh location syntax
This adds the ability to specify an alternative SSH port within
the host variable.
2018-06-06 22:03:24 -04:00
Hugh Nowlan
4725d7835f Correct path 2018-04-30 13:47:30 +01:00
Hugh Nowlan
f2bc6d60aa Error out if archive dir doesn't exist 2018-04-30 12:59:21 +01:00
Lyz
469a3fd5ab
Support LUKS in disk partitions 2018-04-30 12:29:23 +02:00
Glandos
90d0b9ff63 Exclude zram devices, they don't have a partition table 2018-03-24 06:23:26 -07:00
Jerome Charaoui
bc9be0d576 Attempt to sanitize the database identifiers
Removing the quote and semicolon characters should be sufficient to
prevent breaking out of the shell command, but this may need reviewed
by someone more knowlegeable in shell scripting.
2018-03-16 16:52:39 -04:00
Jerome Charaoui
0132b2453a [borg] scrap exclude*, add create_options
This allows the handler to fully make use of the advanced command-line
options for "borg create", such as the replaced "exclude*" options and
others like "--read-special" and "--numeric-owner".
2018-01-26 11:47:47 -05:00
Jerome Charaoui
e114ff68f4 [borg] remove unnecessary test on $keep 2018-01-26 11:37:04 -05:00
Jerome Charaoui
e9824c0d7d [borg] enable BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK
Otherwise the handler may hang, waiting for user input. This may occur
when the borg cache is deleted, or when borg is upgraded.
2018-01-26 11:32:04 -05:00