Commit Graph

921 Commits

Author SHA1 Message Date
Jerome Charaoui
889d280a4e tests: Mount temporary test directories in tmpfs 2021-01-06 22:36:20 -05:00
Jerome Charaoui
b67d785937 tests: Fix scheduling testcase 2021-01-06 22:36:20 -05:00
Jerome Charaoui
427ff780a4 tests: Introduce bats-based testsuite
- Depends on being run inside vagrant
 - Installs dependencies automatically
 - Environment is fully reset for each test
 - Adds testcases for backupninja and sys handler
 - Docs are missing but will be added

Two tests are currently failing due to two bugs, one of which has gone
unreported.
2021-01-06 22:36:20 -05:00
Jerome Charaoui
c86aa8edd9 tests: Modify Vagrantfile to build+install local codebase 2021-01-06 22:36:20 -05:00
Jerome Charaoui
86c6b19119 Merge branch 'bugfix-borg-handler-create-options' into 'master'
Quoting test of $create_options in borg handler

Closes #11304

See merge request liberate/backupninja!39
2021-01-06 19:35:59 -08:00
Jerome Charaoui
85f4ea886a rsync: improve initial directory creation
The code added in the previous commit is only relevant for the mirror
format. With the long format, directories are created correctly in the
rotation routine. For the short format, the directory is initially
created, but only for a local destination, so adding a short snippet to
the rotation routine does it.
2021-01-06 20:52:32 -05:00
Fabian Raab
a2646868b3 Create path hirarchy of source path automatically in destination
Before it was only possible to backup root directories (depth one, eg.
/etc, /var, /usr, …). For example consider the following truncated toy
configuration for rsync:

    [general]
    mountpoint = /media/
    backupdir = backup
    [source]
    include = /foo/bar
    include = /etc

We assume you have prepared the existent but empty base-directory
/media/backup/ at your destination, then against the expectation, the
following error rises:

    rsync: mkdir "/media/backup/foo/bar/" failed: No such file or directory (2)

The reason is that rsync will take the files from the folder /foo/bar/
and tries to put them into the ulitmate – and therefore existent –
destination /media/backup/foo/bar. This is not what you would expect,
since the prepared directory /media/backup exists and the application
should create the rest itself.

This Merge Request adds a little script which creates the required
directories in the destination before running rsync. It assumes that
the base-directory already exists, so that not too much directories
are created automatically (could be a mess if you accidentally type the
wrong path)
2021-01-06 20:49:44 -05:00
Jerome Charaoui
7c8ba1e71e rsync: improve log msgs, add debug for rsync success 2021-01-06 20:40:05 -05:00
Jerome Charaoui
c6d4eee574 add vim temp files to .gitignore 2021-01-06 11:34:28 -05:00
Jerome Charaoui
e017810941 rdiff: add extra sanity check for keep parameter 2021-01-06 11:33:58 -05:00
Jerome Charaoui
db7daa3d59 rdiff: make version check more verbose in debug 2021-01-06 11:33:24 -05:00
Jerome Charaoui
f1d876be82 rdiff: fix typo 2021-01-06 11:33:05 -05:00
Jerome Charaoui
506c0dfb40 remove mention of vserver in manpage 2021-01-05 23:32:27 -05:00
Jerome Charaoui
9f5952ce34 dup: streamline encryption and signing logic 2021-01-05 11:00:54 -05:00
Jerome Charaoui
d949454366 dup: bandwidthlimit is compatible with desturl
However it's not compatible with local filesystem desturl.
2021-01-05 10:59:49 -05:00
Jerome Charaoui
f007227e4c dup: execute ssh connection test in test mode
Also bring in line with the code used in other handlers.
2021-01-05 10:56:26 -05:00
Jerome Charaoui
e657d42aab dup: improve sanity checks
* also validate desthost and destuser when desturl undefined
 * move desturl vs destuser/host/dir warning in sanity checks section
 * ensure includes are specified
 * drop checks on local destination, duplicity autocreates directory
 * add some needed whitespace
2021-01-05 10:52:46 -05:00
Jerome Charaoui
02c28a7899 borg: improve destination checks 2021-01-03 14:31:12 -05:00
Jerome Charaoui
70f11dd701 borg: add envvar changes to debug log 2021-01-03 14:30:20 -05:00
Jerome Charaoui
45e4c5831c borg: raise fatal error for missing includes 2021-01-03 12:18:46 -05:00
Jerome Charaoui
beff3adc1e Remove remaining vserver bits 2021-01-03 12:07:56 -05:00
lavamind
8dd906ff88 Merge branch 'master' into 'master'
Update namespace from Riseup Labs to Liberate (#11314)

See merge request riseuplabs/backupninja!50
2021-01-02 13:03:49 -08:00
Micah Anderson
8e37956c35
Update namespace from Riseup Labs to Liberate (#11314) 2021-01-02 16:01:02 -05:00
Jerome Charaoui
40a346ff13 rsync: change id_dsa to id_rsa in rsync handler, fixes #11315
Also changed in example config.
2021-01-02 15:56:54 -05:00
Jerome Charaoui
eabda5615c Remove vserver support (fixes #11289)
This is probably completely unused by now, as vservers have been
superseeded by alternative technology for many years already.
2021-01-02 15:27:32 -05:00
Jerome Charaoui
076ef20ead borg: fix typos in debug messages 2021-01-02 14:59:58 -05:00
Jerome Charaoui
fe30b5cce3 tar: add support from zstd compression filter 2021-01-02 13:49:41 -05:00
Jerome Charaoui
ff2376f56c tar: properly default to gzip in case of unknown compress filter 2021-01-02 13:49:22 -05:00
Jerome Charaoui
43c0f9d9bf rsync: fix rsync_options example
Copy-pasting this in a rsync handler config will cause an error as the
quotes will be used on the command line directly...
2020-12-31 14:36:04 -05:00
lavamind
ac8b41eb87 Merge branch 'fix_11316' into 'master'
Support LUKS v2 headers, fixes #11316

Closes #11316

See merge request riseuplabs/backupninja!47
2020-12-31 08:12:39 -08:00
lavamind
db1ce74e10 Merge branch 'ionice' into 'master'
Add support for ionice in handlers

See merge request riseuplabs/backupninja!27
2020-12-31 08:12:16 -08:00
Gabriel Filion
7db5c9c3e8 Document the new ionice configuration in example config files 2020-12-31 02:08:50 -05:00
Gabriel Filion
5c6c583181 Enable use of ionice for all handlers to alleviate IO load
On some workloads, a process creating backups should not become so
aggressive on resources that it prevents normal activity of a machine
from running. It is especially important when writing to disk.

One way to make backups have less impact on the main workload is to
force backups to a lower ionice level.

With the new ionicelevel configuration, it becomes possible for users to
activate the use of ionice and to define the level within the
best-effort class to better suit their needs.
2020-12-31 02:08:50 -05:00
Jerome Charaoui
79903cd689 Replace dd with cryptsetup luksHeaderBackup
Since the luks headers backup was introduced in the handler, cryptsetup
has gained the "luksHeaderBackup" command. We should use this instead
of dd, as it is likely more reliable. Plus, it supports backing up luks
version 2 headers, which the current code doesn't handle.
2020-12-30 15:25:30 -05:00
Jerome Charaoui
c655eceaa3 Reuse lsblk command to list devices for luksheaders
The previous sfdisk-based command would also list partitions, leading to
partition-based luks volumes being duplicated. Instead, reuse the same
command used in the partition backup component.
2020-12-30 15:20:02 -05:00
lavamind
0491432147 Merge branch '11293-partition_table_backup' into 'master'
Do not try to dump partition tables for a disk when non are detected

Closes #11293

See merge request riseuplabs/backupninja!22
2020-12-30 09:57:28 -08:00
Emil Breiner
582024f4bf Quoting test of $create_options in borg handler
The check for the string length in the condition for appending the $create_options fails because it isn't a string. Fixed with quoting the variable.

Change-Id: I6bdacb81f25cf924dc61e3aa6f4b8ebfbe09da65
Signed-Off-By: Emil Breiner <emil.breiner@krumedia.com>
2020-09-08 12:13:23 +02:00
Matthias Blümel
ab8d5a3a39 Merge branch 'fix_11272' into 'master'
Fix ninjahelper bug in the mysql helper

Closes #11272

See merge request riseuplabs/backupninja!23
2020-09-01 09:20:15 -07:00
Guillaume Subiron
10b8887b8e Merge branch 'borg_cache_dir_location' into 'master'
Borg cache dir location

See merge request riseuplabs/backupninja!29
2020-08-14 05:46:34 -07:00
LeLutin
e28da8c390 borg: add configuration to displace the cache directory
borg will by default create its cache directory in ~/.cache/borg. This
means that during backup runs, borg will read and write quite
extensively from/to this directory.

In some situations, it is rather undesirable to have this amount of IO
activity in this location and it would make sense to tell borg to place
its cache elsewhere. This can help for example with placing the cache on
a hard drive where the added IO load will not have as big of an impact
on other running activity for the system.

This change also makes sure that the cache directory environment
variable is cleared out when the configuration option is unset. This
should avoid unpleasant surprises when this environment variable is set
to some unknown value in the context where backupninja is called, which
could lead to borg reading and writing to random places on the system.
2020-08-14 05:46:34 -07:00
Guillaume Subiron
edcdf9dc52 Merge branch 'borg-port' into 'master'
handlers/borg: add port support

Closes #11296

See merge request riseuplabs/backupninja!31
2020-08-14 05:32:03 -07:00
Guillaume Subiron
9b6a5dedae Merge branch 'fix_authors' into 'master'
fix AUTHORS email address

See merge request riseuplabs/backupninja!26
2020-08-14 05:23:48 -07:00
Guillaume Subiron
0742c593b9 Merge branch 'patch-1' into 'master'
Update sys.in: add missing "disk" flag

See merge request riseuplabs/backupninja!24
2020-08-14 05:21:42 -07:00
tiredpixel
3d7494bfc2 handlers/borg: add port support
This enables the handlers to be used for hosts which use a non-standard
port, such as is offered by some commercial services. Default to port
22, so it's optional and backwards-compatible.
2020-07-16 08:38:46 +00:00
Benjamin Maisonnas
797c09df36 fix AUTHORS email address 2020-01-26 22:39:08 +01:00
Julien Rabier
ac40f35542 Update sys.in to add missing "disk" flag.
It seems like only the short output of hwinfo --disk is provided to hardware.txt. I'd suggest we also add disk to the flags for the "complete" output.
That's especially useful when you need to indicate the serial number of a dead drive in order for it to be replaced. Once the disk is dead, using hardware.txt can be the only way to retrieve this info.
2019-02-04 09:12:12 -08:00
Aurélien Olivier
82c25f4fee Remove double quotes when parsing $result 2018-11-17 11:03:30 +01:00
Aurélien Olivier
e3c1a7a5e9 checkBox function: use on/off instead of yes/no
This function uses on/off to define default values.
2018-11-17 10:51:02 +01:00
Kienan Stewart
f449c7c656
Do not try to dump partition tables for a disk when non are detected
Refs #11293
2018-10-05 14:25:48 -04:00
Jerome Charaoui
f9529d1d3f Remove zero-padding from nowday and whenday (fixes #9397)
This fixes cases where the day component of the when parameter resolved
to a non-zero padded integer under 10, such as "when = 1st at 01"

The comparison against the nowday variable failed consistently since
the date command format used always returned a zero-padded two digit
integer.

The solution chosen is to avoid generating nowday with zero-padding and
remove any zero-padding from whenday parameter using a more thorough
sed pattern which now also removes any non-alphanumeric characters from
the string.
2018-07-13 21:59:43 -04:00