Commit Graph

1085 Commits

Author SHA1 Message Date
Jerome Charaoui
2b130e171d tests: Add luks1/luks2 specific tests for sys handler 2021-01-06 22:36:20 -05:00
Jerome Charaoui
3988b79b92 tests: Fix mysql-server install on buster 2021-01-06 22:36:20 -05:00
Jerome Charaoui
17ea2c1792 tests: Update testsuite platform to buster64 2021-01-06 22:36:20 -05:00
Jerome Charaoui
0b11f5b1ed tests: Add data samples for pgsql
The samples were created from the MySQL databases, imported into
Postgresql using the pgloader program.
2021-01-06 22:36:20 -05:00
Jerome Charaoui
19f5be20c8 tests: Add mysql test cases
The sample data consists of default Drupal and Wordpress databases with
extra generated data using devel_generate (Drupal) and fakerpress
(WordPress). The content used by both modules is freely licensed.
2021-01-06 22:36:20 -05:00
Jerome Charaoui
ce3e45ab44 tests: Make sys tests cases more efficient, add mbr 2021-01-06 22:36:20 -05:00
Jerome Charaoui
8d9b0418bb tests: replace test_sh() with setup_sh() 2021-01-06 22:36:20 -05:00
Jerome Charaoui
26e530ea9d tests: loglevel 5 is now default for all tests 2021-01-06 22:36:20 -05:00
Jerome Charaoui
4dd6a0971d tests: Fix setconfig() and make more robust 2021-01-06 22:36:20 -05:00
Jerome Charaoui
57b2f217a8 tests: Support component-specific setup & teardown
This introduces support for setup_*, teardown_*, being_* and finish_*
functions in component tests. For example, when testing the sh handler
component in "sh.bats", this will invoke setup_sh and teardown_sh for
each test case, and begin_sh and finish_sh at the beginning of the first
test case and at the end of the last one.
2021-01-06 22:36:20 -05:00
Jerome Charaoui
4c56e50857 tests: Show debug logfile when a test case fails 2021-01-06 22:36:20 -05:00
Jerome Charaoui
48c3da6986 tests: Add testcase for sh handler 2021-01-06 22:36:20 -05:00
Jerome Charaoui
7bfe6f69fd tests: Improve naming is sys test cases 2021-01-06 22:36:20 -05:00
Jerome Charaoui
51a796e0d4 tests: Allow launching tests for specific component 2021-01-06 22:36:20 -05:00
Jerome Charaoui
889d280a4e tests: Mount temporary test directories in tmpfs 2021-01-06 22:36:20 -05:00
Jerome Charaoui
b67d785937 tests: Fix scheduling testcase 2021-01-06 22:36:20 -05:00
Jerome Charaoui
427ff780a4 tests: Introduce bats-based testsuite
- Depends on being run inside vagrant
 - Installs dependencies automatically
 - Environment is fully reset for each test
 - Adds testcases for backupninja and sys handler
 - Docs are missing but will be added

Two tests are currently failing due to two bugs, one of which has gone
unreported.
2021-01-06 22:36:20 -05:00
Jerome Charaoui
c86aa8edd9 tests: Modify Vagrantfile to build+install local codebase 2021-01-06 22:36:20 -05:00
Jerome Charaoui
86c6b19119 Merge branch 'bugfix-borg-handler-create-options' into 'master'
Quoting test of $create_options in borg handler

Closes #11304

See merge request liberate/backupninja!39
2021-01-06 19:35:59 -08:00
Jerome Charaoui
85f4ea886a rsync: improve initial directory creation
The code added in the previous commit is only relevant for the mirror
format. With the long format, directories are created correctly in the
rotation routine. For the short format, the directory is initially
created, but only for a local destination, so adding a short snippet to
the rotation routine does it.
2021-01-06 20:52:32 -05:00
Fabian Raab
a2646868b3 Create path hirarchy of source path automatically in destination
Before it was only possible to backup root directories (depth one, eg.
/etc, /var, /usr, …). For example consider the following truncated toy
configuration for rsync:

    [general]
    mountpoint = /media/
    backupdir = backup
    [source]
    include = /foo/bar
    include = /etc

We assume you have prepared the existent but empty base-directory
/media/backup/ at your destination, then against the expectation, the
following error rises:

    rsync: mkdir "/media/backup/foo/bar/" failed: No such file or directory (2)

The reason is that rsync will take the files from the folder /foo/bar/
and tries to put them into the ulitmate – and therefore existent –
destination /media/backup/foo/bar. This is not what you would expect,
since the prepared directory /media/backup exists and the application
should create the rest itself.

This Merge Request adds a little script which creates the required
directories in the destination before running rsync. It assumes that
the base-directory already exists, so that not too much directories
are created automatically (could be a mess if you accidentally type the
wrong path)
2021-01-06 20:49:44 -05:00
Jerome Charaoui
7c8ba1e71e rsync: improve log msgs, add debug for rsync success 2021-01-06 20:40:05 -05:00
Jerome Charaoui
c6d4eee574 add vim temp files to .gitignore 2021-01-06 11:34:28 -05:00
Jerome Charaoui
e017810941 rdiff: add extra sanity check for keep parameter 2021-01-06 11:33:58 -05:00
Jerome Charaoui
db7daa3d59 rdiff: make version check more verbose in debug 2021-01-06 11:33:24 -05:00
Jerome Charaoui
f1d876be82 rdiff: fix typo 2021-01-06 11:33:05 -05:00
Jerome Charaoui
506c0dfb40 remove mention of vserver in manpage 2021-01-05 23:32:27 -05:00
Jerome Charaoui
9f5952ce34 dup: streamline encryption and signing logic 2021-01-05 11:00:54 -05:00
Jerome Charaoui
d949454366 dup: bandwidthlimit is compatible with desturl
However it's not compatible with local filesystem desturl.
2021-01-05 10:59:49 -05:00
Jerome Charaoui
f007227e4c dup: execute ssh connection test in test mode
Also bring in line with the code used in other handlers.
2021-01-05 10:56:26 -05:00
Jerome Charaoui
e657d42aab dup: improve sanity checks
* also validate desthost and destuser when desturl undefined
 * move desturl vs destuser/host/dir warning in sanity checks section
 * ensure includes are specified
 * drop checks on local destination, duplicity autocreates directory
 * add some needed whitespace
2021-01-05 10:52:46 -05:00
Jerome Charaoui
02c28a7899 borg: improve destination checks 2021-01-03 14:31:12 -05:00
Jerome Charaoui
70f11dd701 borg: add envvar changes to debug log 2021-01-03 14:30:20 -05:00
Jerome Charaoui
45e4c5831c borg: raise fatal error for missing includes 2021-01-03 12:18:46 -05:00
Jerome Charaoui
beff3adc1e Remove remaining vserver bits 2021-01-03 12:07:56 -05:00
lavamind
8dd906ff88 Merge branch 'master' into 'master'
Update namespace from Riseup Labs to Liberate (#11314)

See merge request riseuplabs/backupninja!50
2021-01-02 13:03:49 -08:00
Micah Anderson
8e37956c35
Update namespace from Riseup Labs to Liberate (#11314) 2021-01-02 16:01:02 -05:00
Jerome Charaoui
40a346ff13 rsync: change id_dsa to id_rsa in rsync handler, fixes #11315
Also changed in example config.
2021-01-02 15:56:54 -05:00
Jerome Charaoui
eabda5615c Remove vserver support (fixes #11289)
This is probably completely unused by now, as vservers have been
superseeded by alternative technology for many years already.
2021-01-02 15:27:32 -05:00
Jerome Charaoui
076ef20ead borg: fix typos in debug messages 2021-01-02 14:59:58 -05:00
Jerome Charaoui
fe30b5cce3 tar: add support from zstd compression filter 2021-01-02 13:49:41 -05:00
Jerome Charaoui
ff2376f56c tar: properly default to gzip in case of unknown compress filter 2021-01-02 13:49:22 -05:00
Jerome Charaoui
43c0f9d9bf rsync: fix rsync_options example
Copy-pasting this in a rsync handler config will cause an error as the
quotes will be used on the command line directly...
2020-12-31 14:36:04 -05:00
lavamind
ac8b41eb87 Merge branch 'fix_11316' into 'master'
Support LUKS v2 headers, fixes #11316

Closes #11316

See merge request riseuplabs/backupninja!47
2020-12-31 08:12:39 -08:00
lavamind
db1ce74e10 Merge branch 'ionice' into 'master'
Add support for ionice in handlers

See merge request riseuplabs/backupninja!27
2020-12-31 08:12:16 -08:00
Gabriel Filion
7db5c9c3e8 Document the new ionice configuration in example config files 2020-12-31 02:08:50 -05:00
Gabriel Filion
5c6c583181 Enable use of ionice for all handlers to alleviate IO load
On some workloads, a process creating backups should not become so
aggressive on resources that it prevents normal activity of a machine
from running. It is especially important when writing to disk.

One way to make backups have less impact on the main workload is to
force backups to a lower ionice level.

With the new ionicelevel configuration, it becomes possible for users to
activate the use of ionice and to define the level within the
best-effort class to better suit their needs.
2020-12-31 02:08:50 -05:00
Jerome Charaoui
79903cd689 Replace dd with cryptsetup luksHeaderBackup
Since the luks headers backup was introduced in the handler, cryptsetup
has gained the "luksHeaderBackup" command. We should use this instead
of dd, as it is likely more reliable. Plus, it supports backing up luks
version 2 headers, which the current code doesn't handle.
2020-12-30 15:25:30 -05:00
Jerome Charaoui
c655eceaa3 Reuse lsblk command to list devices for luksheaders
The previous sfdisk-based command would also list partitions, leading to
partition-based luks volumes being duplicated. Instead, reuse the same
command used in the partition backup component.
2020-12-30 15:20:02 -05:00
lavamind
0491432147 Merge branch '11293-partition_table_backup' into 'master'
Do not try to dump partition tables for a disk when non are detected

Closes #11293

See merge request riseuplabs/backupninja!22
2020-12-30 09:57:28 -08:00