Improved sync speed by resolving the N+1 query issues.
Solves #1402 and Solves #1453
With this change there is just one query done to retreive all the
important data, and matching is done in-code/memory.
With a very large database the sync time went down about 3 times.
Also updated misc crates and Github Actions versions.
The armv6 builds need a specific location for the libatomic.a file.
This commit fixes that by adding a RUSTFLAGS argument for this.
Also removed the `link-arg=-s` since this is now already done during via the release profile
And removed the CFLAGS for armv7, this is already fixed by default in the blackdex/rust-musl images.
- Removed all `thread::sleep` and use `tokio::time::sleep` now.
This solves an issue with updating to Bullseye ( Resolves #1998 )
- Updated all Debian images to Bullseye
- Added MiMalloc feature and enabled it by default for Alpine based images
This increases performance for the Alpine images because the default
memory allocator for MUSL based binaries isn't that fast
- Updated `dotenv` to `dotenvy` a maintained and updated fork
- Fixed an issue with a newer jslib (not fully released yet)
That version uses a different endpoint for `prelogin` Resolves #2378 )
Favicon:
- Replaced HTML tokenizer, much faster now.
- Caching the domain blacklist function.
- Almost all functions are async now.
- Fixed bug on minimizing data to parse
- Changed maximum icon download size to 5MB to match Bitwarden
- Added `apple-touch-icon.png` as a second fallback besides `favicon.ico`
SMTP:
- Deprecated SMTP_SSL and SMTP_EXPLICIT_TLS, replaced with SMTP_SECURITY
Misc:
- Fixed issue when `resolv.conf` contains errors and trust-dns panics (Fixes #2283)
- Updated Javscript and CSS files for admin interface
- Fixed an issue with the /admin interface which did not cleared the login cookie correctly
- Prevent websocket notifications during org import, this caused a lot of traffic, and slowed down the import.
This is also the same as Bitwarden which does not trigger this refresh via websockets.
Rust:
- Updated to use v1.59
- Use the new `strip` option and enabled to strip `debuginfo`
- Enabled `lto` with `thin`
- Removed the strip RUN from the alpine armv7, this is now done automatically
- Changed nightly to stable in Dockerfile and Workflow
- Updated Dockerfile to use stable and updated ENV's
- Removed 0.0.0.0 as default addr it now uses ROCKET_ADDRESS or the default
- Updated Github Workflow actions to the latest versions
- Updated Hadolint version
- Re-orderd the Cargo.toml file a bit and put libs together which are linked
- Updated some libs
- Updated .dockerignore file
- Using my own rust-musl build containers we now support all database
types for both Debian and Alpine.
- Added new Alpine containers for armv6 and arm64/aarch64
- The Debian builds can also be done wihout dpkg magic stuff, probably
some fixes in Rust regarding linking (Or maybe OpenSSL or Diesel), in
any case, it works now without hacking dpkg and apt.
- Updated toolchain and crates
- Decreased `recursion_limit` from 512 to 87
Mainly done by optimizing the config macro's.
This fixes an issue with the rust-analyzer which doesn't go beyond 128
- Removed Regex for masking sensitive values and replaced it with a map()
This is much faster then using a Regex.
- Refactored the get_support_json macro's
- All items above also lowered the binary size and possibly compile-time
- Removed `_conn: DbConn` from several functions, these caused unnecessary database connections for functions who didn't used that at all
- Decreased json response for `/plans`
- Updated libraries and where needed some code changes
This also fixes some rare issues with SMTP https://github.com/lettre/lettre/issues/678
- Using Rust 2021 instead of 2018
- Updated rust nightly
Recently the LetsEncrypt DST certificate has expired.
Older versions of OpenSSL like v1.0.x have issues using this certificate.
Recently clux has updated his image to support OpenSSL v1.1.1[a-z].
This solves issues with those certificates.
This issues was disscused on Matrix.
This fixes #1998 where with some checking it seems Bullseye has some
issues with the glibc sleep call. It returns a SIGILL.
The glibc on Buster doesn't seem to have this issue, so revert back for
now until a fix has been released.
- Split Debian and Alpine into different build matrix
This starts building both Debian and Alpine based images at the same time
- Make use of Docker BuildKit, which improves speed also.
- Use BuildKit caching for Rust Cargo across docker images.
This prevents downloading the same crates multiple times.
- Use Github Actions Services to start a docker registry, starting it
via the build script sometimes caused issues.
- Updated the Build workflow to use Ubuntu 20.04 which is more close to
the Bullseye Debian release regarding package versions.
Updated several dependencies and switch to different totp library.
- Switch oath with totp-lite
oauth hasn't been updated in a long while and some dependencies could not be updated any more
It now also validates a preseeding 0, as the previous library returned an int instead of a str which stripped a leading 0
- Updated rust to the current latest nightly (including build image)
- Updated bootstrap css and js
- Updated hadolint to latest version
- Updated default rust image from v1.53 to v1.54
- Updated new nightly build/clippy messages
- Removed azure-pipelines
- Updated gh-actions to run `cargo test` per db feature
- Fail on warnings by adding `RUSTFLAGS` env
- Updated Dockerfile to fix some new hadolint warnings
- Fixed bug when web-vault is disabled.
- Updated sql-server version check to be simpler thx to @weiznich ( https://github.com/dani-garcia/bitwarden_rs/pull/1548#discussion_r604767196 )
- Use `VACUUM INTO` to create a SQLite backup instead of using the external sqlite3 application.
- This also removes the dependancy of having the sqlite3 packages installed on the final image unnecessary, and thus removed it.
- Updated backup filename to also have the current time.
- Add specific bitwarden_rs web-vault version check (to match letter patched versions)
Will work when https://github.com/dani-garcia/bw_web_builds/pull/33 is build (But still works without it also).
Some small changes in general:
- Moved the SQL Version check struct into the function.
- Updated hadolint to 2.0.0
- Fixed hadolint 2.0.0 warnings
- Updated github workflows
- Added .editorconfig for some general shared editor settings.
- Updated rust nightly
- Updated depenencies
- Removed unicode support for regex (less dependencies)
- Fixed dependency and nightly changes/deprications
- Some mail changes for less spam point triggering
The bitwarden_rs code is still cross-compiled exactly as before, but Docker
Buildx is used to rewrite the resulting Docker images with correct platform
metadata (reflecting the target platform instead of the build platform).
Buildx also now handles building and pushing the multi-arch manifest lists.
- Updated the Github Actions to build just one binary with all DB
Backends.
- Created a hadolint workflow to check and verify Dockerfiles.
- Fixed current hadolint errors.
- Fixed a bug in the Dockerfile.j2 which prevented the correct libraries
and tools to be installed on the Alpine images.
- Deleted travis.yml since that is not used anymore
- Updated crates
- Updated rust-toolchain
- Updated Dockerfile to use latest rust 1.48 version
- Updated AMD64 Alpine to use same version as rust-toolchain and support
PostgreSQL.
- Updated Rocket to the commit right before they updated hyper.
Until that update there were some crates updated and some small fixes.
After that build fails and we probably need to make some changes
(which is probably something already done in the async branch)
The runtime image was using a very old Alpine version.
This caused issues with the catatonit install
Now using the Balena armv7hf Alpine image for this.
With some apt/dpkg magic building multidb containers for arm versions
now also works. As long as the build stage and docker-image stage use
the same base (debian buster now) it should all work.
Resolves #530, resolves #1066
This is useful for making local customizations upon container start. To use
this feature, mount a script into the container as `/etc/bitwarden_rs.sh`
and/or a directory of scripts as `/etc/bitwarden_rs.d`. In the latter case,
only files with an `.sh` extension are sourced, so files with other
extensions (e.g., data/config files) can reside in the same dir.
Note that the init scripts are run each time the container starts (not just
the first time), so these scripts should be idempotent.
* Switch healthcheck interval/timeout from 30s/3s to 60s/10s.
30s interval is arguably overkill, and 3s timeout is definitely too short
for lower end machines.
* Use HEALTHCHECK CMD exec form to avoid superfluous `sh` invocations.
* Add `--silent --show-error` flags to curl call to avoid progress meter being
shown in healthcheck logs.
The muslrust images seem to have a workdir of /volume as opposed to / in the
others so doing cargo new like this would create the folder in /volume/app.
No need to use two different base images. Debian buster is pulled later
anyway so we can just use it for the vault stage as well.
My reason for this change is partly to avoid redundancy and partly to
make it easier to build everything yourself. When all the build
environment is based on Debian than you just have to figure out how to
build a Debian Docker base image (ref:
https://github.com/ypid/docker-makefile).
