mangadex_at_home/CHANGELOG.md

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

Unreleased

Added

Changed

Deprecated

Removed

Fixed

Security

2.0.1 - 2021-05-27

Added

• [2021-05-27] Added SNI check to prevent people from simply scanning nodes [@carbotaniuman].

Changed

• [2021-05-21] Update metrics and fix cache directory leak [@carbotaniuman].
• [2021-05-21] Change headers to be wildcards [@carbotaniuman].
• [2021-05-27] Make sending the Server header configurable but off by default [@carbotaniuman].

2.0.0 - 2021-03-11

Changed

• [2021-03-11] Switch back to HTTP/1.1 [@carbotaniuman].
• [2021-03-11] Tune connection pool [@carbotaniuman].

Fixed

• [2021-03-11] Fixed Access-Control-Expose-Headers typo [@lflare].
• [2021-03-11] Throw exceptions less frequently [@carbotaniuman].
• [2021-03-11] Fix various Netty issues [@carbotaniuman].
• [2021-03-11] Don't log IOUring and Epoll errors [@carbotaniuman].
• [2021-03-11] Ignore IPV6 when pinging [@carbotaniuman].

2.0.0-rc14 - 2021-03-02

Changed

• [2021-03-02] Fix Prometheus to 2.24.1 and Grafana to 7.4.0 [@_tde9].
• [2021-03-02] Update and rearrange the embedded dashboard with the new Timeseries panel from Grafana 7.4 [@_tde9].
• [2021-03-02] Update sample dashboard screenshot thanks to DLMSweet 😄 [@_tde9].
• [2021-02-25] Use HTTP/2 to download when possible [@carbotaniuman].

Fixed

• [2021-02-21] Fix pipeline [@_tde9].

2.0.0-rc13 - 2021-02-19

Changed

• [2021-02-19] Back to sqlite we go [@carbotaniuman].

2.0.0-rc12 - 2021-02-11

Fixed

• [2021-02-11] Fixed stupid cross platform bug [@carbotaniuman].

2.0.0-rc11 - 2021-02-11

Fixed

• [2021-02-11] Fixed docker bug [@carbotaniuman].

2.0.0-rc10 - 2021-02-11

Added

• [2021-02-11] Add ability to disable io_uring and epoll [@carbotaniuman].

Changed

• [2021-02-11] Changed databaseFile to databaseFolder in run args [@carbotaniuman].
• [2021-02-11] Added more error recovery and logging [@carbotaniuman].
• [2021-02-11] Enforce stricter bounds on disk size [@carbotaniuman].

Fixed

• [2021-02-11] Fixed issues with Netty not shutting down properly [@carbotaniuman].

2.0.0-rc9 - 2021-02-06

Added

• [2021-02-06] Add Micrometer and Resilience4J dashboards to defaults [@_tde9].

Changed

• [2021-02-06] Use Resilience4J circuit breaker stats for upstream download panel [@_tde9].
• [2021-02-06] Set default docker GC to Shenandoah [@_tde9].
• [2021-02-06] Updated timing of circuit breaker [@carbotaniuman].

2.0.0-rc8 - 2021-02-04

Added

• [2021-02-04] Installed a circuit breaker in order to save upstream [@carbotaniuman].

2.0.0-rc7 - 2021-01-31

Added

• [2021-01-31] Add distinct timerange and total egress stats to dashboard [@_tde9].
• [2021-01-31] Add average cache hitrate panel [@_tde9].

Changed

• [2021-01-31] Add HikariCP connection pool [@carbotaniuman].

Fixed

• [2021-01-31] Fix client reqs dashboard panel [@_tde9].

2.0.0-rc6 - 2021-01-28

Fixed

• [2021-01-27] Upped max Apache threadpool size [@carbotaniuman].
• [2021-01-27] Add ability for nodes to specify external ip [@carbotaniuman].

2.0.0-rc5 - 2021-01-27

Changed

• [2021-01-27] Minor code tweaks [@carbotaniuman].

2.0.0-rc4 - 2021-01-27

Changed

• [2021-01-27] Added threads back with new semantics [@carbotaniuman].

Fixed

• [2021-01-27] Fix regression with Prometheus metrics [@carbotaniuman].

2.0.0-rc3 - 2021-01-26

Changed

• [2021-01-26] Removed threads options in settings [@carbotaniuman].

Fixed

• [2021-01-25] Add privileged: true to mangadex-at-home service in docker-compose to enable use of IOUring for the dockerized version [@_tde9].
• [2021-01-26] Make updated config restart the webserver and apply changes [@carbotaniuman].
• [2021-01-26] Optimize some code to reduce allocations [@carbotaniuman].
• [2021-01-26] Fix stupid bug with MessageDigest [@carbotaniuman].

2.0.0-rc2 - 2021-01-24

Added

• [2021-01-24] Epoll and IOUring HTTP transports [@carbotaniuman].

Changed

• [2021-01-24] Log files are now UTF-8 [@carbotaniuman].

Fixed

• [2021-01-23] Fix sample docker-compose.yml [@_tde9].
• [2021-01-24] Fix another race condition [@carbotaniuman].
• [2021-01-24] Fix validating small (<1024MiB) caches [@carbotaniuman].

2.0.0-rc1 - 2021-01-23

This release contains many breaking changes! Of note are the changes to the cache folders, database location, and settings format.

Added

• [2021-01-23] Added external_max_kilobits_per_second config option [@carbotaniuman].
• [2021-01-23] Various internal tests to ensure stability [@carbotaniuman].
• [2021-01-23] Added /prometheus endpoint for Prometheus stats and eventual integration [@_tde9].
• [2021-01-23] docker-compose for easy spinup of a Prometheus + Grafana stack [@_tde9].

Changed

• [2021-01-23] Changed the settings to a settings.yaml file [@carbotaniuman].
• [2021-01-23] Changed from cache to images for images folder [@carbotaniuman].
• [2021-01-23] Changed folder structure to be simpler [@carbotaniuman].
• [2021-01-23] Coalesced DB writes to reduce DB load [@carbotaniuman].
• [2021-01-23] Store metadata along with the image to reduce IOPS [@carbotaniuman].
• [2021-01-23] Updated internal dependencies to improve performance [@carbotaniuman].

Removed

• [2020-01-23] Unceremoniously removed the old WebUI [@carbotaniuman].

Fixed

• [2021-01-23] Fixed a long-standing cache deadlock [@carbotaniuman].
• [2021-01-23] Fixed another shutdown bug [@carbotaniuman].
• [2021-01-23] Fixed various CPU and memory leaks [@carbotaniuman].
• [2021-01-23] Fixed another shutdown bug [@carbotaniuman].
• [2021-01-23] Fixed data races when storing images [@carbotaniuman].

1.2.4 - 2021-01-09

Fixed

• [2021-01-08] Better exception handling [@carbotaniuman].

1.2.3 - 2021-01-08

Fixed

• [2021-01-08] Fix a bunch of stupid edge cases [@carbotaniuman].
• [2021-01-08] Fix pathological shutdown issues [@carbotaniuman].
• [2021-01-08] Provide messages on request failures [@carbotaniuman].
• [2021-01-08] Exempt well-known test images [@carbotaniuman].
• [2021-01-08] Give actual errors and not 500 [@carbotaniuman].

1.2.2 - 2020-08-21

Changed

• [2020-08-11] Moved to a Java implementation of NaCl [@carbotaniuman].

Fixed

• [2020-08-11] Revert WebUI changes in 1.2.1 [@carbotaniuman].

1.2.1 - 2020-08-11

Added

• [2020-08-11] New CLI for specifying database location, cache folder, and settings [@carbotaniuman].

Changed

• [2020-08-11] Change logging defaults [@carbotaniuman].

Fixed

• [2020-08-11] Bugs relating to settings.json changes [@carbotaniuman].
• [2020-08-11] Logs taking up an absurd amount of space [@carbotaniuman].
• [2020-08-11] Random crashes for no reason [@carbotaniuman].
• [2020-08-11] SQLException is now properly handled [@carbotaniuman].

1.2.0 - 2020-08-10

Added

• [2020-07-13] Added reloading client setting without stopping client by [@radonbark].

Changed

• [2020-07-29] Disallow unsafe ports [@m3ch_mania].

Fixed

• [2020-07-29] Fixed stupid libsodium bugs [@carbotaniuman].
• [2020-07-29] Fixed issues from the Great Cache Propagation [@carbotaniuman].
• [2020-08-03] Fix client_hostname stuff [@carbotaniuman].

1.1.5 - 2020-07-05

Added

• [2020-07-05] Added basic graph creation interface to dash [@RedMatriz].

Changed

• [2020-07-04] Changed GitLab CI to build on every push irregardless of tagging by [@carbotaniuman].
• [2020-07-05] Added mangadex.network as allowed domain for Referer. Allow blank or missing Referer by [@AviKav].
• [2020-07-05] Changed mobile dash to allow for graph creation [@RedMatriz].
• [2020-07-05] Minor improvements to graph load [@RedMatriz].

Fixed

• [2020-07-04] Fixed typo on access-control-allow-methods by [@carbotaniuman].
• [2020-07-05] Fixed minor graph update issues [@RedMatriz].

Security

• [2020-07-05] Prevent Referer matching on subdomains such as mangadex.org.example.com by [@AviKav].

1.1.4 - 2020-07-04

Changed

• [2020-07-04] Logging backbone overhauled by [@carbotaniuman].
• [2020-07-04] Bumped client version to 15 by [@carbotaniuman].

Removed

• [2020-07-04] Rolled back configuration of ServerFilters for CORS by [@carbotaniuman].

1.1.3 - 2020-07-04

Added

• [2020-07-04] Updated to allow for automatic version constant variable by [@carbotaniuman].

Fixed

• [2020-07-04] Updated Dockerfile to include libsodium for token verification by [@FOG_Yamato].

1.1.2 - 2020-07-04

Fixed

• [2020-07-04] Fixed client_hostname default configuration bug by [@carbotaniuman].
• [2020-07-04] Suppressed some non-essential logs by [@carbotaniuman].

1.1.1 - 2020-07-04

Changed

• [2020-07-04] Bumped version number by [@carbotaniuman].

1.1.0 - 2020-07-04

Added

• [2020-06-23] Added Gitlab CI integration by [@lflare].
• [2020-06-28] Added client_external_port setting [@wedge1001].
• [2020-06-29] Added rudimentary support of Referer checking to mitigate hotlinking by [@lflare].
• [2020-06-30] Added read and write timeouts to protect against some attacks [@carbotaniuman].
• [2020-06-30] Added dev_settings to allow for easier development [@wedge1001].
• [2020-07-02] Added stand-ins for future client control in web interface [@RedMatriz].

Changed

• [2020-06-28] Added pasued field in ServerSettings [@carbotaniuman].
• [2020-06-28] Hopefully fixed connection leaks [@carbotaniuman].
• [2020-07-02] Minor fixes and changes to data handling in web interface [@RedMatriz].
• [2020-07-02] Renamed localstorage keys in web interface [@RedMatriz].
• [2020-06-28] Actually report custom client_hostname [@carbotaniuman].

Fixed

• [2020-06-28] Fixed various state transition bugs by [@carbotaniuman].

Security

• [2020-07-02] Added option to enforce strict checks on tokens to prevent hotlinking [@carbotaniuman].

1.0.0 - 2020-06-22

Added

• [2020-06-22] Repository clean-up by [@carbotaniuman]
• [2020-06-22] Comments added to JSON configuration file by [@carbotaniuman]
• [2020-06-22] Added UI graph resizing and dragging by [@RedMatriz].
• [2020-06-22] Added client setting generator in UI by [@RedMatriz].

Changed

• [2020-06-22] Fix incorrect config.sample.json naming to settings.sample.json by [@lflare]
• [2020-06-22] Logging statements improved by [@carbotaniuman]

1.0.0-RC22 - 2020-06-22

Added

• [2020-06-20] Added graceful_shutdown_wait_seconds client setting by [@carbotaniuman]
• [2020-06-20] Added config.sample.json sample client configuration file by [@lflare]
• [2020-06-19] Readded ability to set log files level [@carbotaniuman]

1.0.0-RC21 - 2020-06-20

Changed

• [2020-06-20] Fixed logging for DiskLruCache [@carbotaniuman]
• [2020-06-20] Don't automatically delete on corruption [@carbotaniuman]

1.0.0-RC20 - 2020-06-19

Added

• [2020-06-19] Errored out on invalid settings.json tokens [@carbotaniuman]

Changed

• [2020-06-19] Changed default CPU thread count to 4 by [@lflare].
• [2020-06-19] Removed ability to set log files but increased throughput [@carbotaniuman]

Fixed

• [2020-06-19] Make graceful shutdown work better [@carbotaniuman]
• [2020-06-19] Actually shutdown logback [@carbotaniuman]

1.0.0-RC19 - 2020-06-18

Added

• [2020-06-16] Added WebUI versions to constants by [@RedMatriz].
• [2020-06-16] Added WebUI PWA support for mobile by [@RedMatriz].
• [2020-06-16] Added WebUI local data caching [@RedMatriz].

Changed

• [2020-06-16] Reworked graceful shutdown [@carbotaniuman].
• [2020-06-17] Revamped configuration & units by [@lflare].

Fixed

• [2020-06-17] Fixed minor typo with threads error logging by [@lflare].

1.0.0-RC18 - 2020-06-16

Changed

• [2020-06-16] Changed log level of response timings to INFO by [@lflare].
• [2020-06-16] Added server ping logging [@carbotaniuman].
• [2020-06-16] Added access control headers by [@Fugi].

1.0.0-RC17 - 2020-06-15

Added

• [2020-06-15] Added logging of backend assigned URL to logs by [@lflare].
• [2020-06-15] Added logging of compromised softban to logs by [@lflare].

Changed

• [2020-06-14] Migrated cache metadata over to a sqlite3 handler [@carbotaniuman].
• [2020-06-15] Properly describe dirty builds as dirty by [@lflare].

Deprecated

• [2020-06-14] Removed old cache subdirectory migration system by [@carbotaniuman].

Fixed

• [2020-06-14] Switched cache metadata over to a MySql instance [@carbotaniuman].
• [2020-06-15] Fixed tokenized data-saver parser not working by [@lflare].
• [2020-06-15] Properly synchronised sqlite3 handler across threads by [@lflare].

1.0.0-RC16 - 2020-06-14

Added

• [2020-06-14] Added new client_hostname selector to allow for custom address binding for Netty by [@lflare].
• [2020-06-14] Added new ui_hostname selector to allow for custom address binding for WebUiNetty by [@lflare].
• [2020-06-14] Added response timings to trace logs and response headers by [@lflare].

1.0.0-RC15 - 2020-06-13

Added

• [2020-06-13] Allow for the two log levels to be configurable by [@lflare].
• [2020-06-13] Added X-Cache header to image responses by [@lflare].
• [2020-06-13] Added .gitattributes to help sort out CHANGELOG.md merge conflicts by [@lflare].
• [2020-06-13] Added rudimentary web-ui by [@carbotaniuman & @RedMatriz].
• [2020-06-13] Added additional entry to server ping for network speed by [@lflare].
• [2020-06-13] Added colouring to web-ui pie chart by [@lflare].

Changed

• [2020-06-13] Modified AsyncAppender queue size to 1024 by [@lflare].
• [2020-06-13] Bumped client version to 5 by [@lflare].
• [2020-06-13] Modularized the image server by [@carbotaniuman].
• [2020-06-13] Suppressed log output for IOException by [@carbotaniuman].
• [2020-06-13] Migration of Java to Kotlin for most handlers by [@carbotaniuman]

1.0.0-RC14 - 2020-06-12

Fixed

• [2020-06-12] Fixed not actually creating the directories before moving cache files by [@lflare].

1.0.0-RC13 - 2020-06-12

Added

• [2020-06-12] Added CHANGELOG.md by [@lflare].
• [2020-06-12] Added on-read atomic image migrator to 4-deep subdirectory format by [@lflare].

Changed

• [2020-06-12] Raised ApacheClient socket limit to 2**18 by [@lflare].
• [2020-06-12] Changed gradle versioning to using git describe by [@lflare].
• [2020-06-12] Made Netty thread count global instead of per-cpu by [@lflare].
• [2020-06-12] Store cache files in a 4-deep subdirectory to improve performance by [@lflare].

Fixed

• [2020-06-12] Re-added missing default threads_per_cpu setting by [@lflare].
• [2020-06-12] Replaced exponential calculation for ApacheClient threads by [@lflare].

Security

• [2020-06-12] Update ClientSettings.java changed showing client secret in logs back to hidden by [@dskilly].

1.0.0-RC12 - 2020-06-12

Fixed

• [2020-06-12] Fixed hourly refresh bug by [@carbotaniuman].

1.0.0-RC11 - 2020-06-11

Added

• [2020-06-11] New setting threads_per_cpu to faciliate with Netty multi-threading by [@lflare].

Changed

• [2020-06-11] Swapped threading to Netty instead of ApacheClient by [@lflare].

Fixed

• [2020-06-11] Tweaked logging configuration to reduce log file sizes by [@carbotaniuman].