Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/discussions/2582
If there exist any built-in filter list which last update time
is older than 2 hours, the "Report a filter issue" page will ask
the user to update their filter lists then verify that the issue
still exists.
Once filter lists are updated, the troubleshooting information
will reflect the change in update time.
Before this commit, a network error condition was not reported
when the filter list was imported, it was only reported afterward
when the imported filter list was updated.
This commit fix properly handling toggling off the default
status of a list such that the list will be automatically
turned off when its status change from default to non-default.
Additionally, imported lists which become stock lists will
be properly migrated from imported lists section.
This commit is a rewrite of the static filtering parser into a
tree-based data structure, for easier maintenance and better
abstraction of parsed filters.
This simplifies greatly syntax coloring of filters and also
simplify extending filter syntax.
The minimum version of Chromium-based browsers has been raised
to version 73 because of usage of String.matchAll().
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664
The changes are enough to fulfill the related issue.
A new platform has been added in order to allow for building
a NodeJS package. From the root of the project:
./tools/make-nodejs
This will create new uBlock0.nodejs directory in the
./dist/build directory, which is a valid NodeJS package.
From the root of the package, you can try:
node test
This will instantiate a static network filtering engine,
populated by easylist and easyprivacy, which can be used
to match network requests by filling the appropriate
filtering context object.
The test.js file contains code which is typical example
of usage of the package.
Limitations: the NodeJS package can't execute the WASM
versions of the code since the WASM module requires the
use of fetch(), which is not available in NodeJS.
This is a first pass at modularizing the codebase, and
while at it a number of opportunistic small rewrites
have also been made.
This commit requires the minimum supported version for
Chromium and Firefox be raised to 61 and 60 respectively.
Reported internally:
> STR --
>
> Import https://cdn.statically.io/gh/uBlockOrigin/uAssets/master/filters/filters.txt
> as a Custom filter list.
>
> Observe the filter count at 24K instead of true count
> being 29K.
>
> Force updating is sometimes compiling 24K filters and
> on subsequent updates 29K, so I narrowed it down to
> this host.
Findings:
One of the sublists was erroring when being fetched on
this particular CDN server (reason unknown).
uBO was not properly handling network errors when
fetching a sublist. This commit make it so that if
a sublist can't be fetched, then the error is propagated
as if it affected the whole list, in which case uBO will
use an alternative URL if any.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1365
When compiled data format changes, do not rely on order
of operations at launch to assume deletion of storage
occurs before attempts to access it. It's unclear this
commit will fix the reported issue, as I could not
reproduce it except when outright commenting out the code
to prevent the storage deletion from occurring.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/763
Changes:
From now on, uBO will allow click-to-subscribe on only
a few select domains, currently:
- https://filterlists.com/
- https://github.com/
- https://github.io/
More domains can be added if and only the demonstration
is made that more than a marginal number of filter lists
can be subscribed from those domains.
The browser alert box is no longer used to confirm
subscription to a filter list. Instead, the asset
viewer has been expanded to serve that purpose. This
way, users can peruse at the content of a filter list
before subscribing to it.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1134
CodeMirror's code folding reference:
- https://codemirror.net/doc/manual.html#addon_foldcode
This commit adds support for code-folding to the filter
list editor/viewer.
The following blocks of code are foldable by clicking the
corresponding marker in the gutter:
- !#if/#endif blocks
- !#include blocks
Addtionally, the following changes:
- The `!#include` line is now preserved when importing a
sublist
- The `!#if` directives will be syntax-colored according
to whether they evaluate to true or false on the current
platform
- Double-clicking on a foldable line in the gutter will
select the content of the foldable block
- Minor visual improvement to matching brackets
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1134
Invalid values for `!#if ...` will be highlighted as errors.
Auto completion is now supported for both the directives
themselves and the valid values for `!#if ...`.
For examples, when pressing ctrl-space:
- `!#e` will auto-complete to `!#endif`
- `!#i` will offer to choose between `!#if ` or `!#include `
- `!#if fir` will auto-complete to `!#if env_firefox`
Additionally, support for some of AdGuard preparsing
directives, i.e. `!#if adguard` is now a valid and will be
honoured -- it always evaluate to `false` in uBO.
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/hbpo86/
For unknown reasons at this point, it appears some users
end up with a truncated version of EasyList, leading to
improper blocking in uBO.
This commit adds a heuristic to discard a downloaded list
when its new size is more than 25% smaller than the
currently cached version.
This commit add the ability to fetch from CDN servers
when an asset is fetched as a result of auto-update.
If an asset has a `cdnURLs` entry in `assets.json`,
the asset will be auto-updated using one of those
CDN URLs. When many CDN URLs are specified, those
URLs will be shuffled in order to spread the bandwidth
across all specified CDN servers. If all specified CDN
servers fail to respond, uBO will fall back to usual
`contentURLs` entry.
The `cdnURLs` are used only when an asset is
auto-updated, this ensures a user will get the more
recent available version of an asset when manually
updating.
The motivation of this new feature is to relieve
GitHub from acting as a CDN (which it is not) for
uBO -- an increasing concern with the growing adoption
of uBO along with the growing size of key uBO assets.
The readTime property is unused for compiled or selfie
assets, and thus writing back to storage to persist
this property is useless for those assets, and an undue
overhead when reading such assets, especially at launch
time.
Assets are always loaded from their compiled or selfie
counterparts at launch.
This commit will prevent those unnecessary storage
write operations.
Reported by:
- https://github.com/uBlock-user:
Imported custom list were incorrectly seen as out of
date immediately after import operation.
Regression from:
- e27328f931
A few lines of code were improperly removed during
refactoring.
Related commit:
- e27328f931
The regression was preventing the compiled filter
lists from being properly loaded by uBO, thus
always causing a full parsing/compiling at
launch time.
The ability to redirect xmlhttprequest to binary
resources was lost when redirectable/injectable
resources became immutable in commit
152cea2dfe.
This commit restores the ability to redirect a
xmlhttprequest to a binary resource by making
it possible to derive a data: URI from the
content of binary resources such as images.
Addtionally a redirect to a data: URI can be
forced by prefixing the resource token with `%`.
This is a non-official feature at this point,
i.e. it could be removed at any time.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/682#issuecomment-515197130
The following advanced setting has been added:
updateAssetBypassBrowserCache
Default to `false`. If set to `true`, uBO will ensure the
browser cache is bypassed when fetching a remote resource.
This is for the convenience of filter list maintainers who
may want to test the latest version of their lists when
fetched from their remote location.
Related issue:
- https://github.com/gorhill/uBlock/issues/2394
Additionally, I added a new advanced setting to control
how long after launch an auto-update session should be
started -- value is in seconds:
autoUpdateDelayAfterLaunch 180
