This commit is a rewrite of the static filtering parser into a
tree-based data structure, for easier maintenance and better
abstraction of parsed filters.
This simplifies greatly syntax coloring of filters and also
simplify extending filter syntax.
The minimum version of Chromium-based browsers has been raised
to version 73 because of usage of String.matchAll().
Related discussion:
- a0a9497b4a (commitcomment-62560291)
The new setting, when disabled (enabled by default), allows a user
to prevent uBO from waiting for all filter lists to be loaded
before allowing network activity at launch. The setting is enabled
by default, meaning uBO waits for all filter lists to be loaded in
memory before unsuspending network activity. Some users may find
this behavior undesirable, hence the new setting.
This gives the option to potentially speed up page load at launch,
at the cost of potentially not properly filtering network requests
as per filter lists/rules.
For platforms not supporting the suspension of network activity,
the setting will merely prevent whatever mechanism exists on the
platform to mitigate improper filtering of network requests at
launch. For example, in Chromium-based browsers, unchecking the
new setting will prevent the browser from re-loading tabs for
which there was network activity while in "suspended" state at
launch.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1664
The changes are enough to fulfill the related issue.
A new platform has been added in order to allow for building
a NodeJS package. From the root of the project:
./tools/make-nodejs
This will create new uBlock0.nodejs directory in the
./dist/build directory, which is a valid NodeJS package.
From the root of the package, you can try:
node test
This will instantiate a static network filtering engine,
populated by easylist and easyprivacy, which can be used
to match network requests by filling the appropriate
filtering context object.
The test.js file contains code which is typical example
of usage of the package.
Limitations: the NodeJS package can't execute the WASM
versions of the code since the WASM module requires the
use of fetch(), which is not available in NodeJS.
This is a first pass at modularizing the codebase, and
while at it a number of opportunistic small rewrites
have also been made.
This commit requires the minimum supported version for
Chromium and Firefox be raised to 61 and 60 respectively.
Though Firefox shares a lot of WebExtensions code with Chromium,
these platforms have their own specific code paths, for various
reasons.
The reorganization here makes it clear that Chromium platform is
just one flavor of WebExtensions, and as such all Chromium-specific
code paths should no longer be automatically pulled by other
platforms where these code paths are not needed.
Given that the filepath of many files changed, here is the
parent commit to quickly browse back to the previous directory
layout:
ec7db30b2f
There were spurious error messages in the dev console of
uBO in Firefox, because Firefox does not silently ignore
duplicate contextual menu entries, which could occur
transiently when the contextual menu entries were updated.
The fix simplifies contextual menu code, and actually
fulfill the original goal of avoiding to call extensions
framework API as much as possible.
The managed `userSettings` entry is an array of entries,
where each entry is a name/value pair encoded into an array
of strings.
The first item in the entry array is the name of a setting,
and the second item is the stringified value for the
setting.
This is a more convenient way for administrators to set
specific user settings. The settings set through
`userSettings` policy will always be set at uBO launch
time.
The new entry is an array of strings, each representing a
distinct line, and all entries are used to populate the
"My filters" pane.
This offers an more straightforward way for administrators
to specify a list of custom filters to use for all
installations.
The entry `toOverwrite.filterLists` is an array of
string, where each string is a token identifying a
stock filter list, or a URL for an external filter
list.
This new entry is to make it easier for an
administrator to centrally configure uBO with a
custom set of filter lists.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1433
The new "extraTrustedSiteDirectives" policy is an array
of strings, each of which is parsed as a trusted-site
directive to append to a user's own set of trusted-site
directives at launch time.
The added trusted-site directives will be considered as
part of the default set of directives by uBO.
Related issue:
- https://github.com/gorhill/uBlock/issues/1744
A new context menu entry, "Block element in frame...", will
be present when right-clicking on a frame element. When
this entry is clicked, uBO's element picker will be
launched from within the embedded frame and function the
same way as when launched from within the page.
The condition has been spotted occurring when bringing
up the DOM inspector for a page on which cosmetic filters
are being applied.
Not clear why this happens, but uBO must be ready to
graciously handle such condition.
Reported internally by @gwarser.
In rare occasion, a timing issue could cause uBO to redirect
to a web accessible resource meant to be used for another
network request. This is a regression introduced with the
following commit:
- 2e5d32e967
Additionally, I identified another issue which would cause
cached redirection to fail when a cache entry with redirection
to a web accessible resource was being reused, an issue which
could especially affect pages which are generated dynamically
(i.e. without full page reload).
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/760
The purpose of this new network filter option is to remove
query parameters form the URL of network requests.
The name `queryprune` has been picked over `querystrip`
since the purpose of the option is to remove some
parameters from the URL rather than all parameters.
`queryprune` is a modifier option (like `csp`) in that it
does not cause a network request to be blocked but rather
modified before being emitted.
`queryprune` must be assigned a value, which value will
determine which parameters from a query string will be
removed. The syntax for the value is that of regular
expression *except* for the following rules:
- do not wrap the regex directive between `/`
- do not use regex special values `^` and `$`
- do not use literal comma character in the value,
though you can use hex-encoded version, `\x2c`
- to match the start of a query parameter, prepend `|`
- to match the end of a query parameter, append `|`
`queryprune` regex-like values will be tested against each
key-value parameter pair as `[key]=[value]` string. This
way you can prune according to either the key, the value,
or both.
This commit introduces the concept of modifier filter
options, which as of now are:
- `csp=`
- `queryprune=`
They both work in similar way when used with `important`
option or when used in exception filters. Modifier
options can apply to any network requests, hence the
logger reports the type of the network requests, and no
longer use the modifier as the type, i.e. `csp` filters
are no longer reported as requests of type `csp`.
Though modifier options can apply to any network requests,
for the time being the `csp=` modifier option still apply
only to top or embedded (frame) documents, just as before.
In some future we may want to apply `csp=` directives to
network requests of type script, to control the behavior
of service workers for example.
A new built-in filter expression has been added to the
logger: "modified", which allow to see all the network
requests which were modified before being emitted. The
translation work for this new option will be available
in a future commit.
Months ago, usage of synchronous localStorage was replaced
with asynchronous extension storage. There was code for the
conversion to be seamless by importing the content of now
obsolete localStorage.
This code is no longer needed as majority of users are
assumed to use versions of uBO above 1.25.0.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/763
Changes:
From now on, uBO will allow click-to-subscribe on only
a few select domains, currently:
- https://filterlists.com/
- https://github.com/
- https://github.io/
More domains can be added if and only the demonstration
is made that more than a marginal number of filter lists
can be subscribed from those domains.
The browser alert box is no longer used to confirm
subscription to a filter list. Instead, the asset
viewer has been expanded to serve that purpose. This
way, users can peruse at the content of a filter list
before subscribing to it.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/1226
Related commit:
- 9eb455ab5e
In the previous commit, the element picker dialog was
isolated from the page content. This commit is to also
isolate the svg layers from the page content.
With this commit, there is no longer a need for an anonymous
iframe and the isolated world iframe is now directly
embedded in the page.
As a result, pages are now unable to interfere with any
of the element picker user interface. Pages can now only
see an iframe, but are unable to see the content of that
iframe. The styles applied to the iframe are from a user
stylesheet, so as to ensure pages can't override the
iframe's style properties set by uBO.
This commit fixes deleting all entries when cloud
storage usage is beyond allowed limit.
The issue would prevent pushing new data well within
quota limit because the old data beyond limit was
never removed in the first place.
Cloud storage is a limited resource, and thus it
makes sense to support data compression before
sending the data to cloud storage.
A new hidden setting allows to toggle on
cloud storage compression:
name: cloudStorageCompression
default: false
By default, this hidden setting is `false`, and a
user must set it to `true` to enable compression
of cloud storage items.
This hidden setting will eventually be toggled
to `true` by default, when there is good confidence
a majority of users are using a version of uBO
which can properly handle compressed cloud storage
items.
A cursory assessment shows that compressed items
are roughly 40-50% smaller in size.
The pseudo user styles code served only browsers based
on Chromium 65 and earlier -- Chromium 66 supports
native user styles and was first released more than two
years ago.
In Chromium-based browsers, the pseudo user styles code
is being unconditionally injected in every page/frame
just in case the browser is version 65 or earlier.
Removing pseudo user styles reduce uBO's main content
script in Chromium-based browsers by more than 20K.
Related thread:
- https://github.com/NanoAdblocker/NanoCore/issues/348#issuecomment-653646507
