Using pairs of integers allows the use of a
single integer-only array to store lists of
string indices associated to a specific
hostname.
Memory usage of instances of HostnameBasedDB
as per Chromium's heap snaphshot (bytes):
Before:
2,459,256 => specific cosmetic filters
944,152 => scriptlet filtering
736
---------
3,404,144
After:
1,947,448 => "
757,936 => "
632
---------
2,706,016
Ultimately, using 2 integers for each entry
instead of a single one is still worth it
because this allows the use of one single
integer-only array instead of having to use
an array of arrays for hostnames which have
multiple entries.
This commits make it so that `csp=` filters
are now stored in the same data structures as
all other static network filters rather than
being stored in a separate one.
This internal change is motivated by the wish
to bring session filters to the static network
filtering engine, as has already been done for
the static extended filtering engine in the
following commit:
59c9a34d34
Related commit:
- 87d0e456f1
Ensure that the code which depends on extending
`vapi-client.js` is ready to deal with
`vapi-client-extra.js` failing to load.
This is a feature under development, hidden behind
a new advanced setting, `filterAuthorMode` which
default to `false`.
Ability to point-and-click to create temporary
exception filters for static extended filters (i.e.
cosmetic, scriptlet & html filters) from within
the summary pane in the logger. The button to
toggle on/off temporary exception filter is
labeled `#@#`.
The created exceptions are temporary and will be
lost when restarting uBO, or manually toggling off
the exception filters.
Creating temporary exception filters does not
cause the filter lists to reloaded, and thus there
is no overhead in creating/removing these temporary
exception filters.
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/d6zbqv/
For static filter `popup` filter purpose, the URL of the
embedded frame from which the popup was launched will
be used in the matching algorithm.
As reported by https://github.com/uBlock-user,
adding the two following filters:
example.com##div:style(width: 1016px !important)
example.com##div:style(display: block !important)
The logger would report only one of the filter when
navigating to https://example.com/
While testing for the new `elemhide` option using ABP's
test page[1], I found out that the placeholder of the
blocked image on that page was not properly collapsed
by uBO.
The reason was because the page is very simple and
flushing the blocked-elements cache at
webNavigation.onCommitted time was causing the loss
of information collected between
webRequest.onBeforeRequest and
webNavigation.onCommitted, preventing uBO from
properly collapsing the placeholders.
The blocked-elements cache is now flushed
ONLY at webRequest.onBeforeRequest time.
[1] https://testpages.adblockplus.org/en/exceptions/elemhide
... from platform-independent code.
This should complete the de-chromiumification of
uBO. Next step will be to swap the content of the
`platform/chromium` and `platform/firefox`
folders so as to reflect that Firefox is the
natural platform for uBO.
Related documentation:
- https://help.eyeo.com/en/adblockplus/how-to-write-filters#element-hiding
Related feedback/discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/d6vxzj/
The `elemhide` filter option as per ABP semantic is
now supported. Previously uBO would consider `elemhide`
to be an alias of `generichide`.
The support of `elemhide` is through the convenient
conversion of `elemhide` option into existing
`generichide` option and new `specifichide` option.
The purpose of the new `specifichide` filter option
is to disable all specific cosmetic filters, i.e.
those who target a specific site.
Additionally, for convenience purpose, the filter
options `generichide`, `specifichide` and `elemhide`
can be aliased using the shorter forms `ghide`,
`shide` and `ehide` respectively.
Little-used code from vapi-client.js has been moved
to vapi-client-extra.js. Given that vapi-client.js
is injected in all web pages, this means less dead
code being injected in all pages.
Swathes of code in vapi-client.js was used only in
a few very specific cases, such as when the logger's
DOM inspector is opened or when the "Filter lists"
pane in the dashboard is opened -- and thus to avoid
that little used code to be loaded in every web page
unconditionally, it has been moved to its own
separate file, vapi-client.extra.js.
vapi-client-extra.js is loaded declaratively or
programmatically only where needed.
No need to store mouse coordinates in background
page, thus no need to post mouse coordinates
information for every click.
Rename/group element picker arguments and popup
arguments separately.
Reported by:
- https://github.com/uBlock-user:
Imported custom list were incorrectly seen as out of
date immediately after import operation.
Regression from:
- e27328f931
A few lines of code were improperly removed during
refactoring.
Also, coallesce calls to selfieManager.destroy() so as
to avoid undue repeated calls to storage deletion of
selfie assets.
Related commit:
- e27328f931
Related commit:
- e27328f931
The regression was preventing the compiled filter
lists from being properly loaded by uBO, thus
always causing a full parsing/compiling at
launch time.
Issue reported by @uBlock-user in team channel.
Creating cosmetic procedural exception filters was
causing `cosmetic-logger.js` scriptlet to throw at
and thus further breaking the logging of cosmetic
filters overall.
Reuse permanent instances instead. The trailing `$` is
used to denote these variables are register-like
instances, i.e. their content is valid only for the
duration of the call. (From now on I will use this
convention throughout the code base.)
Entity-based filters where not properly looked-up if
they used subdomains. Example:
- `example.*##^script` => ok
- `www.example.*##^script` => failed on `https://www.example.com/`
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/710
Messages from unprivileged ports (i.e. from content scripts)
are no longer relayed to message handlers which are to be
strictly used to execute privileged code.
The last remaining case of unprivileged messages which
should be converted into a privileged ones will be taken
care of when the following issue is fixed:
- https://github.com/gorhill/uBlock/issues/3497
Related commit:
- fb4e94f92c
A bidi-trie can't store strings longer than 255 characters
because the string segment lengths are encoded into a single
byte. This commit ensures only strings smaller than
256 characters are stored in the bidi-tries.
Plain filters can be any length, while the bidi-trie was
assuming max length of 256. The origin of this error
is from when the bidi-trie code was originally imported
from the hntrie code as start to develop bidi-trie.
The erroneous code could cause issue when the following
conditions were met:
- Plain filter longer than 256 characters
- Free space in bidi-trie's character buffer was less
than 256 bytes
Likely a rare occurrence, but some filter lists do contains
long plain filters, for example:
https://gitlab.com/curben/urlhaus-filter/raw/master/urlhaus-filter.txt
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/cs1y26/
The purpose is to wholly disable scriptlet injection
for a given site without having to create exceptions
for all matching scriptlet injection filters.
The following exception filter will cause scriptlet
injection to be wholly disable for `example.com`:
`example.com#@#+js()`
Or to disable scriptlet injection everywhere:
`#@#+js()`
The following form is meaningless and will be
ignored:
`example.com##+js()`
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/701
The filter option `empty` is converted to `redirect=empty`
by uBO internally; however unlike when the `redirect=`
option is used expressly, the `empty` option does not
require a resource type.
When `empty` is used, only network requests which are meant
to return a text response will be redirected to an empty
response body by uBO -- so `empty` will not work for
resources such as images, media, or other binary resources.
Related commit & feedback:
- 7ff750eaf6
The color value for the icon badge is now
"attached" to the blocking profile value.
Additionally, as per feedback, `3p` rules
will be relaxing before master JavaScript
switch rules.
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/cmh910/
Additionally, the `3p` rule has been made distinct from
`3p-script`/`3p-frame` for the purpose of
"Relax blocking mode" command.
The badge color will hint at the current blocking mode.
There are four colors for the four following blocking
modes:
- JavaScript wholly disabled
- All 3rd parties blocked
- 3rd-party scripts and frames blocked
- None of the above
The default badge color will be used when JavaScript is not
wholly disabled and when there are no rules for `3p`,
`3p-script` or `3p-frame`.
A new advanced setting has been added to let the user choose
the badge colors for the various blocking modes,
`blockingProfileColors`. The value *must* be a sequence of
4 valid CSS color values that match 6 hexadecimal digits
prefixed with`#` -- anything else will be ignored.
The ability to redirect xmlhttprequest to binary
resources was lost when redirectable/injectable
resources became immutable in commit
152cea2dfe.
This commit restores the ability to redirect a
xmlhttprequest to a binary resource by making
it possible to derive a data: URI from the
content of binary resources such as images.
Addtionally a redirect to a data: URI can be
forced by prefixing the resource token with `%`.
This is a non-official feature at this point,
i.e. it could be removed at any time.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/310
The purpose of this new option is to add the ability
to create standalone redirect rule without being forced
to create a block filter (a corresponding block filter
is always created when using the `redirect=`).
Additionally:
The syntax `*$redirect=token,...` is now supported, there
is no need to "trick" the filter parser with
`*/*$redirect=token,...` in order to create redirect rules
which are meant to match all paths.
Filters of the form `|http*://` will be normalized into
two corresponding filters `|https://` and `|http://` so as
to reduce the number of filters in the buckets of
untokenizable filters.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/682#issuecomment-515197130
The following advanced setting has been added:
updateAssetBypassBrowserCache
Default to `false`. If set to `true`, uBO will ensure the
browser cache is bypassed when fetching a remote resource.
This is for the convenience of filter list maintainers who
may want to test the latest version of their lists when
fetched from their remote location.
Related feedback:
- a54cb2e38f (commitcomment-34387041)
Regression from converting uBO resources into
immutable resources. This affects only Chromium-based
browsers.
This is a quick fix for the dev build, to at least unbreak
the reported case.
I need to research whether the root issue (conflict with HTTPS-E)
is still occurring, and if so the fix is not trivial: I will need
to add code to uBO to fetch and convert binary data into `data:`
URIs.
Theoretically, if a selfie is loaded from corrupted
storage resulting in a thrown exception from the
constructor, this would cause an improperly initialized
HostnameBasedDB instance and overall potentially a
broken uBO.
Related feedback:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1543880#c7
Since https://github.com/uBlockOrigin/uBlock-issues/issues/156
won't be fixed in next release, no need to ship
with code which will be unused, and anyways only once
the fix is worked on will it be clear exactly what needs
to be used by scriptlets to deal harmoniously with
property listener collisions.
Make sure the parser is safely compatible with old
resources format -- for those users still using
custom resources (via `userResourcesLocation`).
Prepare code for future fix to
<https://github.com/uBlockOrigin/uBlock-issues/issues/156>:
This commit introduces a new private Map() object,
`uBOSafe`, accessible by all injected scriptlets. This
private safe can be used to store data which can be shared
with different scriptlets. The idea is for scriptlets to
use that safe to graciously deal with the need to install
multiple listeners for the same property.
Since resources are now immutable, by default they are
only compiled once each time uBO updates to a new
version. However I need a way to force a re-compiling
of the resource in the dev build. This commit adds code
to invalidate the resources selfie when forcing the
update of any filter list.
