This commit add the ability to fetch from CDN servers
when an asset is fetched as a result of auto-update.
If an asset has a `cdnURLs` entry in `assets.json`,
the asset will be auto-updated using one of those
CDN URLs. When many CDN URLs are specified, those
URLs will be shuffled in order to spread the bandwidth
across all specified CDN servers. If all specified CDN
servers fail to respond, uBO will fall back to usual
`contentURLs` entry.
The `cdnURLs` are used only when an asset is
auto-updated, this ensures a user will get the more
recent available version of an asset when manually
updating.
The motivation of this new feature is to relieve
GitHub from acting as a CDN (which it is not) for
uBO -- an increasing concern with the growing adoption
of uBO along with the growing size of key uBO assets.
Related commit:
- 453f5450b6
The changes in this commit are mostly related
to the "Filter lists" pane.
An eye icon has been added, which purpose is
to view the content of a list. Clicking on the
name of the list will toggle the list's
checkbox.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/974
Related feedback:
- https://www.reddit.com/r/uBlockOrigin/comments/fuscia/
The race condition was that a content script could
query the main process to retrieve cosmetic filters
while the cosmetic filters had not been yet fully
loaded into memory. The fix ensure that an already
injected content script will re-query once the
cosmetic filters are fully loaded in memory at
browser launch time.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/959
The problematic site does not declare the expected
`disqus_shortname` global variable, so the scriptlet
has been extended to deal with such occurrence.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/911
Since cname-uncloaking is available only on Firefox
at the moment, the fix is relevant only to Firefox.
By default uBO will no longer cname-uncloak when it
detects that network requests are being being proxied.
This default behavior can be overriden by setting the
new advanced setting `cnameUncloakProxied` to `true`.
The new setting default to `false`, i.e. cname-uncloaking
is disabled when uBO detects that a proxy is in use.
This new advanced setting may disappear once the
following Firefox issue is fixed:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1618271
Provide a way to optionally deduplicate filter
instances, useful for filter instances with:
- high likelihood of duplication; and
- non-trivial memory footprint per instance
- For examples, filter instances to implement
`domain=`, `denyallow=`, `csp=`.
Cursory tests show this helps further reduce
uBO's memory footprint.
This concerns the static network filtering engine.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/943
* * *
New static network filter type: `cname`
By default, network requests which are result of
resolving a canonical name are subject to filtering.
This filtering can be bypassed by creating exception
filters using the `cname` option. For example:
@@*$cname
The filter above tells the network filtering engine
to except network requests which fulfill all the
following conditions:
- network request is blocked
- network request is that of an unaliased hostname
Filter list authors are discouraged from using
exception filters of `cname` type, unless there no
other practical solution such that maintenance
burden become the greater issue. Of course, such
exception filters should be as narrow as possible,
i.e. apply to specific domain, etc.
* * *
New static network filter option: `denyallow`
The purpose of `denyallow` is bring
default-deny/allow-exceptionally ability into static
network filtering arsenal. Example of usage:
*$3p,script, \
denyallow=x.com|y.com \
domain=a.com|b.com
The above filter tells the network filtering engine that
when the context is `a.com` or `b.com`, block all
3rd-party scripts except those from `x.com` and `y.com`.
Essentially, the new `denyallow` option makes it easier
to implement default-deny/allow-exceptionally in static
filter lists, whereas before this had to be done with
unwieldy regular expressions[1], or through the mix of
broadly blocking filters along with exception filters[2].
[1] https://hg.adblockplus.org/ruadlist/rev/f362910bc9a0
[2] Typically filters which pattern are of the
form `|http*://`
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/923
Use current page's hostname instead of that of
image URLs to decide whether to reset pattern
union with previous picker sessions.
The fixed issue arose from the fact that the
page uses URLs from different origins in a
single srcset attribute.
Related commit:
- https://github.com/gorhill/uBlock/commit/703c525b01aa
This adds an indentation requirement for line
continuation to take place. The conditions are now
as follow:
- Current line ends with ` \`: ASCII space + backslash
- Next line starts with ` `: four ASCII spaces
If a line in a filter list ends with a space
(ASCII code 32) followed by a backslash
(ASCII code 92), those two characters will be
removed, the line will be trimmed and the next
line will be trimmed and concatenated to form
a new, longer line.
The purpose is to give filter list authors
a way to visually break apart unduly long
filters and thus make maintenance easier.
When line continuation is used, it is suggested
that the extra lines are prepended with four
space so as to make it more visually obvious that
the extra line(s) are the continuation of a
previous line.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/943
The filter referenced in the above issue was the
motivation to implement this feature:
- https://hg.adblockplus.org/ruadlist/rev/f362910bc9a0
I verified and could not find any instance in major
filter lists of lines ending with ` \`, thus the
change should be safe.
The new syntax deprecate the old syntax, though the
old syntax will still be supported until it's no
longer used in mainstream filter lists.
The new syntax is:
example.com##+js(window.open-defuser, pattern, seconds)
`pattern`:
A pattern to match for the defusing to take place.
Patterns which starts and ends with `/` will be
interpreted as regular expressions. To NOT match a
pattern, prefix with `!`.
`seconds`:
If not present, no window will be opened and the
scriptlet will return `null`.
If present and a valid integer value, the defuser
will return a valid window object even though no
popup window is opened. The returned window object
will cease to be valid after the specified number
of seconds.
Reported internally. The following invalid filter was not
discarded by uBO:
123tvnow.com##.123tv-ads
The correct form should be:
123tvnow.com##.\31 23tv-ads
Not discarding invalid CSS selector-based cosmetic
filter may break CSS selector-based cosmetic
filtering.
***
New procedural cosmetic operator: `:remove()`
Related issue:
- https://github.com/gorhill/uBlock/issues/2252
The purpose is to outright remove elements from the
DOM tree. Since `:remove()` is an "action" operator,
it must only be used as a trailing operator (just
like the `:style()` operator).
AdGuard's cosmetic filter syntax `{ remove: true; }`
will be converted to uBO's `:remove()` operator
internally.
***
New procedural cosmetic operator: `:upward(...)`
The purpose is to lookup an ancestor element.
When used with an integer argument, it is synonym of
`:nth-ancestor()`, which will be deprecated and which
will no longer be supported once no longer used in
mainstream filter lists.
Filter lists maintainers must only use `:upward(int)`
instead of `:nth-ancestor(int)` once the new operator
become available in all stable releases of uBO.
`:upward()` can also accept a CSS selector as argument,
in which case the nearest ancestor which matches the
CSS selector will be selected.
Override the Backspace key, not just the Delete key, as Mac keyboards have Backspace as the only delete key and label it Delete.
Source of key value: https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/key/Key_Values#Editing_keys
More background on Mac keyboard shortcuts: Mac keyboards can emulate Delete using fn+Delete, meaning Forward Delete, but Mac software does not use it except in text editing. When deletion is dangerous, Mac software requires holding a modifier key in conjuction with Delete, but I think it’s better to make deletion easy in this case.
This new binding has a potential downside: if the user Backspace key normally goes Back in history (which can differ across OSs and browsers), this will change the behavior to delete the selected element instead. If the user really wants to go back in history, they will have to press Escape to leave the mode and then press Backspace, or they will have to press an alternative keyboard shortcuts such as Alt+Left. I think the user will rarely want to go back in history in the middle of picking an element, though.
That downside could be mitigated by conditioning the key check on `runtime.PlatformOs` (https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/runtime/PlatformOs). But that would complicate the code a lot compared to the importance of this feature, and such detection would still fail to capture the user’s intent accurately in all cases. I think it’s better to unconditionally accept both Backspace (Delete) and Delete (Forward Delete).
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/899
browser.storage.local is now used to store non-critical
local settings.
These settings are all collated under the key
`localStorage`, and vAPI.localStorage is an API to
handle access to these values stored under this key.
vAPI.localStorage.getItem() is still synchronous but
its purpose is to return internally cached values --
this minimizes code changes throughout uBO.
New advanced setting: `benchmarkDatasetURL`
Default value: `unset`
To specify a URL from where the benchmark dataset will be
fetched. This allows to launch benchmark operations from
within published versions of uBO, rather than from just
a locally built version.
A specific cosmetic filter of the following form...
*##.selector
... will be unconditionally injected into all web pages,
whereas a cosmetic filter of the form...
##.selector
... would be injected only when uBO's DOM surveyor finds
at least one matching element in a web page.
The new specific-generic form will also be disabled when a
web page is subject to a `generichide` exception filter,
since the filter is essentially a generic one -- the only
difference from the usual generic form is that the filter
is injected unconditionally instead of through the DOM
surveyor.
Specific-generic cosmetic filters will NOT be discarded
when checking the "Ignore generic cosmetic filters"
option in the "Filter lists" pane -- since the purpose
of this option is primarily to disable the DOM surveyor.
Specific-generic cosmetic filters should be used
parcimoniously and only when using a normal specific
filter is really impractical.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/803
Hostnames which have unpunycoded IDN or canonical
name information will have this information reported
as an extra line under the hostname name in the popup
panel, rather than as a tooltip.
For the fenix UI flavor, the save/revert tools are
now shown regardless of whether the overview area is
visible or not.
More specifically, this commit makes it so that
the new default UI for Firefox Preview will be
automatically selected.
A default platform-specific UI can be cancelled
by setting the advanced setting `uiFlavor` to `-`.
The toolip of a canonical name entry (blue)
in the popup panel will provide the alias
hostname related to the canonical hostname
in order to help with rule-making decisions.
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/882
Related commits:
- https://github.com/gorhill/uBlock/commit/a95ef16e064a
- https://github.com/gorhill/uBlock/commit/7971b223855d
Leading wildcards before valid token characters need to
be kept in order to respect the semantic of the filter.
A leading wildcard in such case changes the semantic of
a filter, i.e. two following filters are semantically
different:
example/abc
*example/abc
As a result, µBlock.BidiTrieContainer.indexOf() is now
able to deal with a needle of length zero -- which is
what happens in FilterPatternLeft(Ex) with filter
patterns starting with `*` (or `^*`) and followed by
valid token characters (0-9, a-z and %).
First draft of changes as discussed with Firefox
Preview people.
In order to allow testing/evaluating these changes,
the new advanced setting `uiFlavor` has been added.
Default to `unset`; and can currently only be set
to `fenix`.
The new setting takes effect at launch only. This
new setting is not to be mentioned in official
documentation for now.
This is ongoing work, not open to external feedback.
This commit make it so that if an *unexpected* error is
thrown when trying to load the selfie at launch, the filter
lists will still be loaded as a result.
Tokens which are as long or longer than the max token
length possible do not need to have the prefix part
evaluated against special regex characters. This will
help increase the likelihood of extracting a valid
token from regex-based filters.
Actual case found in EasyPrivacy:
/^https?:\/\/eulerian..*\/[a-z0-9]{2,12}\.js/$script
Before this commit, uBO was not able to extract a
valid token, while now uBO is able to extract `eulerian`
as a valid token (consequently the regex-based filter
will now be evaluated only when the token `euleria` is
found in a URL).
This is related to the list of domains/subdomains in
the dynamic filtering pane of the popup panel.
Related issue:
- https://github.com/gorhill/uBlock/issues/284
Clicking on the empty space of a row will toggle
the visibility of the subdomains.
Additionally, the root context will always be visible
regardless of the expand/collspase state, along with
a visual indicator that a specific domain or subdomain
is the actual root context. (the root context is the
hostname in which local rules are created).
Related discussion:
- https://github.com/uBlockOrigin/uBlock-issues/issues/805#issuecomment-561500819
uBO was testing for regex-like plain patterns after
prepending `@@` in the case of exception filters, thus
preventing proper detection of regex-like plain
patterns. The filtering engine was not affected, only
the proper rendering of the filter in the logger was
affected.
Advanced setting `cnameAliasList` has been removed.
New advanced settings:
cnameUncloak:
Boolean
Default value:
true
Description:
Whether to CNAME-uncloak hostnames.
cnameIgnoreExceptions:
Boolean
Default value:
true
Description:
Whether to bypass the uncloaking of network requests
which were excepted by filters/rules. This is
necessary so as to avoid undue breakage by having
exception filters being rendered useless as a result
of CNAME-uncloaking.
For example, `google-analytics.com` uncloaks to
`www-google-analytics.l.google.com` and both hostnames
appear in Peter Lowe's list, which means exception
filters for `google-analytics.com` (to fix site
breakage) would be rendered useless as the uncloaking
would cause the network request to be ultimately
blocked.
Related issue:
- https://github.com/uBlockOrigin/uAssets/issues/5132
The change in this commit make it so that it's no
longer required to have an exception filter for
`google-analytics.com/analytics.js` for the page to
render properly.
