1
0
mirror of https://github.com/gorhill/uBlock.git synced 2024-11-18 00:13:30 +01:00
Commit Graph

2750 Commits

Author SHA1 Message Date
Raymond Hill
ae56c4dfe8
Fix whitelist status evaluation of tabless network requests
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/651

The `behnind-the-scene` context was wrongly used to
evaluate the whitelist status of the context of tabless
network requests. The document origin must be used
instead when it's available.

Additionally, much code has been revisited for better
ES6 syntax compliance.
2019-07-05 17:44:08 -04:00
Raymond Hill
9693d07a6d
Code review of https://github.com/gorhill/uBlock/commit/f930da7ad64a 2019-07-05 12:33:14 -04:00
Raymond Hill
f930da7ad6
Fix regression of reverse-lookup of scriptlet filters in logger
Related commit:
- 5552d6717d
2019-07-05 11:44:40 -04:00
Raymond Hill
e107b6bcf1
Fix typo in comment 2019-07-05 10:32:19 -04:00
Raymond Hill
5552d6717d
Implement scriptlet token normalization
The goal is to be able to specify a scriptlet token
without the `.js` part at the end, because that part
is essentially redundant with the `+js` part of
the syntax.

When the next stable release is in widespread use
(to determine), scriptlet tokens will have to be
specified without the `.js` part, and with this
commit the logger will already report the normalized
version of scriptlets.

Eventually, when the migration to sans-`.js` is
complete (also to determine), the internal
normalization of the token will be removed and this
will become official syntax.

Filter list maintainers will have to mind that
uAssets is becoming in use beyond uBO (i.e. Brave)
when skipping the `.js` part -- hopefully Brave will
go along with the change here, which is to remove a
bit of tediousness for filter list maintainers.
2019-07-05 10:10:59 -04:00
Raymond Hill
6220e1d3eb
Add missing newline 2019-07-05 08:22:26 -04:00
Raymond Hill
3e53963be3
Fix window.open-defuser
Matching logic has been erronesouly reverse.
2019-07-05 07:59:02 -04:00
Raymond Hill
a992875c94
Save only modified immediate hidden settings 2019-07-05 07:33:09 -04:00
Raymond Hill
b3ea68a08d
Import translation work from https://crowdin.com/project/ublock 2019-07-04 15:50:20 -04:00
Raymond Hill
1fb9845c35
Remove useless code 2019-07-04 14:10:23 -04:00
Raymond Hill
f9e680f111
Convert more resources as immutable
Related commit:
- 152cea2dfe
2019-07-04 14:08:56 -04:00
Raymond Hill
8e245c8919
Convert more resources as immutable
Related commit:
- 152cea2dfe
2019-07-03 19:26:09 -04:00
Raymond Hill
0ba9a35818
Convert more resources as immutable
Related commit:
- 152cea2dfe
2019-07-03 14:33:06 -04:00
Raymond Hill
5672f54663
Import translation work from https://crowdin.com/project/ublock 2019-07-03 09:57:04 -04:00
Raymond Hill
152cea2dfe
Refactor management of injectable resources
This is a first step, the ultimate goal is to remove
the need for resources.txt, or at least to reduce to
only hotfixes or for trivial resources targeting very
specific websites.

Most resources will become immutable, i.e. they will
be part of uBO's code base. Advantages include easier
code maintenance (jshint, syntax highlight), and to
make scriptlets more easy to code review by external
parties (for example extension store reviewers).

TODO:

- More scriptlets need to be imported before next
  release.
- Need to make legacy versions of uBO use a legacy
  version of resources.txt, as all the now obsolete
  scriptlets will have to be removed once uBO's
  next release become widespread.
- Possibly need to add code to load binary
  resources so that they can be injected as
  data: URI. So far it's unclear whether this is
  really needed. For example, this would be needed
  if a xmlhttprequest is redirected to an image
  resource.
2019-07-03 09:47:56 -04:00
Raymond Hill
41636c59fb
Strict-block only if match is anchored to end of hostname
As per feedback from filter list maintainers.
2019-07-02 11:56:27 -04:00
Raymond Hill
2e04b75ccf
Import translation work from https://crowdin.com/project/ublock 2019-07-02 09:59:26 -04:00
Raymond Hill
730a83377e
Minor code review re. context menu code
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/151

I have been unsuccessful fixing the above issue, but I will
keep the changes made in the process of trying to fix it.
2019-07-02 09:43:26 -04:00
Raymond Hill
cdd1aac442
Add convenience link to network resources in logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/648
2019-06-30 16:15:19 -04:00
Raymond Hill
2bcf671dae
Put back erroneously removed line
Regression from 1dfdc40e09
2019-06-30 12:54:05 -04:00
Raymond Hill
c8860ff61d
Code review of c1bdc123f2 2019-06-30 10:22:06 -04:00
Raymond Hill
1dfdc40e09
Add ability to suspend network request handler at will
This works only for platforms supporting the return of
Promise by network listeners, i.e. only Firefox at this
point.

When filter lists are reloaded[1], there is a small
time window in which some network requests which should
have normally been blocked are not being blocked
because the static network filtering engine may not
have yet loaded all the filters in memory

This is now addressed by suspending the network request
handler when filter lists are reloaded -- again, this
works only on supported platforms.

[1] Examples: when a filter list update session
    completes; when user filters change, when
    adding/removing filter lists.
2019-06-30 10:09:27 -04:00
Raymond Hill
c1bdc123f2
Fix use of sibling-related CSS syntax at prefix position
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/c6iem5/
2019-06-29 14:07:54 -04:00
Raymond Hill
b1ec593b9f
Import translation work from https://crowdin.com/project/ublock 2019-06-29 11:16:25 -04:00
Raymond Hill
cf4345ffc4
Fix some element picker-related issues
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/c5do7w/

Make the element picker better reflect network filters as
parsed by the static network filtering engine. Additionally,
discard single alphanumeric character-based filters.

Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/c62irc/

Inject newly created cosmetic filters into the DOM
filterer, in order for these filters to be enforced by
the DOM filterer in subsequent dynamic DOM changes.
2019-06-29 11:06:03 -04:00
Raymond Hill
dba075af59
Import translation work from https://crowdin.com/project/ublock 2019-06-28 06:27:53 -04:00
Raymond Hill
6c34b3c3c9
Use "relax" instead of "toggle"
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/371
2019-06-27 08:16:18 -04:00
Raymond Hill
388e5df5fd
Import translation work from https://crowdin.com/project/ublock 2019-06-27 08:14:48 -04:00
Raymond Hill
a9659d039d
Add missing command in manifest.json 2019-06-26 17:17:41 -04:00
Raymond Hill
f9c9c2cc14
Import translation work from https://crowdin.com/project/ublock 2019-06-26 10:20:31 -04:00
Raymond Hill
693687fd74
Add keyboard support for toggling down blocking profile
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/371

By default, no specific keyboard shortcut is predefined,
this will have to be assigned by the user. The command
name in English is "Toggle blocking profile".

The default behavior is to toggle down according to one
of the following scenarios.

a) If script execution is disabled through the no-scripting
switch, the no-scripting switch will be locally toggled
so as to allow script execution. The page will be
automatically reloaded.

b) If script execution is not blocked but the 3rd-party
script and/or frame cells are blocked, local no-op rules
will be set so as to no longer block 3rd-party scripts
and/or frames. The page will be automatically reloaded.

Given this, it may take more than one toggle down command
to reach the lowest blocking profile, which is one where
JavaScript execution is not blocked and 3rd-party scripts
and frames resources block rules, if any, are bypassed
with local no-op rules.

TODO: At this point, I haven't yet decided whether
toggling from the lowest profile should restore the
original highest blocking profile.
2019-06-26 07:47:14 -04:00
Raymond Hill
d1df2b5e73
Fix merging multiple URls in element picker
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/c5do7w/

Fixed:
- Expect the differ can return the first input as is when
  there is no difference between the two items.
- Better deal with extraneous whitespaces in `srcset`
2019-06-25 17:09:04 -04:00
Raymond Hill
9065bbdd48
Code review of whitelisting-related code
- Use `Map()` instead of `{}` for internal data
  structure
- Export as array of directives instead of as
  a string
2019-06-25 11:57:14 -04:00
Raymond Hill
8e7384ba84
Prevent duplicate inline-script entries in the logger
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/c4340z/filter_problem/ervpjd8/
2019-06-24 11:40:14 -04:00
Raymond Hill
ce94319e86
Import translation work from https://crowdin.com/project/ublock 2019-06-23 08:44:13 -04:00
Raymond Hill
41685f4cce
Replace exec with transpose in procedural filters
The purpose is to avoid having to iterate through
all input nodes at each operator implementation
level. The `transpose` method deals with only one
input node, and the iteration is performed by the
main procedural filtering entry points.

Additionally:
- Add `:spath` to HTML filtering
- Rename `:watch-attrs` to `:watch-attr`
  - `:watch=attrs` is deprecated and will be kept around
    until it is safe to remove it completely
2019-06-23 08:05:53 -04:00
Raymond Hill
570723f795
Import translation work from https://crowdin.com/project/ublock 2019-06-20 14:42:37 -04:00
Raymond Hill
b428a25c3f
Add new procedural operator: :min-text-length(x)
Where `x` is the minimal text length of the subject
DOM element. DOM elements whose text length is
greater than or equal to `x` will be selected.

The original rationale for such procedural cosmetic
operator[1] is to be able to remove inline script
elements according to a minimum text length using
HTML filtering.

[1] As a result of internal discussion with filter
    list maintainers @ uAssets.
2019-06-20 14:11:54 -04:00
Raymond Hill
822e0a133d
Provide visual feedback for invalid entries in "My rules"
Related issue:
- https://github.com/gorhill/uBlock/issues/1039
2019-06-19 18:28:44 -04:00
Raymond Hill
be2a950541
Code review of HNTrie/staticNetFilteringEngine
- Remove HNTrieContainer class from global context by
  storing it as a property of µBlock.

- Use block scope to isolate HNTrie-related constants
  from global context.

- Prevent filters which are pure IP address from
  being stored in an HNTrie instance -- as this
  could cause false positives.
2019-06-19 10:00:19 -04:00
Raymond Hill
7303776757
Use async/await in Matrix.benchmark() 2019-06-19 08:37:48 -04:00
Raymond Hill
cfc2ce333d
Implement bidirectional plain-string trie
The bidirectional trie allows storing the right
and left parts of a string into a trie given a
pivot position.

Releated issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/528

Additionally, the mandatory token-at-index-0 rule
for FilterPlainHnAnchored has been lifted, thus
allowing the engine to pick a potentially better token
at any position in the filter string.

***

TODO: Eventually rename `strie.js` to `biditrie.js`.

TODO: Fix dump() method, it currently only show the
      right-hand side of a filter string.
2019-06-18 19:16:39 -04:00
Raymond Hill
7e38391a71
Import new Bosnian translations from https://crowdin.com/project/ublock 2019-06-15 09:38:06 -04:00
Raymond Hill
5c05209e7f
Import translation work from https://crowdin.com/project/ublock 2019-06-14 10:59:07 -04:00
Raymond Hill
2a2faa84a0
Import translation work from https://crowdin.com/project/ublock 2019-06-05 08:19:12 -04:00
Raymond Hill
e9bbc33194
Import translation work from https://crowdin.com/project/ublock 2019-06-03 06:42:02 -04:00
Raymond Hill
2eb9b726a5
Fix generichide not being evaluated for local context
Related issue:
- https://github.com/uBlockOrigin/uAssets/issues/5704
2019-06-03 06:37:39 -04:00
Raymond Hill
bd961364cc
Import new en-GB from https://crowdin.com/project/ublock 2019-05-31 09:30:32 -04:00
Raymond Hill
c312ec2cd5
Import translation work from https://crowdin.com/project/ublock 2019-05-31 09:17:16 -04:00
Raymond Hill
27e8c8d468
Normalize tabless xhr to image/media in onHeadersReceived()
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/610

The service worker-related issue affects both
Chromium/Firefox: the type of resources fetched
from a service worker are uniformly set to
`xmlhttprequest`, hence losing a key piece of
information for the purpose of accurate content
filtering.
2019-05-31 09:02:07 -04:00
Raymond Hill
a691d180f2
Import translation work from https://crowdin.com/project/ublock 2019-05-28 07:25:11 -04:00
Raymond Hill
8828522fe8
Fix errors with cosmetic filter exception in the logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/602
2019-05-28 07:21:16 -04:00
Raymond Hill
d99789e107
Import translation work from https://crowdin.com/project/ublock 2019-05-26 08:11:38 -04:00
Raymond Hill
85b89fbe63
Fix broken import-from-file in Whitelist pane
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bt2d1f/
2019-05-26 08:03:44 -04:00
Raymond Hill
ede233abe7
Import translation work from https://crowdin.com/project/ublock 2019-05-25 10:09:53 -04:00
Raymond Hill
a7bfff03d6
Avoid spurious diff at edit time in "My rules"
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/593

The issue was caused by the lack of empty last line, since
the differ taking into account new line characters.
2019-05-25 10:04:31 -04:00
Raymond Hill
80a8750d35
Select existing "Advanced settings" page if any
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/591

Additionally, I added a link to the logger in the
"About" pane in the dashboard in order to be able
to access the logger without having to go through
the popup panel.
2019-05-25 08:31:06 -04:00
Raymond Hill
fb6d69f543
Discard whole filter with bad csp= content
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bshn7z/

uBO was just removing the bad option, while the whole
filter needs to be discarded.
2019-05-24 15:41:37 -04:00
Raymond Hill
1e9528e2a6
Fix regression affecting *$csp=-like filters
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bshn7z/filter_question/

Regression introduced in:
- 3f3a1543ea
2019-05-24 12:15:32 -04:00
Raymond Hill
26708b37c1
Integrate bare-bone filter hit stats in the logger
Related issue:
- https://github.com/gorhill/uBlock/issues/983
- https://github.com/gorhill/uBlock/issues/1353

The current implementation reports statistics for all
static filters, and the presentation/featureset is
intentionally minimal: *Do not open issues about this.*
It's still a work in progress and it will be worked on
slowly and thoughtfully over time and as time allows.

Pausing the logger will not pause the collation of
filter hit statistics, thus it is possible to lower
the logger overhead by pausing logger output without
losing filter hit collation.
2019-05-24 11:18:39 -04:00
Raymond Hill
64636a44cb
Import translation work from https://crowdin.com/project/ublock 2019-05-23 19:35:53 -04:00
Raymond Hill
eef76c49ae
Add a link to the remote asset in asset viewer
The link will be present if and only if the content
of the currently viewed asset has been fetched from
a remote location.
2019-05-23 19:29:59 -04:00
Raymond Hill
294ea41fde
Import emergency fix 5a29a21c81 in dev build 2019-05-23 10:22:51 -04:00
Raymond Hill
1f398134f9
Minor code reivew of 4430ec11e2 2019-05-23 08:15:26 -04:00
Raymond Hill
7b8c087fdd
Start using async/await where it makes sense 2019-05-22 19:23:04 -04:00
Raymond Hill
8d40d41cf7
Import translation work from https://crowdin.com/project/ublock 2019-05-22 18:07:08 -04:00
Raymond Hill
4430ec11e2
Rearrange inner loop of static network filtering engine
The motivations for the re-arrangement:

- Reducing the number of entry points:
  matchStringExactString() has been removed and
  matchString() is simply reused with a modifier parameter
  to enable matching variants.

- Presumption that most matches, if any, occur early with
  the left-most tokens in a URL. This gives a very small
  marginal performance gain as per built-in benchmark.
2019-05-22 17:51:03 -04:00
Raymond Hill
e8c2f7eea3
Fix "Close this window" not working on document-blocked page
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/breeux/
2019-05-21 18:56:59 -04:00
Raymond Hill
6f4ccec836
Import translation work from https://crowdin.com/project/ublock 2019-05-21 15:50:53 -04:00
Raymond Hill
32b04fa262
Re-arrange parsing of type options to be order-independent
Related commit:
- 1888033070

This removes the need to place `all` before any negated
type in the list of options.
2019-05-21 14:04:21 -04:00
Raymond Hill
5eff4a027a
Fix https://github.com/gorhill/uBlock/issues/3541 2019-05-20 18:29:28 -04:00
Raymond Hill
cdfbc1ae02
Import translation work from https://crowdin.com/project/ublock 2019-05-20 14:10:21 -04:00
Raymond Hill
1888033070
Add support for all filter option
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bqnsoa/

The `all` option is equivalent to specifying all
network-based types + `popup`, `document`,
`inline-font`, `inline-script`.

Example from discussion:

    ||bet365.com^$all

Above will block all network requests, block all popups,
prevent inline fonts/scripts from `bet365.com`. EasyList-
compatible syntax does not allow to accomplish that
semantic when using only `||bet365.com^`.

If using specific negated type options along with `all`,
the order in which the options appear is important. In
such case `all` should always be first, followed by
the negated type option(s).
2019-05-20 13:46:36 -04:00
Raymond Hill
4b85838745
Import translation from https://crowdin.com/project/ublock 2019-05-19 18:42:32 -04:00
Raymond Hill
72d9758faa
Ensure the "Filter lists" pane is in sync with update status
Related issue:
- https://github.com/gorhill/uBlock/issues/2394

Additionally, I added a new advanced setting to control
how long after launch an auto-update session should be
started -- value is in seconds:

    autoUpdateDelayAfterLaunch 180
2019-05-19 18:31:12 -04:00
Raymond Hill
a0ac1b7ee8
Fix handling of data: for filtering purpose in logger
Related issue:
- https://github.com/gorhill/uBlock/issues/2469
2019-05-19 17:00:49 -04:00
Raymond Hill
f677443878
Warn when navigating away from pane with unsaved changes
Related issue:
- https://github.com/gorhill/uBlock/issues/3271

When navigating away by clicking another pane tab button,
there will be an embedded warning, which can be ignore
in order to proceed to the new pane, or dismissed by
either clicking on the "Stay" button or anywhere else
in the dashboard.

When navigating away by trying to close the tab, there will
be a built-in browser warning asking for confirmation.
2019-05-19 15:35:00 -04:00
Raymond Hill
1caff7429e
Add optional support for generic procedural cosmetic filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/131

The new advanced setting and its default value is:

    allowGenericProceduralFilters false

Whenever this setting is toggled, the user is responsible
of forcing a reload of all filter lists so as to allow uBO
to process differently any existing generic procedural
cosmetic filters.
2019-05-18 18:57:32 -04:00
Raymond Hill
e66e4496ed
Fix https://github.com/uBlockOrigin/uBlock-issues/issues/360 2019-05-18 17:50:58 -04:00
Raymond Hill
ca34bc4f3e
Fix "Revert" button not resetting after saving changes
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/367
2019-05-18 17:48:19 -04:00
Raymond Hill
3cf71835c4
Set default delay for creating selfie to 3 minutes
Related discussion:
- https://www.reddit.com/r/uBlockOrigin/comments/bq49zi/
2019-05-18 14:43:44 -04:00
Raymond Hill
f7bbc80717
Improve "Whitelist pane"; remove now useless built-in switch rule
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/214

Built-in whitelist directives are now rendered differently
than user-defined whitelist directives. Also, removing a
built-in whitelist directive will only cause that directive
to be commented out, so that users do not have to remember
built-in directives should they want to bring them back.

Related issue:
 https://github.com/uBlockOrigin/uBlock-issues/issues/494

The built-in per-site switch rule
`no-scripting: behind-the-scene false` has been removed,
it should not ever be needed since there will always be a
valid root context for main- and sub-frames.
2019-05-18 14:20:05 -04:00
Raymond Hill
de41c1bf53
Fix parsing of recursive !#if-`!#endif directives
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/270
2019-05-18 10:31:04 -04:00
Raymond Hill
62387fb87a
Prevent picker's preview mode from modifying style attribute
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/551

The issue fixes previewing the hiding/unhiding of targeted
elements in the element picker.

However it does not address the case of previewing
`:style(...)` operators -- this would require a much
more complex fix, which I am not sure is worth the
amount of work and increased code complexity.
2019-05-17 19:26:48 -04:00
Raymond Hill
9cbdff1a9d
Import translation work from https://crowdin.com/project/ublock 2019-05-17 15:02:00 -04:00
Raymond Hill
9bfbbfec84
Adjust visual of cosmetic exception filters in logger
The invariant prefixes `##` and `#@#` are now hidden,
allowing to reveal more of the filter itself when the
logger view is narrow.
2019-05-17 11:45:07 -04:00
Raymond Hill
0ca44b847c
Avoid duplicated strings in filterOrigin w/ new approach
The new approach is simpler and should benefit selfie
serialization/unserialization.

This renders stringDeduplicater obsolete -- it has been
removed.
2019-05-17 10:13:58 -04:00
Raymond Hill
1386429382
Fix regression in applying procedural cosmetic filters
Related commit:
- 3573b6b32c
2019-05-16 17:22:20 -04:00
Raymond Hill
3573b6b32c
Add ability to report exception cosmetic filters in the logger
Related issue:
- https://github.com/gorhill/uBlock/issues/127

Additionally, the extended exception filters in the
logger will be rendered with a line-through to more
easily distinguish them from non-exception ones.

Also, opportunistically converted revisited code to
ES6 syntax.
2019-05-16 13:44:49 -04:00
Raymond Hill
fc109c8b7c
Revisit code to benefit from ES6 syntax 2019-05-15 14:49:12 -04:00
Raymond Hill
1fe3b54acc
Fix cosmetic exception filters not applying
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/575

Regression from:
- 93f80eedfa

Specific cosmetic exception filters need to be returned so that
they can be applied to generic cosmetic filters.
2019-05-15 14:43:59 -04:00
Raymond Hill
39e2a03edb
Fix comment 2019-05-14 09:31:51 -04:00
Raymond Hill
a14dcecf8f
Do not assume wildcards fall on label boundaries
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492223980
2019-05-14 09:29:45 -04:00
Raymond Hill
93f80eedfa
Refactor runtime storage of specific cosmetic filters
This was a TODO item:
- 07cbae66a4/src/js/cosmetic-filtering.js (L375)

µBlock.staticExtFilteringEngine.HostnameBasedDB has been
re-factored to accomodate the storing of specific cosmetic
filters.

As a result of this refactoring:

- Memory usage has been further decreased
- Performance of selector retrieval marginally
  improved
- New internal representation opens the door
  to use a specialized version of HNTrie, which
  should further improve performance/memory
  usage
2019-05-14 08:52:34 -04:00
Raymond Hill
8a312b9bbb
Support cases with more than one wildcard
Related commit:
- fe0b7a0e0f

Related feedback:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572#issuecomment-492147440
2019-05-14 06:52:13 -04:00
Raymond Hill
fe0b7a0e0f
Relax destination hostname requirements in redirect filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/572

Wildcards are now allowed in the hostname part of redirect
filters. There will be an attempt to find the longest
right-hand portion of the hostname with no wildcard. If
no non-empty hostname can be extracted, `*` will be used.
2019-05-13 20:19:10 -04:00
Raymond Hill
1e40f50eb3
Add benchmark method to cosmetic filtering engine
To measure retrieval of site-specific selectors. From
uBO's own dev console:

    µBlock.cosmeticFilteringEngine.benchmark();
2019-05-12 11:41:47 -04:00
Raymond Hill
57890d60ff
Fix incorrect use of this in static method
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/568

Regression from:
- 19ece97b0c
2019-05-11 17:40:55 -04:00
Raymond Hill
8a7e704080
Add support for nth-ancestor operator in HTML filtering
Also opportunitisically converted some code to
ES6's `class`.
2019-05-11 13:21:23 -04:00
Raymond Hill
915c1f1f3c
Report resources blocked by csp= option in logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/552
2019-05-11 10:40:34 -04:00
Raymond Hill
12bdd01595
Ensure "Ignore generic cosmetic filters" sticks on Fennec
Related issue:
- https://www.reddit.com/r/uBlockOrigin/comments/blkudl/

The setting was not sticking at first-install time.
2019-05-11 09:04:13 -04:00
Raymond Hill
ac1a323918
Import translation work from https://crowdin.com/project/ublock 2019-05-10 17:26:34 -04:00
Raymond Hill
e59bdb1485
Defuse fixed position on body element in element zapper
The `fixed` style property on the `body` element will be
defused if an overlay element is removed using the element
zapper.

Related:
- https://www.reddit.com/r/uBlockOrigin/comments/bktxtb/scrolling_doesnt_work/emlscyz
2019-05-06 13:32:55 -04:00
Raymond Hill
26237d6d40
Import translation work from https://crowdin.com/project/ublock 2019-05-06 11:39:41 -04:00
Raymond Hill
3692bb4ada
Add HNTrieRef.dump() and STrieRef.dump() as dev tool
To be used at the console, as an investigation tool for
development purpose.

Using it to verify the content of the largest
FilterHostnameDict instance, I spotted an all-uppercase
hostname in the HNTrieRef instance:

µBlock.staticNetFilteringEngine.categories.get(0).get(0x10000000).dict.dump();

Thus the changes to static-net-filtering.js are to fix
the erroneous insertion of filters with uppercase
characters. The single instance found was a hostname entry
in Malware Domain List (TRIANGLESERVICESLTD dot COM).
2019-05-06 11:12:39 -04:00
Raymond Hill
ce488d9fc8
Import translation work from https://crowdin.com/project/ublock 2019-05-01 20:05:16 -04:00
Raymond Hill
0e4fbefd07
Remove unecessary null placeholders FilterOriginHitSet et al.
The `null` placeholder are not necessary, we can just use
default arguments instead, and add the HNTrieContainer
references if and only if they are instanciated.
2019-05-01 18:54:11 -04:00
Raymond Hill
9e4385243c
Web accessible secrets can be used for at most one second
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/550

Related Chromium issue (I can't access it):
- https://bugs.chromium.org/p/chromium/issues/detail?id=957866

Findings so far: affects browsers based on Chromium 74.
I could not reproduce the issue with either Chromium 73 or
Google Chrome 75.

This commit is a mitigation: to prevent sites from using
uBO's internal WAR secret for tracking purpose. A secret
can be used for at most one second, after which a new secret
is generated.

The original issue related to the implementation of
secret-gated web accessible resources is:
- https://github.com/gorhill/uBlock/issues/2823
2019-04-30 14:36:07 -04:00
Raymond Hill
73e2f25e95
Add new cosmetic procedural operator: :nth-ancestor(n)
The purpose of this new `:nth-ancestor(n)` operator is to
lookup the nth ancestor relative to the currently selected
node.

It is essentially equivalent to `:xpath(..)`, where
ancestor distance is expressed as a number rather than a
sequence of slash-separated `..`.

The rationale to introduce this new procedural selector
is to have a low overhead way to accomplish ancestor
selection.
2019-04-30 09:02:14 -04:00
Raymond Hill
42bf659695
Revert "Order HNTrie nodes alphabetically to allow for early bailout"
This reverts commit f5f9e05071.
2019-04-30 07:00:52 -04:00
Raymond Hill
f5f9e05071
Order HNTrie nodes alphabetically to allow for early bailout
This commit implements the alphabetical ordering of HNTrie
nodes, so as to make it possible to bail out early at
HNTrie.matches() time.

Contrary to what I expected, there is no performance gain
observed to HNTrie.matches() as per benchmarks -- I find
the results perplexing.

Because of this I will revert this commit immediately.
The purpose of this commit is to record the changes so
that I can bring them back to life in the future whenever
I want to investigate further.
2019-04-30 06:47:54 -04:00
Raymond Hill
adabb56dc9
Do not store impossible to match filters in HNTrie
Consider the two following filters:

    example.com
    www.example.com

This commit make it so that if the first filter is
already present in a given HNTrie, the second filter
will not be stored, since HNTrie will _always_
return the first filter as a match whenever the
hostname to match is example.com or any subdomain
of example.com.

The detection of such pointless filters is
virtually free when adding a hostname to an HNTrie
instance (given how data is stored in the trie), so
in practice no overhead is incurred to detect such
pointless filters.

The ability to ignore impossible to match filters
in HNTrie instances will _especially_ benefit those
using large hosts files.

Examples of how this helps using real configurations:

- Default lists:
  444 filters out of 100,382 were ignored as a result
  of this commit.

- Default lists + "Energized Ultimate Protection":
  283,669 filters out of 903,235 were ignored as a
  result of this commit.

Side note: There was no measurable difference between
the two configurations above in the performance of
the matching algorithm as reported by the built-in
benchmark tool.
2019-04-29 13:15:16 -04:00
Raymond Hill
c4f9ae706a
Fix alternate code path introduced in 295f08da97 (oops) 2019-04-28 14:18:09 -04:00
Raymond Hill
295f08da97
Implement code path for when TextDecoder() is not available
The primary purpose is to unbreak
https://github.com/cliqz-oss/adblocker/tree/master/bench/comparison
2019-04-28 14:07:21 -04:00
Raymond Hill
ac58b8e688
Make token hashes fit within a 32-bit integer
The staticNetFilteringEngine uses token hashes to store/lookup
filters into Map objects.

Before this commit, the tokens were encoded into token hashes
as JS numbers (not exceeding MAX_SAFE_INTEGER) using at most
the 8 first characters of the token.

With this commit, token hashes are now restricted to fit
into 32-bit integers, and are derived from at most the 7 first
characters. This improves filter look-up performance as per
built-in benchmark().
2019-04-28 10:15:15 -04:00
Raymond Hill
510cda0bc5
Support in logger's filter expression picker 2019-04-28 07:23:14 -04:00
Raymond Hill
96dce22218
Increase resolution of known-token lookup table
Related commit:
- 69a43e07c4

Using 32 bits of token hash rather than just the 16 lower
bits does help discard more unknown tokens.

Using the default filter lists, the known-token lookup
table is populated by 12,276 entries, out of 65,536, thus
making the case that theoretically there is a lot of
possible tokens which can be discarded.

In practice, running the built-in
staticNetFilteringEngine.benchmark() with default filter
lists, I find that 1,518,929 tokens were skipped out of
4,441,891 extracted tokens, or 34%.
2019-04-27 08:18:01 -04:00
Raymond Hill
a8946c8d73
Fix list lookup of multi-hostname domain= filters in logger
Related commit:
- 3f3a1543ea

The regression was preventing uBO to find from which list a filter
originated. This affected only filters for which the `domain=`
option had multiple hostnames.
2019-04-27 07:04:43 -04:00
Raymond Hill
4c9e760a10
Import translation work from https://crowdin.com/project/ublock 2019-04-26 17:30:19 -04:00
Raymond Hill
69a43e07c4
Ignore unknown tokens in urlTokenizer.getTokens()
Given that all tokens extracted from one single URL are potentially
iterated multiple times in a single URL-matching cycle, it pays to
ignore extracted tokens which are known to not be used anywhere in
the static filtering engine.

The gain in processing a single network request in the static
filtering engine can become especially high when dealing with
long and random-looking URLs, which URLs have a high likelihood
of containing a majority of tokens which are known to not be in
use.
2019-04-26 17:14:00 -04:00
Raymond Hill
19ece97b0c
Leverage compile-time token information in new fitler classes
Related commit:
- 99390390fc

The token information available at compile time can be stored
in the filter to be used at match() time. This allows the use of
startsWith() rather than a more costly indexOf() call as a first
quick test to detect mismatches.
2019-04-26 11:16:47 -04:00
Raymond Hill
f667fc2d65
Fix page count computation in publicSuffixList.enableWASM() 2019-04-25 19:40:07 -04:00
Raymond Hill
e0d2285da0
Convert HNTrie code to ES6 class 2019-04-25 19:38:07 -04:00
Raymond Hill
155abfba18
Cache and reuse result of HNTrieRef.matches() when possible
Due to how web pages typically load secondary resources and due
to how HNTrieContainer instances are used in uBO, there is a
great likelihood that the result of a previous call to
HNTrieRef.matches() can be reused in a subsequent call.
This has been confirmed by instrumenting HNTrieRef.matches().

Since uBO uses distinct HNTrieContainer instances to either
match against the request or the origin hostnames, this
means a high likelihood of repeated calls to HNTrieRef.matches()
with the same hostname as argument, hence a performance gain
when caching the argument+result -- as despite that
HNTrie.matches() is fast, comparing two short strings is even
faster if this allows to skip HNTrie.matches() altogether.
2019-04-25 18:36:03 -04:00
Raymond Hill
99390390fc
Introduce three more specialized filter classes to avoid regexes
Performance- and memory-related work. Three more classes have
been created to avoid regex-based filters internally.

Purpose is to enforce filters which have only one single
wildcard in their pattern, a common occurrence. The filter
pattern is split in two literal string segments.

Similar as above, with the added condition that the filter is
hostname-anchored (`||`). The "Wildcard2" variant is a further
specialization to enforce filters where the only wildcard
is immediately preceded by the `^` special character, again
a very common occurrence.

Using two literal string segments in lieu of regexes allows to
quickly detect a mismatch by just testing the first segment.
Additionally, this reduces memory footprint as regexes are
much more expensive memory-wise than plain strings.

These three new filter classes allow to replace the use of
5276 regex-based filters internally with plain string-based
filters.

Often-called isHnAnchored() has been further fine-tuned to
avoid as much work as possible. I have also observed that
using an arrow function for closure-purpose helps measurably
performance, as per built-in benchmark.
2019-04-25 17:48:08 -04:00
Raymond Hill
fff2bb6290
Assume media elements with no Content-Length header to be of size 0
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/543
2019-04-24 08:30:54 -04:00
Raymond Hill
72bbcdd93c
Prevent search expression in CodeMirror editor from crossing line boundaries
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/493
2019-04-23 19:26:02 -04:00
Raymond Hill
cd1a11fa9d
Update to CodeMirror version 5.46 2019-04-23 19:06:03 -04:00
Raymond Hill
c535c624bd
Import translation work from https://crowdin.com/project/ublock 2019-04-23 09:32:15 -04:00
Raymond Hill
3c5102811a
Fix the logger's rendering of hostnames starting with digits
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/541
2019-04-23 09:28:00 -04:00
Raymond Hill
16a76aa524
Add filter expressions in logger's expression picker
- Added `media`
- Include `generichide` in `dom` filter expression
- Include `beacon`/`csp_report`/`ping` in `other filter expression
2019-04-22 10:23:58 -04:00
Raymond Hill
43ecffc295
Fix overzealous strict blocking (regression)
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/536

Regression from:
- 3f3a1543ea (diff-522a16ddeed280252d7c3a351261b441R2767)
2019-04-21 09:17:31 -04:00
Raymond Hill
f10b100379
Fix the handling of pseudoclass-based generic cosmetic filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/464

Regression from:
261ef8c510 (diff-3b15596213ed9ba37fb5b8bb1402a6c2R599)

Pseudoclass-based generic cosmetic filters were improperly seen
as invalid following the regression.
2019-04-21 07:49:44 -04:00
Raymond Hill
7735b35e21
Fix uncaught rejected promise in assets.fetchText()
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/534

Regression from
a52b07ff6e
2019-04-21 06:12:20 -04:00
Raymond Hill
97f91f8be9
Small code review of a52b07ff6e 2019-04-20 19:10:34 -04:00
Raymond Hill
f0d5205bd7
Discard existing lines when importing from file in "My filters"
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/519
2019-04-20 18:57:16 -04:00
Raymond Hill
537271f26b
Fix how *$, |https://, http:// filters are reported in logger
This was a regression introduced in
3f3a1543ea

Reported in issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/528#issuecomment-485163348
2019-04-20 17:25:32 -04:00
Raymond Hill
a52b07ff6e
Make userResourcesLocation able to support multiple URLs
The URLs must be space-separated.

Reminders:
- The additional resources will be updated at the same time
  the built-in resource file is updated
- Purging the cache of 'uBlock filters' will also purge the
  cache of the built-in resource file -- and hence force a
  reload of the user's custom resources if any

Related issues:
- https://github.com/gorhill/uBlock/issues/3307
- https://github.com/uBlockOrigin/uAssets/issues/5184#issuecomment-475875189

Addtionally:
- Opportunitically promisified assets.fetchText()
- Fixed https://github.com/gorhill/uBlock/issues/3586
2019-04-20 17:16:49 -04:00
Raymond Hill
fa83744b58
Use a sequence of base 64 numbers to encode array buffers
The purpose of using a custom base128 encoder is to
convert array buffers into strings, to allow a direct
string-to-array buffer conversion at load time:

  string => array buffer

Whereas a JSON array would require an extra step:

  JSON array as string => JS array => array buffer

Turns out that the current use of a custom base128 encoding
results in a significantly larger selfie storage usage when
converting array buffers into strings.

Speculation: possibly the browser convert the strings to
save into JSON strings internally. Since the custom base128
encoder is likely to cause the resulting string to contain
a lot of unprintable ASCII characters, these will need to
be escaped when converted to JSON -- escaped characters
occupy more space than non-escaped ones.

Using a sequence of base 64 numbers means only printable
will be present in the output string, hence no escaping
necessary. I have observed significant reduction in
storage usage for selfie purpose.
2019-04-20 09:06:54 -04:00
Raymond Hill
69cb5d8abd
Import translation work from https://crowdin.com/project/ublock 2019-04-19 17:07:27 -04:00
Raymond Hill
3f3a1543ea
Add HNTrie-based filter classes to store origin-only filters
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/528#issuecomment-484408622

Following STrie-related work in above issue, I noticed that a large
number of filters in EasyList were filters which only had to match
against the document origin. For instance, among just the top 10
most populous buckets, there were four such buckets with over
hundreds of entries each:

- bits: 72, token: "http", 146 entries
- bits: 72, token: "https", 139 entries
- bits: 88, token: "http", 122 entries
- bits: 88, token: "https", 118 entries

These filters in these buckets have to be matched against all
the network requests.

In order to leverage HNTrie for these filters[1], they are now handled
in a special way so as to ensure they all end up in a single HNTrie
(per bucket), which means that instead of scanning hundreds of entries
per URL, there is now a single scan per bucket per URL for these
apply-everywhere filters.

Now, any filter which fulfill ALL the following condition will be
processed in a special manner internally:

- Is of the form `|https://` or `|http://` or `*`; and
- Does have a `domain=` option; and
- Does not have a negated domain in its `domain=` option; and
- Does not have `csp=` option; and
- Does not have a `redirect=` option

If a filter does not fulfill ALL the conditions above, no change
in behavior.

A filter which matches ALL of the above will be processed in a special
manner:

- The `domain=` option will be decomposed so as to create as many
  distinct filter as there is distinct value in the `domain=` option
- This also apply to the `badfilter` version of the filter, which
  means it now become possible to `badfilter` only one of the
  distinct filter without having to `badfilter` all of them.
- The logger will always report these special filters with only a
  single hostname in the `domain=` option.

***

[1] HNTrie is currently WASM-ed on Firefox.
2019-04-19 16:33:46 -04:00
Raymond Hill
b70302c0fc
Cleanup comments following changes in 34f3cfe5e7 2019-04-16 19:20:56 -04:00
Raymond Hill
34f3cfe5e7
Add filterClassHistogram() method to µBlock.staticNetFilteringEngine
As a development tool for investigation purpose. To use, enter the
following at uBO's dev console:

    µBlock.staticNetFilteringEngine.filterClassHistogram()
2019-04-16 19:01:14 -04:00
Raymond Hill
4940cda154
Categorize google as a bad token for map key purpose
In the static network filtering engine, `google` token is too
generic and probably leads to too many false positives, beside
causing too large filter bucket.
2019-04-16 06:52:13 -04:00
Raymond Hill
60858b6719
Fix handling of backslashes in string expressions for :has-text() 2019-04-15 18:56:28 -04:00
Raymond Hill
a594b3f3d1
Add µBlock.staticNetFilteringEngine.bucketHistogram() as investigative dev tool
Additionally, lower the treshold of trieability to 4 for FilterPlainPrefix1.
2019-04-15 11:45:33 -04:00
Raymond Hill
c9c21f9cbf
Add more languages for list selection at install/reset time
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/501

Also, the handling of 3-letter language codes has been fixed.
2019-04-14 18:20:57 -04:00
Raymond Hill
7652808806
Improve handling of srcset-based images in element picker
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/517
2019-04-14 17:37:48 -04:00
Raymond Hill
b73480b4c5
Update fix for https://github.com/uBlockOrigin/uBlock-issues/issues/468
As suggested by @jspenguin2017:
https://github.com/uBlockOrigin/uBlock-issues/issues/468#issuecomment-482863195
2019-04-14 16:57:09 -04:00
Raymond Hill
c229003d31
Performance + code maintenance work on static network filtering engine
Implement a plain string trie container class: STrieContainer.

Make use of STrieContainer where beneficial

  Some filter buckets can grow quite large, and in such case
  coalescing "trieable" filter classes into a single trie reduces
  lookup performance and memory usage.

  For instance, at time of commit, the filter bucket for the
  `ad` keyword contains 919 entries[1].

  Coalescing trieable filters of the same class into a single plain
  string trie reduced the size of the bucket into 50 entries + two
  tries which are scanned only once each whenever the bucket is
  visited.

  [1] Enter the following code at uBO's dev console:
      µBlock.staticNetFilteringEngine.categories.get(0).get(µBlock.urlTokenizer.tokenHashFromString('ad'))

Refactor static network filtering engine code to make use of
ES6's syntactic sugar `class`.

Change first auto-update run from 7 to 5 minutes.
2019-04-14 16:45:20 -04:00
Raymond Hill
92c5f17b78
Improve usefulness of FilterContainer.benchmark()
Add ability to test/record results. This allows to compare against
output after code changes to detect and more accurately report
regressions.
2019-04-14 09:44:24 -04:00
Raymond Hill
813d96175d
Fix https://github.com/uBlockOrigin/uBlock-issues/issues/468 2019-04-13 08:10:55 -04:00
Raymond Hill
d2cb0f17ea
Report block count in benchmark()
The block count can be used for testing against regression after
code changes.
2019-04-12 10:19:38 -04:00
Raymond Hill
563f6fddde
Import translation work from https://crowdin.com/project/ublock 2019-04-08 12:36:19 -03:00
Noelle Leigh
0bb7b76338 Fixed wrong method for number of elements in a Map (#3755) 2019-04-06 16:42:24 -03:00
Raymond Hill
1a7a3298e2
Be prepared to deal with failure to read user settings
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/507
2019-04-03 13:18:47 -03:00
Raymond Hill
f47ad8366e
Fix https://github.com/NanoAdblocker/NanoCore/issues/257 2019-04-03 00:18:16 -03:00
Raymond Hill
d505adfa38
Upgrade CodeMirror to 5.45 2019-03-31 00:31:29 -03:00
Raymond Hill
f62d866b36
Code review implementation of cacheStorage.clear()
Possibly related issue:
- https://old.reddit.com/r/firefox/comments/b3u4nj/what_is_the_estimated_time_period_for_reviewing_a/

@gwarser has been able to reproduce at will, while I have been
unable to reproduce at all. The change here is to clear the
content of the database instead of outright deleting it before
restoring backed up settings.
2019-03-28 10:17:47 -03:00
Raymond Hill
f52e77b0c8
Import translation work from https://crowdin.com/project/ublock 2019-03-23 18:08:49 -03:00
Raymond Hill
2fd587b7ae
Simplyfy code to gather storage used with StorageManager.estimate()
Documentation:
https://developer.mozilla.org/docs/Web/API/StorageManager
2019-03-22 22:09:27 -03:00
Raymond Hill
ac71d6577a
Visually emphasize directive syntax (!#if/!#endif) in list viewer/editor 2019-03-21 19:53:04 -03:00
Raymond Hill
26c57feee8
Code review of IndexedDB usage for cache storage purpose
Use Promise.prototype.catch() to deal with potential exceptions.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/416
2019-03-21 17:49:19 -03:00
Raymond Hill
34a138e3ef
Add unlimitedStorage to Firefox manifest; add timeout to IndexedDB access
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/416

The Chromium version of uBO has declared `unlimitedStorage` since the
extension was first published in 2014. Declaring this permission in
Firefox brings uBO inline with the Chromium version. I suspect some
reported errors could be caused by IndexedDB eviction due to the lack
of `unlimitedStorage` permission.

Additionally, a timeout has been added when uBO tries to access its
indexedDB storage. It's unclear whether this will help with the
mentioned related issue though, the root cause is still to be
identified.
2019-03-17 09:45:28 -04:00
Raymond Hill
008370e4b9
Fix https://github.com/uBlockOrigin/uBlock-issues/issues/461
uBO will fallback using a JSON string when trying to encode an array
buffer in Chromium version 59 and earlier.
2019-03-16 09:00:31 -04:00
Raymond Hill
580c3885df
Fix typo which could lead to improper filtering context
Related discussion:
- 354ac4f57b (commitcomment-32715209)
2019-03-15 07:47:36 -04:00
Raymond Hill
875542c964
Code review of fix for https://github.com/uBlockOrigin/uBlock-issues/issues/459
Relocate workaround to the code responsible to compute filtering context, such
that the workaround will also be applied in other code paths, for example  also
for webRequest.onHeadersReceived.
2019-03-14 11:24:13 -04:00
Raymond Hill
9a7887eb39
Better English in comment 2019-03-13 17:21:30 -04:00
Raymond Hill
f5974a500b
Fix https://github.com/uBlockOrigin/uBlock-issues/issues/459 2019-03-13 17:17:37 -04:00
Raymond Hill
e49debd5dd
Properly report :spath operator of procedural cosmetic filters in logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/453
2019-03-08 07:26:55 -05:00
Raymond Hill
3a8b68ea76
Remove obsolete code related to assets storage refactoring in 1.11.0
The removed code was quite old, and was about how user filters were
persisted before/after uBO version 1.11, related to the following
issue:
- https://github.com/gorhill/uBlock/pull/2314

The assets storage refactoring was released in:
- https://github.com/gorhill/uBlock/releases/tag/1.11.0
2019-03-06 08:59:13 -05:00
Raymond Hill
67d143ec4e
Fix https://github.com/uBlockOrigin/uBlock-issues/issues/448 2019-03-05 12:42:59 -05:00
Raymond Hill
07d4a6b60c
Import translation work from https://crowdin.com/project/ublock 2019-03-05 10:52:56 -05:00
Raymond Hill
388c1c98ec
Fix parsing of AdGuard's #$?#-based cosmetic filters
As reported in the following commit:
- https://github.com/AdguardTeam/AdguardFilters/commit/4fe02d73cee6
2019-03-05 10:10:40 -05:00
Raymond Hill
5a2133de2a
Import translation work from https://crowdin.com/project/ublock 2019-02-26 11:29:22 -05:00
Raymond Hill
337b1f81b6
Code review of indexedDB-based cache storage 2019-02-26 10:37:25 -05:00
Raymond Hill
eaa0094e53
Fix uncaugh exception at launch when indexedDB is forbidden
Possibly related to:
- https://www.reddit.com/r/uBlockOrigin/comments/atbjvg/web_pages_dont_load_in_waterfox_56271_64bit_with/
2019-02-24 11:28:28 -05:00
Raymond Hill
6aa2074e2f
Import translation work from https://crowdin.com/project/ublock 2019-02-24 09:08:19 -05:00
Raymond Hill
c161d45230
Use const, let instead of var 2019-02-24 09:01:58 -05:00
Raymond Hill
1c26afe874
Remove caching the result of domain extraction from hostname
With the new PSL implementation, benchmarks do not show benefit
from caching the domain extracted from a hostname for later
reuse -- the caching seems to even add an overhead instead with
the new publicSuffixList implementation.
2019-02-20 08:51:14 -05:00
Raymond Hill
8edc3bf3e7
Import translation work from https://crowdin.com/project/ublock 2019-02-20 07:25:56 -05:00
Raymond Hill
c83b6776da
Remove support for implicit scriptlet injection
Related issue:
- https://github.com/gorhill/uBlock/issues/3550
2019-02-20 07:18:37 -05:00
Raymond Hill
b585518c00
Do not fall back to alternative cache backend if selected one fails
Default behavior is to fall back to an alternative backend
if the uBO-selected one is not available. However there will be
no fall back when the `cacheStorageAPI` is set to one specific
backend by the user.
2019-02-20 07:05:45 -05:00
Raymond Hill
87feb47b51
Support disabling suspendTabsUntilReady in Firefox
The value of `suspendTabsUntilReady` was disregarded in Firefox and
uBO defaulted to always defer tab loading until it was ready.

This commit allows to disable the deferring of tab loading in
Firefox. The new valid values for `suspendTabsUntilReady` are:
- `unset`: leave it to the platform to pick the optimal
  behavior (default)
- `no`: do no suspend tab loading at launch time
- `yes`: suspend tab loading at launch time
2019-02-19 12:30:37 -05:00
Raymond Hill
928ab91ab8
Add support to benchmark the dynamic filtering pane
From uBO's dev console, type:
- `µBlock.sessionFirewall.benchmark();`

Keep in mind that it's the temporary ruleset being benchmarked.
2019-02-19 10:46:33 -05:00
Raymond Hill
93842a3f9c
Improve detection of invalid CSS selectors
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/389

Additionally, fix case of using potentially uninitialized variable
in preview mode. Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/425
2019-02-18 16:00:42 -05:00
Raymond Hill
426a6ea9a7
Fix spurious output at uBO's dev console
Regression from https://github.com/gorhill/uBlock/commit/0d369cda21bb
2019-02-18 14:41:04 -05:00
Raymond Hill
f7a632c94a
Fix regression preventing new install or reseting to factory default
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/431
2019-02-18 06:18:30 -05:00
Raymond Hill
27f7de7b96
Import translation work from https://crowdin.com/project/ublock 2019-02-17 16:11:36 -05:00
Raymond Hill
0d369cda21
Allow use of browser.storage.local as cache storage backend in Firefox
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/409

By default `indexedDB` is used in Firefox for purpose of cache storage
backend.

This commit allows to force the use of `browser.storage.local` instead
as cache storage backend. For this to happen, set `cacheStorageAPI` to
`browser.storage.local` in advanced settings.

Additionally, should `indexedDB` not be available for whatever reason,
uBO will automatically fallback to `browser.storage.local`.
2019-02-17 15:40:09 -05:00
Raymond Hill
3b81841dc0
Properly set resource URL in benchmark loop 2019-02-17 07:45:05 -05:00
Raymond Hill
d63592b11e
Remove obsolete code to translate |blob: filters into CSP filters
These filters are to be considered obsolete since they can't be
matched against network requests in the webRequest API.

They were probably meant to work when ABP was pre-webext, which
means they are quite probably obsolete and there is no longer
a point for uBO to conveniently translate them into CSP directives.
2019-02-16 19:25:15 -05:00
Raymond Hill
e93062bcdf
Spin-off FilterOrigin flavors into standalone classes
This removes the derivation of FilterOrigin flavors from
FilterOrigin itself and simplify code paths. FilterOrigin
flavors are small specialized classes, no need to
overcomplicate with derivation.

Specifically, this removes an indirect call to reach the
match() method.
2019-02-16 12:16:30 -05:00
Raymond Hill
5733439f62
Leverage whotracks.me's huge dataset of URLs for benchmark purpose
As seen at:
  https://whotracks.me/blog/adblockers_performance_study.html

The requests.json.gz file can be downloaded from:
  https://cdn.cliqz.com/adblocking/requests_top500.json.gz

Copy the file into ./tmp/requests.json.gz

If the file is present when you build uBO using `make-[target].sh` from
the shell, the resulting package will contain `./assets/requests.json`,
which will be looked-up by the method below to launch a benchmark
session.

From uBO's dev console, launch the benchmark:
  µBlock.staticNetFilteringEngine.benchmark();

The usual browser dev tools can be used to obtain useful profiling
data, i.e. start the profiler, call the benchmark method from the
console, then stop the profiler when it completes.

Keep in mind that the measurements at the blog post above where obtained
with ONLY EasyList. The CPU reportedly used was:
  https://www.cpubenchmark.net/cpu.php?cpu=Intel+Core+i7-6600U+%40+2.60GHz&id=2608

Rename ./tmp/requests.json.gz to something else if you no longer want
./assets/requests.json in the build.
2019-02-15 16:18:03 -05:00
Raymond Hill
b1f7739cba
Import translation work from https://crowdin.com/project/ublock 2019-02-15 07:50:28 -05:00
Raymond Hill
515b7cdcfc
Use proper embedding context when evluating inline-script
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/422
2019-02-15 07:37:43 -05:00
Raymond Hill
ed7e34fb07
Refactor selfie generation into a more flexible persistence mechanism
The motivation is to address the higher peak memory usage at launch
time with 3rd-gen HNTrie when a selfie was present.

The selfie generation prior to this change was to collect all
filtering data into a single data structure, and then to serialize
that whole structure at once into storage (using JSON.stringify).

However, HNTrie serialization requires that a large UintArray32 be
converted into a plain JS array, which itslef would be indirectly
converted into a JSON string. This was the main reason why peak
memory usage would be higher at launch from selfie, since the JSON
string would need to be wholly unserialized into JS objects, which
themselves would need to be converted into more specialized data
structures (like that Uint32Array one).

The solution to lower peak memory usage at launch is to refactor
selfie generation to allow a more piecemeal approach: each filtering
component is given the ability to serialize itself rather than to be
forced to be embedded in the master selfie. With this approach, the
HNTrie buffer can now serialize to its own storage by converting the
buffer data directly into a string which can be directly sent to
storage. This avoiding expensive intermediate steps such as
converting into a JS array and then to a JSON string.

As part of the refactoring, there was also opportunistic code
upgrade to ES6 and Promise (eventually all of uBO's code will be
proper ES6).

Additionally, the polyfill to bring getBytesInUse() to Firefox has
been revisited to replace the rather expensive previous
implementation with an implementation with virtually no overhead.
2019-02-14 13:33:55 -05:00
Raymond Hill
2fb4dc8358
Update PSL library to latest official version 2019-02-12 12:00:20 -05:00
Raymond Hill
1e2969c3b0
Update PSL library to last version 2019-02-12 07:59:38 -05:00
Raymond Hill
0c976992ba
Import translation work from https://crowdin.com/project/ublock 2019-02-10 12:26:48 -05:00
Raymond Hill
4da340384a
Update publicSuffixLibrary to latest (WASM-able) version
See https://github.com/gorhill/publicsuffixlist.js
2019-02-10 12:19:05 -05:00
Raymond Hill
29b10d2151
Tone down logger visual for tab-less network requests 2019-02-05 12:05:00 -05:00
Raymond Hill
d432f78b66
Import translation work from https://crowdin.com/project/ublock 2019-02-05 07:04:52 -05:00
Raymond Hill
4c428edeef
Import translation work from https://crowdin.com/project/ublock 2019-02-02 08:14:47 -05:00
Raymond Hill
656203adc7
Minor generic code review of changes since 1.18.2 2019-02-02 08:09:34 -05:00
Raymond Hill
fc03782985
Ensure that WASM module was actually loaded 2019-02-01 09:09:51 -05:00
Raymond Hill
ff269d793d
Use indexedDB instead of for cacheStorageAPI value 2019-02-01 08:27:28 -05:00
Raymond Hill
69c87c5117
Fix Promise chain of WASM module load operations
The Promise chain was not properly designed for WASM module
loading. This became apparent when removing WASM modules
from Opera build[1].

The problem was that errors thrown by fetch() -- used to
load WASM modules -- were not properly handled.

[1] Opera refuses updating uBO if there are unrecognized file
types in the package, and `.wasm`/`.wat` files are not
recognized by Opera uploader.
2019-02-01 08:20:43 -05:00
Raymond Hill
923c5ce5bd
Compute URL of sublists as relative to URL of parent list
Related issue:
- https://github.com/NanoAdblocker/NanoCore/issues/239

The erroneous behavior was to compute the URL of a sublist as
relative to the URL of the root list, which may differ from the
URL of a parent list.
2019-01-29 11:52:16 -05:00
Raymond Hill
3195f554f7
Fix partyness evaluation for cases of base domain-less hostnames
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/402
2019-01-29 10:34:58 -05:00
Raymond Hill
98307e07cb
Import translation work from https://crowdin.com/project/ublock 2019-01-28 16:22:41 -05:00
Raymond Hill
d7c169c587
Import translation work from https://crowdin.com/project/ublock 2019-01-27 17:24:26 -05:00
Raymond Hill
15100459b3
Harden content script's message Port against spurious disconnections
Those spurious disconnections have been observed to occur at
uBO's launch time.

Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/403

I have observed that this fixes an issue observed on Firefox 64
(current stable).

The reported Waterfox issue *may* be fixed as a result. If not,
the issue he still considered fixed as Waterfox is not
officially supported.
2019-01-27 17:07:40 -05:00
Raymond Hill
85d6339b07
Code review for a026e9ae54: no need to be so flexible 2019-01-25 18:58:49 -05:00
Raymond Hill
a026e9ae54
Fix reverting use of IndexedDB as default cache storage on Chromium
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/399

The advanced setting `cacheStorageAPI` has been added to allow
a user to force the use of IndexedDB as cache storage. Set to
`IndexedDB` to force use of IndexedDB. Default to `unset`.
2019-01-25 18:49:30 -05:00
Raymond Hill
185ebf0d7d
Import translation work from https://crowdin.com/project/ublock 2019-01-25 08:47:39 -05:00
Raymond Hill
ba7baec1f4
Import translation work from https://crowdin.com/project/ublock 2019-01-24 07:06:43 -05:00
Raymond Hill
b214ec1bae
Ensure pointer-events style is set to auto at element picker launch
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/393
2019-01-23 17:11:07 -05:00
Raymond Hill
be76714cfc
Import translation work from https://crowdin.com/project/ublock 2019-01-22 06:53:09 -05:00
Raymond Hill
9e89dd4f26
Fix missing partyness in logger when empty domain in resource URL 2019-01-22 06:13:30 -05:00
Raymond Hill
a668893427
Import translation work from https://crowdin.com/project/ublock 2019-01-21 12:18:10 -05:00
Raymond Hill
64e0498084
Import translation work from https://crowdin.com/project/ublock 2019-01-20 12:35:25 -05:00
Raymond Hill
16a0ebbfb0
Import translation work from https://crowdin.com/project/ublock 2019-01-19 09:09:37 -05:00
Raymond Hill
1de821d99b
Apply stricter rejection of usage of url() in :style rules 2019-01-18 19:07:31 -05:00
Raymond Hill
6d19c4be88
Import translation work from https://crowdin.com/project/ublock 2019-01-18 06:57:47 -05:00
Raymond Hill
ff6991c292
Import translation work from https://crowdin.com/project/ublock 2019-01-17 15:54:43 -05:00
Raymond Hill
52526b74d9
Minor visual change to logger 2019-01-17 15:49:18 -05:00
Raymond Hill
299afacd99
Import translation work from https://crowdin.com/project/ublock 2019-01-17 08:07:24 -05:00
Raymond Hill
4df8e9be41
Add logger "Export"-related strings for i18n 2019-01-17 08:03:50 -05:00
Raymond Hill
801eb43572
Prevent info entries from polluting logger output
These entries are meant to be shown only when using "All"
mode, not when the output is narrowed to a specific tab.
2019-01-16 13:29:34 -05:00
Raymond Hill
0c80994124
Fine tune visuals in logger's export-to-clipboard dialog 2019-01-16 12:02:16 -05:00
Raymond Hill
5f341e02ed
Import translation work from https://crowdin.com/project/ublock 2019-01-16 07:32:18 -05:00
Raymond Hill
e8ff6a2abf
Prevent sites from disabling mouse events for element picker
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/380

Previous fix was incorrect:
84818fcd84

Additionally, I fixed the "Network filters" or "Cosmetic filters"
sections not being properly hidden when there was not valid
filters found.
2019-01-15 16:34:57 -05:00
Raymond Hill
86e5d0384c
Fix minor logger quirks as per feedback 2019-01-15 15:12:47 -05:00
Raymond Hill
84818fcd84
Prevent sites from disabling mouse events for the element picker
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/380
2019-01-15 14:41:23 -05:00
Raymond Hill
0b73808bf7
Import translation work from https://crowdin.com/project/ublock 2019-01-15 08:42:28 -05:00
Raymond Hill
3b206fd224
Put back 'Static filter' instead of just 'Filter' 2019-01-15 08:32:39 -05:00
Raymond Hill
b0fb6d678d
Fix regression from 0edf53f508 (diff-a51c8c4aedf62ea9aec5e71f3b1427c3R624) 2019-01-15 08:18:45 -05:00
Raymond Hill
5d5b05b387
Fix an issue in new logger export feature
Specifically, if selecting table+markdown, the export dialog
would no longer be accessible once closed.
2019-01-14 16:52:13 -05:00
Raymond Hill
ac5bae6a65
Import translation work from https://crowdin.com/project/ublock 2019-01-14 15:10:28 -05:00
Raymond Hill
0edf53f508
Add export-to-clipboard feature to logger
Related issue:
- https://github.com/uBlockOrigin/uBlock-issues/issues/334

Additionally, a number of smallish issues following
refactoring of the logger code were addressed.
2019-01-14 14:57:31 -05:00
Raymond Hill
88a54d442d
Import translation work from https://crowdin.com/project/ublock 2019-01-14 08:08:09 -05:00
Raymond Hill
3be59f172d
Use blocked/allowed colors in logger's "Details" pane
Additionally, another attempt at making the logger render
better on small-screen devices.
2019-01-13 08:34:17 -05:00
Raymond Hill
10f1fdcb7c
Import translation work from https://crowdin.com/project/ublock 2019-01-12 17:22:55 -05:00
Raymond Hill
9fd63bdbe6
Merge branch 'master' of github.com:gorhill/uBlock into nextdev 2019-01-12 17:06:07 -05:00
Raymond Hill
be0c7186a2
Import translation worl from https://crowdin.com/project/ublock 2019-01-12 17:05:41 -05:00
Raymond Hill
ed5d63df69
Grand refactoring of the logger
Performance-related work: the logger data has been decoupled
from the DOM -- inspired from CodeMirror's way of efficiently
handling large amout of text data.

This decoupling now makes the logger highly efficient CPU- and
memory-wise, and open the way to more possibilities.

Ability to configure some aspect of the logger behavior and
visuals:
- The hard-coded limit of 5000 entries has been
  removed and is now replaced with a variety of
  user-configurable settings to enforce the discarding of
  logger entries.
- Some columns in the logger output can now be hidden.

The filter list look-up feature has been merged into the
existing overlay dialog used to create URL rules or static
filters, as an entry in a new "Details" pane.

Other issues addressed during refactoring:
- https://github.com/uBlockOrigin/uBlock-issues/issues/280
- https://github.com/gorhill/uBlock/issues/1999

The minimum version supported on Firefox has been bumped
up to 55.0.
2019-01-12 16:36:20 -05:00
Raymond Hill
407aa68272
Merge branch 'master' of github.com:gorhill/uBlock into nextdev 2019-01-08 08:08:16 -05:00
Raymond Hill
f1f1892233
Import translation work from https://crowdin.com/project/ublock 2019-01-08 07:57:49 -05:00
Raymond Hill
64bea27881
Add ability to control auto-commenting at filter creation time
Related issues:
- https://github.com/uBlockOrigin/uBlock-issues/issues/372
- https://github.com/gorhill/uBlock/issues/93

A new advanced settings has been added: `autoCommentFilterTemplate`.

Default value is `{{date}} {{origin}}`.

Placeholders are identified by `{{...}}`. There are currently
only three placeholders supported:

- `{{date}}`: will be replaced with current date
- `{{time}}`: will be replaced with current time
- `{{origin}}`: will be replaced with site information on which
  the filter(s) was created

If no placeholder is found in `autoCommentFilterTemplate`, this
will disable auto-commenting. So one can use `-` to disable
auto-commenting.

Additionally, if auto-commenting is enabled, uBO will not emit a
comment if an emitted comment would be a duplicate of the last
one found in the user filter list.
2019-01-08 07:37:50 -05:00