Mike Fährmann
50acf2ac84
[danbooru] add 'artist-search' extractor ( #5348 )
2024-11-17 16:58:54 +01:00
Mike Fährmann
ca4b2a0760
[danbooru] add 'artist' extractor ( #5348 )
2024-11-17 16:23:16 +01:00
Mike Fährmann
9184a5643a
[danbooru] move all initialization code into '_init()'
2024-11-17 16:15:53 +01:00
Mike Fährmann
55afd712d6
[pp] allow inheriting settings from global 'postprocessor' entries
...
No idea how to properly explain/document this, so here's an example:
The extractor.postprocessors object
gets its options from postprocessor.jl
and adds 'filename' itself.
{
"extractor": {
"postprocessors": {
"type": "jl",
"filename": "meta.jsonl"
}
},
"postprocessor": {
"jl": {
"name": "metadata",
"mode": "jsonl",
"open": "a"
}
}
}
2024-11-16 21:16:13 +01:00
Mike Fährmann
80454460ce
[config] support accumulating non-list values
...
fixes 1264fc518b
2024-11-16 21:13:57 +01:00
Mike Fährmann
bced143750
[tests] add workaround for compile_expression_defaultdict in pypy3
2024-11-16 19:35:28 +01:00
Mike Fährmann
f0419574a5
merge #6475 : [imagechest] fix extractors
2024-11-16 09:26:41 +01:00
Mike Fährmann
75612997fe
[imagechest] simplify
...
and fix user pagination end condition
2024-11-16 09:17:13 +01:00
Mike Fährmann
f7246f025f
[weibo] simplify 'livephoto' extraction ( #6471 )
...
continuation of 396b52aef7
2024-11-16 08:19:02 +01:00
Mike Fährmann
cb09273670
[koharu] implement 'tags' option
2024-11-15 23:49:58 +01:00
Mike Fährmann
ddd325b435
merge #6432 : [koharu] update domain ( #6430 )
2024-11-15 22:41:46 +01:00
Mike Fährmann
e5c2882320
[koharu] cleanup
...
- update BASE_PATTERN formatting
- fix groups indices
- add tests for new domains
- update docs/supportedsites
2024-11-15 22:41:40 +01:00
K0ng2
a09d9edaa6
[koharu] updat root and root_api change
2024-11-15 22:14:33 +01:00
Mike Fährmann
0d1469f229
[exhentai] implement 'tags' option ( #2117 )
...
allow splitting tags into categories,
e.g. 'tags_parody', 'tags_group', etc.
2024-11-15 21:47:13 +01:00
Mike Fährmann
1264fc518b
allow 'postprocessors' to be a single dict/str
...
do not require it to be a list with just one element
"postprocessors": "metadata"
"postprocessors": {"name": "metadata"}
2024-11-15 21:15:00 +01:00
Mike Fährmann
c82f3db098
[common] add 'proxy-env' option
...
(#6134 , #6455 )
disable using environment proxies by default
2024-11-15 18:03:56 +01:00
Mike Fährmann
0a72a5009c
[common] disable Authorization header injection from .netrc auth
...
(#6134 , #6455 )
2024-11-15 17:37:04 +01:00
Mike Fährmann
a3dbc58172
[pillowfort] provide 'count' metadata field ( #6478 )
2024-11-15 08:27:52 +01:00
Mike Fährmann
9821503226
[misc] 'api_root' -> 'root_api'
2024-11-14 23:44:15 +01:00
Mike Fährmann
e763efd36c
[bilibili] add workarounds for getting rate-limited ( #6443 )
...
- set 3-6 second request_interval by default
- retry request after waiting 5 minutes
2024-11-14 23:06:26 +01:00
Mike Fährmann
5bc3657c59
[util] implement 'compile_filter()' ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-2477029728
allow (theoretically*) all filter expression statements
to be a list of individual filters
(*) except for 'filename' and 'directory' conditionals,
as dict keys cannot be lists
2024-11-14 22:47:36 +01:00
Mike Fährmann
0b99d9e6b9
[util] add "defaultdict" filters-environment
...
allows accessing undefined values without raising an exception,
but preserves other errors like TypeError, AttributeError, etc
2024-11-14 22:47:25 +01:00
Mike Fährmann
cfe24a9e31
[twitter] make 'source' metadata extraction non-fatal ( #6472 )
2024-11-14 18:59:01 +01:00
Mike Fährmann
396b52aef7
[weibo] fix livephoto 'filename' & 'extension' ( #6471 )
2024-11-14 18:56:18 +01:00
Achim
917e873c63
fix imagechest extractor
2024-11-14 16:54:59 +01:00
Achim
b2fa149598
fix imagechest extractor
2024-11-14 16:50:06 +01:00
Mike Fährmann
a3276e3b5d
[hentaifoundry] add 'tag' extractor ( #6465 )
2024-11-13 20:56:37 +01:00
Mike Fährmann
b62c466c14
[flickr] fix video download URLs ( #6464 )
...
continuation of 0e18fa395d
2024-11-13 20:56:37 +01:00
Mike Fährmann
cd6d6ea8be
[options] fix passing negative numbers as arguments ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-2468677453
fixes regression introduced in 9e729681
2024-11-11 19:07:37 +01:00
Mike Fährmann
30ff775382
[workflows:tests] change job name to 'test'
...
… and use alternate list syntax for Python versions
2024-11-10 21:29:48 +01:00
Mike Fährmann
2b96d638dc
[bunkr] support 'bunkr.cr' URLs
2024-11-10 20:43:33 +01:00
Mike Fährmann
096b9f1d26
[bunkr] fix album names containing <>&
...
unescaping HTML entities once is not good enough
2024-11-10 20:38:21 +01:00
Mike Fährmann
c61c0461a9
[urlgalleries] fix 'root' and update 'request_interval'
2024-11-10 20:28:55 +01:00
Mike Fährmann
73d6e56a8f
merge #6443 : [bilibili] add support for articles ( #2824 )
2024-11-10 18:01:51 +01:00
Mike Fährmann
82d561e825
[bilibili] update
...
- use self.groups[…] to access matched values
- extract more metadata (count, width, height, size)
- remove type hint
- add tests
- update docs/supportedsites
2024-11-10 17:59:24 +01:00
hdk5
fc59e0fb14
[bilibili] support large articles
2024-11-10 15:18:03 +02:00
Mike Fährmann
74f1e9a1ac
[poipiku] return 'count' as proper number ( #6445 )
2024-11-10 08:26:43 +01:00
hdk5
6eef3e3495
[bilibili] initial support ( #2824 )
2024-11-10 00:21:27 +02:00
Mike Fährmann
7916c8bf77
allow passing cookies to OAuth extractors
...
partially revert ce54b8c04c
2024-11-09 18:06:27 +01:00
Mike Fährmann
0e18fa395d
[flickr] use "download" URLs ( #6360 )
2024-11-09 17:33:27 +01:00
Mike Fährmann
1ddbcda58b
[nhentai] support ',webp' files ( #6442 )
2024-11-08 17:46:38 +01:00
Mike Fährmann
9e72968154
add '-4/--force-ipv4' and '-6/--force-ipv6' command-line options
2024-11-08 16:37:14 +01:00
Mike Fährmann
b6cf348658
[webtoons] extract 'episode_no' for comic results ( #6439 )
2024-11-08 14:19:17 +01:00
Mike Fährmann
77f761d320
merge #6437 : [philomena:ponybooru] switch default filter
...
… to get everything by default
2024-11-08 08:20:10 +01:00
Mike Fährmann
6205e255f4
merge #6394 : [tumblr] add 'search' extractor
2024-11-08 08:17:46 +01:00
Mike Fährmann
33778d35ba
[tumblr] update
...
- simplify
- fix search pagination
- support custom search mode and post types
2024-11-08 08:15:13 +01:00
Shelvacu
f8e707b92c
[philomena] switch default ponybooru filter to get everything by default
...
The system filter mislabeled "Everything" hides 4 tags https://ponybooru.org/filters/2
There are [many public filters that don't hide anything](https://ponybooru.org/filters?fq=spoilered_count%3A0%2C+hidden_count%3A0 ), I just picked [the oldest one](https://ponybooru.org/filters/3 ).
2024-11-07 20:08:42 -08:00
Mike Fährmann
ce90566c56
[pinterest] detect video/audio by block content ( #6421 )
...
story blocks from search/board results do not always contain a 'type'
2024-11-05 15:55:24 +01:00
Mike Fährmann
a9a9f3a180
[pinterest] support 'story_pin_music_block' blocks ( #6421 )
2024-11-05 15:55:24 +01:00
Mike Fährmann
0b3ddd01af
[hiperdex] update domain to 'hipertoon.com' ( #6420 )
...
and fix 'description' extraction
2024-11-05 15:54:42 +01:00
