Mike Fährmann
ae3706286a
[speakerdeck] inherit from GalleryExtractor
2024-06-15 21:56:51 +02:00
Mike Fährmann
9c65db2a92
consistent 'with open(…) as fp:' syntax
2024-06-14 01:22:00 +02:00
Mike Fährmann
3fc2e61818
add '--config-open' and '--config-status' ( #5713 )
...
adapted from 119a25a59c
2024-06-13 23:22:56 +02:00
Mike Fährmann
a92c0bef96
merge #5730 : [speakerdeck] fix extraction
2024-06-13 15:55:57 +02:00
Yuta Hayashibe
f272b5d2ac
fix: extractor for speakerdeck.com
2024-06-13 15:54:54 +02:00
Mike Fährmann
c699ce8ebb
[twitter] add 'username-alt' option ( #5715 )
2024-06-13 00:54:43 +02:00
Mike Fährmann
c6fc0281e8
[newgrounds] extend 'format' option ( #5709 )
...
- check more extensions for original formats (mp4, webm, m4v, mov, mkv)
- allow specifying which extensions and recoded formats to check
2024-06-12 20:46:45 +02:00
Mike Fährmann
86f0c3baaf
[szurubooru] support empty tag searches ( #5711 )
2024-06-11 20:25:06 +02:00
Mike Fährmann
40c685dc43
merge #5712 : [hentainexus] update key
2024-06-10 19:58:03 +02:00
KageNiHisomu
87859ff200
fix: static method defines hostname, update url link
2024-06-10 14:35:03 +10:00
KageNiHisomu
68540aef2a
update nexus key
2024-06-10 14:25:51 +10:00
Mike Fährmann
2e11b6e756
[nijie] support downloading videos ( #5707 , #5617 )
2024-06-08 22:55:28 +02:00
Mike Fährmann
e17b1d5073
[deviantart] fix 2c8aa9b2 ( #5696 )
2024-06-08 02:00:36 +02:00
Mike Fährmann
f160859c5c
[hitomi] extract 'title_jpn' metadata ( #5706 )
2024-06-08 00:05:19 +02:00
Mike Fährmann
722b4b513d
merge #5609 : [instagram] extract 'liked' metadata
2024-06-07 01:43:13 +02:00
Mike Fährmann
7b0c07ee69
[instagram] rename to 'liked', enable for GraphQL results
2024-06-07 01:41:56 +02:00
Mike Fährmann
aa9be75d44
[twitter] fix duplicate ArkoseLogin check
...
forgot to replace this in 9e5d65fbf3
2024-06-06 19:44:42 +02:00
Mike Fährmann
2c8aa9b2de
[deviantart] fix watching module ID extraction ( #5696 )
2024-06-06 15:21:04 +02:00
Mike Fährmann
9abeab5ecf
[shimmie2] support 'vidya.pics' ( #5632 )
2024-06-06 15:08:56 +02:00
Mike Fährmann
162d4269ec
[twitter] extend 'ratelimit' option ( #5532 )
...
allow waiting for a set amount of seconds
2024-06-06 01:18:08 +02:00
Mike Fährmann
2cab87c5b6
[kemonoparty:favorite] fix exception when sorting None ( #5692 )
2024-06-05 16:49:30 +02:00
Mike Fährmann
7614bc458e
[util] extend CustomNone with comparison operators
2024-06-05 16:49:30 +02:00
Mike Fährmann
1ce5de0290
[formatter] implement 'C' format specifier ( #5647 )
...
to apply a conversion after ':' or
to apply multiple conversions
for example {tags:CSl} or {tags:J - /Cl}
to convert list to string and lowercase it
2024-06-05 16:49:29 +02:00
Mike Fährmann
9b99d2c886
[philomena] support downloading SVG files ( #5643 )
2024-06-05 16:48:51 +02:00
Mike Fährmann
8fce9ea6d5
[hentainexus] restore module ( #5275 )
...
revert 97641cd151
2024-06-05 16:48:25 +02:00
Mike Fährmann
3ec7ee6baa
[downloader:ytdl] fix exception due to missing 'ext' ( #5675 )
2024-06-02 18:16:53 +02:00
Mike Fährmann
5d3d03a1f1
fix 6cfbc107
...
the former condition would return True for 2.31.*
6cfbc1071f (commitcomment-142642913)
2024-06-02 18:16:53 +02:00
Mike Fährmann
4d11cd9ffb
[vichan] remove 'wikieat.club'
...
redirects to some non-vichan site
2024-06-02 18:16:24 +02:00
Mike Fährmann
31133b97fb
[nitter] remove instances
2024-06-02 18:15:53 +02:00
Mike Fährmann
ce228ee163
[photobucket] remove module
...
had been broken for years and the new site is payed access only
2024-06-02 01:40:31 +02:00
Mike Fährmann
612435480a
[workflows:tests] restore Python 3.5 usage
...
https://github.com/actions/setup-python/issues/866
2024-06-02 01:32:54 +02:00
Mike Fährmann
31bdb288ef
release version 1.27.0
2024-06-01 21:06:12 +02:00
Mike Fährmann
6cfbc1071f
workaround for requests 2.32.3 ( #5665 )
...
manually call 'load_default_certs()' for SSLContexts
in custom HTTPAdapter instances
2024-06-01 16:02:18 +02:00
Mike Fährmann
807e2f7094
[realbooru] fix videos and provide fallback URLs ( #2530 )
...
revert acc94ac187
2024-05-31 23:58:40 +02:00
Mike Fährmann
deacbcbfdf
[tbscans] remove excess whitespace in 'manga'
2024-05-31 22:32:59 +02:00
Mike Fährmann
00440b8fb1
[slideshare] fix extraction
2024-05-31 21:05:50 +02:00
Mike Fährmann
281ab33f87
[4archive] fix extraction
2024-05-31 17:42:53 +02:00
Mike Fährmann
a53db09ca0
[artstation] disable TLS 1.2 ciphers by default ( #5564 , #5658 )
2024-05-30 23:54:19 +02:00
Mike Fährmann
7ba2b209f1
fix exit status of --clear-cache/--list-modules/--list-extractors
...
All three would always cause gallery-dl to exit with status 1,
regardless of error or success.
2024-05-29 02:56:25 +02:00
Mike Fährmann
3d033a48ea
[workflows:executables] change dev/linux binary name to …_linux
...
This reverts commit 05f8435e09
2024-05-29 02:46:29 +02:00
Mike Fährmann
53aadb0c44
[update] implement '--update-to'
...
to switch between update channels (stable or dev (alias nightly & master))
or to install a specific version
Examples:
--update-to dev
--update-to dev@2024.05.25
--update-to v1.25.2
2024-05-28 20:40:03 +02:00
Mike Fährmann
5018bd42f8
[update] implement '--update-check'
...
to check if a new update is available without downloading it
2024-05-28 03:42:58 +02:00
Mike Fährmann
2e323ef0a2
[update] implement '-U/--update'
...
to auto-update executable files in-place
2024-05-28 02:26:13 +02:00
Mike Fährmann
601f5becc8
[version] add __variant__
...
Specifies origin and OS of executable files.
For example 'stable/windows'.
2024-05-27 21:37:01 +02:00
Mike Fährmann
020050ea8b
merge #5641 : [pixeldrain] add support for single file album download
2024-05-25 23:43:43 +02:00
Mike Fährmann
154a890399
[pixeldrain] integrate into 'album' extractor
2024-05-25 23:42:23 +02:00
HornyQT
24e70b956b
[pixeldrain] add support for single file album download
2024-05-25 16:06:50 +02:00
Mike Fährmann
0761b22a7f
[hiperdex] update domain to 'hiperdex.top' ( #5635 )
2024-05-24 17:13:10 +02:00
Mike Fährmann
ea434963ae
[subscribestar] fix file URLs ( #5631 )
2024-05-23 19:12:01 +02:00
Mike Fährmann
6198dfa424
[seiga] re-implement login with username & password
2024-05-23 18:52:53 +02:00
Mike Fährmann
329da4db5e
[newgrounds] update and fix login procedure ( #5109 )
2024-05-23 17:06:22 +02:00
Mike Fährmann
895e633c44
implement 'keywords-eval' option ( #5621 )
...
to allow evaluating 'keywords' values as format strings
2024-05-22 22:53:34 +02:00
Mike Fährmann
bc25636e63
[patreon] fix bootstrap data extraction ( #5624 )
...
again
2024-05-22 00:17:49 +02:00
Mike Fährmann
141efc2ad3
[twitter] transfer 'twitter.com' cookies to 'x.com' ( #5597 )
2024-05-21 23:46:42 +02:00
Jan Wikholm
ece3837408
Enable user_has_liked as IG download filter
...
Background and context:
1. Instagram by default requires you to be logged in.
2. Your account can LIKE posts.
3. Getting all of your likes from IG can be a pain.
As an active IG user, I want to be able to download only posts I've liked.
The information of whether or not your logged-in user has liked the target post is available in the API results, but was not available to the end-user for filtering.
This small change allows `user_has_liked` to be used to e.g. download only liked posts of an entire IG account.
Example use:
```
$ gallery-dl --cookies-from-browser FIREFOX--filter 'user_has_liked' https://www.instagram.com/imgur/
```
This will only download the posts that have `has_liked: True` set in the returned REST API data.
Note: it will still need to loop through the entire account's posts since this is done per-post basis instead of looping just the liked posts, but at least you are able to use this IG feature as a download marker.
2024-05-18 12:05:08 +03:00
Mike Fährmann
6276f5cd94
[twitter] update domain to x.com ( #5597 )
2024-05-17 22:59:52 +02:00
Mike Fährmann
bfcb66af05
[poipiku] send LANG cookie to ensure English UI ( #5590 )
2024-05-17 22:59:31 +02:00
Mike Fährmann
f651b3b6ab
merge #5601 : [twitter] match '/video/' Tweet URLs
2024-05-17 22:49:12 +02:00
Mike Fährmann
2dec4bef60
merge #5591 : [poipoku] avoid language-specific extraction
2024-05-17 22:47:51 +02:00
Mike Fährmann
7f1ed909d5
[imgur] match gallery/album/image URLs with title slugs ( #5593 )
2024-05-17 22:44:37 +02:00
Delphox
8ba73e2ec9
[twitter] match /video/ tweet urls
2024-05-17 16:50:51 -03:00
fireattack
831f922c1c
[poipoku] avoid language-specific extr
2024-05-14 02:42:02 +08:00
Mike Fährmann
33b07c4603
[twitter] wait for rate limit reset before 429 error ( #5532 )
2024-05-12 01:13:30 +02:00
Mike Fährmann
3b6f306a8b
[exhentai] fix blank page detection
2024-05-11 22:01:34 +02:00
Mike Fährmann
90b3741643
[cookies] set proper 'expires' value for Chrome session cookies
...
https://github.com/yt-dlp/yt-dlp/pull/9747
2024-05-11 19:46:35 +02:00
Mike Fährmann
f178839fc2
[exhentai] fix multi-page viewer detection ( #4969 )
2024-05-11 18:32:52 +02:00
Mike Fährmann
5b6b5dd81b
[8chan] fix downloaded files by sending 'TOS' cookie ( #5578 )
2024-05-11 15:34:21 +02:00
Mike Fährmann
d2f50ecf09
add 'skip-filter' option ( #5255 )
2024-05-10 22:59:52 +02:00
Mike Fährmann
fd734b9222
[archive] add 'archive-mode' option ( #5255 )
2024-05-10 22:59:51 +02:00
Mike Fährmann
28039229fe
[common] use 'create_urllib3_context' for creating SSLContexts
...
enables dumping TLS session keys by setting SSLKEYLOGFILE (#5215 )
as well as other potentially useful settings.
2024-05-10 22:59:29 +02:00
Mike Fährmann
33006fe126
[common] disable 'check_hostname' for non-urllib3 SSLContexts
...
e.g. when 'browser' is set to a non-empty value and gallery-dl creates
its own SSLContext instance instead of using requests' and urllib3's
defaults.
urllib3 disables this option for its default contexts,
since it does this check on its own.
Fixes "ValueError: Cannot set verify_mode to CERT_NONE when
check_hostname is enabled" when using --no-check-certificate.
(#3614 , #4891 , #5576 )
2024-05-10 18:20:08 +02:00
Mike Fährmann
215abbc3e4
[archive] implement DownloadArchiveMemory class ( #5255 )
...
keeps archive IDs in memory
and only writes them to disk in a 'finalize' step
2024-05-10 01:15:44 +02:00
Mike Fährmann
88f94190f4
[archive] move DownloadArchive into its own module
2024-05-10 01:05:28 +02:00
Mike Fährmann
2ee9ffeed6
merge #5568 : [furaffinity] match 'xfuraffinity' URLs
2024-05-09 19:20:12 +02:00
Mike Fährmann
b27ffd790a
[poipiku] fix downloading R-18 posts ( #5567 )
...
… by automatically sending a `POIPIKU_CONTENTS_VIEW_MODE=1` cookie
to enable "adult" mode.
2024-05-09 15:14:08 +02:00
Delphox
11109d5bad
[furaffinity] match xfuraffinity.com
2024-05-08 12:15:47 -03:00
Mike Fährmann
dc9d83e64b
[output] support 'NO_COLOR' environment variable
2024-05-03 12:25:41 +02:00
Mike Fährmann
625740c3d2
merge #5538 : [cookies] optimize _find_most_recently_used_file for exact profiles
2024-05-02 18:19:19 +02:00
Jan Wikholm
06d102f19a
optimize _find_most_recently_used_file for exact profile
...
When reading cookies from the browser, the user is able to give either just the browser name, or also provide profile/container information.
If an exact profile is provided, there is no need to find the latest profile with `os.walk` which is very expensive.
This change optimizes that case and the performance increase is significant (~8 sec to 0.6 sec).
```
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
./imgur_OO4UNqJ.jpg
real 0m8.429s
user 0m0.216s
sys 0m0.431s
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX:bgamf5r6.default-release https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
./imgur_OO4UNqJ.jpg
real 0m0.456s
user 0m0.183s
sys 0m0.011s
$ gallery-dl --version
1.26.9
```
2024-05-02 18:16:13 +02:00
Mike Fährmann
bd8e4797e5
[vsco] add 'avatar' extractor ( #5341 )
2024-05-02 18:12:19 +02:00
Mike Fährmann
d0cead105b
[formatter] allow dots etc in '…' literals ( #5539 )
...
don't parse fields starting with '
this disables the ability to directly apply […] to '…' literals,
but that's not really useful anyway and can still be done with _lit
2024-05-02 17:43:24 +02:00
Mike Fährmann
619bf5c644
[inkbunny] retry API calls with a loop instead of recursion
2024-05-01 22:46:24 +02:00
Mike Fährmann
d11ec00908
[common] fix _cfgpath for BaseExtractor objects
...
After the changes in 0c17884673
2024-05-01 16:00:07 +02:00
Mike Fährmann
6db1837a6d
[bluesky] filter reposts only for user timelines ( #5528 )
2024-05-01 16:00:07 +02:00
Mike Fährmann
3cf5366143
[mastodon] add support for card images
2024-05-01 16:00:07 +02:00
Mike Fährmann
9b1995dda3
[mastodon] add 'favorite', 'list', and 'hashtag' extractors ( #5529 )
2024-05-01 15:59:34 +02:00
Mike Fährmann
7fd31aaf8e
merge #5516 : [mastodon] send canonical 'true'/'false' boolean values
...
instead of '1'/'0' to be compatible with Mitra instances or others that
do not fully implement Mastodon's boolean value semantics.
2024-04-27 01:34:56 +02:00
cenodis
3ba5fd9efd
[mastodon] Use boolean instead of integer keys for accounts/statuses endpoint
2024-04-26 22:51:56 +02:00
Mike Fährmann
6c57958806
merge #5511 : [twitter] [furaffinity] match fixvx.com and fxfuraffinity/fxraffinity.net URLs
2024-04-25 22:00:19 +02:00
Mike Fährmann
5aefa9465c
[furaffinity] simplify pattern, match original domain first
2024-04-25 21:58:34 +02:00
Delphox
6969963125
[furaffinity] support fxfuraffinity and fxraffinity urls
2024-04-25 13:25:39 -03:00
Delphox
85550a37c4
[twitter] support fixvx urls
2024-04-25 13:19:40 -03:00
Mike Fährmann
cd241bea0a
[downloader:http] add MIME type and signature for .m4v files ( #5505 )
2024-04-25 01:01:35 +02:00
Mike Fährmann
bef0bd3b49
add '--no-colors' command-line option
2024-04-20 21:06:59 +02:00
Mike Fährmann
20e2c0042b
[output] enable colors by default
2024-04-20 20:51:20 +02:00
Mike Fährmann
14b38264e0
[output] extend 'output.colors' ( #2566 )
...
allow specifying ANSI colors for all loglevels
(debug, info, warning, error)
2024-04-20 20:50:58 +02:00
Mike Fährmann
ddffeeaa7b
[exhentai] detect CAPTCHAs during login ( #5492 )
2024-04-20 00:19:22 +02:00
Mike Fährmann
347af7f5c8
[twitter] handle missing 'expanded_url' fields ( #5463 , #5490 )
2024-04-19 21:42:22 +02:00
Mike Fährmann
c9d3b5e5d9
[pixiv] change 'sanity_level' debug message to a warning ( #5180 )
2024-04-19 16:41:31 +02:00
Mike Fährmann
bffadf35b7
[hotleak] download files with 404 status code ( #5395 )
2024-04-19 16:08:31 +02:00
Mike Fährmann
a7d8cbab0e
[common] show full URL in Extractor.request() error messages
2024-04-18 15:45:36 +02:00
Mike Fährmann
a5071c9ca0
[common] fix NameError in Extractor.request()
...
… when accessing 'code' after an requests exception was raised.
Caused by the changes in 566472f080
2024-04-18 15:42:53 +02:00
Mike Fährmann
566472f080
[common] add 'sleep-429' option ( #5160 )
2024-04-16 18:41:28 +02:00
Mike Fährmann
923c6f3214
[common] simplify 'status_code' check in Extractor.request()
2024-04-16 18:39:47 +02:00
Mike Fährmann
68f4208251
[common] update Extractor.wait() message format
2024-04-16 17:51:14 +02:00
Mike Fährmann
9e5d65fbf3
[twitter] improve username & password login procedure ( #5445 )
...
- handle more subtasks
- support 2FA
- support email verification codes
2024-04-16 01:25:04 +02:00
Mike Fährmann
b38a917355
[common] add Extractor.input() method
2024-04-16 00:02:48 +02:00
Mike Fährmann
257e9fb435
[gelbooru] improve pagination logic for meta tags ( #5478 )
...
similar to 494acabd38
2024-04-15 23:14:48 +02:00
Aidan Harris
a2affdcef3
re-order verbose and warning options
2024-04-14 13:15:07 +00:00
Aidan Harris
a3f580254a
Add warnings flag
...
This commit adds a warnings flag
It can be combined with -q / --quiet to display warnings.
The intent is to provide a silent option that still surfaces
warning and error messages so that they are visible in logs.
2024-04-14 08:55:50 +00:00
Mike Fährmann
0d72789aa3
merge #5461 : [cookies] use tempfile when saving cookies.txt files
2024-04-13 19:02:39 +02:00
Mike Fährmann
63ac06643f
compute tempfile path only once
2024-04-13 18:59:18 +02:00
Mike Fährmann
5227bb6b1d
[text] catch general Exceptions
2024-04-13 18:51:40 +02:00
Mike Fährmann
141a93c8fd
[docs] update docs/configuration links ( #5059 , #5369 , #5423 )
2024-04-13 02:18:44 +02:00
Mike Fährmann
85bbb59483
[twitter] implement 'relogin' option ( #5445 )
2024-04-12 23:01:22 +02:00
Mike Fährmann
e02d2ff45d
[tapas] add 'creator' extractor ( #5306 )
2024-04-11 23:41:50 +02:00
Mike Fährmann
35d4a706ae
[pixiv:novel] add 'covers' option ( #5373 )
2024-04-11 22:27:49 +02:00
Mike Fährmann
b57051719f
[wikimedia] support wiki.gg wikis
2024-04-09 19:24:01 +02:00
Mike Fährmann
40c1a8e471
[wikimedia] fix exception for files with empty 'metadata'
2024-04-09 19:12:15 +02:00
Mike Fährmann
0e730ba980
[pp:mtime] do not overwrite '_mtime' for None values ( #5439 )
2024-04-07 02:33:19 +02:00
Mike Fährmann
647a87d17c
[twitter] match '/photo/' Tweet URLs ( #5443 )
...
fixes regression introduced in 40c05535
2024-04-06 17:56:21 +02:00
Mike Fährmann
3346a377b3
[pp:ugoira] log errors for general exceptions
2024-04-06 17:00:48 +02:00
Mike Fährmann
40bd145637
remove 'contextlib' imports
2024-04-06 16:59:09 +02:00
Mike Fährmann
9a8403917a
restore LD_LIBRARY_PATH for PyInstaller builds ( #5421 )
2024-04-06 16:58:33 +02:00
Mike Fährmann
86a97d8e27
[kemonoparty] fix crash on posts with missing datetime info ( #5422 )
2024-04-05 00:25:23 +02:00
Mike Fährmann
ef8f02c3d9
[oauth] use Extractor.request() for HTTP requests ( #5433 )
...
Enables using proxies and general network options.
2024-04-04 20:51:54 +02:00
Mike Fährmann
095e5ded6f
[reddit] support comment embeds ( #5366 )
2024-04-01 23:35:42 +02:00
Mike Fährmann
64948f2c09
[foolfuuka] improve 'board' pattern & support pages ( #5408 )
2024-04-01 22:31:25 +02:00
Mike Fährmann
0c17884673
store 'match' and 'groups' in Extractor objects
2024-04-01 03:07:52 +02:00
Mike Fährmann
bdbc8be5f3
[twitter] prevent crash when extracting 'birdwatch' metadata ( #5403 )
2024-03-30 18:13:38 +01:00
Mike Fährmann
ef0c90414c
[wikimedia] suppress exception for entries without 'imageinfo' ( #5384 )
2024-03-26 15:33:26 +01:00
Mike Fährmann
9cce461627
[kemonoparty] add 'announcements' option ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-2015919188
2024-03-26 15:20:14 +01:00
Mike Fährmann
72ac2c750d
[kemonoparty:favorite] support 'sort' and 'order' query params ( #5375 )
2024-03-26 02:27:36 +01:00
Mike Fährmann
d1d017ab5d
merge #5372 : [twitter] match Tweet URLs with query parameters ( #5371 )
...
fixes regression introduced in 40c05535
2024-03-25 22:01:46 +01:00
fireattack
423599ce95
[twitter] fix pattern for single tweet ( #5371 )
...
- Add optional slash
- Update tests to include some non-standard tweet URLs
2024-03-25 21:57:35 +01:00
Mike Fährmann
15a4bc2584
[kemonoparty] fix KeyError for empty files ( #5368 )
2024-03-24 02:21:38 +01:00
Mike Fährmann
c3bafd6a2b
release version 1.26.9
2024-03-23 17:41:29 +01:00
Mike Fährmann
925123e007
[deviantart] handle CloudFront blocks in general ( #5363 )
...
This was already done for non-OAuth requests (#655 )
but CF is now blocking OAuth API requests as well.
2024-03-23 15:50:24 +01:00
Mike Fährmann
31e7ca73b6
[gelbooru] add 'order-posts' option for favorites ( #5220 )
2024-03-23 13:30:09 +01:00
Mike Fährmann
4b6f47e571
[pornhub:gif] extract 'viewkey' and 'timestamp' metadata ( #4463 )
...
https://github.com/mikf/gallery-dl/issues/4463#issuecomment-2014550302
2024-03-22 18:00:20 +01:00
Mike Fährmann
9f73fac5ef
[steamgriddb] raise proper exception for deleted assets
2024-03-22 17:53:06 +01:00
Mike Fährmann
fe9171508f
[kemonoparty] fix exception for '/revision/' URLs
...
caused by 03a9ce98
2024-03-22 03:10:12 +01:00
Mike Fährmann
a650fd3177
[deviantart] improve 'index' extraction for stash files ( #5335 )
2024-03-22 02:53:54 +01:00
Mike Fährmann
1d6260f456
[bunkr] remove 'description' metadata
...
album descriptions are no longer available on album pages
and the previous code erroneously returned just '0'
2024-03-22 02:14:41 +01:00
Mike Fährmann
32262a048b
[idolcomplex] fix metadata extraction
...
- replace legacy 'id' vales with alphanumeric ones, since the former are
no longer available
- approximate 'vote_average', since the real value is no longer
available
- fix 'vote_count'
2024-03-22 01:43:05 +01:00
Mike Fährmann
77ab015df2
[idolcomplex] support new pool URLs
2024-03-22 01:38:25 +01:00
Mike Fährmann
1b34d5ac40
[subscribestar] fix 'date' metadata
2024-03-22 00:45:09 +01:00
Mike Fährmann
ddb2edfd32
[formatter] fix local DST datetime offsets for ':O'
...
'O' would get the *current* local UTC offset and apply it to all
'datetime' objects it gets applied to.
This would result in a wrong offset if the current offset includes
DST and the target 'datetime' does not or vice-versa.
'O' now determines the correct local UTC offset while respecting DST for
each individual 'datetime'.
2024-03-21 20:45:46 +01:00
wankio
56f2d5a5f2
fapello.py Fullsize image
...
by remove ".md" and ".th" in image url, it will download fullsize of images
2024-03-21 18:08:18 +01:00
wankio
eb673a7204
Update fapello.py
...
get fullsize image instead resized
2024-03-19 02:48:53 +07:00
Mike Fährmann
6d93295fea
[gelbooru] add 'date_favorited' metadata field
2024-03-18 20:46:11 +01:00
Mike Fährmann
0d69af94d5
[gelbooru] detect returned favorites order ( #5220 )
2024-03-18 20:45:06 +01:00
Mike Fährmann
e1c51c0dfb
[vipergirls] add 'domain' option ( #4166 )
2024-03-18 17:36:16 +01:00
Mike Fährmann
ae116812cc
[vipergirls] add 'like' option ( #4166 )
2024-03-18 17:36:16 +01:00
Mike Fährmann
da6ba60331
[bluesky] add 'instance' metadata field ( #4438 )
2024-03-18 17:36:16 +01:00
Mike Fährmann
5c1f5861b6
[flickr] add 'contexts' option ( #5324 )
2024-03-18 17:36:16 +01:00
Mike Fährmann
26bc2d55f4
[hiperdex] update URL patterns & fix 'manga' metadata ( #5340 )
2024-03-18 17:36:16 +01:00
Mike Fährmann
8e694d85c4
[twitter] add 'birdwatch' metadata field ( #5317 )
...
should probably get a better name,
but this is what it's called internally by Twitter
2024-03-18 17:36:02 +01:00
Mike Fährmann
b8e7be225c
merge #5333 : [imagefap] fix folder extractor
2024-03-15 23:46:43 +01:00
Herp
99c53f7fa8
Fix imagefap extrcator
2024-03-15 23:44:25 +01:00
Mike Fährmann
03a9ce9832
[kemonoparty] add 'order-revisions' option ( #5334 )
2024-03-15 23:06:05 +01:00
Mike Fährmann
1418c0ce38
[kemonoparty] add 'revision_count' metadata field ( #5334 )
2024-03-15 22:28:15 +01:00
Mike Fährmann
ddee5ae7c4
[gofile] fix extraction
2024-03-15 20:34:54 +01:00
Mike Fährmann
5716430c35
[deviantart:stash] recognize 'deviantart.com/stash/…' URLs
2024-03-15 18:14:55 +01:00
Mike Fährmann
76683c5f5c
[deviantart:stash] fix 'index' metadata ( #5335 )
2024-03-15 18:10:59 +01:00
Mike Fährmann
5158cbb4c1
[weibo] rework pagination logic ( #4168 )
...
don't automatically stop when receiving an empty status list
shouldn't improve 'tabtype=feed' results, but at least 'tabtype=album'
ones and others using cursors won't end prematurely
2024-03-14 00:06:25 +01:00
Mike Fährmann
d53db6e11a
[twitter] handle "account is temporarily locked" errors ( #5300 )
...
- display proper error message
- add 'locked' option
2024-03-13 22:34:25 +01:00
Mike Fährmann
108abab537
[twitter] add 'protected' metadata field ( #5327 )
...
for 'author' and 'user'
2024-03-13 14:46:03 +01:00
Mike Fährmann
dde822e69a
[instagram] change 'posts are private' exception to a warning ( #5322 )
2024-03-12 18:58:15 +01:00
Mike Fährmann
e40973118d
merge #5323 : [instagram] default 'likes' to '0'
2024-03-12 18:47:24 +01:00
Mike Fährmann
b188cb2729
merge #5321 : [mastodon] fix handling null 'moved' account field
2024-03-12 18:45:53 +01:00
Martin Contento
c210c7a4ed
[instagram] default posts like_count to zero
...
I don't know when/why this happens and for private account reasons can't
add an example here, but i had this error out and it feels like a safe
fix to default to `0` here.
2024-03-12 15:12:07 +00:00
blankie
225d849139
[mastodon] fix handling null 'moved' account field
2024-03-12 11:44:25 +11:00
Mike Fährmann
106dfdb4c3
cleanup sleep-request retry delay code
...
more lines but easier to read I'd say
2024-03-11 21:38:06 +01:00
Mike Fährmann
4565de3c8e
[reddit] provide 'gif' and 'mp4' variant fallbacks ( #5315 )
2024-03-10 20:50:55 +01:00
Mike Fährmann
5842e4928d
[imgur] fail downloads when redirected to 'removed.png' ( #5308 )
2024-03-09 23:35:23 +01:00
Mike Fährmann
ac4e29f70a
[lensdump] support more direct link formats ( #5293 )
2024-03-09 23:33:58 +01:00
Mike Fährmann
146459056c
[reddit] provide 'fallback_url' as video fallback ( #5296 )
2024-03-07 15:58:01 +01:00
Mike Fährmann
d3003f8531
merge #5270 : [imagefap] add 'folder' metadata
2024-03-07 01:31:40 +01:00
Mike Fährmann
05331f9cf1
[imagefap] flake8, cleanup, tests
2024-03-07 01:29:19 +01:00
Mike Fährmann
40c0553523
[twitter] add 'quotes' extractor ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-1981571924
It's implemented as a search for 'quoted_tweet_id:…' on Twitter.
2024-03-07 00:52:50 +01:00
Mike Fährmann
790c0ffb8d
[lensdump] recognize direct image links ( #5293
2024-03-06 22:56:57 +01:00
Mike Fährmann
6d9e3c0eb1
[skeb] add extractor for followed users ( #5290 )
...
needs 'Authorization' header from browser session
-o headers.Authorization="Bearer ey…"
2024-03-06 22:43:01 +01:00
Mike Fährmann
ace16f00f5
[weibo] fix retweets ( #2825 , #3874 , #5263 )
...
- handle 快转 retweets
- disable 'retweets' by default
- skip all retweet media when 'retweets' are disabled
- extract all retweet media when 'retweets' is set to "original"
2024-03-06 19:36:53 +01:00
Mike Fährmann
0676a9d6ec
[weibo] fix 'livephoto' filename extensions ( #5287 )
2024-03-06 19:36:32 +01:00
Mike Fährmann
a8027745e3
[downloader:http] add MIME type and signature for .mov files ( #5287 )
2024-03-06 14:00:24 +01:00
Mike Fährmann
db507e30c7
[pixiv] fix novel text extraction ( #5285 )
...
change to '/webview/v2/novel'
since '/v1/novel/text' does not work anymore
2024-03-06 02:31:26 +01:00
Mike Fährmann
296f20e630
[warosu] fix 'board_name' metadata
2024-03-06 01:28:47 +01:00
Mike Fährmann
24873c2724
[warosu] fix crash for threads with deleted posts ( #5289 )
2024-03-06 01:27:45 +01:00
Mike Fährmann
f296067797
[naver] unescape post 'title' and 'description'
2024-03-06 00:46:19 +01:00
Mike Fährmann
a71cdab53e
merge #5126 : [naver] fix EUC-KR encoding issue in old image URLs
2024-03-06 00:22:33 +01:00
Mike Fährmann
a8d3efbb99
[naver] simplify code + add test
2024-03-06 00:21:23 +01:00
Johann Hong
f64fb8f239
[naver] EUC-KR encoding issue in old image URLs Fix
...
Around October 2010, the image server URL format and file name
encoding changed from EUC-KR to UTF-8.
Modified to detect old URL format and decode image URLs into EUC-KR
- (lint with flake8) Customize conditions
Wrap lines smaller than 79 characters
- (lint with flake8) Customize conditions (2nd try)
- One import per line
- Indent on consecutive lines
- (lint with flake8) Customize conditions (3rd try)
- E128 continuation line under-indented for visual indent
- E123 closing bracket does not match indentation of opening bracket's line
- Update naver.py
Check encoding for all image URLs
2024-03-06 00:21:23 +01:00
Mike Fährmann
7b28418f69
[naver] recognize '.naver' URLs
...
https://blog.naver.com/PostView.naver ?…
2024-03-05 22:30:29 +01:00
Mike Fährmann
a767832332
[deviantart:avatar] ignore default avatars ( #5276 )
2024-03-04 23:11:30 +01:00
Mike Fährmann
0cbc910905
[deviantart:avatar] fix 'index' for avatars without '?' ( #5276 )
2024-03-04 22:31:35 +01:00
Mike Fährmann
6482bbc525
[bluesky] handle different 'embed' structure
2024-03-03 20:41:01 +01:00
Mike Fährmann
1115dccd0d
[bluesky] fix feeds and lists
...
bug introduced in 495c9ee1
2024-03-03 20:22:34 +01:00
Mike Fährmann
1a9b9aa310
[artstation] support video clips ( #2566 , #3309 , #3911 )
...
- add 'videos' and 'previews' options
- fix 403 errors for video previews
2024-03-03 18:00:45 +01:00
termvacycurtocs
f8b037ed40
[Imagefap] Add folder metadata
...
[Imagefap] Add "folder" metadata when downloading a folder or user profile.
No additional request is made to the server.
Use for example with the following configuration :
"parent-metadata": true
"directory":["{category}", "{uploader}", "{folder}", "{gallery_id} {title}"]
2024-03-02 22:15:45 +01:00
Mike Fährmann
982880615d
[deviantart] prevent unnecessary API requests ( #4995 )
...
… when using 'comments-avatars'
This also has the added benefit of making it possible to download
comment avatars from users without a valid user profile entry,
like deleted users.
2024-03-02 21:59:16 +01:00
Mike Fährmann
25d2854272
[deviantart] add 'comments-avatars' option ( #4995 )
2024-03-02 21:59:16 +01:00
Mike Fährmann
218ec1a9ee
[instagram] raise proper error for missing 'reels_media' ( #5257 )
2024-03-02 21:58:59 +01:00
Mike Fährmann
82c73c77b0
[redgifs] make 'date' available for directories ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-1973975415
2024-03-01 23:39:16 +01:00
Mike Fährmann
cf9e99c07b
[artstation] support collections ( #146 )
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1972101003
2024-03-01 20:21:21 +01:00
Mike Fährmann
32ec695195
merge #5256 : [wikimedia] add azurlane.koumakan.jp
2024-02-29 21:50:24 +01:00
thatfuckingbird
88a06df165
[wikimedia] add azurlane.koumakan.jp to presets
2024-02-29 19:28:50 +01:00
Mike Fährmann
1db0a587f3
[nitter] ignore invalid Tweets ( #5253 )
...
like "Load newest"
2024-02-29 16:31:37 +01:00
Mike Fährmann
76581c13f7
handle URLs without '/' after their TLD ( #5252 )
2024-02-29 15:05:46 +01:00
Mike Fährmann
a00b171d4e
[bluesky] wait unitl 'RateLimit-Reset' on 429 responses
2024-02-28 18:13:16 +01:00
Mike Fährmann
7d874e2497
[bluesky] improve API error messages
2024-02-28 14:45:36 +01:00
Mike Fährmann
d921d860f1
automatically create directory path for logging files ( #5249 )
2024-02-28 02:11:20 +01:00
Mike Fährmann
495c9ee126
[bluesky] add 'reposts' option ( #4438 , #5248 )
2024-02-27 18:42:29 +01:00
Mike Fährmann
c8b591303f
[paheal] cleanup
2024-02-27 02:27:20 +01:00
Mike Fährmann
8a11b72253
remove extractor/test.py ( #4504 )
2024-02-27 01:37:57 +01:00
Mike Fährmann
fde9e25c9f
[tests:kemonoparty] '.party' -> '.su'
2024-02-26 22:25:04 +01:00
Mike Fährmann
311a21bfb2
[bluesky] fix '/follows' not spawning child extractors ( #5246 )
2024-02-26 15:38:31 +01:00
Mike Fährmann
d3dca68225
[xvideos] fix galleries with more than 500 images ( #5244 )
2024-02-26 15:36:41 +01:00
Mike Fährmann
13443f40a3
[xvideos] support '/channels/' URLs ( #5244 )
2024-02-26 00:08:37 +01:00
Mike Fährmann
c60ebc6519
[deviantart] improve fetching extended metadata ( #5175 )
...
use multiple metadata API calls per chunk of deviations if necessary
2024-02-25 03:36:00 +01:00
Mike Fährmann
cc6b9e4c18
[zerochan] use API by default ( #3669 )
...
add 'pagination' option
2024-02-25 00:36:14 +01:00
Mike Fährmann
a2b55d5dde
[skeb] retry 429 responses containing a 'request_key' cookie ( #5210 )
2024-02-24 00:54:15 +01:00
Mike Fährmann
e51ee6b132
fix HttpError.status value
...
'response' with error status code evaluates to False
2024-02-24 00:04:24 +01:00
Mike Fährmann
b4c46de4b8
merge #5224 : [artstation] update URL patterns to recognize usernames with dashes
2024-02-21 14:41:02 +01:00
blankie
962f55cc68
[artstation] fix handling usernames with dashes
2024-02-21 17:39:37 +11:00
Mike Fährmann
fe7e2281ac
[nijie] increase default delay between requests ( #5221 )
...
1-2s is not enough
2024-02-20 18:19:49 +01:00
Mike Fährmann
a34312e3ac
[instagram] make accessing 'like_count' non-fatal ( #5218 )
2024-02-19 19:24:51 +01:00
Mike Fährmann
741fd00cec
[deviantart] extend 'metadata' option ( #5175 )
...
alloe fetching extended metadata in addition to the usual
'description', 'tags', etc by setting 'metadata' to a list of
'camera', 'stats', 'submission', 'collection', and 'gallery'
for example "metadata": "stats,submission"
2024-02-18 23:14:14 +01:00
Mike Fährmann
fc46177578
release version 1.26.8
2024-02-17 21:03:19 +01:00
Mike Fährmann
8a63801311
[vsco] add 'spaces' extractor ( #5202 )
...
for spaces listed on a user page
2024-02-17 18:20:48 +01:00
Mike Fährmann
ccb413df71
[wikimedia] support 'pidgi.net' and 'bulbapedia.bulbagarden.net' ( #5205 , #5206 )
2024-02-17 17:35:10 +01:00
Mike Fährmann
7033cc14e9
[vsco] add 'space' extractor ( #5202 )
2024-02-17 01:54:05 +01:00
Mike Fährmann
770aec922d
[fapachi] ignore empty entries
2024-02-16 22:43:37 +01:00
Mike Fährmann
ee7c054855
[bluesky] add 'search' extractor ( #4438 )
...
Both https://bsky.app/search?q=QUERY and https://bsky.app/search/QUERY
are recognized as search URLs, where QUERY gets forwarded unmodified as
'q' parameter for app.bsky.feed.searchPosts .
User searches are not supported yet.
2024-02-16 15:58:47 +01:00
Mike Fährmann
91e5c4fdfe
[bluesky] add 'avatar' and 'background' extractors ( #4438 )
2024-02-16 15:41:19 +01:00
Mike Fährmann
24c1317e0d
[batoto] fix crash when manga/chapter contains a '-' ( #5200 )
2024-02-16 00:10:08 +01:00
Mike Fährmann
0abd9723af
[bluesky] add 'metadata' option ( #4438 )
...
allow extracting 'user' metadata and
make 'facets' extraction optional
2024-02-15 23:30:16 +01:00
Mike Fährmann
7e036ea290
[bluesky] add 'depth' option ( #4438 )
...
and reduce default depth and parentHeight values
2024-02-15 22:26:05 +01:00
Mike Fährmann
42335ea880
[zerochan] fix skipping every other post
2024-02-15 02:51:01 +01:00
Mike Fährmann
c97b92cc35
[fanbox] add 'home' and 'supporting' extractors ( #5138 )
2024-02-14 23:25:39 +01:00
Mike Fährmann
04e4ffc64c
[deviantart] combine 'png' option with 'quality' ( #4846 )
...
"quality": "png" to download PNGs instead og JPEGs
2024-02-14 22:07:29 +01:00
Mike Fährmann
9cc4ec2c58
[deviantart] add 'png' option ( #4846 )
2024-02-14 01:03:15 +01:00
Mike Fährmann
966c8608e6
[deviantart] move image content extraction into separate function
2024-02-14 00:30:06 +01:00
Mike Fährmann
1d1ffe3317
[pornpics] update 'channel' extraction & add test
...
change 'channel' to a list, since extracting both 'channel' and
'channels' does not really work with text.extract_from()
2024-02-13 23:48:46 +01:00
cc1234
32472d7d6c
Add support for multi channels
2024-02-13 18:34:04 +00:00
Mike Fährmann
139ff3f6ab
[kemonoparty] add 'posts' extractor ( #5194 )
2024-02-13 15:41:34 +01:00
Mike Fährmann
814ad9321e
[deviantart] skip locked/blurred posts ( #4567 , #5193 )
2024-02-13 14:15:12 +01:00
Mike Fährmann
f7f8ef8684
[twitter] support communities ( #4913 )
2024-02-13 01:30:23 +01:00
Mike Fährmann
cae77e85f8
[twitter] update query hashes
...
... as well as 'variables' and 'features' values
also remove unused legacy API code
2024-02-12 23:19:13 +01:00
Mike Fährmann
06cb518d97
[bunkr] fix extraction ( #5088 , #5151 , #5153 )
...
- remove legacy code
- map legacy domains to bunkr.sk
- use input URL domain for newer domains
- update tests (some files got slightly modified or deleted)
2024-02-11 22:36:03 +01:00
Mike Fährmann
dcc6e3f65c
merge #5134 : [bunkr] add new bunkr domains ( #5130 )
2024-02-11 21:10:06 +01:00
Mike Fährmann
4641937ca3
[imagetwist] add 'gallery' extractor ( #5190 )
2024-02-11 18:41:02 +01:00
Mike Fährmann
fde82ab0ce
[imagechest] add 'user' extractor ( #5143 )
2024-02-11 18:38:33 +01:00
Mike Fährmann
4474cea31b
merge #5187 : [skeb] add 'num' and 'count' metadata fields
2024-02-10 19:36:59 +01:00
Mike Fährmann
4cfceb23cb
[skeb] rename 'data' -> 'file' & add tests
2024-02-10 19:35:50 +01:00
Mike Fährmann
44a1a66dac
merge #5186 : Fix filename formatting silently failing under certain circumstances
2024-02-10 19:22:41 +01:00
Mike Fährmann
c83d0a1596
[weibo] add 'gifs' option ( #5183 )
2024-02-10 18:17:07 +01:00
blankie
f9a8e8cacf
[skeb] add 'num' and 'count' metadata fields
2024-02-10 21:51:23 +11:00
blankie
909830f8ea
fix filename formatting silently failing under certain circumstances
2024-02-10 21:18:57 +11:00
Mike Fährmann
af61d2b037
[wikimedia] combine most wikimedia.org sites ( #1443 )
...
add wikidata.org and wikivoyage.org
2024-02-10 03:00:58 +01:00
Mike Fährmann
c7d17f1111
[bluesky] extract 'hashtags', 'mentions', and 'uris' metadata ( #4438 )
2024-02-10 00:01:55 +01:00
Mike Fährmann
55bbd49a0e
[bluesky] download images in original resolution ( #4438 )
...
at least up to 2000 px
2024-02-09 21:33:33 +01:00
Mike Fährmann
6414dc6bca
[idolcomplex] fix pagination for tags containing ':' ( #5171 )
2024-02-09 17:51:08 +01:00
Mike Fährmann
5c2a2321a2
[bluesky] update refresh token after using it ( #4438 )
2024-02-08 22:33:34 +01:00
Mike Fährmann
9c10be54fb
[bluesky] add 'following' extractor ( #4438 )
2024-02-08 21:58:17 +01:00
Mike Fährmann
86ce35d6a1
[bluesky] simplify 'pattern'
2024-02-08 21:28:21 +01:00
Mike Fährmann
da292ded4e
[bluesky] add 'list' extractor ( #4438 )
2024-02-08 21:24:07 +01:00
Mike Fährmann
004bf7bb38
[bluesky] add 'feed' extractor ( #4438 )
2024-02-08 21:01:44 +01:00
Mike Fährmann
6aea818d4e
[bluesky] allow using DIDs as user handles ( #4438 )
2024-02-08 20:15:54 +01:00
Mike Fährmann
aee5580c62
[idolcomplex] extract 'id_alnum' metadata ( #5171 )
2024-02-08 18:29:54 +01:00
Mike Fährmann
cf7d6be2d4
[bluesky] initial support ( #4438 , #4708 , #4722 , #5047 )
2024-02-07 19:09:33 +01:00
Mike Fährmann
6ef143ea31
[idolcomplex] support alphanumeric post IDs ( #5171 )
2024-02-07 14:57:13 +01:00
Mike Fährmann
6e928300bc
[flickr] handle non-JSON errors ( #5131 )
2024-02-06 21:22:10 +01:00
Mike Fährmann
90ac6d7375
[wikimedia] use '/api.php' as default API path
2024-02-06 00:36:51 +01:00
Mike Fährmann
d7823b9f81
[pinterest] fix section URLs for boards with /?# in name ( #5104 )
2024-02-05 15:54:06 +01:00
Mike Fährmann
de752eb7b1
[naverwebtoon] support '/webtoon/' paths for all comics ( #5123 )
2024-02-04 21:38:46 +01:00
Mike Fährmann
0dacb2b24c
[downloader:http] remove 'pyopenssl' import ( #5156 )
2024-02-04 21:06:54 +01:00
Jeff Mercado
d9d0601ab1
break up line to fit 80 char
2024-01-29 20:31:58 -08:00
Jeff Mercado
6bcd3c9380
[bunkr] add new bunkr domains ( #5130 )
2024-01-29 20:25:33 -08:00
Mike Fährmann
62d6f5f8d2
[luscious] fix IndexError for files without thumbnail ( #5122 )
2024-01-28 01:43:29 +01:00
Mike Fährmann
22647c2626
[naverwebtoon] fix 'title' for comics with empty tags ( #5120 )
2024-01-27 16:24:03 +01:00
Mike Fährmann
3433481dd2
[gofile] update 'website_token' extraction
2024-01-27 01:10:14 +01:00
Mike Fährmann
1f7101d606
[archivedmoe] fix thebarchive webm URLs ( #5116 )
2024-01-27 00:24:41 +01:00
Mike Fährmann
34a4ddc399
[sankaku] add 'id-format' option ( #5073 )
2024-01-26 17:56:08 +01:00
Mike Fährmann
afd20ef42c
[kemonoparty] implement filtering duplicate revisions ( #5013 )
...
set 'revisions' to '"unique"' to have it ignore duplicate revisions
2024-01-26 14:44:15 +01:00
Mike Fährmann
c28475d325
[kemonoparty] fix deleting 'name' in orginal objects ( #5103 )
...
... when computing 'revision_hash'
regression caused by 3d68eda4
2024-01-25 23:46:19 +01:00
Mike Fährmann
beacfa7436
[bunkr] update domain to 'bunkr.sk' ( #5114 )
2024-01-25 23:45:41 +01:00
Mike Fährmann
0502256251
release version 1.26.7
2024-01-21 23:02:50 +01:00
Mike Fährmann
67c99b1366
[patreon] prevent HttpError for stream.mux.com URLs
2024-01-21 22:50:40 +01:00
Mike Fährmann
f3ad91b44f
[bunkr] update domain ( #5088 )
2024-01-21 03:00:57 +01:00
Mike Fährmann
c7a42880ab
[wikimedia] support fandom wikis ( #1443 , #2677 , #3378 )
...
Wikis hosted on fandom.com are just wikimedia instances
and support its API.
2024-01-21 00:52:02 +01:00
Mike Fährmann
5bf156f0b1
merge #5094 : [webtoons] fix extracting comic and episode name with commas
2024-01-21 00:47:26 +01:00
blankie
df718887c2
[webtoons] fix extracting comic and episode name with commas
2024-01-21 09:50:27 +11:00
Wiiplay123
6eb62f2140
Combine lh*(-**).googleusercontent.com URL regex into one line.
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2024-01-20 15:53:11 -06:00
Wiiplay123
a6fed628dd
[blogger] Fix lh*.googleusercontent.com forward slash bug, add support for lh*-**.googleusercontent.com
...
Some URLs use "lh(number)-(locale).googleusercontent.com" format, so I added support for those.
Also, "lh(number).googleusercontent.com" formats were broken because the regex was looking for a second forward slash.
Examples:
lh7.googleusercontent.com
lh7-us.googleusercontent.com
2024-01-20 15:07:52 -06:00
Mike Fährmann
6f8592eaff
[hbrowse] remove from modules list
2024-01-20 18:25:38 +01:00
Mike Fährmann
acc94ac187
[realbooru] fix extraction
...
revert ac97aca99c
2024-01-20 17:56:07 +01:00
Mike Fährmann
9599151118
[issuu] fix extraction
2024-01-20 16:44:48 +01:00
Mike Fährmann
9ca6117c67
[hbrowse] remove module
...
website gone
2024-01-20 02:53:44 +01:00
Mike Fährmann
375eefb886
[chevereto] remove 'pixl.li'
...
"Pixl is closing down"
"All images will be deleted January 1st."
2024-01-20 02:21:40 +01:00
Mike Fährmann
321861af7e
[erome] fix 'count' metadata
2024-01-20 00:26:41 +01:00
Mike Fährmann
b41d9bf616
[paheal] fix 'source' metadata
2024-01-19 22:24:39 +01:00
Mike Fährmann
b0a441f1e3
[nitter] remove 'nitter.lacontrevoie.fr'
...
"Fermeture de Nitter / Closing down Nitter"
2024-01-19 19:34:16 +01:00
Mike Fährmann
a1c1e80f67
[giantessbooru] update domain
2024-01-19 14:21:56 +01:00
Mike Fährmann
2007cb2f59
[tests] check extractor category values
2024-01-19 14:21:09 +01:00
Mike Fährmann
fc4e737f67
[wikimedia] include 'sha1' in default filenames
2024-01-19 03:08:43 +01:00
Mike Fährmann
44f2c15a04
[wikimedia] handle 'File:' paths
2024-01-19 03:05:45 +01:00
Mike Fährmann
93b4120e77
[gelbooru] support 'all' and empty tag ( #5076 )
2024-01-18 21:49:33 +01:00
Mike Fährmann
a416d4c3d5
[sankaku] support post URLs with alphanumeric IDs ( #5073 )
2024-01-18 16:23:14 +01:00
Mike Fährmann
ea553a1d55
[wikimedia] generalize ( #1443 )
...
- support mediawiki.org
- support mariowiki.com (#3660 )
- combine code into a single extractor
(use prefix as subcategory)
- handle non-wiki instances
- unescape titles
2024-01-18 15:36:16 +01:00
Mike Fährmann
89066844f4
add 'config_instance' method
...
to allow for a more streamlined access to BaseExtractor instance options
2024-01-18 03:20:36 +01:00
Mike Fährmann
c3c1635ef3
[wikimedia] update
...
- rewrite using BaseExtractor
- support most Wiki* domains
- update docs/supportedsites
- add tests
2024-01-17 00:08:06 +01:00
Ailothaen
221f54309c
[wikimedia] Improved archive identifiers
2024-01-16 02:32:32 +01:00
Ailothaen
e33056adcd
[wikimedia] Add Wikipedia/Wikimedia extractor
2024-01-16 02:32:25 +01:00
Mike Fährmann
3d68eda4ab
[kemonoparty] add 'revision_hash' metadata ( #4706 , #4727 , #5013 )
...
A SHA1 hexdigest of other relevant metadata fields like
title, content, file and attachment URLs.
This value does NOT reflect which revisions are listed on the website.
Neither does 'edited' or any other metadata field (combinations).
2024-01-16 00:38:10 +01:00
Mike Fährmann
799a8206ad
merge #5061 : [webtoons] extract more metadata
...
- author_name
- comic_name
- episode_name
- username
2024-01-15 18:27:12 +01:00
Mike Fährmann
8ffa0cd3c8
[webtoons] small optimization
...
don't extract the entire 'author_area' and
avoid creating a second 'text.extract_from()' object
2024-01-15 18:24:47 +01:00
Mike Fährmann
59cf4b3884
merge #4444 : [2ch] add 'thread' and 'board' extractors ( #1009 , #3540 )
2024-01-15 17:50:34 +01:00
Mike Fährmann
90b382304a
[deviantart] fix KeyError: 'premium_folder_data' ( #5063 )
2024-01-15 17:30:03 +01:00
Mike Fährmann
4cedf378d5
[deviantart] fix AttributeError for URLs without username ( #5065 )
...
caused by 4f367145
2024-01-15 16:28:57 +01:00
Mike Fährmann
68196589c4
[2ch] update
...
- simplify extractor code
- more metadata
- add tests
2024-01-15 04:09:05 +01:00
hunter-gatherer8
6c4abc982e
[2ch] add 'thread' and 'board' extractors
...
- [2ch] add thread extractor
- [2ch] add board extractor
- [2ch] add new entry to supported sites
2024-01-15 03:51:03 +01:00
blankie
bb446b1598
[webtoons] extract more metadata
2024-01-14 19:26:49 +11:00
Mike Fährmann
355b909f46
merge #5041 : [steamgriddb] add support ( #5033 )
2024-01-13 00:59:15 +01:00
Mike Fährmann
71e2c3e5a2
merge #5037 : [hatenablog] add support ( #5036 )
2024-01-13 00:57:21 +01:00
blankie
9f53daabb8
[hatenablog] implement additional suggestion
2024-01-13 10:43:25 +11:00
blankie
293f1559df
[hatenablog] implement suggestions
2024-01-13 10:42:22 +11:00
blankie
65f42442f5
[steamgriddb] implement another suggestion
2024-01-13 10:12:15 +11:00
blankie
8995fd5f01
[steamgriddb] implement suggestions
2024-01-13 09:55:39 +11:00
Mike Fährmann
b1c175fdd1
allow using an empty string as argument for -D/--directory
2024-01-12 16:39:16 +01:00
Mike Fährmann
2dcfb012ea
[patreon] download 'm3u8' manifests with ytdl
2024-01-12 02:33:27 +01:00
Mike Fährmann
1c68b7df01
[patreon] fix KeyError ( #5048 )
2024-01-11 17:56:47 +01:00
Mike Fährmann
2191e29e14
[nijie] fix image URL for single image posts ( #5049 )
2024-01-11 05:07:38 +01:00
Mike Fährmann
bbf96753e2
[gelbooru] only log "Incomplete API response" for favorites ( #5045 )
2024-01-10 17:27:46 +01:00
Mike Fährmann
39904c9e4e
[deviantart:avatar] add 'formats' option ( #4995 )
2024-01-10 17:13:34 +01:00
Mike Fährmann
5c43098a1a
[twitter] revert to using 'media' timeline by default ( #4953 )
...
This reverts commit a94f944148
2024-01-09 23:19:39 +01:00
Mike Fährmann
5f9a98cf0f
[deviantart:avatar] fix exception when 'comments' are enabled ( #4995 )
2024-01-09 20:04:46 +01:00
Mike Fährmann
887ade30a5
[batoto] support more mirror domains ( #5042 )
2024-01-09 18:02:49 +01:00
Mike Fährmann
0a382a5092
[batoto] improve 'manga_id' extraction ( #5042 )
2024-01-09 17:25:04 +01:00
blankie
100966b122
[steamgriddb] fix linting error
2024-01-09 17:18:31 +11:00
blankie
2ccb7d3bd3
[steamgriddb] add support
2024-01-09 17:12:56 +11:00
Mike Fährmann
ec958a26bc
[fuskator] make metadata extraction non-fatal ( #5039 )
...
- prevent KeyErrors
- prevent HTTP redirect
- return file URLs as list
2024-01-08 19:18:12 +01:00
blankie
2cfe788f93
[hatenablog] fix extractor naming errors
2024-01-09 01:42:57 +11:00
blankie
be6949c55d
[hatenablog] fix linting error
2024-01-09 01:36:52 +11:00
blankie
61f3b2f820
[hatenablog] add support
2024-01-09 01:29:47 +11:00
Mike Fährmann
657ed93a22
[batoto] improve v2 manga URL pattern
...
and add tests
2024-01-07 22:23:30 +01:00
Mike Fährmann
50eef1b5cc
merge #5029 : [pixiv] update App API headers
2024-01-07 16:27:13 +01:00
Mike Fährmann
33f228756a
[mangadex] add 'list' extractor ( #5025 )
...
supports listing manga and chapters from list feed
2024-01-07 02:59:35 +01:00
Mike Fährmann
db8de13537
[vk] transform image URLs to non-blurred versions ( #5017 )
...
apply the same filter from before d85e66bc
2024-01-07 00:19:10 +01:00
Mike Fährmann
6e10260fb0
release version 1.26.6
2024-01-06 17:48:58 +01:00
