Mike Fährmann
fd734b9222
[archive] add 'archive-mode' option ( #5255 )
2024-05-10 22:59:51 +02:00
Mike Fährmann
28039229fe
[common] use 'create_urllib3_context' for creating SSLContexts
...
enables dumping TLS session keys by setting SSLKEYLOGFILE (#5215 )
as well as other potentially useful settings.
2024-05-10 22:59:29 +02:00
Mike Fährmann
33006fe126
[common] disable 'check_hostname' for non-urllib3 SSLContexts
...
e.g. when 'browser' is set to a non-empty value and gallery-dl creates
its own SSLContext instance instead of using requests' and urllib3's
defaults.
urllib3 disables this option for its default contexts,
since it does this check on its own.
Fixes "ValueError: Cannot set verify_mode to CERT_NONE when
check_hostname is enabled" when using --no-check-certificate.
(#3614 , #4891 , #5576 )
2024-05-10 18:20:08 +02:00
Mike Fährmann
215abbc3e4
[archive] implement DownloadArchiveMemory class ( #5255 )
...
keeps archive IDs in memory
and only writes them to disk in a 'finalize' step
2024-05-10 01:15:44 +02:00
Mike Fährmann
88f94190f4
[archive] move DownloadArchive into its own module
2024-05-10 01:05:28 +02:00
Mike Fährmann
2ee9ffeed6
merge #5568 : [furaffinity] match 'xfuraffinity' URLs
2024-05-09 19:20:12 +02:00
Mike Fährmann
b27ffd790a
[poipiku] fix downloading R-18 posts ( #5567 )
...
… by automatically sending a `POIPIKU_CONTENTS_VIEW_MODE=1` cookie
to enable "adult" mode.
2024-05-09 15:14:08 +02:00
Delphox
11109d5bad
[furaffinity] match xfuraffinity.com
2024-05-08 12:15:47 -03:00
Mike Fährmann
dc9d83e64b
[output] support 'NO_COLOR' environment variable
2024-05-03 12:25:41 +02:00
Mike Fährmann
699592498b
[tests] use random port number for local HTTP server
...
… and explicitly bind to 127.0.0.1 instead of all interfaces
2024-05-02 22:54:15 +02:00
Mike Fährmann
625740c3d2
merge #5538 : [cookies] optimize _find_most_recently_used_file for exact profiles
2024-05-02 18:19:19 +02:00
Jan Wikholm
06d102f19a
optimize _find_most_recently_used_file for exact profile
...
When reading cookies from the browser, the user is able to give either just the browser name, or also provide profile/container information.
If an exact profile is provided, there is no need to find the latest profile with `os.walk` which is very expensive.
This change optimizes that case and the performance increase is significant (~8 sec to 0.6 sec).
```
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
./imgur_OO4UNqJ.jpg
real 0m8.429s
user 0m0.216s
sys 0m0.431s
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX:bgamf5r6.default-release https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
./imgur_OO4UNqJ.jpg
real 0m0.456s
user 0m0.183s
sys 0m0.011s
$ gallery-dl --version
1.26.9
```
2024-05-02 18:16:13 +02:00
Mike Fährmann
bd8e4797e5
[vsco] add 'avatar' extractor ( #5341 )
2024-05-02 18:12:19 +02:00
Mike Fährmann
d0cead105b
[formatter] allow dots etc in '…' literals ( #5539 )
...
don't parse fields starting with '
this disables the ability to directly apply […] to '…' literals,
but that's not really useful anyway and can still be done with _lit
2024-05-02 17:43:24 +02:00
Mike Fährmann
619bf5c644
[inkbunny] retry API calls with a loop instead of recursion
2024-05-01 22:46:24 +02:00
Mike Fährmann
8ed70b3256
[tests] mark tests with missing auth as 'only_matching'
...
… instead of skipping them completely
2024-05-01 16:00:07 +02:00
Mike Fährmann
d11ec00908
[common] fix _cfgpath for BaseExtractor objects
...
After the changes in 0c17884673
2024-05-01 16:00:07 +02:00
Mike Fährmann
6db1837a6d
[bluesky] filter reposts only for user timelines ( #5528 )
2024-05-01 16:00:07 +02:00
Mike Fährmann
3cf5366143
[mastodon] add support for card images
2024-05-01 16:00:07 +02:00
Mike Fährmann
9b1995dda3
[mastodon] add 'favorite', 'list', and 'hashtag' extractors ( #5529 )
2024-05-01 15:59:34 +02:00
Mike Fährmann
7fd31aaf8e
merge #5516 : [mastodon] send canonical 'true'/'false' boolean values
...
instead of '1'/'0' to be compatible with Mitra instances or others that
do not fully implement Mastodon's boolean value semantics.
2024-04-27 01:34:56 +02:00
cenodis
3ba5fd9efd
[mastodon] Use boolean instead of integer keys for accounts/statuses endpoint
2024-04-26 22:51:56 +02:00
Mike Fährmann
6c57958806
merge #5511 : [twitter] [furaffinity] match fixvx.com and fxfuraffinity/fxraffinity.net URLs
2024-04-25 22:00:19 +02:00
Mike Fährmann
5aefa9465c
[furaffinity] simplify pattern, match original domain first
2024-04-25 21:58:34 +02:00
Delphox
1886721d82
update tests
2024-04-25 13:28:30 -03:00
Delphox
6969963125
[furaffinity] support fxfuraffinity and fxraffinity urls
2024-04-25 13:25:39 -03:00
Delphox
85550a37c4
[twitter] support fixvx urls
2024-04-25 13:19:40 -03:00
Mike Fährmann
cd241bea0a
[downloader:http] add MIME type and signature for .m4v files ( #5505 )
2024-04-25 01:01:35 +02:00
Mike Fährmann
bef0bd3b49
add '--no-colors' command-line option
2024-04-20 21:06:59 +02:00
Mike Fährmann
20e2c0042b
[output] enable colors by default
2024-04-20 20:51:20 +02:00
Mike Fährmann
14b38264e0
[output] extend 'output.colors' ( #2566 )
...
allow specifying ANSI colors for all loglevels
(debug, info, warning, error)
2024-04-20 20:50:58 +02:00
Mike Fährmann
ddffeeaa7b
[exhentai] detect CAPTCHAs during login ( #5492 )
2024-04-20 00:19:22 +02:00
Mike Fährmann
068ccfe0b3
[tests] allow filtering extractor result tests by URL or comment
...
python test_results.py twitter:+/i/web/
python test_results.py twitter:~twitpic
2024-04-19 23:02:55 +02:00
Mike Fährmann
347af7f5c8
[twitter] handle missing 'expanded_url' fields ( #5463 , #5490 )
2024-04-19 21:42:22 +02:00
Mike Fährmann
c9d3b5e5d9
[pixiv] change 'sanity_level' debug message to a warning ( #5180 )
2024-04-19 16:41:31 +02:00
Mike Fährmann
bffadf35b7
[hotleak] download files with 404 status code ( #5395 )
2024-04-19 16:08:31 +02:00
Mike Fährmann
a7d8cbab0e
[common] show full URL in Extractor.request() error messages
2024-04-18 15:45:36 +02:00
Mike Fährmann
a5071c9ca0
[common] fix NameError in Extractor.request()
...
… when accessing 'code' after an requests exception was raised.
Caused by the changes in 566472f080
2024-04-18 15:42:53 +02:00
Mike Fährmann
566472f080
[common] add 'sleep-429' option ( #5160 )
2024-04-16 18:41:28 +02:00
Mike Fährmann
923c6f3214
[common] simplify 'status_code' check in Extractor.request()
2024-04-16 18:39:47 +02:00
Mike Fährmann
68f4208251
[common] update Extractor.wait() message format
2024-04-16 17:51:14 +02:00
Mike Fährmann
9e5d65fbf3
[twitter] improve username & password login procedure ( #5445 )
...
- handle more subtasks
- support 2FA
- support email verification codes
2024-04-16 01:25:04 +02:00
Mike Fährmann
b38a917355
[common] add Extractor.input() method
2024-04-16 00:02:48 +02:00
Mike Fährmann
257e9fb435
[gelbooru] improve pagination logic for meta tags ( #5478 )
...
similar to 494acabd38
2024-04-15 23:14:48 +02:00
Mike Fährmann
4cf24f15c2
merge #5474 : add '-w/--warning' command-line option
2024-04-14 19:38:23 +02:00
Aidan Harris
a2affdcef3
re-order verbose and warning options
2024-04-14 13:15:07 +00:00
Aidan Harris
a3f580254a
Add warnings flag
...
This commit adds a warnings flag
It can be combined with -q / --quiet to display warnings.
The intent is to provide a silent option that still surfaces
warning and error messages so that they are visible in logs.
2024-04-14 08:55:50 +00:00
Mike Fährmann
0d72789aa3
merge #5461 : [cookies] use tempfile when saving cookies.txt files
2024-04-13 19:02:39 +02:00
Mike Fährmann
63ac06643f
compute tempfile path only once
2024-04-13 18:59:18 +02:00
Mike Fährmann
5227bb6b1d
[text] catch general Exceptions
2024-04-13 18:51:40 +02:00
