1
0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-23 03:02:50 +01:00
Commit Graph

5529 Commits

Author SHA1 Message Date
Mike Fährmann
7b0c07ee69
[instagram] rename to 'liked', enable for GraphQL results 2024-06-07 01:41:56 +02:00
Jan Wikholm
ece3837408
Enable user_has_liked as IG download filter
Background and context:

1. Instagram by default requires you to be logged in.
2. Your account can LIKE posts.
3. Getting all of your likes from IG can be a pain.

As an active IG user, I want to be able to download only posts I've liked.

The information of whether or not your logged-in user has liked the target post is available in the API results, but was not available to the end-user for filtering.

This small change allows `user_has_liked` to be used to e.g. download only liked posts of an entire IG account.

Example use:

```
$ gallery-dl --cookies-from-browser FIREFOX--filter 'user_has_liked' https://www.instagram.com/imgur/
```

This will only download the posts that have `has_liked: True` set in the returned REST API data.

Note: it will still need to loop through the entire account's posts since this is done per-post basis instead of looping just the liked posts, but at least you are able to use this IG feature as a download marker.
2024-05-18 12:05:08 +03:00
Mike Fährmann
6276f5cd94
[twitter] update domain to x.com (#5597) 2024-05-17 22:59:52 +02:00
Mike Fährmann
bfcb66af05
[poipiku] send LANG cookie to ensure English UI (#5590) 2024-05-17 22:59:31 +02:00
Mike Fährmann
f651b3b6ab
merge #5601: [twitter] match '/video/' Tweet URLs 2024-05-17 22:49:12 +02:00
Mike Fährmann
2dec4bef60
merge #5591: [poipoku] avoid language-specific extraction 2024-05-17 22:47:51 +02:00
Mike Fährmann
7f1ed909d5
[imgur] match gallery/album/image URLs with title slugs (#5593) 2024-05-17 22:44:37 +02:00
Delphox
8ba73e2ec9
[twitter] match /video/ tweet urls 2024-05-17 16:50:51 -03:00
fireattack
831f922c1c [poipoku] avoid language-specific extr 2024-05-14 02:42:02 +08:00
Mike Fährmann
07d962d60a
[workflows:tests] remove Python 3.5
currently causing errors in the setup phase:

  Error: Could not find a version that satisfies the requirement pip
  Error: No matching distribution found for pip
  Error: The process '/usr/bin/bash' failed with exit code 1
2024-05-12 16:52:38 +02:00
Mike Fährmann
33b07c4603
[twitter] wait for rate limit reset before 429 error (#5532) 2024-05-12 01:13:30 +02:00
Mike Fährmann
3b6f306a8b
[exhentai] fix blank page detection 2024-05-11 22:01:34 +02:00
Mike Fährmann
90b3741643
[cookies] set proper 'expires' value for Chrome session cookies
https://github.com/yt-dlp/yt-dlp/pull/9747
2024-05-11 19:46:35 +02:00
Mike Fährmann
f178839fc2
[exhentai] fix multi-page viewer detection (#4969) 2024-05-11 18:32:52 +02:00
Mike Fährmann
5b6b5dd81b
[8chan] fix downloaded files by sending 'TOS' cookie (#5578) 2024-05-11 15:34:21 +02:00
Mike Fährmann
d2f50ecf09
add 'skip-filter' option (#5255) 2024-05-10 22:59:52 +02:00
Mike Fährmann
fd734b9222
[archive] add 'archive-mode' option (#5255) 2024-05-10 22:59:51 +02:00
Mike Fährmann
28039229fe
[common] use 'create_urllib3_context' for creating SSLContexts
enables dumping TLS session keys by setting SSLKEYLOGFILE (#5215)
as well as other potentially useful settings.
2024-05-10 22:59:29 +02:00
Mike Fährmann
33006fe126
[common] disable 'check_hostname' for non-urllib3 SSLContexts
e.g. when 'browser' is set to a non-empty value and gallery-dl creates
its own SSLContext instance instead of using requests' and urllib3's
defaults.

urllib3 disables this option for its default contexts,
since it does this check on its own.

Fixes "ValueError: Cannot set verify_mode to CERT_NONE when
check_hostname is enabled" when using --no-check-certificate.

(#3614, #4891, #5576)
2024-05-10 18:20:08 +02:00
Mike Fährmann
215abbc3e4
[archive] implement DownloadArchiveMemory class (#5255)
keeps archive IDs in memory
and only writes them to disk in a 'finalize' step
2024-05-10 01:15:44 +02:00
Mike Fährmann
88f94190f4
[archive] move DownloadArchive into its own module 2024-05-10 01:05:28 +02:00
Mike Fährmann
2ee9ffeed6
merge #5568: [furaffinity] match 'xfuraffinity' URLs 2024-05-09 19:20:12 +02:00
Mike Fährmann
b27ffd790a
[poipiku] fix downloading R-18 posts (#5567)
… by automatically sending a `POIPIKU_CONTENTS_VIEW_MODE=1` cookie
to enable "adult" mode.
2024-05-09 15:14:08 +02:00
Delphox
11109d5bad
[furaffinity] match xfuraffinity.com 2024-05-08 12:15:47 -03:00
Mike Fährmann
dc9d83e64b
[output] support 'NO_COLOR' environment variable 2024-05-03 12:25:41 +02:00
Mike Fährmann
699592498b
[tests] use random port number for local HTTP server
… and explicitly bind to 127.0.0.1 instead of all interfaces
2024-05-02 22:54:15 +02:00
Mike Fährmann
625740c3d2
merge #5538: [cookies] optimize _find_most_recently_used_file for exact profiles 2024-05-02 18:19:19 +02:00
Jan Wikholm
06d102f19a
optimize _find_most_recently_used_file for exact profile
When reading cookies from the browser, the user is able to give either just the browser name, or also provide profile/container information.

If an exact profile is provided, there is no need to find the latest profile with `os.walk` which is very expensive.

This change optimizes that case and the performance increase is significant (~8 sec to 0.6 sec).

```
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
 ./imgur_OO4UNqJ.jpg

real    0m8.429s
user    0m0.216s
sys     0m0.431s

$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX:bgamf5r6.default-release https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
 ./imgur_OO4UNqJ.jpg

real    0m0.456s
user    0m0.183s
sys     0m0.011s

$ gallery-dl --version
1.26.9
```
2024-05-02 18:16:13 +02:00
Mike Fährmann
bd8e4797e5
[vsco] add 'avatar' extractor (#5341) 2024-05-02 18:12:19 +02:00
Mike Fährmann
d0cead105b
[formatter] allow dots etc in '…' literals (#5539)
don't parse fields starting with '

this disables the ability to directly apply […] to '…' literals,
but that's not really useful anyway and can still be done with _lit
2024-05-02 17:43:24 +02:00
Mike Fährmann
619bf5c644
[inkbunny] retry API calls with a loop instead of recursion 2024-05-01 22:46:24 +02:00
Mike Fährmann
8ed70b3256
[tests] mark tests with missing auth as 'only_matching'
… instead of skipping them completely
2024-05-01 16:00:07 +02:00
Mike Fährmann
d11ec00908
[common] fix _cfgpath for BaseExtractor objects
After the changes in 0c17884673,
_cfgpath was mssing its 'category' value
since that hadn't been initialized yet.
2024-05-01 16:00:07 +02:00
Mike Fährmann
6db1837a6d
[bluesky] filter reposts only for user timelines (#5528) 2024-05-01 16:00:07 +02:00
Mike Fährmann
3cf5366143
[mastodon] add support for card images 2024-05-01 16:00:07 +02:00
Mike Fährmann
9b1995dda3
[mastodon] add 'favorite', 'list', and 'hashtag' extractors (#5529) 2024-05-01 15:59:34 +02:00
Mike Fährmann
7fd31aaf8e
merge #5516: [mastodon] send canonical 'true'/'false' boolean values
instead of '1'/'0' to be compatible with Mitra instances or others that
do not fully implement Mastodon's boolean value semantics.
2024-04-27 01:34:56 +02:00
cenodis
3ba5fd9efd
[mastodon] Use boolean instead of integer keys for accounts/statuses endpoint 2024-04-26 22:51:56 +02:00
Mike Fährmann
6c57958806
merge #5511: [twitter] [furaffinity] match fixvx.com and fxfuraffinity/fxraffinity.net URLs 2024-04-25 22:00:19 +02:00
Mike Fährmann
5aefa9465c
[furaffinity] simplify pattern, match original domain first 2024-04-25 21:58:34 +02:00
Delphox
1886721d82
update tests 2024-04-25 13:28:30 -03:00
Delphox
6969963125
[furaffinity] support fxfuraffinity and fxraffinity urls 2024-04-25 13:25:39 -03:00
Delphox
85550a37c4
[twitter] support fixvx urls 2024-04-25 13:19:40 -03:00
Mike Fährmann
cd241bea0a
[downloader:http] add MIME type and signature for .m4v files (#5505) 2024-04-25 01:01:35 +02:00
Mike Fährmann
bef0bd3b49
add '--no-colors' command-line option 2024-04-20 21:06:59 +02:00
Mike Fährmann
20e2c0042b
[output] enable colors by default 2024-04-20 20:51:20 +02:00
Mike Fährmann
14b38264e0
[output] extend 'output.colors' (#2566)
allow specifying ANSI colors for all loglevels
(debug, info, warning, error)
2024-04-20 20:50:58 +02:00
Mike Fährmann
ddffeeaa7b
[exhentai] detect CAPTCHAs during login (#5492) 2024-04-20 00:19:22 +02:00
Mike Fährmann
068ccfe0b3
[tests] allow filtering extractor result tests by URL or comment
python test_results.py twitter:+/i/web/
python test_results.py twitter:~twitpic
2024-04-19 23:02:55 +02:00
Mike Fährmann
347af7f5c8
[twitter] handle missing 'expanded_url' fields (#5463, #5490) 2024-04-19 21:42:22 +02:00