Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-22 10:42:34 +01:00
Code Issues Releases Wiki Activity
6,047 Commits 6 Branches 164 Tags 15 MiB
1ddbcda58b
Commit Graph

5535 Commits

Author SHA1 Message Date
Mike Fährmann
3cbf15a107
[8chan] don't make network requests in '_init()' (#6387) 2024-10-27 19:36:49 +01:00
Mike Fährmann
6693ae19e8
[civitai] add 'metadata' option (#6383) 2024-10-27 15:46:00 +01:00
Mike Fährmann
a4791f5243
[bluesky] add 'hashtag' extractor (#4438) 
https://github.com/mikf/gallery-dl/issues/4438#issuecomment-2439979958
 2024-10-27 13:59:46 +01:00
Mike Fährmann
8dce52471a
[pixiv] restructure file extraction code 2024-10-27 09:01:12 +01:00
Mike Fährmann
ae448f8ca3
[pixiv] fix/improve PHPSESSID warning code 2024-10-27 08:59:46 +01:00
Mike Fährmann
cc07c81063
[mangadex] apply 'lang' only to chapter results (#6372) 2024-10-27 07:55:14 +01:00
Mike Fährmann
7e43c9292d
release version 1.27.7 2024-10-25 18:59:39 +02:00
Mike Fährmann
8f396cfc57
[bluesky] add 'quoted' option (#6323) 2024-10-25 17:22:33 +02:00
Mike Fährmann
d34e2d56aa
[mangadex] use '/manga' endpoint to get manga by author (#6372) 
allows for more than 10 manga to be returned
 2024-10-24 20:07:11 +02:00
Mike Fährmann
0fd98f67ba
[mangadex] add 'author' extractor (#6372) 2024-10-24 14:57:17 +02:00
Mike Fährmann
c243a7b060
[lensdump] fix extraction (#6313) 
- support custom sort orders & query parameters
- rewrite pagination logic
 2024-10-24 13:59:39 +02:00
Mike Fährmann
8f0b0b0735
[wikimedia] fix non-English Fandom/wiki.gg articles (#6370) 2024-10-24 13:00:02 +02:00
Mike Fährmann
bce3c4b424
[bunkr] try different domain when encountering a CF challenge page 
(#6344, #6352, #6368)
 2024-10-23 21:17:01 +02:00
Mike Fährmann
75674944f0
[pixiv] implement 'sanity_level' workaround for user artworks results 
(#4327, #5435, #6339)
 2024-10-22 20:38:00 +02:00
Mike Fährmann
b08da4ffc7
[reddit] add 'embeds' option (#6357) 2024-10-22 17:06:54 +02:00
Mike Fährmann
33161da121
[pixiv] add 'captions' option (#4327) 
make extra requests for empty captions independent of 'sanity'
 2024-10-22 16:31:37 +02:00
Mike Fährmann
2d15e64ea9
[pixiv] fix exception when processing deleted sanity_level work (#6339) 
https://github.com/mikf/gallery-dl/issues/6339#issuecomment-2427350524
 2024-10-21 21:13:26 +02:00
Mike Fährmann
9a2bccd0d9
[bunkr] update to new site layout (#6344, #6352) 2024-10-21 14:17:28 +02:00
Mike Fährmann
66aa514c25
[scrolller] add initial support (#295, #3418, #5051) 2024-10-21 14:17:18 +02:00
Mike Fährmann
eacd4479a2
[patreon] support '/c/' before creator name in URLs (#6348) 
and avoid '/CREATOR/posts' redirects
 2024-10-21 14:11:19 +02:00
Mike Fährmann
cff3d37551
[civitai] update "My Reactions" handling (#6263) 
- use "reactions" subcategory
- raise exception when no api-key or cookies are present
 2024-10-20 14:56:56 +02:00
Mike Fährmann
10c56a561d
add '--print' and '--print-to-file' command-line options 2024-10-20 10:06:04 +02:00
Mike Fährmann
2e1dab3036
[pp] add 'error' event 2024-10-19 20:30:34 +02:00
Mike Fährmann
5d984f35aa
[pinterest] support 'story' pins (#6188, #6078, #4229) 2024-10-19 17:47:31 +02:00
Mike Fährmann
5477ed181d
[pinterest] move file extraction into separate method 2024-10-18 20:55:20 +02:00
Mike Fährmann
9751b716ef
merge #6328: [vk] add 'offset' option 2024-10-17 21:21:52 +02:00
Mike Fährmann
0e4e40c9d2
[vk] document 'offset', add '--range' support 2024-10-17 21:20:21 +02:00
Mike Fährmann
4667833195
[util] add 'std' object to global eval namespace (#6330) 
allows accessing standard library modules (and other external modules)
in a more straightforward manner than '__import__(...)'

* std.os.getcwd()
* std["os"].getcwd()
 2024-10-17 21:08:07 +02:00
Mike Fährmann
d3dcc44bd1
use child fallbacks only when a non-user error occurs (#6329) 2024-10-17 08:04:41 +02:00
Mike Fährmann
4b31601dc0
[civitai] fix "My Reactions" results (#6263) 
requires 'api-key' or authenticated cookies to work
 2024-10-16 23:18:00 +02:00
Mike Fährmann
69a75b1de2
[civitai] add extractors for global 'models' and 'images' (#6310) 2024-10-16 23:00:51 +02:00
Mike Fährmann
a37ac3c748
[civitai] add 'uuid' metadata field & use as archive format (#6326) 2024-10-16 19:56:09 +02:00
Mike Fährmann
a78f33d0fc
[civitai] fix 'model' file download URLs for tRPC API 2024-10-16 15:18:28 +02:00
Mike Fährmann
1824267447
[dl:ytdl] implement explicit HLS/DASH handling 
add '_ytdl_manifest' to specify a manifest type to process
 2024-10-16 15:16:21 +02:00
Sergey Kolesnik
67be377c77
Update vk.py: add "offset" option 2024-10-16 02:07:58 +03:00
Mike Fährmann
67547a659f
[cookies] convert Chromium expires_utc values to Unix timestamps 2024-10-15 08:31:46 +02:00
Mike Fährmann
36883e458e
use 'v[0] == "c"' instead of 'v.startswith("c")' 2024-10-15 08:24:06 +02:00
Mike Fährmann
d68bb78f44
use 'True if COND else False' for bool conversions 
faster than 'bool(COND)', especially on older Pythons < 3.11
 2024-10-14 20:57:47 +02:00
Mike Fährmann
c148e200a4
[bunkr] send proper Referer headers for file downloads (#6319) 2024-10-14 16:28:24 +02:00
Mike Fährmann
6506afb257
[bunkr] support 'bunkr.ps' URLs 2024-10-14 16:27:09 +02:00
Mike Fährmann
4358799833
[newgrounds] support page numbers in URLs (#6320) 
https://USER.newgrounds.com/art/?page=5
https://USER.newgrounds.com/art/page/5
 2024-10-14 14:25:09 +02:00
Mike Fährmann
d7e34e1dc3
[8chan] automatically determine TOS cookie name (#6318) 2024-10-14 14:25:09 +02:00
Mike Fährmann
93265db9b3
[8chan] support '/last/' thread URLs (#6318) 2024-10-14 14:25:09 +02:00
Mike Fährmann
c7f0d8945b
[pixiv] make retrieving ugoira metadata non-fatal (#6297) 
update to 6ae3a5cdb0
catch all exceptions instead of just StopException
 2024-10-14 14:25:09 +02:00
Mike Fährmann
3946fe5ac4
[cookies] return loaded cookies as list 
don't set_cookie() them immediately into a CookieJar
also, give some more consistent names to chrome/chromium functions
 2024-10-14 14:24:27 +02:00
Mike Fährmann
39b7d748f6
[urlgalleries] fix extraction 
accept HTTP status 500 as valid pages
 2024-10-11 20:16:32 +02:00
Mike Fährmann
1023b2d85d
[exception] implement constructing HttpError from response 2024-10-11 19:57:38 +02:00
Mike Fährmann
6d8d882dbf
[common] allow request() to accept all HTTP status codes 
by passing Ellipsis/... as 'fatal' argument
 2024-10-11 19:49:16 +02:00
Mike Fährmann
1df09292b5
consistent __exit__() argument names 2024-10-11 16:43:28 +02:00
Mike Fährmann
bca9a1a1e5
release version 1.27.6 2024-10-11 14:31:17 +02:00
Mike Fährmann
f8bb0fd6cb
[pixiv][danbooru] set '_ugoira_original' for ugoira results 2024-10-11 13:19:45 +02:00
Mike Fährmann
5dd17910d1
[deviantart] fix sta.sh folder extraction 2024-10-11 09:51:20 +02:00
Mike Fährmann
1f5f04e673
[pixiv] make '_request_ajax()' non-fatal 2024-10-11 07:55:02 +02:00
Mike Fährmann
bc954eb7e4
[civitai] update default archive format (#6302) 2024-10-10 20:29:07 +02:00
Mike Fährmann
0f7d032773
[shimmie2] use 'self.groups' to access matched URL values 2024-10-10 19:01:30 +02:00
Mike Fährmann
09d4c281b6
[shimmie2] remove 'loudbooru.com' 2024-10-10 18:32:42 +02:00
Mike Fährmann
51ceb33b6c
[8chan] also set TOS cookie for the previous day 2024-10-10 18:19:37 +02:00
Mike Fährmann
2818973981
[gelbooru_v02] unescape categorized tags 2024-10-10 17:30:55 +02:00
Mike Fährmann
f9e76ba6eb
[myhentaigallery] fix 'tags' extraction 2024-10-10 17:06:17 +02:00
Mike Fährmann
6807bf9c11
[komikcast] update domain to 'komikcast.cz' 2024-10-10 16:52:40 +02:00
Mike Fährmann
bcd920e24d
[lolisafe] remove 'xbunkr.com' 2024-10-10 16:19:08 +02:00
Mike Fährmann
4a1cbe94a9
[pururin] remove module 
"This domain name has been seized in accordance with a seizure warrant
 issued by the United States District Court for the District of Idaho"
 2024-10-10 15:57:17 +02:00
Mike Fährmann
04df813e1b
[pixiv] detect works requiring 'My pixiv' access 2024-10-10 15:57:02 +02:00
Mike Fährmann
82269fdc09
[pixiv] disable 'sanity' and 'comments' for avatar and background 
fixes regression introduced in e05b9b10 and 8bcf7bf5
 2024-10-10 15:00:11 +02:00
Mike Fährmann
3194bcbccc
[blogger] remove 'micmicidol.club' 2024-10-10 14:23:58 +02:00
Mike Fährmann
73f77a543a
[generic] ensure 'path' metadata is always defined 
fixes None directory names when 'path' is empty and got removed
 2024-10-10 14:06:47 +02:00
Mike Fährmann
a09cef79c6
[imagefap] fix '{num}' in default filenames for single images 
fixes regression introduced in 304bb4bb
 2024-10-10 13:40:50 +02:00
Mike Fährmann
a25aa26577
[chevereto] remove 'deltaporno.com' 2024-10-10 13:07:08 +02:00
Mike Fährmann
7c43f9e152
[flickr] update default API credentials (#6300) 2024-10-10 11:50:34 +02:00
Mike Fährmann
c681e94736
[cohost] limit 'headline' length in default filenames (#6262) 2024-10-10 09:37:15 +02:00
Mike Fährmann
4b7a24122e
[pp:ugoira] prevent BadZipFile exceptions (#6285) 
reset '_convert_…' flags  on every 'prepare' event
 2024-10-10 09:20:06 +02:00
Mike Fährmann
a389b8aeaa
[rule34us] make 'file_url' extraction more robust (#6289) 
update to b64bede9ae
 2024-10-10 09:11:38 +02:00
Mike Fährmann
cfb7b3dd71
[deviantart] improve 'tiptap' conversion (#6207) 
- support literature link embeds
- support @ mentions
- support more text styles
 2024-10-09 20:59:36 +02:00
Mike Fährmann
e05b9b101e
[pixiv] retrieve 'caption' from AJAX API when empty (#4327, #5191) 2024-10-09 17:50:27 +02:00
Mike Fährmann
4adb1df55a
[pornhub] fix KeyError when album images are missing (#6299) 2024-10-09 17:18:06 +02:00
Mike Fährmann
d14d04adbf
[boosty] improve signedQuery & isMigrated handling (#2387) 2024-10-08 17:12:10 +02:00
Mike Fährmann
cc38042500
[pp:ugoira] forward frame data as '_ugoira_frame_data' (#6154, #6285) 2024-10-08 11:28:44 +02:00
Mike Fährmann
c0835a0cd1
[pp:ugoira] catch all exceptions when extracting zip archive (#6285) 2024-10-08 10:06:19 +02:00
Mike Fährmann
b64bede9ae
[rule34us] fix extraction (#6289) 2024-10-07 08:10:08 +02:00
Mike Fährmann
02ca1ac602
[fanbox] add 'comments' option, extend 'metadata' option (#6287) 2024-10-06 22:31:41 +02:00
Mike Fährmann
8bcf7bf5ee
[pixiv] add 'comments' option (#6287) 2024-10-06 20:41:36 +02:00
Mike Fährmann
9757eacce1
[civitai] add 'post' extractors (#6279) 
- https://civitai.com/posts/12345
- https://civitai.com/user/USER/posts
 2024-10-06 17:48:48 +02:00
Mike Fährmann
b12d65ade2
[civitai] use tRPC API by default (#6279) 2024-10-06 08:57:58 +02:00
Mike Fährmann
a9671f160b
[deviantart] support converting 'tiptap' markup to HTML (#6207) 2024-10-05 23:43:32 +02:00
Mike Fährmann
7f945c44f5
[pixiv] support unlisted artworks (#5162) 2024-10-05 17:10:03 +02:00
Mike Fährmann
c5be50fdaa
[pixiv] implement workaround for 'limit_sanity_level' works 
(#4327, #4747, #5054, #5435, #5651, #5655)

Metadata should be ~95% identical (there might be some 'date' differences)
and there could be issues with R-18 works, as these require some URL
manipulation to transform /c/250x250_80_a2/ thumbnail URLs into
/img-original/ ones.
 2024-10-04 21:07:56 +02:00
Mike Fährmann
d1432d02a1
[pixiv] move file/ugoira extraction into a separate function 2024-10-04 16:21:19 +02:00
Mike Fährmann
3fa639fc2d
[boosty] support audio files (#2387) 2024-10-04 09:20:25 +02:00
Mike Fährmann
57c30fdc5b
[bunkr] support 'bunkr.pk' URLs (#6272) 2024-10-04 09:16:31 +02:00
Mike Fährmann
5b968a0a7c
[boosty] extend image URLs with 'signedQuery' (#2387) 2024-10-03 20:25:12 +02:00
Mike Fährmann
8548fc3a70
[boosty] add 'bought' option (#2387) 2024-10-03 20:25:12 +02:00
Mike Fährmann
ee8c4e2e49
[boosty] fix typo causing 'links' to always be empty (#2387) 2024-10-03 20:25:12 +02:00
Mike Fährmann
274d99e7d6
[boosty] add 'feed' and 'following' extractors (#2387) 2024-10-03 18:09:31 +02:00
Mike Fährmann
aa31c153e3
[patreon] include 'attachments_media' in API results (#6268) 
continuation of fb6be2dd34
 2024-10-03 10:24:05 +02:00
Mike Fährmann
1ad58cab84
[boosty] add initial support (#2387) 2024-10-02 20:39:55 +02:00
Mike Fährmann
7dbd53e9b4
[deviantart] extract journal HTML from webpage (#6254, #6207, #6196) 2024-10-01 21:47:14 +02:00
Mike Fährmann
fb6be2dd34
[patreon] extract 'attachments_media' files (#6241) 
https://github.com/mikf/gallery-dl/issues/6241#issuecomment-2386343160
 2024-10-01 20:28:30 +02:00
Mike Fährmann
b03ee3c4c4
[text] implement 'parse_query_list()' 2024-10-01 20:28:30 +02:00
Mike Fährmann
9f49cf16e8
[text] implement 'parse_query()' without using 'urllib.parse.parse_qsl' 
doesn't support bytes anymore, but is twice as fast
 2024-10-01 20:28:11 +02:00
Mike Fährmann
a120295632
[util] use minimal separators for 'json_dumps()' 2024-10-01 17:03:13 +02:00