Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-23 11:12:40 +01:00
Code Issues Releases Wiki Activity
3,872 Commits 6 Branches 164 Tags 16 MiB
67a2efb885
Commit Graph

780 Commits

Author SHA1 Message Date
Mike Fährmann
6742f3bc1e
implement --cookies-from-browser (#1606) 
most of the code is adapted from yt-dlp's implementation
and *should* work the same.
 2022-05-07 23:06:37 +02:00
Mike Fährmann
61887c895b
implement 'output.colors' options (#2532) 2022-05-02 12:41:14 +02:00
Mike Fährmann
52b47c3cf9
[gelbooru_v01] add 'favorite' extractor (#2546) 2022-05-02 11:33:28 +02:00
Mike Fährmann
84756982e9
[pixiv] implement 'include' option 
- split 'user' extractor and its 'avatar' and 'background' options into
  separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
  https://www.pixiv.net/en/users/ID/avatar
  as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
  the same effect as before
 2022-05-02 09:03:35 +02:00
Mike Fährmann
d11e2191ae
[nijie] support /history_nuita.php listings (closes #2541) 2022-05-02 09:03:34 +02:00
Mike Fährmann
61afa973b7
use extended format strings for archive keys 2022-04-28 20:26:12 +02:00
Federico Ravasio
0381752575
[photovogue] switch to .com, update api endpoint (#2494) 2022-04-27 22:37:53 +02:00
Mike Fährmann
9e6ff42a9d
[pixiv] implement 'background' option (#623, #1124, #2495) 2022-04-21 13:53:02 +02:00
Mike Fährmann
97e8a15295
[deviantart] implement 'pagination' option (#2488) 2022-04-18 18:08:01 +02:00
Mike Fährmann
12bd9ba33a
[readcomiconline] add 'quality' option (#2467) 2022-04-15 18:10:37 +02:00
Mike Fährmann
24ea1398bc
[postprocessor:ugoira] do not auto-select image2 demuxer (#2492) 2022-04-12 21:47:47 +02:00
Mike Fährmann
ffa1dd1d91
add 'coomer.party' to supported sites (closes #2462) 2022-04-06 21:27:31 +02:00
Mike Fährmann
9c5d2d7af3
[pinterest] add extractor for created pins (#2452) 2022-04-01 16:59:58 +02:00
Mike Fährmann
1171911dc3
[twitter] add 'syndication' option (#2354) 
to fetch age-restricted content using Twitter's  syndication API
 2022-04-01 16:56:47 +02:00
loragja
7e545a3ae9
[gofile] add gofile.io extractor (#2364) 
* Add gofile extractor

* add gofile extractor to module list

* add support for tiny monitors and ancient python versions

* seriously, f-strings are not *that* new...

* i love flake8 :)

* add 'api-token' and 'recursive' options
* add tests
 2022-03-29 17:31:57 +02:00
Layerex
625f4d4cc4
[telegraph] Add telegra.ph extractor (#2312) 2022-03-28 19:18:13 +02:00
Mike Fährmann
5c2d953766
[postprocessor:ugoira] auto-select demuxer by default 2022-03-26 21:10:59 +01:00
Mike Fährmann
37d584a9b2
[hitomi] update metadata extraction (fixes #2444) 
remove 'hitomi.metadata' option, as it is no longer necessary
to make additional HTTP requests to fetch all metadata.
 2022-03-26 12:46:18 +01:00
Mike Fährmann
e718dd7b32
[postprocessor:ugoira] support setting timecodes with mkvmerge 
by selecting the "mkvmerge" demuxer

(#1550)
 2022-03-26 12:41:55 +01:00
Mike Fährmann
71bba774da
respect 'output.private' in '-K/--list-keywords' output 2022-03-25 22:19:37 +01:00
Mike Fährmann
ba69fb669d
[kemonoparty] add 'duplicates' option (closes #2440) 2022-03-24 11:58:38 +01:00
Mike Fährmann
40ce505420
[postprocessor:ugoira] add 'mtime' option (#2307) 2022-03-21 10:02:29 +01:00
Mike Fährmann
9bd27b1b8d
[postprocessor:metadata] implement archive options (#2421) 
'archive', 'archive-format', and 'archive-prefix'
 2022-03-20 21:16:46 +01:00
Mike Fährmann
f8230dde43
[instagram] add 'previews' option (#2135) 2022-03-19 15:26:40 +01:00
Mike Fährmann
cf44aba333
[formatter] allow evaluating f-string literals 
by starting a format string with '\fF'.

This was technically already possible with '\fE',
but this makes it a bit more convenient.
 2022-03-18 13:31:01 +01:00
Mike Fährmann
58e0b17211
add note about using '\f' in shells to docs/formatting.md 
(closes #2398, closes #2406)
 2022-03-17 23:09:25 +01:00
lingegrino
b3d8b40690
Rename meme.museum (#2393) 2022-03-14 08:00:35 +01:00
Mike Fährmann
fc277fa45f
[seiga] require authentication with 'user_session' cookie (#2372) 
Login with username & password would now require entering a 2FA token.

see also 7b009cc893
 2022-03-11 02:10:15 +01:00
Mike Fährmann
47cf05c4ab
refactor proxy handling code (#2357) 
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
  - add 'downloader.proxy' option
  - '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
    now has the same effect as youtube-dl's '--geo-verification-proxy'
 2022-03-10 23:55:35 +01:00
Mike Fährmann
acbb0f5a57
clarify 'metadata.mtime' usage and shortcomings (#2307) 2022-03-08 21:11:06 +01:00
Mike Fährmann
bfa5e61900
[patreon] add explicit 'image_large' file type (#2257) 
to allow more control over when and if to download 'large_url' images

4fee3a0e52 forced them to be downloaded
instead of regular images, even though 'large_url' images are most likely
an upscaled version of the original.
 2022-03-06 17:07:13 +01:00
Alice
f1cab23724
[skeb] add 'sent-requests' option (#2322) (#2330) 
* Update skeb.py

* Update configuration.rst

* flake8
 2022-02-28 22:42:15 +01:00
Mike Fährmann
4385a34e05
[twitter] fix handling of 429 responses (fixes #2339) 
Twitter doesn't return a valid JSON response for 429 errors anymore.
 2022-02-28 16:42:55 +01:00
Mike Fährmann
5a50569360
[toyhouse] support 'art' listings (#1546, #2331) 2022-02-27 16:22:50 +01:00
Mike Fährmann
59749555b1
[postprocessor:mtime] add 'event' option (#2307) 2022-02-22 23:27:40 +01:00
Mike Fährmann
7958995398
[postprocessor:metadata] implement 'mtime' option (#2307) 2022-02-22 23:02:13 +01:00
Mike Fährmann
fdfdc1b614
[kissgoddess] add 'gallery' and 'model' extractors 
(closes #1052, #2304)
 2022-02-20 04:45:37 +01:00
Mike Fährmann
79a461a2c1
[mememuseum] add 'tag' and 'post' extractors (closes #2264) 2022-02-20 02:15:38 +01:00
Mike Fährmann
254a5b26e0
[twibooru] add extractors for searches, galleries, and posts 
(#2219)
 2022-02-18 23:43:57 +01:00
Mike Fährmann
f5b2a9fcc6
add 'signals-ignore' option (#2296) 2022-02-13 23:00:50 +01:00
David Hoppenbrouwers
b17e2dcf93
[wallpapercave] add extractor for images (#2205) 2022-02-11 23:44:51 +01:00
Thomas Jost
a7de819aca
[lightroom] add Lightroom gallery extractor (#2263) 2022-02-11 21:30:59 +01:00
Mike Fährmann
563bd0ecf4
[danbooru] inherit from BaseExtractor 
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283)
- remove support for old e621 tag search URLs
 2022-02-11 21:01:51 +01:00
Mike Fährmann
f1c853c6ef
[furaffinity] add 'layout' option (#2277) 
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
 2022-02-11 00:28:47 +01:00
Mike Fährmann
4e4cb40663
do not sort BaseExtractor instances by name 2022-02-10 01:44:16 +01:00
Mike Fährmann
f351746483
release version 1.20.4 2022-02-06 22:02:36 +01:00
HRXN
f4a7c6eca2
Add two examples for custom instances of gelbooru (#2262) 
* Add two examples for custom instances of gelbooru

- First example uses some custom options at the "gelbooru_v01" basecategory
   These apply to all instances used herein
- Second example adds all options to the specific "gelbooru_v02" basecategory of TBIB
    These settings are only used by TBIB

* move tbib settings into their own block
 2022-02-04 23:38:22 +01:00
Mike Fährmann
cc7dce5755
[sexcom] add 'pins' extractor (closes #2265) 2022-02-04 20:55:00 +01:00
Mike Fährmann
02e18f56be
[e621] add 'favorite' extractor (closes #2250) 2022-02-04 20:54:48 +01:00
Mike Fährmann
86fa412b47
[hitomi] add 'format' option (#2260) 
default is 'webp' since downloading original files is no longer allowed
 2022-02-03 23:32:19 +01:00
Mike Fährmann
492436f936
[twitter] add 'warnings' option (#2258) 
disable reporting any non-fatal errors by default
 2022-02-02 18:37:19 +01:00
Mike Fährmann
9a221494c3
[twitter] add 'event' extractor (closes #2109) 2022-01-22 20:55:50 +01:00
Mike Fährmann
de754590e0
add --source-address command-line option (closes #2206) 2022-01-21 17:07:56 +01:00
Mike Fährmann
f2e8aedd74
[twitter] changes to 'cards' option 
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
  when 'cards' is set to "ytdl"

"cards": true   --> only download card images
"cards": "ytdl" --> download card images and
                    use youtube_dl on otherwise unsupported cards
 2022-01-15 22:02:57 +01:00
ImportTaste
c559c49781
formatting.md: add environment variable syntax (#2065) 2022-01-13 17:47:19 +01:00
Mike Fährmann
3c79c9b271
document extended blacklist/whitelist syntax (#2025) 
and not just in the commit message of 010d65dc
 2022-01-06 23:36:57 +01:00
Mike Fährmann
37beb1298e
[newgrounds] add 'search' extractor (closes #2161) 2022-01-06 19:32:39 +01:00
Mike Fährmann
fa7d92f7a9
add docs for 'extractor.generic.enabled' 2021-12-29 22:46:17 +01:00
Mike Fährmann
89bebe1bef
[500px] add 'favorite' extractor (closes #1927) 2021-12-25 17:16:45 +01:00
Mike Fährmann
882c614281
add album extractor for lolisafe/chibisafe instances 
- support bunkr.is (closes #2038)
- support zz.ht    (closes #2105)
 2021-12-21 19:24:17 +01:00
Mike Fährmann
64cf26eaf4
allow specifying sleep-* options as string 
either as single value or as range: "3.5", "2.1 - 5.0"
 2021-12-18 23:28:56 +01:00
Mike Fährmann
62692c6842
[exhentai] add 'source' option 
setting it to "hitomi" downloads the corresponding gallery from
hitomi.la; might be extended to other sources in the future
 2021-12-16 23:16:19 +01:00
Mike Fährmann
099ed72de7
[hitomi] disable extra 'metadata' by default 
safes one HTTP request that not needed with default filename settings
 2021-12-16 22:21:07 +01:00
Mike Fährmann
f587458a3c
[twitter] include '4096x4096' as a default image fallback 
(closes #2107, closes #1881)
 2021-12-15 23:19:30 +01:00
Mike Fährmann
299bd2f1f5
[rule34us] add 'tag' and 'post' extractors (#1527) 2021-12-14 00:27:46 +01:00
Mike Fährmann
3cf1075d86
[inkbunny] add 'search' extractor (closes #2094) 2021-12-12 03:08:14 +01:00
Mike Fährmann
f4e3cee6ac
use yt-dlp by default (#1850, #2028) 2021-11-29 18:24:26 +01:00
Mike Fährmann
f1b142e993
{kemonoparty[ change default 'files' order to attachments,file,inline 
(#1991)
 2021-11-29 04:41:30 +01:00
Mike Fährmann
addb72e1bb
[reactor] support thatpervert.com (closes #2029) 2021-11-26 18:58:07 +01:00
Mike Fährmann
d8d9502e1e
[reactor] inherit from BaseExtractor 2021-11-26 18:58:07 +01:00
Mike Fährmann
f4ea216c95
[shopify] support loungeunderwear.com (closes #2053) 2021-11-26 18:58:06 +01:00
Alice
612850438e
[skeb] add 'thumbnails' option (#2047) (#2051) 2021-11-23 21:16:42 +01:00
Mike Fährmann
11a3d96d13
[mangadex] load additional metadata using includes[] directives 
- always provide 'artist', 'author', and 'group' metadata fields (#2049)
- remove 'metadata' option
 2021-11-22 01:16:33 +01:00
Mike Fährmann
c67756e187
[kemonoparty] add 'dms' option (#2008) 2021-11-20 23:36:16 +01:00
Mike Fährmann
3842cdcd8f
[formatter] implement 'D' format specifier 
To be able to parse any string into a 'datetime' object
and format it as necessary.

Example:

{created_at:D%Y-%m-%dT%H:%M:%S%z}
->
"2010-01-01 00:00:00"

{created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p}
->
"Jan 01 2010 12:00 AM"

with 'created_at' == "2010-01-01T01:00:00+0100"
 2021-11-20 23:04:34 +01:00
Mike Fährmann
3a7a19c7b9
[dynastyscans] add 'manga' extractor (closes #2035) 2021-11-19 22:51:26 +01:00
Mike Fährmann
9bc83af3a6
[kemonoparty] 'postfile' -> 'file' (#1991) 
to stay consistent with the existing file types for kemono
 2021-11-19 01:50:48 +01:00
Mike Fährmann
d433735750
[kemonoparty] skip duplicate files (#2032, #1991, #1899) 
Extract the SHA-256 file hash from URLs
and skip files with the same hash in the same post.

- provide a 'hash' metadata field (empty string if not available)
- remove 'patreon-skip-file' option
 2021-11-17 22:44:15 +01:00
Mike Fährmann
d4ec245554
[kemonoparty] implement a 'files' option (#1991) 
similar to 8d676151
 2021-11-17 22:43:41 +01:00
Mike Fährmann
cfa4876848
[philomena] support furbooru.org (closes #1995) 2021-11-15 20:57:51 +01:00
Mike Fährmann
7c50f473e4
[ytdl] document 'cmdline-args' and 'config-file' options (#1680) 2021-11-12 18:54:04 +01:00
Mike Fährmann
37c9dedee1
[seisoparty] remove module 2021-11-09 22:41:04 +01:00
Mike Fährmann
b315a0ecef
[redgifs] update to API v2 (#1984) 2021-11-04 21:31:20 +01:00
Mike Fährmann
f0fc3b0ba1
[kemonoparty] add 'comments' option (#1980) 2021-11-03 23:02:13 +01:00
Mike Fährmann
4c49174579
[mangakakalot] update domain and fix extraction 2021-10-31 02:16:13 +01:00
ImportTaste
69cd573809
add special type format strings to docs (#1987) 2021-10-30 21:46:26 +02:00
Mike Fährmann
9156e90f1f
[twitter] add 'pinned' option 2021-10-29 22:10:58 +02:00
Mike Fährmann
06b414c9a3
[redgifs] 'gfyId' -> 'id' (#1984) 2021-10-29 02:05:39 +02:00
Alice
bfd7401b1e
[skeb] add 'user' and 'post' extractors (#1031) (#1971) 
* Create skeb.py

* Update __init__.py

* Update supportedsites.py

* Update supportedsites.md

* Update supportedsites.py

* Update skeb.py
 2021-10-26 20:00:41 +02:00
Mike Fährmann
02a247f4e5
[deviantart] full resolution for non-downloadable images (#293) 
Many thanks to @Ironchest337 for discovering this method
and providing a well-documented implementation.
 2021-10-24 21:11:12 +02:00
Mike Fährmann
c19e762fdf
[vk] add 'album' extractor (#474, fixes #1952) 
todo: better metadata for albums
 2021-10-23 00:46:20 +02:00
Mike Fährmann
8bb442f20d
[redgifs][gfycat] provide fallback URLs (fixes #1962) 
and extend the 'format' option
 2021-10-22 22:47:29 +02:00
Mike Fährmann
8d676151b7
[patreon] implement 'files' option (#1935) 2021-10-17 04:17:10 +02:00
Mike Fährmann
80314e981b
small corrections to gallery-dl-example.conf 
use "'_reddit' in locals()"
instead of ""locals().get('_reddit')"
since the reddit metadata dict could be empty. The latter expression
would evaluate to False in that case, even though it has a reddit parent.
 2021-10-16 20:50:39 +02:00
Mike Fährmann
3ecf4a9729
update gallery-dl-example.conf 
- add comments
- remove/replace some older options
- display more post processor settings
 2021-10-16 17:35:56 +02:00
Mike Fährmann
50098762e3
[nhentai] add 'tag' extractor (closes #1950) 2021-10-14 16:23:47 +02:00
Mike Fährmann
fe6ce5495a
[kemonoparty] add 'discord' extractor (#1827, #1940) 2021-10-13 20:33:05 +02:00
Mike Fährmann
918fc9974d
[picarto] add 'gallery' extractor (closes #1931) 2021-10-13 01:22:10 +02:00
Mike Fährmann
e33125ad39
[pixiv] add 'sketch' extractor (#1497) 2021-10-13 00:02:11 +02:00
Mike Fährmann
b93915c113
[inkbunny] add 'pool' extractor (#1937) 2021-10-09 22:07:07 +02:00