Mike Fährmann
15a4d7696f
[docs] update 'adjust-extensions' explanation ( #3031 )
2022-10-11 10:48:28 +02:00
Mike Fährmann
1696f68a68
[8chan] add 'thread' and 'board' extractors ( #2938 )
2022-10-11 10:47:19 +02:00
Mike Fährmann
560f7b41d8
[vk] add 'tagged' extractor ( #2997 )
2022-10-08 15:41:39 +02:00
Mike Fährmann
7f30a0d7a7
add 'path-extended' option ( #3021 )
2022-10-08 15:41:39 +02:00
Mike Fährmann
14717f3fc9
[deviantart] add 'group' option ( #3018 )
...
disabling this option allows to better download from deleted accounts
2022-10-08 15:41:21 +02:00
enduser420
f0321f423d
[2chen] Add 2chen.moe extractor ( #2707 )
...
* [2chen] Add 2chen.moe extractor
* change "==" to is
* fix for "test_unique_pattern_matches"
* fix regex pattern and group matching
* fix regex again
* [2chen] add 'reply_no' and 'hash' metadata and change 'filename_fmt'
also made an entry in supportedsites.md
* [2chen] unescape 'title'
* [2chen] partition() -> rpartition()
* [2chen] extract 'date' and 'name' metadata
* [2chen] remove 'offset' argument
* [2chen] do some changes
* [2chen] do some more changes
* [2chen] unescape 'name' and 'filename'
2022-10-04 22:18:13 +02:00
enduser420
f7ba19a1c0
[nana] add 'nana' extractors ( #2967 )
2022-10-04 09:23:24 +02:00
Mike Fährmann
4089bceddd
[sankaku] implement 'refresh' option ( #2958 )
2022-09-30 19:55:48 +02:00
Mike Fährmann
63e0924927
[pixiv] add 'series' extractor ( #2964 )
2022-09-27 23:24:03 +02:00
Mike Fährmann
aa49bf13d2
[instagram] add 'api' option
2022-09-26 22:05:29 +02:00
Mike Fährmann
ac45ed2764
[skeb] implement 'filters' option ( #2945 )
2022-09-26 15:17:28 +02:00
Mike Fährmann
08b0c4c0b1
[docs] add kemono text content example ( #2963 )
2022-09-25 14:07:41 +02:00
Mike Fährmann
0393e59535
[newgrounds] add 'games' extractor ( #2955 )
2022-09-24 12:34:37 +02:00
Mike Fährmann
4b2a006871
[skeb] add 'search' extractor ( #2945 )
2022-09-21 17:57:55 +02:00
Mike Fährmann
eef50c1f28
[blogger] split 'search' extractor ( #2930 )
2022-09-19 21:01:21 +02:00
enduser420
bd846abba0
[hotleak] add hotleak extractor ( #2909 ) ( #2890 )
2022-09-18 13:37:16 +02:00
Mike Fährmann
e99a9b2aff
[twitter] improve 'cards-blacklist' ( #2875 )
...
allow blacklisting domains and 'name:domain',
where 'domain' depends on a card's 'vanity_url' value
2022-09-17 17:46:34 +02:00
Mike Fährmann
46fe469c53
[tumblr] implement 'ratelimit' option ( #2919 )
2022-09-17 14:10:33 +02:00
Mike Fährmann
b36125333f
[postprocessor:zip] implement 'files' option ( #2872 )
2022-09-09 11:41:27 +02:00
Mike Fährmann
3cb8327c60
[zerochan] add 'metadata' option ( #2861 )
2022-09-02 23:25:19 +02:00
blankie
9745b48830
[tumblr] attempt to fetch high-quality inline images ( #2877 )
...
* [tumblr] attempt to fetch high-quality images (again)
Fixes #1846 , and fixes #1344
* slight refactor
* update configuration.rst entry
2022-08-31 10:53:50 +02:00
Mike Fährmann
4d78ca89db
[twitter] add 'cards-blacklist' option ( #2875 )
2022-08-31 10:28:25 +02:00
Mike Fährmann
8839b0d2ee
add section about global replacement fields to formatting.md
...
(#2862 )
2022-08-30 21:32:22 +02:00
Mike Fährmann
f16fbe9f93
document 'extractor.twitter.expand' ( #2848 )
2022-08-30 18:16:20 +02:00
Mike Fährmann
7e385ed63e
[foolfuuka] update domains
...
- remove nyafuu
- add rozenarcana (https://archive.alice.al/ )
- add tokyochronos (https://www.tokyochronos.net )
2022-08-26 17:57:17 +02:00
Mike Fährmann
67bad04dda
[formatter] add 'g' conversion to sluGify a string ( #2410 )
2022-08-26 17:57:17 +02:00
enduser420
3d87cedc58
[jpgchurch] rework the image extractor
...
now the image extractor can recognize if an image if from an album
also removed some unnecessary methods
2022-08-26 15:58:19 +05:30
enduser420
574e38a287
[kemonoparty] add 'favorites' option ( #2826 ) ( #2831 )
...
* [kemonoparty] add 'favorites' option (#2826 )
* [kemonoparty] add regex for the url parameter and fallback on the config
option
* [kemonoparty] simplify
2022-08-18 18:01:42 +02:00
Mike Fährmann
a799fae2df
[catbox] add 'album' extractor ( #2410 )
...
adapted from https://github.com/mikf/gallery-dl/pull/2805
- rewrite using GalleryExtractor
- extract more metadata
- match lolisafe names
- add test
2022-08-18 18:00:24 +02:00
Martin Juhasz
9ea521bec5
[oauth] add host config option ( #2806 )
2022-08-14 17:08:01 +02:00
Mike Fährmann
df1c643dda
[tumblr] attempt to extract full-resolution photos
...
- for photos with apparent width == 2048 or height == 3072
- can be disabled with 'original' option
2022-08-10 20:01:46 +02:00
Mike Fährmann
a69f198ac9
[wallhaven] implement 'metadata' option ( #2803 )
2022-08-08 21:46:36 +02:00
Mike Fährmann
7d1a95ada6
implement 'path-metadata' option ( #2734 )
2022-07-30 12:31:45 +02:00
Mike Fährmann
8cf5981ded
[skeb] add option to download 'article' images ( #1031 )
2022-07-29 16:32:00 +02:00
Mike Fährmann
98af5a0409
[zerochan] implement login with username & password ( #1434 )
2022-07-29 12:56:20 +02:00
Mike Fährmann
3a8addfe45
[zerochan] add 'tag' and 'image' extractors ( #1434 )
2022-07-27 22:58:23 +02:00
Chew Shee Yang
977d53b640
[Instagram] Add support for user's saved collection ( #2769 )
...
* [Instagram] Add support for user's saved collection
* [Instagram] Run formatter
* [Instagram] Simplify collection_id retrieval and add metadata
* [Instagram] Fix bug when params is not passed to _pagination_api
2022-07-27 10:49:45 +02:00
Mike Fährmann
0c73914848
[postprocessor:metadata] implement 'mode: modify' ( #2640 )
2022-07-19 12:24:26 +02:00
Mike Fährmann
f3de6b7a87
[postprocessor:metadata] implement 'mode: delete' ( #2640 )
2022-07-19 00:57:29 +02:00
Mike Fährmann
eb68d45544
add global 'warnings' option ( #2762 )
2022-07-18 22:20:30 +02:00
Mike Fährmann
f225247670
[gelbooru] add support for api_key and user_id ( #2767 )
2022-07-18 18:46:31 +02:00
Mike Fährmann
46f11a3118
[bunkr] fix extraction ( #2732 )
...
move bunkr.is code to its own module
2022-07-15 13:00:57 +02:00
Mike Fährmann
fe2b3d57d4
[komikcast] update domain
2022-07-12 23:07:58 +02:00
Mike Fährmann
2dc57637cf
[foolfuuka] remove archive.wakarimasen.moe
2022-07-10 23:13:49 +02:00
Mike Fährmann
74865adae5
implement 'format-separator' option ( #2737 )
...
a global option, that servers as a workaround for shortcomings due to
lack of a proper format string parser
2022-07-10 13:31:43 +02:00
bradenhilton
117eeefda0
[postprocessor:mtime] add 'value' option ( #2739 )
2022-07-08 20:56:01 +02:00
Mike Fährmann
90ae48c40c
[formatter] implement 'O' format specifier ( #2736 )
...
to apply a UTC offset to 'date' values and other datetime objects
2022-07-08 12:51:03 +02:00
Mike Fährmann
1d14928bd9
[twitter] ignore previously seen Tweets ( #2712 )
...
occurs primarily for /with_replies results when logged in
2022-07-03 16:13:53 +02:00
Mike Fährmann
4b2a0a0eda
[twitter] implement 'strategy' option ( #2712 )
...
to be able to better control what Tweets get used an returned
for twitter.com/USER URLs.
2022-07-03 14:29:15 +02:00
Mike Fährmann
124b5c6410
[postprocessor:ugoira] enable 'mtime' by default ( #2714 )
2022-07-01 20:59:19 +02:00
Mike Fährmann
3a5d5c3a91
update default User-Agent header to Firefox 102 ESR
...
snd update headers and ciphers for "browser": "firefox"
2022-06-28 17:38:58 +02:00
Mike Fährmann
54525d2e21
[formatter] implement slice operator as format specifier
...
this allows using a slice operator alongside other (special) format
specifiers like J, to first join list elements to a string and then
trimming that with a slice.
{tags:J, /[:50]}
2022-06-25 16:52:58 +02:00
Mike Fährmann
241e82e18d
[horne] add support for horne.red ( #2700 )
2022-06-25 16:52:16 +02:00
Mike Fährmann
9d8e99af80
[itaku] support videos ( #1842 )
2022-06-20 19:47:53 +02:00
Mike Fährmann
27e8078fb7
[poipiku] add 'user' and 'post' extractors ( #1602 )
2022-06-20 11:32:02 +02:00
Mike Fährmann
fa902cd54d
[itaku] add 'gallery' and 'image' extractors ( #1842 )
2022-06-20 11:31:44 +02:00
Mike Fährmann
9c8d895d19
[twitter] implement 'csrf' option ( #2676 )
2022-06-13 18:36:39 +02:00
Mike Fährmann
6db77d4656
[weibo] support '?tabtype=video' listings ( #2601 )
2022-06-12 17:55:23 +02:00
Mike Fährmann
4b78bd423f
[paheal] add 'metadata' option ( #2641 )
2022-06-04 16:05:49 +02:00
Mike Fährmann
57508d3bb7
[weibo] support all different 'tabtype' listings ( #686 , #2601 )
2022-06-03 16:36:22 +02:00
HRXN
6d0aa8c0f3
[docs/configuration.rst] Fix small typo ( #2649 )
2022-06-01 18:31:39 +02:00
Mike Fährmann
e0ac358aa5
[gofile] fix 401 Unauthorized errors ( #2632 )
2022-06-01 13:02:34 +02:00
Mike Fährmann
8a42d859bf
[bunkr] change domain to 'app.bunkr.is' ( #2634 )
2022-06-01 11:30:27 +02:00
Mike Fährmann
4bf5bc2403
[weibo] support 'livephoto' entries ( #2146 )
2022-05-31 15:35:24 +02:00
Mike Fährmann
5b43faffed
[postprocessor:metadata] write to stdout by setting filename to "-"
...
(#2624 )
2022-05-30 21:17:31 +02:00
Mike Fährmann
603af48265
implement 'output.ansi' option ( #2628 )
2022-05-29 19:15:25 +02:00
Mike Fährmann
ff5e10a86d
[hypnohub] move to gelbooru_v02 instances ( #2631 )
2022-05-28 21:10:05 +02:00
Mike Fährmann
d6e744bf0f
[gfycat] add 'collection' extractor ( #2629 )
2022-05-28 16:53:27 +02:00
Mike Fährmann
d4e9d51760
[reddit] add 'home' extractor ( #2614 )
2022-05-26 15:28:33 +02:00
Mike Fährmann
915dba8345
[twitter] improve results for regular user URLs
...
- continuation of 3346f58ahttps://twitter.com/USER/tweets )
although there should be no need to ever do that
2022-05-23 18:33:52 +02:00
Mike Fährmann
9df4e0f65b
[twitter] disable 'cards' by default
2022-05-21 15:39:25 +02:00
Mike Fährmann
79dce8ae68
[weasyl] implement 'metadata' option ( #2610 )
2022-05-20 22:32:35 +02:00
Mike Fährmann
feb470d19a
[shopifx] natively support a few more sites ( closes #2089 )
...
- chelseacrew.com
- michaels.com.au
- modcloth.com
- pinupgirlclothing.com
- raidlondon.com (loveraid.com)
- unique-vintage.com
2022-05-10 15:49:36 +02:00
Mike Fährmann
60f4d59b1e
[gelbooru_v01] remove 'tlb.booru.org' from supported domains
...
403 Forbidden
nginx
it is also no longer listed on https://booru.org/top
2022-05-10 12:23:05 +02:00
Mike Fährmann
6b6eb0b8f6
[lolisafe] implement 'domain' option ( #2575 )
2022-05-10 12:17:59 +02:00
Mike Fährmann
6742f3bc1e
implement --cookies-from-browser ( #1606 )
...
most of the code is adapted from yt-dlp's implementation
and *should* work the same.
2022-05-07 23:06:37 +02:00
Mike Fährmann
61887c895b
implement 'output.colors' options ( #2532 )
2022-05-02 12:41:14 +02:00
Mike Fährmann
52b47c3cf9
[gelbooru_v01] add 'favorite' extractor ( #2546 )
2022-05-02 11:33:28 +02:00
Mike Fährmann
84756982e9
[pixiv] implement 'include' option
...
- split 'user' extractor and its 'avatar' and 'background' options into
separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
https://www.pixiv.net/en/users/ID/avatar
as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
the same effect as before
2022-05-02 09:03:35 +02:00
Mike Fährmann
d11e2191ae
[nijie] support /history_nuita.php listings ( closes #2541 )
2022-05-02 09:03:34 +02:00
Mike Fährmann
61afa973b7
use extended format strings for archive keys
2022-04-28 20:26:12 +02:00
Federico Ravasio
0381752575
[photovogue] switch to .com, update api endpoint ( #2494 )
2022-04-27 22:37:53 +02:00
Mike Fährmann
9e6ff42a9d
[pixiv] implement 'background' option ( #623 , #1124 , #2495 )
2022-04-21 13:53:02 +02:00
Mike Fährmann
97e8a15295
[deviantart] implement 'pagination' option ( #2488 )
2022-04-18 18:08:01 +02:00
Mike Fährmann
12bd9ba33a
[readcomiconline] add 'quality' option ( #2467 )
2022-04-15 18:10:37 +02:00
Mike Fährmann
24ea1398bc
[postprocessor:ugoira] do not auto-select image2 demuxer ( #2492 )
2022-04-12 21:47:47 +02:00
Mike Fährmann
ffa1dd1d91
add 'coomer.party' to supported sites ( closes #2462 )
2022-04-06 21:27:31 +02:00
Mike Fährmann
9c5d2d7af3
[pinterest] add extractor for created pins ( #2452 )
2022-04-01 16:59:58 +02:00
Mike Fährmann
1171911dc3
[twitter] add 'syndication' option ( #2354 )
...
to fetch age-restricted content using Twitter's syndication API
2022-04-01 16:56:47 +02:00
loragja
7e545a3ae9
[gofile] add gofile.io extractor ( #2364 )
...
* Add gofile extractor
* add gofile extractor to module list
* add support for tiny monitors and ancient python versions
* seriously, f-strings are not *that* new...
* i love flake8 :)
* add 'api-token' and 'recursive' options
* add tests
2022-03-29 17:31:57 +02:00
Layerex
625f4d4cc4
[telegraph] Add telegra.ph extractor ( #2312 )
2022-03-28 19:18:13 +02:00
Mike Fährmann
5c2d953766
[postprocessor:ugoira] auto-select demuxer by default
2022-03-26 21:10:59 +01:00
Mike Fährmann
37d584a9b2
[hitomi] update metadata extraction ( fixes #2444 )
...
remove 'hitomi.metadata' option, as it is no longer necessary
to make additional HTTP requests to fetch all metadata.
2022-03-26 12:46:18 +01:00
Mike Fährmann
e718dd7b32
[postprocessor:ugoira] support setting timecodes with mkvmerge
...
by selecting the "mkvmerge" demuxer
(#1550 )
2022-03-26 12:41:55 +01:00
Mike Fährmann
71bba774da
respect 'output.private' in '-K/--list-keywords' output
2022-03-25 22:19:37 +01:00
Mike Fährmann
ba69fb669d
[kemonoparty] add 'duplicates' option ( closes #2440 )
2022-03-24 11:58:38 +01:00
Mike Fährmann
40ce505420
[postprocessor:ugoira] add 'mtime' option ( #2307 )
2022-03-21 10:02:29 +01:00
Mike Fährmann
9bd27b1b8d
[postprocessor:metadata] implement archive options ( #2421 )
...
'archive', 'archive-format', and 'archive-prefix'
2022-03-20 21:16:46 +01:00
Mike Fährmann
f8230dde43
[instagram] add 'previews' option ( #2135 )
2022-03-19 15:26:40 +01:00
Mike Fährmann
cf44aba333
[formatter] allow evaluating f-string literals
...
by starting a format string with '\fF'.
This was technically already possible with '\fE',
but this makes it a bit more convenient.
2022-03-18 13:31:01 +01:00
Mike Fährmann
58e0b17211
add note about using '\f' in shells to docs/formatting.md
...
(closes #2398 , closes #2406 )
2022-03-17 23:09:25 +01:00
lingegrino
b3d8b40690
Rename meme.museum ( #2393 )
2022-03-14 08:00:35 +01:00
Mike Fährmann
fc277fa45f
[seiga] require authentication with 'user_session' cookie ( #2372 )
...
Login with username & password would now require entering a 2FA token.
see also 7b009cc893
2022-03-11 02:10:15 +01:00
Mike Fährmann
47cf05c4ab
refactor proxy handling code ( #2357 )
...
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
- add 'downloader.proxy' option
- '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
now has the same effect as youtube-dl's '--geo-verification-proxy'
2022-03-10 23:55:35 +01:00
Mike Fährmann
acbb0f5a57
clarify 'metadata.mtime' usage and shortcomings ( #2307 )
2022-03-08 21:11:06 +01:00
Mike Fährmann
bfa5e61900
[patreon] add explicit 'image_large' file type ( #2257 )
...
to allow more control over when and if to download 'large_url' images
4fee3a0e52
2022-03-06 17:07:13 +01:00
Alice
f1cab23724
[skeb] add 'sent-requests' option ( #2322 ) ( #2330 )
...
* Update skeb.py
* Update configuration.rst
* flake8
2022-02-28 22:42:15 +01:00
Mike Fährmann
4385a34e05
[twitter] fix handling of 429 responses ( fixes #2339 )
...
Twitter doesn't return a valid JSON response for 429 errors anymore.
2022-02-28 16:42:55 +01:00
Mike Fährmann
5a50569360
[toyhouse] support 'art' listings ( #1546 , #2331 )
2022-02-27 16:22:50 +01:00
Mike Fährmann
59749555b1
[postprocessor:mtime] add 'event' option ( #2307 )
2022-02-22 23:27:40 +01:00
Mike Fährmann
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
2022-02-22 23:02:13 +01:00
Mike Fährmann
fdfdc1b614
[kissgoddess] add 'gallery' and 'model' extractors
...
(closes #1052 , #2304 )
2022-02-20 04:45:37 +01:00
Mike Fährmann
79a461a2c1
[mememuseum] add 'tag' and 'post' extractors ( closes #2264 )
2022-02-20 02:15:38 +01:00
Mike Fährmann
254a5b26e0
[twibooru] add extractors for searches, galleries, and posts
...
(#2219 )
2022-02-18 23:43:57 +01:00
Mike Fährmann
f5b2a9fcc6
add 'signals-ignore' option ( #2296 )
2022-02-13 23:00:50 +01:00
David Hoppenbrouwers
b17e2dcf93
[wallpapercave] add extractor for images ( #2205 )
2022-02-11 23:44:51 +01:00
Thomas Jost
a7de819aca
[lightroom] add Lightroom gallery extractor ( #2263 )
2022-02-11 21:30:59 +01:00
Mike Fährmann
563bd0ecf4
[danbooru] inherit from BaseExtractor
...
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283 )
- remove support for old e621 tag search URLs
2022-02-11 21:01:51 +01:00
Mike Fährmann
f1c853c6ef
[furaffinity] add 'layout' option ( #2277 )
...
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
2022-02-11 00:28:47 +01:00
Mike Fährmann
4e4cb40663
do not sort BaseExtractor instances by name
2022-02-10 01:44:16 +01:00
Mike Fährmann
f351746483
release version 1.20.4
2022-02-06 22:02:36 +01:00
HRXN
f4a7c6eca2
Add two examples for custom instances of gelbooru ( #2262 )
...
* Add two examples for custom instances of gelbooru
- First example uses some custom options at the "gelbooru_v01" basecategory
These apply to all instances used herein
- Second example adds all options to the specific "gelbooru_v02" basecategory of TBIB
These settings are only used by TBIB
* move tbib settings into their own block
2022-02-04 23:38:22 +01:00
Mike Fährmann
cc7dce5755
[sexcom] add 'pins' extractor ( closes #2265 )
2022-02-04 20:55:00 +01:00
Mike Fährmann
02e18f56be
[e621] add 'favorite' extractor ( closes #2250 )
2022-02-04 20:54:48 +01:00
Mike Fährmann
86fa412b47
[hitomi] add 'format' option ( #2260 )
...
default is 'webp' since downloading original files is no longer allowed
2022-02-03 23:32:19 +01:00
Mike Fährmann
492436f936
[twitter] add 'warnings' option ( #2258 )
...
disable reporting any non-fatal errors by default
2022-02-02 18:37:19 +01:00
Mike Fährmann
9a221494c3
[twitter] add 'event' extractor ( closes #2109 )
2022-01-22 20:55:50 +01:00
Mike Fährmann
de754590e0
add --source-address command-line option ( closes #2206 )
2022-01-21 17:07:56 +01:00
Mike Fährmann
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
2022-01-15 22:02:57 +01:00
ImportTaste
c559c49781
formatting.md: add environment variable syntax ( #2065 )
2022-01-13 17:47:19 +01:00
Mike Fährmann
3c79c9b271
document extended blacklist/whitelist syntax ( #2025 )
...
and not just in the commit message of 010d65dc
2022-01-06 23:36:57 +01:00
Mike Fährmann
37beb1298e
[newgrounds] add 'search' extractor ( closes #2161 )
2022-01-06 19:32:39 +01:00
Mike Fährmann
fa7d92f7a9
add docs for 'extractor.generic.enabled'
2021-12-29 22:46:17 +01:00
Mike Fährmann
89bebe1bef
[500px] add 'favorite' extractor ( closes #1927 )
2021-12-25 17:16:45 +01:00
Mike Fährmann
882c614281
add album extractor for lolisafe/chibisafe instances
...
- support bunkr.is (closes #2038 )
- support zz.ht (closes #2105 )
2021-12-21 19:24:17 +01:00
Mike Fährmann
64cf26eaf4
allow specifying sleep-* options as string
...
either as single value or as range: "3.5", "2.1 - 5.0"
2021-12-18 23:28:56 +01:00
Mike Fährmann
62692c6842
[exhentai] add 'source' option
...
setting it to "hitomi" downloads the corresponding gallery from
hitomi.la; might be extended to other sources in the future
2021-12-16 23:16:19 +01:00
Mike Fährmann
099ed72de7
[hitomi] disable extra 'metadata' by default
...
safes one HTTP request that not needed with default filename settings
2021-12-16 22:21:07 +01:00
Mike Fährmann
f587458a3c
[twitter] include '4096x4096' as a default image fallback
...
(closes #2107 , closes #1881 )
2021-12-15 23:19:30 +01:00
Mike Fährmann
299bd2f1f5
[rule34us] add 'tag' and 'post' extractors ( #1527 )
2021-12-14 00:27:46 +01:00
Mike Fährmann
3cf1075d86
[inkbunny] add 'search' extractor ( closes #2094 )
2021-12-12 03:08:14 +01:00
Mike Fährmann
f4e3cee6ac
use yt-dlp by default ( #1850 , #2028 )
2021-11-29 18:24:26 +01:00
Mike Fährmann
f1b142e993
{kemonoparty[ change default 'files' order to attachments,file,inline
...
(#1991 )
2021-11-29 04:41:30 +01:00
Mike Fährmann
addb72e1bb
[reactor] support thatpervert.com ( closes #2029 )
2021-11-26 18:58:07 +01:00
Mike Fährmann
d8d9502e1e
[reactor] inherit from BaseExtractor
2021-11-26 18:58:07 +01:00
Mike Fährmann
f4ea216c95
[shopify] support loungeunderwear.com ( closes #2053 )
2021-11-26 18:58:06 +01:00
Alice
612850438e
[skeb] add 'thumbnails' option ( #2047 ) ( #2051 )
2021-11-23 21:16:42 +01:00
Mike Fährmann
11a3d96d13
[mangadex] load additional metadata using includes[] directives
...
- always provide 'artist', 'author', and 'group' metadata fields (#2049 )
- remove 'metadata' option
2021-11-22 01:16:33 +01:00
Mike Fährmann
c67756e187
[kemonoparty] add 'dms' option ( #2008 )
2021-11-20 23:36:16 +01:00
Mike Fährmann
3842cdcd8f
[formatter] implement 'D' format specifier
...
To be able to parse any string into a 'datetime' object
and format it as necessary.
Example:
{created_at:D%Y-%m-%dT%H:%M:%S%z}
->
"2010-01-01 00:00:00"
{created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p}
->
"Jan 01 2010 12:00 AM"
with 'created_at' == "2010-01-01T01:00:00+0100"
2021-11-20 23:04:34 +01:00
Mike Fährmann
3a7a19c7b9
[dynastyscans] add 'manga' extractor ( closes #2035 )
2021-11-19 22:51:26 +01:00
Mike Fährmann
9bc83af3a6
[kemonoparty] 'postfile' -> 'file' ( #1991 )
...
to stay consistent with the existing file types for kemono
2021-11-19 01:50:48 +01:00
Mike Fährmann
d433735750
[kemonoparty] skip duplicate files ( #2032 , #1991 , #1899 )
...
Extract the SHA-256 file hash from URLs
and skip files with the same hash in the same post.
- provide a 'hash' metadata field (empty string if not available)
- remove 'patreon-skip-file' option
2021-11-17 22:44:15 +01:00
Mike Fährmann
d4ec245554
[kemonoparty] implement a 'files' option ( #1991 )
...
similar to 8d676151
2021-11-17 22:43:41 +01:00
Mike Fährmann
cfa4876848
[philomena] support furbooru.org ( closes #1995 )
2021-11-15 20:57:51 +01:00
Mike Fährmann
7c50f473e4
[ytdl] document 'cmdline-args' and 'config-file' options ( #1680 )
2021-11-12 18:54:04 +01:00
Mike Fährmann
37c9dedee1
[seisoparty] remove module
2021-11-09 22:41:04 +01:00
Mike Fährmann
b315a0ecef
[redgifs] update to API v2 ( #1984 )
2021-11-04 21:31:20 +01:00
Mike Fährmann
f0fc3b0ba1
[kemonoparty] add 'comments' option ( #1980 )
2021-11-03 23:02:13 +01:00
Mike Fährmann
4c49174579
[mangakakalot] update domain and fix extraction
2021-10-31 02:16:13 +01:00
ImportTaste
69cd573809
add special type format strings to docs ( #1987 )
2021-10-30 21:46:26 +02:00
Mike Fährmann
9156e90f1f
[twitter] add 'pinned' option
2021-10-29 22:10:58 +02:00
Mike Fährmann
06b414c9a3
[redgifs] 'gfyId' -> 'id' ( #1984 )
2021-10-29 02:05:39 +02:00
Alice
bfd7401b1e
[skeb] add 'user' and 'post' extractors ( #1031 ) ( #1971 )
...
* Create skeb.py
* Update __init__.py
* Update supportedsites.py
* Update supportedsites.md
* Update supportedsites.py
* Update skeb.py
2021-10-26 20:00:41 +02:00
Mike Fährmann
02a247f4e5
[deviantart] full resolution for non-downloadable images ( #293 )
...
Many thanks to @Ironchest337 for discovering this method
and providing a well-documented implementation.
2021-10-24 21:11:12 +02:00
Mike Fährmann
c19e762fdf
[vk] add 'album' extractor ( #474 , fixes #1952 )
...
todo: better metadata for albums
2021-10-23 00:46:20 +02:00
Mike Fährmann
8bb442f20d
[redgifs][gfycat] provide fallback URLs ( fixes #1962 )
...
and extend the 'format' option
2021-10-22 22:47:29 +02:00
Mike Fährmann
8d676151b7
[patreon] implement 'files' option ( #1935 )
2021-10-17 04:17:10 +02:00
Mike Fährmann
80314e981b
small corrections to gallery-dl-example.conf
...
use "'_reddit' in locals()"
instead of ""locals().get('_reddit')"
since the reddit metadata dict could be empty. The latter expression
would evaluate to False in that case, even though it has a reddit parent.
2021-10-16 20:50:39 +02:00
Mike Fährmann
3ecf4a9729
update gallery-dl-example.conf
...
- add comments
- remove/replace some older options
- display more post processor settings
2021-10-16 17:35:56 +02:00
Mike Fährmann
50098762e3
[nhentai] add 'tag' extractor ( closes #1950 )
2021-10-14 16:23:47 +02:00
Mike Fährmann
fe6ce5495a
[kemonoparty] add 'discord' extractor ( #1827 , #1940 )
2021-10-13 20:33:05 +02:00
Mike Fährmann
918fc9974d
[picarto] add 'gallery' extractor ( closes #1931 )
2021-10-13 01:22:10 +02:00
Mike Fährmann
e33125ad39
[pixiv] add 'sketch' extractor ( #1497 )
2021-10-13 00:02:11 +02:00
Mike Fährmann
b93915c113
[inkbunny] add 'pool' extractor ( #1937 )
2021-10-09 22:07:07 +02:00
Mike Fährmann
373d3e1c57
[seisoparty] implement login with username & password ( #1906 )
2021-10-08 22:44:31 +02:00
Mike Fährmann
7c5f62b453
[seisoparty] add 'favorite' extractor ( #1906 )
2021-10-08 22:26:34 +02:00
Mike Fährmann
d93b5474c3
[mangadex] update parameter handling for API requests
...
- move common parameters into '_pagination()'
- add 'ratings' (#1908 ) and 'api-parameters' options
2021-10-05 19:46:48 +02:00
Mike Fährmann
cd66c3c415
[twitter] add 'size' option ( #1881 )
2021-10-05 19:14:54 +02:00
Mike Fährmann
df8050b81d
[postprocessor:compare] add 'equal' option ( #1592 )
...
Move functionality from cdd72e14
2021-10-05 03:03:37 +02:00
Mike Fährmann
cdd72e1413
[postprocessor:compare] extend 'action' option ( #1592 )
...
allow setting it to "abort", "terminate", or "exit" as with 'skip'
2021-10-01 20:18:36 +02:00
Mike Fährmann
e0bdacd932
[fappic] add 'image' extractor ( closes #1898 )
2021-09-28 23:35:29 +02:00
Mike Fährmann
8c29a6e491
add example for 'url-metadata' option ( closes #1884 )
2021-09-28 22:59:37 +02:00
Mike Fährmann
d0761454b1
implement a download progress indicator ( #1519 )
2021-09-28 22:48:58 +02:00
Mike Fährmann
140a7bfdcd
update --clear-cache command in deviantart client-id instructions
2021-09-23 18:02:19 +02:00
Mike Fährmann
e69ee41f25
implement 'page-reverse' option ( #1854 )
2021-09-23 18:02:19 +02:00
Mike Fährmann
e4684c5cb9
[desktopography] simplify ( #1740 )
2021-09-17 20:09:24 +02:00
Mike Fährmann
7b05600056
[pinterest] only mention cookies for authentication ( #1856 )
...
regular login is broken due to CAPTCHA
2021-09-16 16:43:05 +02:00
Mike Fährmann
266ed9b62e
[aryion] add 'tag' extractor ( closes #1849 )
2021-09-14 23:33:33 +02:00
Mike Fährmann
c9e6693530
allow specifying a minimum/maximum for 'sleep-*' options ( #1835 )
...
for example '"sleep-request": [5.0, 10.0]' to wait between 5 and 10
seconds between each HTTP request
2021-09-14 17:40:05 +02:00
Mike Fährmann
bd845303ad
implement a way to shorten filenames with east-asian characters
...
(#1377 )
Setting 'output.shorten' to "eaw" (East-Asian Width) uses a slower
algorithm that also considers characters with a width > 1.
2021-09-13 21:38:33 +02:00
Mike Fährmann
7bbb1f92d7
[gelbooru_v02] add 'favorite' extractor ( closes #1834 )
2021-09-10 20:43:59 +02:00
Mike Fährmann
4ec11af6a4
[kemonoparty] implement login with username & password ( #1824 )
2021-09-09 01:06:25 +02:00
Mike Fährmann
83bbb628d8
[kemonoparty] add 'favorite' extractor ( #1824 )
2021-09-08 00:32:49 +02:00
Mike Fährmann
44f572c27f
[deviantart] implement a 'auto-unwatch' option ( #1466 , #1757 )
2021-09-08 00:27:28 +02:00
Mike Fährmann
268c9ca034
update default gallery-dl.conf
2021-09-03 19:37:12 +02:00
Mike Fährmann
3e36543c98
[nhentai] add 'favorite' extractor ( #1814 )
2021-09-02 18:26:22 +02:00
Mike Fährmann
96fec14ef7
[deviantart] rename 'watch' option to 'auto-watch'
...
(#1466 , #1757 )
Similar reason as in e05a96db
2021-08-29 21:10:53 +02:00
Mike Fährmann
e75f2de9da
[deviantart] add 'comments' option ( #1800 )
2021-08-29 20:56:34 +02:00
Mike Fährmann
6ce16c6d31
[deviantart] add 'tag' extractor ( closes #1803 )
2021-08-28 23:42:10 +02:00
Mike Fährmann
4e9f8fe395
[shopify] support windsorstore.com ( #1793 )
2021-08-28 18:34:12 +02:00
Mike Fährmann
292fffc83c
add 'j' format string conversion
...
to convert to a JSON formatted string
2021-08-28 01:19:36 +02:00
Mike Fährmann
ecc8da4704
[deviantart] implement a 'watch' option ( #1466 , #1757 )
2021-08-27 18:39:43 +02:00
Mike Fährmann
d3eab417ed
implement a 'path-strip' option
2021-08-24 23:23:12 +02:00
Mike Fährmann
c04f7ab139
[foolfuuka] add 'gallery' extractor ( #1785 )
2021-08-21 22:46:23 +02:00
Mike Fährmann
20ee091289
[429chan] add 'thread' and 'board' extractors ( closes #1773 )
2021-08-21 22:46:22 +02:00
Mike Fährmann
51eb50749f
[foolslide] remove entry for kobato.hologfx.com
2021-08-21 22:46:22 +02:00
Mike Fährmann
d320ee6251
implement a 'fallback' option ( closes #1770 )
2021-08-16 01:47:59 +02:00
Mike Fährmann
c866fcba48
[twitter] fix 'logout' ( #1719 )
...
delete 'auth_token' cookie and cookies.txt path
2021-08-16 01:36:34 +02:00
Mike Fährmann
ed4b3c48cb
fix flake8 and other tests
2021-08-12 16:05:26 +02:00
Nyasume
fa6af46756
Added ability to download GIFs instead of mp4 from Luscious and Reactor ( #1701 )
2021-08-12 15:12:42 +02:00
Mike Fährmann
e5a93e113f
[twitter] extend 'replies' option ( #1254 )
...
Allow setting 'replies to '"self"' to only download from self-replies.
2021-08-10 22:14:00 +02:00
Mike Fährmann
cadfad4eea
[danbooru] add 'external' option ( closes #1747 )
2021-08-08 19:18:41 +02:00
Mike Fährmann
5b1c62bfa9
[furaffinity] add 'external' option ( closes #1492 )
2021-08-08 19:16:02 +02:00
Mike Fährmann
a316e44f8e
[bbc] add 'width' option ( #1706 )
2021-07-30 01:09:32 +02:00
Mike Fährmann
7a0da4f93f
[newgrounds] add 'format' option ( closes #1729 )
2021-07-29 19:11:20 +02:00
Mike Fährmann
486474800f
[kemonoparty] skip duplicated patreon files ( closes #1689 )
...
this behavior can be disabled with the 'patreon-skip-file' option
2021-07-23 16:38:57 +02:00
Mike Fährmann
da7297c0b9
[comicvine] add extractor ( closes #1712 )
2021-07-23 16:17:06 +02:00
Mike Fährmann
e4788fa663
[bbc] add 'gallery' and 'programme' extractors ( closes #1706 )
2021-07-22 20:37:05 +02:00
Mike Fährmann
13d4045a8a
add 'archive-prefix' option ( #1711 )
2021-07-20 20:21:33 +02:00
Mike Fährmann
74b457b2b5
remove unused entries
2021-07-20 20:21:32 +02:00
Mike Fährmann
3868ec02d1
[pururin] update domain and fix extraction
2021-07-18 04:11:59 +02:00
Mike Fährmann
b9783403d9
add 'url-metadata' option ( #1659 , #1073 )
2021-07-14 03:08:49 +02:00
Mike Fährmann
e95f99882f
extend 'parent-metadata' functionality ( #1687 , #1651 , #1364 )
2021-07-14 02:53:41 +02:00
Mike Fährmann
dff0da60f9
[ytdl] add 'generic' option ( #1680 )
2021-07-11 23:48:18 +02:00
Mike Fährmann
36ac2197db
[ytdl] add extractor for sites supported by youtube-dl
...
(#1680 , #878 )
Can be used by prefixing any URL with 'ytdl:',
or by setting 'extractor,ytdl.enabled' to 'true'.
2021-07-10 20:55:47 +02:00
Mike Fährmann
8b036778e3
[kemonoparty] add 'max-posts' option ( #1674 )
2021-07-09 18:19:02 +02:00
Mike Fährmann
8ecca3af58
[pixiv] add extractor for 'pixivision' articles ( #1672 )
2021-07-07 16:27:16 +02:00
Mike Fährmann
312a28e78a
[mastodon] add 'replies' option ( #1669 )
2021-07-07 00:59:02 +02:00
Mike Fährmann
a1f5b78039
[mastodon] add 'reblogs' option ( #1669 )
2021-07-06 23:27:32 +02:00
Mike Fährmann
317ecc8180
use HTML tables in docs/formatting.md
2021-07-05 23:29:03 +02:00
Mike Fährmann
7f591c78cb
[mangafox] cleanup
2021-07-04 03:21:02 +02:00
Mike Fährmann
93d356712c
[mastodon] implement 'text-posts' option ( #1569 )
...
similar to Twitter's 'text-tweets'
2021-07-02 22:12:41 +02:00
Mike Fährmann
963d177a68
document format string syntax
...
or at least attempt to
2021-06-29 19:35:07 +02:00
Mike Fährmann
267bbf5996
[mangasee] add 'chapter' and 'manga' extractors
2021-06-27 02:03:03 +02:00
Mike Fährmann
f74cf52e2b
[seisoparty] add 'user' and 'post' extractors ( #1635 )
2021-06-25 18:40:11 +02:00
Mike Fährmann
e661607e8b
[mangadex] document 'metadata' and 'lang' options ( #1535 )
2021-06-20 22:44:02 +02:00
Mike Fährmann
fd00d47116
implement conditional directories ( #1394 )
...
They work the same way as conditional filenames (84d2e640
2021-06-20 20:09:35 +02:00
Mike Fährmann
a751afdfb3
[twitter] change some defaults
...
- 'retweets' option: true -> false
- 'quoted' option : true -> false
i.e. disable downloading tweets from other user's timelines by default
- search directory:
'["{category}", "Search", "{search}"]' ->
'["{category}", "{user[name]}"]'
i.e. change it to the same as other twitter extractors (#1308 )
2021-06-11 21:26:11 +02:00
Mike Fährmann
84d2e64024
combine conditional filenames into filename option ( #1394 )
2021-06-08 18:00:06 +02:00
Mike Fährmann
1197ee2c20
[mangadex] add extractor for a user's followed feed ( #1535 )
2021-06-08 02:45:36 +02:00
Mike Fährmann
07c8adbd8b
[mangadex] implement login with username & password ( #1535 )
2021-06-08 02:12:57 +02:00
Mike Fährmann
3cbbefd4ed
support 'filter' option for post processors ( #1460 )
2021-06-04 18:23:32 +02:00
Mike Fährmann
4cf40434d7
initial support for conditional filenames ( #1394 )
2021-06-04 16:45:32 +02:00
Mike Fährmann
f3d692a378
improve docs for danbooru and e621 passwords ( #1593 , #1596 )
2021-06-02 23:02:23 +02:00
Mike Fährmann
407627ec86
[foolfuuka] support 'archive.wakarimasen.moe' ( closes #1595 )
2021-06-02 15:45:43 +02:00
Mike Fährmann
52052a0e1a
[manganelo] update domain to 'manganato.com'
2021-05-28 17:59:21 +02:00
Mike Fährmann
c80b18a477
[weibo] extend 'retweets' option ( closes #1542 )
...
Setting 'retweets' to "original" will use metadata from the
original posts, and not from the retweeted ones.
2021-05-27 23:09:42 +02:00
Mike Fährmann
74d97e6c6b
[postprocessor:ugoira] add 'repeat-last-frame' option ( #1550 )
2021-05-26 15:13:11 +02:00
Mike Fährmann
be9547a5f3
[postprocessor:ugoira] use 'image2' demuxer ( #1550 )
...
Ensures exact frame timecodes with no duplicate frames.
Possible issues are the duration the last frame in an Ugoira with variable
frame durations is shown and insufficient timestamp precision of the
underlying file system (e.g. FAT32, ext3; works on ext4, tmpfs, NTFS).
2021-05-26 14:59:51 +02:00
