Mike Fährmann
5f9a98cf0f
[deviantart:avatar] fix exception when 'comments' are enabled ( #4995 )
2024-01-09 20:04:46 +01:00
Mike Fährmann
887ade30a5
[batoto] support more mirror domains ( #5042 )
2024-01-09 18:02:49 +01:00
Mike Fährmann
0a382a5092
[batoto] improve 'manga_id' extraction ( #5042 )
2024-01-09 17:25:04 +01:00
blankie
100966b122
[steamgriddb] fix linting error
2024-01-09 17:18:31 +11:00
blankie
2ccb7d3bd3
[steamgriddb] add support
2024-01-09 17:12:56 +11:00
Mike Fährmann
ec958a26bc
[fuskator] make metadata extraction non-fatal ( #5039 )
...
- prevent KeyErrors
- prevent HTTP redirect
- return file URLs as list
2024-01-08 19:18:12 +01:00
blankie
2cfe788f93
[hatenablog] fix extractor naming errors
2024-01-09 01:42:57 +11:00
blankie
be6949c55d
[hatenablog] fix linting error
2024-01-09 01:36:52 +11:00
blankie
61f3b2f820
[hatenablog] add support
2024-01-09 01:29:47 +11:00
Mike Fährmann
657ed93a22
[batoto] improve v2 manga URL pattern
...
and add tests
2024-01-07 22:23:30 +01:00
Mike Fährmann
50eef1b5cc
merge #5029 : [pixiv] update App API headers
2024-01-07 16:27:13 +01:00
Mike Fährmann
33f228756a
[mangadex] add 'list' extractor ( #5025 )
...
supports listing manga and chapters from list feed
2024-01-07 02:59:35 +01:00
Mike Fährmann
db8de13537
[vk] transform image URLs to non-blurred versions ( #5017 )
...
apply the same filter from before d85e66bc
2024-01-07 00:19:10 +01:00
Se AKi
d0d199414f
modify useragent of pixiv
2024-01-06 23:15:15 +09:00
Mike Fährmann
cbfb7bfdf1
[gelbooru] display error for invalid API responses ( #4903 )
2024-01-06 14:28:35 +01:00
Mike Fährmann
c25bdbae91
[komikcast] fix 'manga' extractor ( #5027 )
2024-01-06 14:19:44 +01:00
Mike Fährmann
8e1a2b5446
[komikcast] update domain to 'komikcast.lol' ( #5027 )
2024-01-06 02:16:43 +01:00
Mike Fährmann
a441249ea2
merge #4979 : [batoto] add 'chapter' and 'manga' extractors ( #1434 , #2111 )
2024-01-06 01:53:26 +01:00
Mike Fährmann
b11c352d66
[bato] rename to 'batoto'
...
to use the same category name as the previous bato.to site
2024-01-06 01:49:34 +01:00
Mike Fährmann
3aa24c3744
[bato] simplify and update
2024-01-06 01:10:04 +01:00
Mike Fährmann
11150a7d72
[nudecollect] remove module
2024-01-05 21:32:04 +01:00
Mike Fährmann
c158927c38
merge #5016 : [zzup] add 'gallery' extractor ( #4517 , #4604 , #4659 , #4863 )
2024-01-05 21:25:46 +01:00
Mike Fährmann
e61f016465
[szurubooru] support 'snootbooru.com' ( #5023 )
2024-01-05 17:56:39 +01:00
Mike Fährmann
b4bcf40278
[weibo] fix AttributeError in 'user' extractor ( #5022 )
...
yet another bug caused by a383eca7
2024-01-05 17:18:33 +01:00
Mike Fährmann
0ab0a10d2d
[jpgfish] update domain
2024-01-05 02:27:20 +01:00
enduser420
0f30136109
[zzup] add 'gallery' extractor
2024-01-04 21:38:59 +05:30
Mike Fährmann
a86775f617
[gelbooru] fix 'favorite' extractor ( #4903 )
...
lots of +1/-1 and </<= mistakes
2024-01-04 15:17:49 +01:00
Mike Fährmann
7eaf648f2e
[fanbox] add 'metadata' option ( #4921 )
...
extracts 'plan' and extended 'user' metadata
2024-01-04 15:01:33 +01:00
Mike Fährmann
4f3671458e
[deviantart] add 'avatar' and 'background' extractors ( #4995 )
2024-01-03 00:07:55 +01:00
Mike Fährmann
9fa4f54c24
[twitter] raise error for invalid 'strategy' values ( #4953 )
2024-01-01 22:58:42 +01:00
Mike Fährmann
516c69297d
[manganelo] fix extraction & recognize '.to' TLDs ( #5005 )
2024-01-01 22:05:21 +01:00
Mike Fährmann
63f649cd92
[idolcomplex] fix extraction & update URL patterns ( #5002 )
2024-01-01 17:38:32 +01:00
Mike Fährmann
b6903a4c90
[nijie] add 'count' metadata field
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1812849102
2023-12-30 22:25:59 +01:00
Mike Fährmann
b93b351db9
merge #4962 : [poringa] add support ( #4675 )
2023-12-30 20:39:35 +01:00
Mike Fährmann
9f21c839ad
[poringa] improvements and fixes
...
- add 'num' and 'count' metadata fields
- prevent crash for "private" posts
- prevent crash when there's no 'main-info'
- update tests
2023-12-30 20:37:09 +01:00
Mike Fährmann
00d83d9588
[rule34us] add fallback for 'video-cdn1' videos ( #4985 )
2023-12-30 18:33:46 +01:00
Mike Fährmann
085411f3f1
[rule34] recognize URLs with 'www' subdomain ( #4984 )
2023-12-30 16:07:56 +01:00
Mike Fährmann
9f5051e4ed
merge #4981 : [pinterest] add 'count' metadata field
2023-12-30 15:53:35 +01:00
bug-assassin
f6ce870885
Better variable names
2023-12-28 17:25:15 -05:00
bug-assassin
3553025584
Removed f-strings
2023-12-28 17:07:41 -05:00
Mike Fährmann
f36dafad06
improve 'include' handling ( #4982 )
...
- remove spaces when given as string
- warn about invalid vales
2023-12-28 19:07:04 +01:00
blankie
375f2db4c2
[pinterest] add count metadata field
2023-12-28 01:07:04 +11:00
Antonio
e348da7a06
[poringa] add support
2023-12-27 00:07:23 -06:00
bug-assassin
2c3f171d65
Fix python 3.5 linting issue
2023-12-26 23:55:47 -05:00
bug-assassin
06ff1d3a3c
Replace text.extract with extr
2023-12-26 23:47:30 -05:00
bug-assassin
9c1ce28f68
[bato] Added mangatoto alias
2023-12-26 23:44:27 -05:00
bug-assassin
663b8d789a
Fix linting
2023-12-26 23:41:37 -05:00
bug-assassin
74c225f94e
[bato] add support
2023-12-26 22:33:33 -05:00
Mike Fährmann
f9544194c0
[paheal] restore 'extension' metadata ( #4976 )
2023-12-26 16:09:26 +01:00
Mike Fährmann
77d46e6f0c
[lynxchan] update 'bbw-chan' domain ( #4970 )
2023-12-25 15:29:05 +01:00
Mike Fährmann
766316e436
[imagechest] fix loading more than 10 images in a gallery ( #4469 )
2023-12-25 15:14:43 +01:00
Mike Fährmann
108c978073
merge #4919 : [postmill] add support ( #4917 )
2023-12-23 15:23:56 +01:00
blankie
8a42ea736a
[postmill] implement suggestions
2023-12-23 13:28:36 +11:00
Mike Fährmann
c184454efb
[shimmie2] small optimizations
...
- unroll/remove loop
- avoid copy
2023-12-22 20:29:22 +01:00
Mike Fährmann
7cd0211cc9
[shimmie2] autodetect single or double quotes
2023-12-22 15:25:28 +01:00
Mike Fährmann
2a60645095
[deviantart] set 'is_original' for intermediary URLs to 'false'
2023-12-22 14:49:10 +01:00
Mike Fährmann
01bb75f6cb
merge #4945 : {shimmie2[ support 'rule34hentai.net' ( #861 , #4789 )
2023-12-22 00:10:26 +01:00
Mike Fährmann
79e4606893
[rule34hentai] cleanup
...
- fix using 'self._posts_rule34hentai'
- fix 'file_url' for posts
- update docs/supportedsites
- add tests
2023-12-22 00:01:36 +01:00
bun-dev
ef370df41d
[shimmie2] support 'rule34hentai.net'
...
- Add files via upload
- Update shimmie2.py
- Update shimme2.py
- Delete gallery_dl/extractor/shimme2.py
- spacefix
- Update shimmie2.py
- Update shimmie2.py
- flask warnings1
- Update shimmie2.py
- Update shimmie2.py
2023-12-21 23:00:58 +01:00
Mike Fährmann
627ed794a2
[danbooru] provide 'tags' as list ( #4942 )
...
keep the old 'tag_string' values around, similar to sankaku
a lot of repeat code ...
would be a lot less bad if "".split(" ") returned an empty list
2023-12-21 14:39:38 +01:00
Mike Fährmann
fbebc58189
[deviantart] add 'intermediary' option ( #4955 )
2023-12-21 02:23:22 +01:00
Mike Fährmann
75fa1a5553
[pinterest] remove login code
...
this has been broken since forever
and is still "protected" by an invisible recaptcha check
2023-12-20 20:59:18 +01:00
Mike Fährmann
92ff99c8e5
[twitter] remove 'syndication' option ( #3889 )
2023-12-20 14:38:36 +01:00
Mike Fährmann
a75f85a2c2
[twitter] remove 'date_liked' ( #3850 , #4108 , #4657 )
...
Twitter's 'sortIndex' can't be used to calculate the timestamp
of when a Tweet was liked anymore.
2023-12-20 14:12:49 +01:00
Mike Fährmann
a94f944148
[twitter] default to 'tweets' timeline when 'replies' are enabled ( #4953 )
2023-12-20 01:57:18 +01:00
Mike Fährmann
a30a3e44d5
[nijie] move 'username required' out of _login_impl
2023-12-18 23:57:44 +01:00
Mike Fährmann
57fc6fcf83
replace '24*3600' with '86400'
...
and generalize cache maxage values
2023-12-18 23:57:22 +01:00
Mike Fährmann
1f9b16a70b
replace static 'sleep-request' defaults with dynamic ones
2023-12-18 22:06:26 +01:00
Mike Fährmann
b127321b5c
[exhentai] only show 'using e-hentai.org' warning for exh domains
2023-12-18 15:29:54 +01:00
Mike Fährmann
e097aaf64a
[exhentai] output continuation URL when interrupted ( #4782 )
2023-12-17 23:25:47 +01:00
Mike Fährmann
99aa923322
[inkbunny] improve '/submissionsviewall.php' patterns ( #4934 )
...
allow 'mode=…' to be in any position
don't require it to be somewhere in the middle
2023-12-16 19:21:20 +01:00
Mike Fährmann
3f9c113d78
[mastodon] Support non-numeric status IDs ( #4936 )
2023-12-16 01:52:31 +01:00
Mike Fährmann
2852404e49
[inkbunny] add 'unread' extractor ( #4934 )
2023-12-15 21:20:12 +01:00
Mike Fährmann
8b87a5330d
[inkbunny] stop pagination on empty results
2023-12-15 21:15:02 +01:00
Mike Fährmann
6cd5e6adad
[patreon] fix bootstrap data extraction ( #4904 )
2023-12-15 18:24:52 +01:00
Mike Fährmann
aac8bb4eae
[deviantart] simplify 9951c112
2023-12-15 17:39:42 +01:00
Mike Fährmann
9951c112f8
[deviantart] workaround for integer client_id values ( #4924 )
2023-12-14 16:25:06 +01:00
Mike Fährmann
a37b7759bc
[myhentaigallery] recognize '/g/' URLs ( #4920 )
2023-12-12 20:02:28 +01:00
Mike Fährmann
da76e13e3b
[tumblr] fix exception after waiting for rate limit ( #4916 )
...
use a loop instead of recursive function calls
2023-12-12 19:14:06 +01:00
blankie
fbe14a2745
[postmill] add support
2023-12-12 21:36:52 +11:00
Mike Fährmann
d59d4ebff4
[tumblr] support infinite 'fallback-retries'
2023-12-11 23:40:13 +01:00
Mike Fährmann
2d5cda2b92
[exhentai] fix TypeError for infinite 'fallback-retries' ( #4911 )
2023-12-11 23:39:56 +01:00
Mike Fährmann
92fbf09643
remove single quotes in some logging messages ( #4908 )
...
('FileNotFoundError: [Errno 2] No such file or directory: ''')
->
(FileNotFoundError: [Errno 2] No such file or directory: '')
2023-12-11 19:13:45 +01:00
Luc Ritchie
7dd79eee93
save cookies to tempfile, then rename
...
avoids wiping the cookies file if the disk is full
2023-12-11 00:47:42 -05:00
Mike Fährmann
9a001fa6e4
merge #4906 : [patreon] fix bootstrap data extraction ( #4904 )
2023-12-10 16:53:54 +01:00
Tobi823
66cbe9da41
- fix style check failure "line to long"
2023-12-10 16:24:00 +01:00
Tobi823
244444b194
- adapt code to current code style
2023-12-10 16:22:32 +01:00
Tobi823
fd06255f93
- reformat and refactor to pass tests
2023-12-10 16:17:34 +01:00
Tobi823
5ff7106d4f
- add code for the situation when Patreon is using window.patreon = wrapInProxy({"bootstrap":' to store metadata
...
- refactor code to make it more readable
- output page content when the HTML structure is unknown (to make debugging easier)
2023-12-10 16:10:46 +01:00
Mike Fährmann
ac22bbe80c
[twitter] retry API requests only for Timeout errors ( #4811 )
2023-12-09 22:30:48 +01:00
Mike Fährmann
c55955db03
[twitter] quick and dirty fix for /media changes ( #4898 )
2023-12-09 15:38:42 +01:00
Mike Fährmann
9a8dc6b02b
[exhentai] add 'fallback-retries' option ( #4792 )
2023-12-09 02:01:59 +01:00
Mike Fährmann
bf74eb5c46
merge #4886 : [urlgalleries] add 'gallery' extractor ( #919 , #1184 , #2905 )
2023-12-08 22:55:58 +01:00
Mike Fährmann
c29ae9af08
[urlgalleries] simplify + resolve redirects
2023-12-08 22:43:56 +01:00
Mike Fährmann
e256434c9e
use custom HTTPBasicAuth class
...
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
2023-12-07 01:40:31 +01:00
Mike Fährmann
6a4218aa23
handle 'json' parameter in Extractor.request() manually
...
Mainly to allow passing custom classes like util.LazyPrompt,
but also to simplify and streamline how requests handles it.
2023-12-06 22:13:13 +01:00
Mike Fährmann
9dd5cb8c8a
interactively prompt for passwords on login when none is provided
2023-12-06 22:12:59 +01:00
Mike Fährmann
4eb3590103
[nijie] fix image URLs of multi-image posts ( #4876 )
2023-12-05 17:48:50 +01:00
Mike Fährmann
a4e6ea667b
[twitter] retry API calls when their response contains errors ( #4811 )
2023-12-05 15:57:26 +01:00
Mike Fährmann
cf5702c843
[twitter] generalize "Login Required" error ( #4734 , #4324 )
2023-12-05 15:13:58 +01:00
jsouthgb
ecaa0feb5d
[urlgalleries] add support
2023-12-05 07:08:11 -05:00
jsouthgb
1770c31e63
[urlgalleries] add support
2023-12-05 07:07:06 -05:00
Mike Fährmann
da0da0faaa
[exhentai] store more cookies when logging in ( #4881 )
...
include 'igneous', 'hath_perks', etc
and not just 'ipb_member_id' and 'ipb_pass_hash' like before
2023-12-04 23:24:11 +01:00
Mike Fährmann
c83fbe6c2d
merge #4855 : [nitter] fix video extraction ( #4853 )
2023-11-27 18:39:05 +01:00
Mike Fährmann
013ca21543
[idolcomplex] update to site layout changes
2023-11-27 18:27:08 +01:00
enduser420
1e9bacd169
[nitter] fix video extraction
2023-11-27 21:58:06 +05:30
Mike Fährmann
9f3368c46f
[pornhub] fix 'user' metadata for gifs
2023-11-26 23:52:24 +01:00
Mike Fährmann
bdb3ce7217
[foolslide] remove 'powermanga.org'
2023-11-26 23:19:05 +01:00
Mike Fährmann
d9734ce008
[cyberdrop] update to site layout changes
2023-11-26 18:03:13 +01:00
Mike Fährmann
8ac68ffba2
[hentaicosplays] force 'https://' for download URLs
2023-11-26 02:08:12 +01:00
Mike Fährmann
fc1101779c
[hiperdex] fix 'manga' metadata
2023-11-26 01:24:42 +01:00
Mike Fährmann
d119507037
[imagefap] fix single image resolution
...
Downloading from a single image page like
https://www.imagefap.com/photo/123456789/
returned only the thumbnail URL.
2023-11-26 00:30:52 +01:00
Mike Fährmann
311ec1d9ef
[mangaread] fix extraction
2023-11-25 23:53:27 +01:00
Mike Fährmann
7608201a44
[tumblr] fix 'day' extractor
...
another bug caused by a383eca7
2023-11-25 00:51:14 +01:00
Mike Fährmann
c8c744a7c0
[webtoons] fix pagination when receiving an HTTP redirect
2023-11-24 22:17:34 +01:00
Mike Fährmann
23cd17997d
[wallpapercave] fix extraction
2023-11-24 21:54:21 +01:00
Mike Fährmann
5b979b5706
[xvideos] fix metadata extraction
2023-11-24 21:27:19 +01:00
Mike Fährmann
adc3aa0b77
[zerochan] fix metadata extraction
...
author, path, tags
2023-11-24 21:21:14 +01:00
Mike Fährmann
f9dac43be9
[warosu] fix file URLs
2023-11-24 02:44:55 +01:00
Mike Fährmann
645b4627ef
[sankaku] update URL patterns
2023-11-24 02:41:52 +01:00
Mike Fährmann
1ae43d8123
merge #4841 : [fapello] support '.su' TLD ( #4840 )
2023-11-22 20:18:32 +01:00
Mike Fährmann
b43be67206
[exhentai] add 'gp' option ( #4576 )
2023-11-22 20:15:00 +01:00
Mike Fährmann
cb9a1176e6
[pixeldrain] add 'api-key' option ( #4839 )
2023-11-22 19:13:51 +01:00
Mike Fährmann
e1404827a6
[pixeldrain] add 'file' and 'album' extractors ( #4839 )
2023-11-22 19:01:19 +01:00
enduser420
2402162e8a
[fapello] support '.su' TLD
2023-11-22 19:35:43 +05:30
Mike Fährmann
725c8dd55a
[tmohentai] 'categories' -> 'genres'
...
quite likely that the site meant 'genres' by "Genders"
2023-11-21 22:11:43 +01:00
Mike Fährmann
ce7c4cb544
merge #4832 : [tmohentai] add 'gallery' extractor ( #4808 )
2023-11-21 20:25:49 +01:00
Mike Fährmann
c4a201ed42
[tmohentai] simplify + tests
2023-11-21 20:24:07 +01:00
Mike Fährmann
e17a48fe56
[blogger] inherit from BaseExtractor
...
- support www.micmicidol.club (#4759 )
2023-11-21 16:52:25 +01:00
jsouthgb
714b1a7089
[tmohentai] simplify url matching
2023-11-21 10:46:48 -05:00
jsouthgb
31963fa947
[tmohentai] inherit from GalleryExtractor. refactor metadata.
2023-11-20 21:35:32 -05:00
Mike Fährmann
0fa85360a0
merge #4812 : [erome] add 'count' metadata field
2023-11-20 22:42:02 +01:00
Mike Fährmann
a43cf78bb7
[erome] tests
2023-11-20 22:41:12 +01:00
Mike Fährmann
34a387b6e2
support 'metadata-*' names for '*-metadata' options
...
For example, instead of 'url-metadata' it is now also possible to use
'metadata-url' as option name.
- metadata-url
- metadata-path
- metadata-http
- metadata-version
- metadata-parent
2023-11-18 23:52:10 +01:00
Mike Fährmann
e97d7b1c85
[exhentai] fix empty api_url with '"source": "hitomi"' ( #4829 )
2023-11-18 21:06:41 +01:00
jsouthgb
ed965eecbb
[tmohentai] refactor to str.format for backwards compatibility
2023-11-18 14:39:17 -05:00
jsouthgb
dad7ba1d58
[tmohentai] fix edge cases. updated archive_fmt and filename_fmt
2023-11-17 21:08:34 -05:00
jsouthgb
286d0cb098
[tmohentai] add support
2023-11-17 19:34:34 -05:00
Mike Fährmann
07cb584231
[behance] add 'modules' option ( #4799 )
2023-11-17 22:54:38 +01:00
Mike Fährmann
6a753d9ff3
[behance] support 'text' modules ( #4799 )
2023-11-17 22:54:38 +01:00
jsouthgb
c6ad9bcd9b
[erome] add "count" for albums
2023-11-15 07:08:52 -05:00
Mike Fährmann
17e710c4bf
[oauth] warn when cache is enabled but not writeable ( #4771 )
2023-11-13 16:24:54 +01:00
Mike Fährmann
2e4bf54644
[hentaifoundry] check for and update expired sessions ( #4694 )
2023-11-13 00:27:22 +01:00
Mike Fährmann
0435c6e603
[exhentai] handle 'Downloading … requires GP' errors ( #4576 , #4763 )
2023-11-12 22:47:40 +01:00
Mike Fährmann
4288cea94a
[mastodon] fix reblogs ( #4580 )
2023-11-11 00:34:49 +01:00
Mike Fährmann
7a0f145cbe
[twitter] ignore promoted Tweets ( #4790 , #3894 )
...
add 'ads' option in case someone actually wants to
download promoted content for whatever reason
2023-11-10 23:46:46 +01:00
Mike Fährmann
e8b5e59a08
[weibo] detect redirects to login page ( #4773 )
2023-11-10 19:35:29 +01:00
Mike Fährmann
5e58d2b455
[instagram] fix exception on empty 'video_versions' ( #4795 )
2023-11-10 14:49:29 +01:00
Mike Fährmann
61d6558322
[exhentai] try to avoid 'DH_KEY_TOO_SMALL' errors ( #1021 , #4593 )
2023-11-04 17:30:27 +01:00
Mike Fährmann
69b931b9bb
[exhentai] provide fallback URLs ( #1021 , #4745 )
2023-11-04 17:06:46 +01:00
Mike Fährmann
007c433677
[patreon] support 'id:<campaign_id>' in place of a user name
...
https://patreon.com/id:12345
… and remove 'campaign-id' config option
2023-11-04 00:17:41 +01:00
Mike Fährmann
3984a49abf
[nijie] set 1-2s delay between requests to avoid 429 errors
2023-11-03 23:44:47 +01:00
Mike Fährmann
dd14adccf6
[pixiv] allow cookies for non-OAuth URLs ( #4760 )
2023-11-03 23:39:58 +01:00
Mike Fährmann
caf31e751c
[kemonoparty] limit 'title' length ( #4741 )
2023-11-02 15:53:23 +01:00
Mike Fährmann
43d0c49d7e
[exhentai] fix original image URLs ( #4754 )
2023-11-02 15:46:01 +01:00
Mike Fährmann
43a3d93467
merge #4755 : [twitter] recognize fixupx.com URLs
2023-11-02 15:33:29 +01:00
Mike Fährmann
fc8f86bf24
[hitomi] recognize 'imageset' gallery URLs ( #4756 )
2023-11-02 15:29:44 +01:00
Mike Fährmann
91e20eb59b
[fantia] simplify 'tags' to a list of strings ( #4752 )
2023-11-02 15:25:01 +01:00
thatfuckingbird
44d7964c09
[twitter] recognize fixupx.com URLs
2023-11-01 15:50:36 +01:00
Mike Fährmann
56cd9d408d
[weibo] fix Sina Visitor request
2023-10-30 22:14:52 +01:00
Mike Fährmann
68e72a836c
[exhentai] fix extraction ( #4730 )
...
- update to new API response layout
- use proper API server URL
- fix 'filesize' metadata
2023-10-30 13:38:49 +01:00
Mike Fährmann
fd8f58ad76
[behance] unescape embed URLs ( #4742 )
2023-10-30 13:38:49 +01:00
Mike Fährmann
ca1d5c2c0c
merge #4738 : [patreon] parse new bootstrap data format ( #4736 )
2023-10-29 15:58:23 +01:00
Mike Fährmann
4730de163f
[patreon] refactor _extract_bootstrap()
2023-10-29 15:57:21 +01:00
Mike Fährmann
c9a2be36d4
[sankaku] support '/posts/' tag search URLs ( #4740 )
2023-10-29 13:48:42 +01:00
Tobias Hellmann
28ada11cba
Try to parse newer HTTP response from Patreon
2023-10-28 22:27:26 +02:00
Mike Fährmann
fd36eafe32
[twitter] restore truncated retweet text ( #3430 , #4690 )
2023-10-27 23:26:21 +02:00
Mike Fährmann
218295a4c6
[twitter] fix avatars without 'date' information ( #4696 )
2023-10-27 17:58:02 +02:00
Mike Fährmann
969be65d0b
[instagram] update API headers
2023-10-27 15:33:00 +02:00
Mike Fährmann
d0effcae20
[kemonoparty] add 'revision_index' metadata field ( #4727 )
2023-10-26 22:26:38 +02:00
Mike Fährmann
3bbaa875f1
[kemonoparty] fix parsing of non-standard 'dates' ( #4676 )
2023-10-26 21:50:18 +02:00
Mike Fährmann
75dec71253
[idolcomplex] disable Referer headers by default ( #4726 )
2023-10-26 18:02:31 +02:00
Mike Fährmann
a09df34bcf
merge #4714 : [4archive] add 'thread' and 'board' extractors
...
(#1262 , #2418 , #4400 , #4710 )
2023-10-25 20:12:07 +02:00
enduser420
acb713b95a
[4archive] update
2023-10-25 23:08:45 +05:30
Mike Fährmann
6766877524
merge #4693 : [reddit] support Reddit Mobile share links
2023-10-25 17:54:32 +02:00
Mike Fährmann
1042278bec
[misskey] support 'misskey.design' ( #4713 )
2023-10-25 17:47:03 +02:00
Mike Fährmann
12a800ce21
[patreon] improve 'campaign_id' handling ( #4699 , #4715 )
...
- add ways to directly specify a 'campaign_id'
- 'campaign-id' config option
- 'c' or 'campaign_id' URL query parameter
- more descriptive error messages
- show 'campaign_id' value in debug log
2023-10-25 17:18:06 +02:00
Mike Fährmann
31dbbffc0b
[twitter] cache 'user_by_…' results ( #4719 )
2023-10-25 16:45:27 +02:00
enduser420
c0714d5585
[4archive] add 'thread' and 'board' extractors
2023-10-24 23:05:28 +05:30
inty
b68aad3dab
[reddit] implement Reddit Mobile share links
2023-10-22 10:38:05 +00:00
Mike Fährmann
7958ab1946
[newgrounds] support 'imageData' files ( #4642 )
2023-10-21 13:22:55 +02:00
Mike Fährmann
b52fd91ac6
[sankaku] support '/posts/' URLs ( #4688 )
2023-10-21 13:20:35 +02:00
Mike Fährmann
b8674776e9
[4chanarchives] disable Referer headers by default ( #4686 )
2023-10-20 17:39:50 +02:00
Mike Fährmann
78493f0870
[bunkr] fix '/d/' file URLs ( #4685 )
2023-10-20 17:30:23 +02:00
Mike Fährmann
b2c3db3e24
[bunkr] add extractor for media URLs ( #4684 )
2023-10-20 15:22:44 +02:00
Mike Fährmann
0d52b775cb
[kemonoparty] add 'revisions' option ( #4498 , #4597 )
2023-10-20 15:20:49 +02:00
Mike Fährmann
6e830ffc9e
[kemonoparty] support post searches ( #3385 , #4057 )
2023-10-19 23:06:06 +02:00
Mike Fährmann
aaf539009b
[kemonoparty] initial support for post revisions ( #4498 , #4597 )
...
- single revision
https://kemono.party/SERVICE/user/12345/post/12345/revision/12345
- all revisions
https://kemono.party/SERVICE/user/12345/post/12345/revisions
2023-10-19 22:32:51 +02:00
Mike Fährmann
174191cb79
[kemonoparty] restore discord pagination ( #4676 )
2023-10-19 21:57:27 +02:00
Mike Fährmann
c9a976d8a6
[kemonoparty] various updates and fixes ( #4676 , #4681 )
...
- fix pagination
- fix 'date' metadata
- fix discord channel API endpoint
2023-10-19 17:36:16 +02:00
Klion Xu
dc1c2139b1
fix line too long
2023-10-19 10:54:08 +08:00
Klion Xu
6b22af9720
[kemonoparty] update API endpoint ( #4676 )
2023-10-19 10:32:59 +08:00
Mike Fährmann
bfdc07632a
[deviantart] expand nested comment replies ( #4653 )
2023-10-17 19:40:53 +02:00
Mike Fährmann
390d14dbcc
[chevereto] support 'img.kiwi' and 'deltaporno.com' ( #4664 , #1381 )
2023-10-16 18:14:30 +02:00
Mike Fährmann
727c8eec6c
merge #4667 : [redgifs] fix 'niches' extraction ( #4666 )
2023-10-16 14:20:01 +02:00
Mike Fährmann
2911ed1240
[chevereto] add generic extractors ( #4664 )
...
- support jpgfish
- support pixl.li / pixl.is (#3179 , #4357 )
2023-10-16 14:15:39 +02:00
enduser420
db3363ac0b
[redgifs] fix 'niches' extraction
2023-10-16 16:51:30 +05:30
Mike Fährmann
ade8347ead
[kemonoparty] fix DM dates
2023-10-15 19:54:28 +02:00
Mike Fährmann
6dfe200ae4
[kemonoparty] support discord URLs with channel IDs ( #4662 )
2023-10-15 19:45:22 +02:00
Mike Fährmann
c6a3892210
[imgbb] update username extraction ( #4626 )
2023-10-14 20:55:39 +02:00
Mike Fährmann
830a48bca4
[fantia] bad workaround for 833dce14 ( #4627 )
...
at least this makes "filter": "content_num == content_count+1"
with "event": "post-after" work
2023-10-13 23:04:47 +02:00
Mike Fährmann
13ce3a9acb
[warosu] fix extraction ( #4634 )
2023-10-13 23:03:39 +02:00
Mike Fährmann
c4c4e4d2f4
[newgrounds] improve 'art-image' extraction ( #4642 )
...
- download files in original resolution
- replace .webp with extension of first file
2023-10-13 20:10:55 +02:00
Mike Fährmann
833dce141f
[fantia] add 'content_count' and 'content_num' metadata fields ( #4627 )
2023-10-13 20:10:55 +02:00
Mike Fährmann
2d41702762
[deviantart] implement '"group": "skip"' ( #4630 )
2023-10-12 22:14:20 +02:00
Mike Fährmann
992e86ec94
[deviantart] disable 'jwt' ( #4652 )
2023-10-12 21:37:10 +02:00
Mike Fährmann
2974b8e3c8
[moebooru] add 'metadata' option ( #4646 )
...
for extended 'pool' metadata
2023-10-12 21:34:25 +02:00
Mike Fährmann
8bb7243c10
[reddit] fix wrong previews ( #4649 )
...
caused by a failed comment URL
using the main submission's preview as fallback
14af15bd4963bb9b
2023-10-10 18:33:01 +02:00
Mike Fährmann
08bdde5aac
merge #4619 : [twitter] add 'sensitive' metadata field
2023-10-09 15:40:58 +02:00
Mike Fährmann
f3d6aaff13
[twitter] rename to 'sensitive'; use 'tget()'
2023-10-09 15:39:09 +02:00
Mike Fährmann
95c280c59b
[imgbb] update pagination end condition ( #4626 )
2023-10-09 15:33:25 +02:00
Mike Fährmann
2e350dd82a
merge #4626 : [imgbb] fix 'user' extraction, add 'displayname'
2023-10-09 15:31:23 +02:00
Mike Fährmann
a2daa9befe
[imgbb] fix flake8 and username order
2023-10-09 15:29:17 +02:00
Mike Fährmann
9a008523ac
[hentaifoundry] fix '.swf' file downloads ( #4641 )
2023-10-09 11:45:55 +02:00
Mike Fährmann
15f940819b
[newgrounds] support 'art-image' files ( #4642 )
2023-10-09 11:20:10 +02:00
Mike Fährmann
63db54b905
[patreon] update 'campaign_id' path ( #4639 )
2023-10-09 09:41:38 +02:00
HRXN
b846f56c3a
[imgbb] Fix user extraction, add displayname
2023-10-05 12:18:58 +02:00
Mike Fährmann
efaab4fbfa
[twitter] fix crash due to missing 'source' ( #4620 )
...
regression caused by 06aaedde
2023-10-04 23:01:04 +02:00
Nahida
3438a3098d
[twitter] add possible_sensitive field
2023-10-04 10:34:02 +08:00
Mike Fährmann
64dbc58a5a
[deviantart] update Eclipse API endpoints 2 ( #4615 )
2023-10-03 14:58:52 +02:00
Mike Fährmann
84fbbd96aa
[shimmie2] remove 'meme.museum'
2023-10-02 20:41:25 +02:00
Mike Fährmann
aa77fda78c
[instagram] better error message for invalid users ( #4606 )
2023-10-02 20:29:10 +02:00
Mike Fährmann
482f002e1f
[nsfwalbum] detect '/error.jpg' images ( #4598 )
2023-10-02 15:38:22 +02:00
Mike Fährmann
eb230e4b77
[nsfwalbum] disable Referer headers by default ( #4598 )
2023-10-01 13:55:17 +02:00
Mike Fährmann
b92645cd37
[bunkr] fix extraction ( #4514 , #4532 , #4529 , #4540 )
2023-09-30 18:05:12 +02:00
Mike Fährmann
be17103e21
[regifs] support 'order' parameter for user URLs ( #4583 )
2023-09-28 21:37:58 +02:00
HRXN
ec91eeb7ef
Update gallery_dl/extractor/reddit.py
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2023-09-26 14:00:29 +02:00
HRXN
66613c3a32
[reddit] ignore '/message/compose' URLs without www subdomain
2023-09-25 02:24:44 +02:00
Mike Fährmann
bb39779e1a
[deviantart] use private tokens for 'is_mature' posts ( #4563 )
2023-09-24 19:15:35 +02:00
Mike Fährmann
0c5d8b1505
[deviantart] re-add 'quality' option and 'intermediary' transform
2023-09-24 17:36:05 +02:00
Mike Fährmann
20d1683c47
[deviantart] fix JWT replacement ( #293 , #4548 , #4563 )
...
And again, a huge thank you to @Ironchest337
for discovering this.
2023-09-24 15:06:16 +02:00
Mike Fährmann
d7aac9fc06
[reddit] ignore '/message/compose' URLs ( #4482 )
2023-09-23 00:13:05 +02:00
Mike Fährmann
1e31fce37b
[pillowfort] support '/tagged/' URLs ( #4570 )
2023-09-23 00:11:01 +02:00
Mike Fährmann
1d2fd0b831
[pillowfort] extract 'b2_lg_url' media ( #4570 )
2023-09-23 00:05:26 +02:00
Mike Fährmann
50e2ebaff0
[danbooru] support 'donmai.moe' URLs
2023-09-22 20:58:38 +02:00
Mike Fährmann
82296b1f05
[reddit] add 'previews' option ( #4322 )
...
another way to disable this new behavior
2023-09-22 18:10:44 +02:00
Mike Fährmann
918ba4f847
[redgifs] match gfycat image URLs ( #4558 )
2023-09-22 18:02:55 +02:00
Mike Fährmann
2ad75bab05
[deviantart] add 'is_original' metadata field ( #4559 )
...
true for 'downloadable' content, journals, flash animations,
and images without '/v1/' in their URL; false otherwise
2023-09-22 17:37:11 +02:00
Mike Fährmann
9d8317d963
[deviantart] disable JWT updates ( #4548 , #4563 )
...
back to lowres images ...
2023-09-22 17:13:05 +02:00
Mike Fährmann
8064663bda
[deviantart] update Eclipse API endpoints ( #4553 )
2023-09-22 16:30:16 +02:00
Mike Fährmann
2cd801232b
fix --range causing crashes ( #4557 )
...
regression caused by a383eca7
2023-09-22 16:28:20 +02:00
Mike Fährmann
3528974459
[instagram] handle exceptions due to missing media ( #4555 )
2023-09-21 20:58:31 +02:00
Mike Fährmann
4963bb9b30
[reddit] improve comment metadata v2 ( #4482 )
...
provide main submission metadata at the top level
and comment metadata inside the 'comment' field,
i.e. the other way round than in 1710f1e9
2023-09-20 22:12:40 +02:00
Mike Fährmann
7592c5e566
[patreon] fix extraction ( #4547 )
2023-09-19 13:18:47 +02:00
Mike Fährmann
0655ce1bae
[mangakakalot] update domain
...
the old one still works, but it incurs a redirect
2023-09-19 00:18:21 +02:00
Mike Fährmann
3ecb512722
send Referer headers by default
2023-09-19 00:02:04 +02:00
Mike Fährmann
cb4798f07a
[architizer] fix extraction ( #4537 )
2023-09-17 15:19:19 +02:00
Mike Fährmann
6178177227
[twitter] fix '_extractor' of following results ( #4536 )
...
regression from 20ed647f
2023-09-15 23:04:30 +02:00
Mike Fährmann
d13c82eff1
[kemonoparty] update favorites API endpoint ( #4522 )
2023-09-14 14:57:01 +02:00
Mike Fährmann
27ec653991
fix bug in test_init and update example URLs
2023-09-14 13:27:03 +02:00
Mike Fährmann
24a1d46391
[mastodon] support '/@USER/following' URLs
...
Previously, only '/users/USER/following' got matched.
2023-09-13 23:42:51 +02:00
Mike Fährmann
9f75713e00
[recursive] simplify
2023-09-13 21:47:20 +02:00
Mike Fährmann
6ae92da57e
Merge branch 'tests'
2023-09-13 21:34:28 +02:00
Mike Fährmann
32da3c70d3
[behance] handle videos without 'renditions' ( #4523 )
2023-09-12 22:00:04 +02:00
Mike Fährmann
ae5e049c4f
[redgifs] provide 'collection' metadata in a separate field ( #4508 )
...
instead of overwriting the actual metadata
2023-09-12 22:00:04 +02:00
Mike Fährmann
1710f1e983
[reddit] improve comment metadata ( #4482 )
...
- provide 'date'
- make metadata of the main submission available as 'submission[…]'
2023-09-12 22:00:04 +02:00
Mike Fährmann
4cdab8074e
update/fix --list-extractors
2023-09-11 17:32:59 +02:00
Mike Fährmann
a453335a9f
remove test results in extractor modules
...
and add generic example URLs
2023-09-11 16:30:55 +02:00
Mike Fährmann
1d2b5d0c60
update test comment positions
...
always put them above the test they're referring to
2023-09-06 18:16:09 +02:00
Mike Fährmann
f856987297
[subscribestar] fix preview detection ( #4468 )
...
and show a warning message when posts contain previews
2023-09-04 22:21:14 +02:00
Mike Fährmann
4c0b3d5dc5
[twitter] fix crash when 'sortIndex' is None ( #4499 )
2023-09-04 18:28:43 +02:00
Mike Fährmann
f2de70f254
[gfycat] remove module
2023-09-04 18:27:11 +02:00
Mike Fährmann
6eca1fab9b
[gelbooru_v02] support 'xbooru.com' ( #4493 )
2023-09-03 15:39:02 +02:00
Mike Fährmann
23bac772f2
[jpgfish] update domain to 'jpg1.su' ( #4494 )
2023-09-03 15:03:47 +02:00
Mike Fährmann
ceb59e176f
fix default Firefox user agent string
...
note to self: do not trust some random third-party website
2023-09-02 22:22:23 +02:00
Mike Fährmann
8259a5abe4
flake8
2023-09-02 13:03:11 +02:00
Mike Fährmann
0b6e5b8161
[hiperdex] send Referer headers during file downloads ( #4490 )
2023-09-02 12:59:30 +02:00
Mike Fährmann
a05821f8b4
[hiperdex] fix 'manga' metadata
...
remove trailing ' Manga'
2023-09-02 12:59:30 +02:00
Mike Fährmann
03d471a0d4
merge #4481 : [pixiv] handle errors for private novels
2023-09-02 12:50:23 +02:00
Cisney-Gassai
8c477f7146
[bunkr] Fixes media-files-pizza.bunkr.ru failed to resolve.
2023-09-02 01:45:59 -06:00
johnsmith1202gmail
c7e31b2724
Update pixiv.py
2023-08-30 12:42:31 -07:00
johnsmith1202gmail
d3046561d4
continue downloading when the item is made private on pixiv
2023-08-30 18:21:06 +00:00
Mike Fährmann
28798594e8
[gfycat] update pagination logic ( #4479 )
...
Some searches do not use cursor based pagination
but an offset based one.
2023-08-30 18:54:15 +02:00
Mike Fährmann
a783c4f0fe
[pornhub] add 'gif' support ( #4463 )
2023-08-29 19:34:27 +02:00
Mike Fährmann
ba842981af
[imagevenue] fix extraction ( #4473 )
2023-08-29 12:06:30 +02:00
Mike Fährmann
7defb24e1e
[reddit] provide video previews if available ( #4322 )
2023-08-28 22:22:10 +02:00
Mike Fährmann
fd65f27ede
[reddit] fix 'preview.redd.it' URLs ( #4470 )
2023-08-28 17:17:03 +02:00
Mike Fährmann
06aaedded5
[twitter] extract 'source' metadata ( #4459 )
2023-08-28 16:31:57 +02:00
Mike Fährmann
14af15bd18
[reddit] download preview for 404ed imgur links ( #4322 )
...
This is a pretty ugly hack as the internal infrastructure doesn't
really support switching from external URL to regular download in
case the former fails, but it kind of works ...
Can be disabled by setting 'reddit.fallback' to 'false'.
2023-08-24 15:41:05 +02:00
Mike Fährmann
d12a5e440a
update docs/supportedsites
2023-08-24 15:01:26 +02:00
Mike Fährmann
3a27150479
[instagram] add 'following' extractor ( #1848 )
2023-08-23 23:58:12 +02:00
Mike Fährmann
e0829ff0fd
[twitter] add 'date_original' metadata for retweets ( #4337 , #4443 )
2023-08-23 23:58:11 +02:00
Mike Fährmann
5ed245317d
[exhentai] add 'fav' option ( #4409 )
...
The name 'favorite' is already taken as extractor subcategory
2023-08-23 23:58:11 +02:00
Mike Fährmann
fd6b413f3c
[exhentai] fix 'domain' option ( #4458 )
...
regression from a383eca7
2023-08-23 23:58:04 +02:00
Mike Fährmann
fdfb22c91f
[instagram] fix video preview archive IDs ( #2135 , #4455 )
2023-08-23 12:29:32 +02:00
Mike Fährmann
2b88ad19e9
[twitter] accept 'x.com' URLs ( #4452 )
2023-08-21 19:47:07 +02:00
Mike Fährmann
8dceea3384
[shimme2] move 'giantessbooru' back into shimmie module ( #4373 )
...
Do the same thing as for 'realbooru' and override 'posts()'
insteadd of using a separate module.
2023-08-18 15:25:28 +02:00
Mike Fährmann
6482f9453b
[behance] fix cookie usage ( #4417 )
2023-08-18 14:48:20 +02:00
Mike Fährmann
d34195b41d
[behance] fix and update 'user' extractor ( #4417 )
2023-08-17 16:06:35 +02:00
Mike Fährmann
4d3cf709da
[behance] add 'date' metadata field ( #4417 )
2023-08-17 15:33:47 +02:00
Mike Fährmann
c689cd9720
[behance] show error for mature content ( #4417 )
2023-08-17 15:31:37 +02:00
Mike Fährmann
33d912490f
merge #4419 : [bunkr] Fix extracting wmv files
2023-08-17 15:28:29 +02:00
Mike Fährmann
01610a6e9e
merge #4412 : [bunkr] fix media domain for cdn9
2023-08-17 15:18:49 +02:00
ClosedPort22
6dc8be5e48
[issuu] fix extraction
2023-08-13 21:13:50 +08:00
Luc Ritchie
85a070b9e6
[bunkr] Fix extracting wmv files
2023-08-12 16:53:14 -04:00
Mike Fährmann
3f8ff692a7
[bunkr] fix media domain for cdn9
...
Fixes #4386
2023-08-11 18:14:47 -04:00
Mike Fährmann
391a7d74c8
[giantessbooru] fix and move to separate module ( #4373 )
...
too many differences to the other shimmie2 sites
2023-08-09 18:36:56 +02:00
Mike Fährmann
089d1a4f67
[twitter] fix 'TweetWithVisibilityResults' ( #4369 )
2023-08-06 22:08:50 +02:00
Mike Fährmann
a4f7f7da17
add '_dump()' convenience method to Extractor
2023-08-06 17:03:09 +02:00
Mike Fährmann
df5c7ee03e
[deviantart] fix search ( #4384 )
...
send correct usernames instead of 'u'
2023-08-04 17:16:04 +02:00
