3913 Commits

Author	SHA1	Message	Date
Mike Fährmann	50eef1b5cc	merge #5029: [pixiv] update App API headers	2024-01-07 16:27:13 +01:00
Mike Fährmann	33f228756a	[mangadex] add 'list' extractor (#5025) ... supports listing manga and chapters from list feed	2024-01-07 02:59:35 +01:00
Mike Fährmann	db8de13537	[vk] transform image URLs to non-blurred versions (#5017) ... apply the same filter from before d85e66bc	2024-01-07 00:19:10 +01:00
Se AKi	d0d199414f	modify useragent of pixiv	2024-01-06 23:15:15 +09:00
Mike Fährmann	cbfb7bfdf1	[gelbooru] display error for invalid API responses (#4903)	2024-01-06 14:28:35 +01:00
Mike Fährmann	c25bdbae91	[komikcast] fix 'manga' extractor (#5027)	2024-01-06 14:19:44 +01:00
Mike Fährmann	8e1a2b5446	[komikcast] update domain to 'komikcast.lol' (#5027)	2024-01-06 02:16:43 +01:00
Mike Fährmann	a441249ea2	merge #4979: [batoto] add 'chapter' and 'manga' extractors (#1434, #2111)	2024-01-06 01:53:26 +01:00
Mike Fährmann	b11c352d66	[bato] rename to 'batoto' ... to use the same category name as the previous bato.to site	2024-01-06 01:49:34 +01:00
Mike Fährmann	3aa24c3744	[bato] simplify and update	2024-01-06 01:10:04 +01:00
Mike Fährmann	11150a7d72	[nudecollect] remove module	2024-01-05 21:32:04 +01:00
Mike Fährmann	c158927c38	merge #5016: [zzup] add 'gallery' extractor (#4517, #4604, #4659, #4863)	2024-01-05 21:25:46 +01:00
Mike Fährmann	e61f016465	[szurubooru] support 'snootbooru.com' (#5023)	2024-01-05 17:56:39 +01:00
Mike Fährmann	b4bcf40278	[weibo] fix AttributeError in 'user' extractor (#5022) ... yet another bug caused by a383eca7	2024-01-05 17:18:33 +01:00
Mike Fährmann	0ab0a10d2d	[jpgfish] update domain	2024-01-05 02:27:20 +01:00
enduser420	0f30136109	[zzup] add 'gallery' extractor	2024-01-04 21:38:59 +05:30
Mike Fährmann	a86775f617	[gelbooru] fix 'favorite' extractor (#4903) ... lots of +1/-1 and </<= mistakes	2024-01-04 15:17:49 +01:00
Mike Fährmann	7eaf648f2e	[fanbox] add 'metadata' option (#4921) ... extracts 'plan' and extended 'user' metadata	2024-01-04 15:01:33 +01:00
Mike Fährmann	4f3671458e	[deviantart] add 'avatar' and 'background' extractors (#4995)	2024-01-03 00:07:55 +01:00
Mike Fährmann	9fa4f54c24	[twitter] raise error for invalid 'strategy' values (#4953)	2024-01-01 22:58:42 +01:00
Mike Fährmann	516c69297d	[manganelo] fix extraction & recognize '.to' TLDs (#5005)	2024-01-01 22:05:21 +01:00
Mike Fährmann	63f649cd92	[idolcomplex] fix extraction & update URL patterns (#5002)	2024-01-01 17:38:32 +01:00
Mike Fährmann	b6903a4c90	[nijie] add 'count' metadata field ... https://github.com/mikf/gallery-dl/issues/146#issuecomment-1812849102	2023-12-30 22:25:59 +01:00
Mike Fährmann	b93b351db9	merge #4962: [poringa] add support (#4675)	2023-12-30 20:39:35 +01:00
Mike Fährmann	9f21c839ad	[poringa] improvements and fixes ... - add 'num' and 'count' metadata fields - prevent crash for "private" posts - prevent crash when there's no 'main-info' - update tests	2023-12-30 20:37:09 +01:00
Mike Fährmann	00d83d9588	[rule34us] add fallback for 'video-cdn1' videos (#4985)	2023-12-30 18:33:46 +01:00
Mike Fährmann	085411f3f1	[rule34] recognize URLs with 'www' subdomain (#4984)	2023-12-30 16:07:56 +01:00
Mike Fährmann	9f5051e4ed	merge #4981: [pinterest] add 'count' metadata field	2023-12-30 15:53:35 +01:00
bug-assassin	f6ce870885	Better variable names	2023-12-28 17:25:15 -05:00
bug-assassin	3553025584	Removed f-strings	2023-12-28 17:07:41 -05:00
Mike Fährmann	f36dafad06	improve 'include' handling (#4982) ... - remove spaces when given as string - warn about invalid vales	2023-12-28 19:07:04 +01:00
blankie	375f2db4c2	[pinterest] add count metadata field	2023-12-28 01:07:04 +11:00
Antonio	e348da7a06	[poringa] add support	2023-12-27 00:07:23 -06:00
bug-assassin	2c3f171d65	Fix python 3.5 linting issue	2023-12-26 23:55:47 -05:00
bug-assassin	06ff1d3a3c	Replace text.extract with extr	2023-12-26 23:47:30 -05:00
bug-assassin	9c1ce28f68	[bato] Added mangatoto alias	2023-12-26 23:44:27 -05:00
bug-assassin	663b8d789a	Fix linting	2023-12-26 23:41:37 -05:00
bug-assassin	74c225f94e	[bato] add support	2023-12-26 22:33:33 -05:00
Mike Fährmann	f9544194c0	[paheal] restore 'extension' metadata (#4976)	2023-12-26 16:09:26 +01:00
Mike Fährmann	77d46e6f0c	[lynxchan] update 'bbw-chan' domain (#4970)	2023-12-25 15:29:05 +01:00
Mike Fährmann	766316e436	[imagechest] fix loading more than 10 images in a gallery (#4469)	2023-12-25 15:14:43 +01:00
Mike Fährmann	108c978073	merge #4919: [postmill] add support (#4917)	2023-12-23 15:23:56 +01:00
blankie	8a42ea736a	[postmill] implement suggestions	2023-12-23 13:28:36 +11:00
Mike Fährmann	c184454efb	[shimmie2] small optimizations ... - unroll/remove loop - avoid copy	2023-12-22 20:29:22 +01:00
Mike Fährmann	7cd0211cc9	[shimmie2] autodetect single or double quotes	2023-12-22 15:25:28 +01:00
Mike Fährmann	2a60645095	[deviantart] set 'is_original' for intermediary URLs to 'false'	2023-12-22 14:49:10 +01:00
Mike Fährmann	01bb75f6cb	merge #4945: {shimmie2[ support 'rule34hentai.net' (#861, #4789)	2023-12-22 00:10:26 +01:00
Mike Fährmann	79e4606893	[rule34hentai] cleanup ... - fix using 'self._posts_rule34hentai' - fix 'file_url' for posts - update docs/supportedsites - add tests	2023-12-22 00:01:36 +01:00
bun-dev	ef370df41d	[shimmie2] support 'rule34hentai.net' ... - Add files via upload - Update shimmie2.py - Update shimme2.py - Delete gallery_dl/extractor/shimme2.py - spacefix - Update shimmie2.py - Update shimmie2.py - flask warnings1 - Update shimmie2.py - Update shimmie2.py	2023-12-21 23:00:58 +01:00
Mike Fährmann	627ed794a2	[danbooru] provide 'tags' as list (#4942) ... keep the old 'tag_string' values around, similar to sankaku a lot of repeat code ... would be a lot less bad if "".split(" ") returned an empty list	2023-12-21 14:39:38 +01:00
Mike Fährmann	fbebc58189	[deviantart] add 'intermediary' option (#4955)	2023-12-21 02:23:22 +01:00
Mike Fährmann	75fa1a5553	[pinterest] remove login code ... this has been broken since forever and is still "protected" by an invisible recaptcha check	2023-12-20 20:59:18 +01:00
Mike Fährmann	92ff99c8e5	[twitter] remove 'syndication' option (#3889)	2023-12-20 14:38:36 +01:00
Mike Fährmann	a75f85a2c2	[twitter] remove 'date_liked' (#3850, #4108, #4657) ... Twitter's 'sortIndex' can't be used to calculate the timestamp of when a Tweet was liked anymore.	2023-12-20 14:12:49 +01:00
Mike Fährmann	a94f944148	[twitter] default to 'tweets' timeline when 'replies' are enabled (#4953)	2023-12-20 01:57:18 +01:00
Mike Fährmann	a30a3e44d5	[nijie] move 'username required' out of _login_impl	2023-12-18 23:57:44 +01:00
Mike Fährmann	57fc6fcf83	replace '24*3600' with '86400' ... and generalize cache maxage values	2023-12-18 23:57:22 +01:00
Mike Fährmann	1f9b16a70b	replace static 'sleep-request' defaults with dynamic ones	2023-12-18 22:06:26 +01:00
Mike Fährmann	b127321b5c	[exhentai] only show 'using e-hentai.org' warning for exh domains	2023-12-18 15:29:54 +01:00
Mike Fährmann	e097aaf64a	[exhentai] output continuation URL when interrupted (#4782)	2023-12-17 23:25:47 +01:00
Mike Fährmann	99aa923322	[inkbunny] improve '/submissionsviewall.php' patterns (#4934) ... allow 'mode=…' to be in any position don't require it to be somewhere in the middle	2023-12-16 19:21:20 +01:00
Mike Fährmann	3f9c113d78	[mastodon] Support non-numeric status IDs (#4936)	2023-12-16 01:52:31 +01:00
Mike Fährmann	2852404e49	[inkbunny] add 'unread' extractor (#4934)	2023-12-15 21:20:12 +01:00
Mike Fährmann	8b87a5330d	[inkbunny] stop pagination on empty results	2023-12-15 21:15:02 +01:00
Mike Fährmann	6cd5e6adad	[patreon] fix bootstrap data extraction (#4904)	2023-12-15 18:24:52 +01:00
Mike Fährmann	aac8bb4eae	[deviantart] simplify 9951c112	2023-12-15 17:39:42 +01:00
Mike Fährmann	9951c112f8	[deviantart] workaround for integer client_id values (#4924)	2023-12-14 16:25:06 +01:00
Mike Fährmann	a37b7759bc	[myhentaigallery] recognize '/g/' URLs (#4920)	2023-12-12 20:02:28 +01:00
Mike Fährmann	da76e13e3b	[tumblr] fix exception after waiting for rate limit (#4916) ... use a loop instead of recursive function calls	2023-12-12 19:14:06 +01:00
blankie	fbe14a2745	[postmill] add support	2023-12-12 21:36:52 +11:00
Mike Fährmann	d59d4ebff4	[tumblr] support infinite 'fallback-retries'	2023-12-11 23:40:13 +01:00
Mike Fährmann	2d5cda2b92	[exhentai] fix TypeError for infinite 'fallback-retries' (#4911)	2023-12-11 23:39:56 +01:00
Mike Fährmann	92fbf09643	remove single quotes in some logging messages (#4908) ... ('FileNotFoundError: [Errno 2] No such file or directory: ''') -> (FileNotFoundError: [Errno 2] No such file or directory: '')	2023-12-11 19:13:45 +01:00
Mike Fährmann	9a001fa6e4	merge #4906: [patreon] fix bootstrap data extraction (#4904)	2023-12-10 16:53:54 +01:00
Tobi823	66cbe9da41	- fix style check failure "line to long"	2023-12-10 16:24:00 +01:00
Tobi823	244444b194	- adapt code to current code style	2023-12-10 16:22:32 +01:00
Tobi823	fd06255f93	- reformat and refactor to pass tests	2023-12-10 16:17:34 +01:00
Tobi823	5ff7106d4f	- add code for the situation when Patreon is using window.patreon = wrapInProxy({"bootstrap":' to store metadata ... - refactor code to make it more readable - output page content when the HTML structure is unknown (to make debugging easier)	2023-12-10 16:10:46 +01:00
Mike Fährmann	ac22bbe80c	[twitter] retry API requests only for Timeout errors (#4811)	2023-12-09 22:30:48 +01:00
Mike Fährmann	c55955db03	[twitter] quick and dirty fix for /media changes (#4898)	2023-12-09 15:38:42 +01:00
Mike Fährmann	9a8dc6b02b	[exhentai] add 'fallback-retries' option (#4792)	2023-12-09 02:01:59 +01:00
Mike Fährmann	bf74eb5c46	merge #4886: [urlgalleries] add 'gallery' extractor (#919, #1184, #2905)	2023-12-08 22:55:58 +01:00
Mike Fährmann	c29ae9af08	[urlgalleries] simplify + resolve redirects	2023-12-08 22:43:56 +01:00
Mike Fährmann	e256434c9e	use custom HTTPBasicAuth class ... to support LazyPrompt as password and to generate the Authorization header only once instead of for every request	2023-12-07 01:40:31 +01:00
Mike Fährmann	6a4218aa23	handle 'json' parameter in Extractor.request() manually ... Mainly to allow passing custom classes like util.LazyPrompt, but also to simplify and streamline how requests handles it.	2023-12-06 22:13:13 +01:00
Mike Fährmann	9dd5cb8c8a	interactively prompt for passwords on login when none is provided	2023-12-06 22:12:59 +01:00
Mike Fährmann	4eb3590103	[nijie] fix image URLs of multi-image posts (#4876)	2023-12-05 17:48:50 +01:00
Mike Fährmann	a4e6ea667b	[twitter] retry API calls when their response contains errors (#4811)	2023-12-05 15:57:26 +01:00
Mike Fährmann	cf5702c843	[twitter] generalize "Login Required" error (#4734, #4324)	2023-12-05 15:13:58 +01:00
jsouthgb	ecaa0feb5d	[urlgalleries] add support	2023-12-05 07:08:11 -05:00
jsouthgb	1770c31e63	[urlgalleries] add support	2023-12-05 07:07:06 -05:00
Mike Fährmann	da0da0faaa	[exhentai] store more cookies when logging in (#4881) ... include 'igneous', 'hath_perks', etc and not just 'ipb_member_id' and 'ipb_pass_hash' like before	2023-12-04 23:24:11 +01:00
Mike Fährmann	c83fbe6c2d	merge #4855: [nitter] fix video extraction (#4853)	2023-11-27 18:39:05 +01:00
Mike Fährmann	013ca21543	[idolcomplex] update to site layout changes	2023-11-27 18:27:08 +01:00
enduser420	1e9bacd169	[nitter] fix video extraction	2023-11-27 21:58:06 +05:30
Mike Fährmann	9f3368c46f	[pornhub] fix 'user' metadata for gifs	2023-11-26 23:52:24 +01:00
Mike Fährmann	bdb3ce7217	[foolslide] remove 'powermanga.org'	2023-11-26 23:19:05 +01:00
Mike Fährmann	d9734ce008	[cyberdrop] update to site layout changes	2023-11-26 18:03:13 +01:00
Mike Fährmann	8ac68ffba2	[hentaicosplays] force 'https://' for download URLs	2023-11-26 02:08:12 +01:00
Mike Fährmann	fc1101779c	[hiperdex] fix 'manga' metadata	2023-11-26 01:24:42 +01:00
Mike Fährmann	d119507037	[imagefap] fix single image resolution ... Downloading from a single image page like https://www.imagefap.com/photo/123456789/ returned only the thumbnail URL.	2023-11-26 00:30:52 +01:00
Mike Fährmann	311ec1d9ef	[mangaread] fix extraction	2023-11-25 23:53:27 +01:00
Mike Fährmann	7608201a44	[tumblr] fix 'day' extractor ... another bug caused by a383eca7	2023-11-25 00:51:14 +01:00
Mike Fährmann	c8c744a7c0	[webtoons] fix pagination when receiving an HTTP redirect	2023-11-24 22:17:34 +01:00
Mike Fährmann	23cd17997d	[wallpapercave] fix extraction	2023-11-24 21:54:21 +01:00
Mike Fährmann	5b979b5706	[xvideos] fix metadata extraction	2023-11-24 21:27:19 +01:00
Mike Fährmann	adc3aa0b77	[zerochan] fix metadata extraction ... author, path, tags	2023-11-24 21:21:14 +01:00
Mike Fährmann	f9dac43be9	[warosu] fix file URLs	2023-11-24 02:44:55 +01:00
Mike Fährmann	645b4627ef	[sankaku] update URL patterns	2023-11-24 02:41:52 +01:00
Mike Fährmann	1ae43d8123	merge #4841: [fapello] support '.su' TLD (#4840)	2023-11-22 20:18:32 +01:00
Mike Fährmann	b43be67206	[exhentai] add 'gp' option (#4576)	2023-11-22 20:15:00 +01:00
Mike Fährmann	cb9a1176e6	[pixeldrain] add 'api-key' option (#4839)	2023-11-22 19:13:51 +01:00
Mike Fährmann	e1404827a6	[pixeldrain] add 'file' and 'album' extractors (#4839)	2023-11-22 19:01:19 +01:00
enduser420	2402162e8a	[fapello] support '.su' TLD	2023-11-22 19:35:43 +05:30
Mike Fährmann	725c8dd55a	[tmohentai] 'categories' -> 'genres' ... quite likely that the site meant 'genres' by "Genders"	2023-11-21 22:11:43 +01:00
Mike Fährmann	ce7c4cb544	merge #4832: [tmohentai] add 'gallery' extractor (#4808)	2023-11-21 20:25:49 +01:00
Mike Fährmann	c4a201ed42	[tmohentai] simplify + tests	2023-11-21 20:24:07 +01:00
Mike Fährmann	e17a48fe56	[blogger] inherit from BaseExtractor ... - support www.micmicidol.club (#4759)	2023-11-21 16:52:25 +01:00
jsouthgb	714b1a7089	[tmohentai] simplify url matching	2023-11-21 10:46:48 -05:00
jsouthgb	31963fa947	[tmohentai] inherit from GalleryExtractor. refactor metadata.	2023-11-20 21:35:32 -05:00
Mike Fährmann	0fa85360a0	merge #4812: [erome] add 'count' metadata field	2023-11-20 22:42:02 +01:00
Mike Fährmann	a43cf78bb7	[erome] tests	2023-11-20 22:41:12 +01:00
Mike Fährmann	34a387b6e2	support 'metadata-*' names for '*-metadata' options ... For example, instead of 'url-metadata' it is now also possible to use 'metadata-url' as option name. - metadata-url - metadata-path - metadata-http - metadata-version - metadata-parent	2023-11-18 23:52:10 +01:00
Mike Fährmann	e97d7b1c85	[exhentai] fix empty api_url with '"source": "hitomi"' (#4829)	2023-11-18 21:06:41 +01:00
jsouthgb	ed965eecbb	[tmohentai] refactor to str.format for backwards compatibility	2023-11-18 14:39:17 -05:00
jsouthgb	dad7ba1d58	[tmohentai] fix edge cases. updated archive_fmt and filename_fmt	2023-11-17 21:08:34 -05:00
jsouthgb	286d0cb098	[tmohentai] add support	2023-11-17 19:34:34 -05:00
Mike Fährmann	07cb584231	[behance] add 'modules' option (#4799)	2023-11-17 22:54:38 +01:00
Mike Fährmann	6a753d9ff3	[behance] support 'text' modules (#4799)	2023-11-17 22:54:38 +01:00
jsouthgb	c6ad9bcd9b	[erome] add "count" for albums	2023-11-15 07:08:52 -05:00
Mike Fährmann	17e710c4bf	[oauth] warn when cache is enabled but not writeable (#4771)	2023-11-13 16:24:54 +01:00
Mike Fährmann	2e4bf54644	[hentaifoundry] check for and update expired sessions (#4694)	2023-11-13 00:27:22 +01:00
Mike Fährmann	0435c6e603	[exhentai] handle 'Downloading … requires GP' errors (#4576, #4763)	2023-11-12 22:47:40 +01:00
Mike Fährmann	4288cea94a	[mastodon] fix reblogs (#4580)	2023-11-11 00:34:49 +01:00
Mike Fährmann	7a0f145cbe	[twitter] ignore promoted Tweets (#4790, #3894) ... add 'ads' option in case someone actually wants to download promoted content for whatever reason	2023-11-10 23:46:46 +01:00
Mike Fährmann	e8b5e59a08	[weibo] detect redirects to login page (#4773)	2023-11-10 19:35:29 +01:00
Mike Fährmann	5e58d2b455	[instagram] fix exception on empty 'video_versions' (#4795)	2023-11-10 14:49:29 +01:00
Mike Fährmann	61d6558322	[exhentai] try to avoid 'DH_KEY_TOO_SMALL' errors (#1021, #4593)	2023-11-04 17:30:27 +01:00
Mike Fährmann	69b931b9bb	[exhentai] provide fallback URLs (#1021, #4745)	2023-11-04 17:06:46 +01:00
Mike Fährmann	007c433677	[patreon] support 'id:<campaign_id>' in place of a user name ... https://patreon.com/id:12345 … and remove 'campaign-id' config option	2023-11-04 00:17:41 +01:00
Mike Fährmann	3984a49abf	[nijie] set 1-2s delay between requests to avoid 429 errors	2023-11-03 23:44:47 +01:00
Mike Fährmann	dd14adccf6	[pixiv] allow cookies for non-OAuth URLs (#4760)	2023-11-03 23:39:58 +01:00
Mike Fährmann	caf31e751c	[kemonoparty] limit 'title' length (#4741)	2023-11-02 15:53:23 +01:00
Mike Fährmann	43d0c49d7e	[exhentai] fix original image URLs (#4754)	2023-11-02 15:46:01 +01:00
Mike Fährmann	43a3d93467	merge #4755: [twitter] recognize fixupx.com URLs	2023-11-02 15:33:29 +01:00
Mike Fährmann	fc8f86bf24	[hitomi] recognize 'imageset' gallery URLs (#4756)	2023-11-02 15:29:44 +01:00
Mike Fährmann	91e20eb59b	[fantia] simplify 'tags' to a list of strings (#4752)	2023-11-02 15:25:01 +01:00
thatfuckingbird	44d7964c09	[twitter] recognize fixupx.com URLs	2023-11-01 15:50:36 +01:00
Mike Fährmann	56cd9d408d	[weibo] fix Sina Visitor request	2023-10-30 22:14:52 +01:00
Mike Fährmann	68e72a836c	[exhentai] fix extraction (#4730) ... - update to new API response layout - use proper API server URL - fix 'filesize' metadata	2023-10-30 13:38:49 +01:00