4122 Commits

Author	SHA1	Message	Date
Mike Fährmann	ee7c054855	[bluesky] add 'search' extractor (#4438) ... Both https://bsky.app/search?q=QUERY and https://bsky.app/search/QUERY are recognized as search URLs, where QUERY gets forwarded unmodified as 'q' parameter for app.bsky.feed.searchPosts . User searches are not supported yet.	2024-02-16 15:58:47 +01:00
Mike Fährmann	91e5c4fdfe	[bluesky] add 'avatar' and 'background' extractors (#4438)	2024-02-16 15:41:19 +01:00
Mike Fährmann	24c1317e0d	[batoto] fix crash when manga/chapter contains a '-' (#5200)	2024-02-16 00:10:08 +01:00
Mike Fährmann	0abd9723af	[bluesky] add 'metadata' option (#4438) ... allow extracting 'user' metadata and make 'facets' extraction optional	2024-02-15 23:30:16 +01:00
Mike Fährmann	7e036ea290	[bluesky] add 'depth' option (#4438) ... and reduce default depth and parentHeight values	2024-02-15 22:26:05 +01:00
Mike Fährmann	42335ea880	[zerochan] fix skipping every other post	2024-02-15 02:51:01 +01:00
Mike Fährmann	c97b92cc35	[fanbox] add 'home' and 'supporting' extractors (#5138)	2024-02-14 23:25:39 +01:00
Mike Fährmann	04e4ffc64c	[deviantart] combine 'png' option with 'quality' (#4846) ... "quality": "png" to download PNGs instead og JPEGs	2024-02-14 22:07:29 +01:00
Mike Fährmann	9cc4ec2c58	[deviantart] add 'png' option (#4846)	2024-02-14 01:03:15 +01:00
Mike Fährmann	966c8608e6	[deviantart] move image content extraction into separate function	2024-02-14 00:30:06 +01:00
Mike Fährmann	1d1ffe3317	[pornpics] update 'channel' extraction & add test ... change 'channel' to a list, since extracting both 'channel' and 'channels' does not really work with text.extract_from()	2024-02-13 23:48:46 +01:00
cc1234	32472d7d6c	Add support for multi channels	2024-02-13 18:34:04 +00:00
Mike Fährmann	139ff3f6ab	[kemonoparty] add 'posts' extractor (#5194)	2024-02-13 15:41:34 +01:00
Mike Fährmann	814ad9321e	[deviantart] skip locked/blurred posts (#4567, #5193)	2024-02-13 14:15:12 +01:00
Mike Fährmann	f7f8ef8684	[twitter] support communities (#4913)	2024-02-13 01:30:23 +01:00
Mike Fährmann	cae77e85f8	[twitter] update query hashes ... ... as well as 'variables' and 'features' values also remove unused legacy API code	2024-02-12 23:19:13 +01:00
Mike Fährmann	06cb518d97	[bunkr] fix extraction (#5088, #5151, #5153) ... - remove legacy code - map legacy domains to bunkr.sk - use input URL domain for newer domains - update tests (some files got slightly modified or deleted)	2024-02-11 22:36:03 +01:00
Mike Fährmann	dcc6e3f65c	merge #5134: [bunkr] add new bunkr domains (#5130)	2024-02-11 21:10:06 +01:00
Mike Fährmann	4641937ca3	[imagetwist] add 'gallery' extractor (#5190)	2024-02-11 18:41:02 +01:00
Mike Fährmann	fde82ab0ce	[imagechest] add 'user' extractor (#5143)	2024-02-11 18:38:33 +01:00
Mike Fährmann	4474cea31b	merge #5187: [skeb] add 'num' and 'count' metadata fields	2024-02-10 19:36:59 +01:00
Mike Fährmann	4cfceb23cb	[skeb] rename 'data' -> 'file' & add tests	2024-02-10 19:35:50 +01:00
Mike Fährmann	c83d0a1596	[weibo] add 'gifs' option (#5183)	2024-02-10 18:17:07 +01:00
blankie	f9a8e8cacf	[skeb] add 'num' and 'count' metadata fields	2024-02-10 21:51:23 +11:00
Mike Fährmann	af61d2b037	[wikimedia] combine most wikimedia.org sites (#1443) ... add wikidata.org and wikivoyage.org	2024-02-10 03:00:58 +01:00
Mike Fährmann	c7d17f1111	[bluesky] extract 'hashtags', 'mentions', and 'uris' metadata (#4438)	2024-02-10 00:01:55 +01:00
Mike Fährmann	55bbd49a0e	[bluesky] download images in original resolution (#4438) ... at least up to 2000 px	2024-02-09 21:33:33 +01:00
Mike Fährmann	6414dc6bca	[idolcomplex] fix pagination for tags containing ':' (#5171)	2024-02-09 17:51:08 +01:00
Mike Fährmann	5c2a2321a2	[bluesky] update refresh token after using it (#4438)	2024-02-08 22:33:34 +01:00
Mike Fährmann	9c10be54fb	[bluesky] add 'following' extractor (#4438)	2024-02-08 21:58:17 +01:00
Mike Fährmann	86ce35d6a1	[bluesky] simplify 'pattern'	2024-02-08 21:28:21 +01:00
Mike Fährmann	da292ded4e	[bluesky] add 'list' extractor (#4438)	2024-02-08 21:24:07 +01:00
Mike Fährmann	004bf7bb38	[bluesky] add 'feed' extractor (#4438)	2024-02-08 21:01:44 +01:00
Mike Fährmann	6aea818d4e	[bluesky] allow using DIDs as user handles (#4438)	2024-02-08 20:15:54 +01:00
Mike Fährmann	aee5580c62	[idolcomplex] extract 'id_alnum' metadata (#5171)	2024-02-08 18:29:54 +01:00
Mike Fährmann	cf7d6be2d4	[bluesky] initial support (#4438, #4708, #4722, #5047)	2024-02-07 19:09:33 +01:00
Mike Fährmann	6ef143ea31	[idolcomplex] support alphanumeric post IDs (#5171)	2024-02-07 14:57:13 +01:00
Mike Fährmann	6e928300bc	[flickr] handle non-JSON errors (#5131)	2024-02-06 21:22:10 +01:00
Mike Fährmann	90ac6d7375	[wikimedia] use '/api.php' as default API path	2024-02-06 00:36:51 +01:00
Mike Fährmann	d7823b9f81	[pinterest] fix section URLs for boards with /?# in name (#5104)	2024-02-05 15:54:06 +01:00
Mike Fährmann	de752eb7b1	[naverwebtoon] support '/webtoon/' paths for all comics (#5123)	2024-02-04 21:38:46 +01:00
Jeff Mercado	d9d0601ab1	break up line to fit 80 char	2024-01-29 20:31:58 -08:00
Jeff Mercado	6bcd3c9380	[bunkr] add new bunkr domains (#5130)	2024-01-29 20:25:33 -08:00
Mike Fährmann	62d6f5f8d2	[luscious] fix IndexError for files without thumbnail (#5122)	2024-01-28 01:43:29 +01:00
Mike Fährmann	22647c2626	[naverwebtoon] fix 'title' for comics with empty tags (#5120)	2024-01-27 16:24:03 +01:00
Mike Fährmann	3433481dd2	[gofile] update 'website_token' extraction	2024-01-27 01:10:14 +01:00
Mike Fährmann	1f7101d606	[archivedmoe] fix thebarchive webm URLs (#5116)	2024-01-27 00:24:41 +01:00
Mike Fährmann	34a4ddc399	[sankaku] add 'id-format' option (#5073)	2024-01-26 17:56:08 +01:00
Mike Fährmann	afd20ef42c	[kemonoparty] implement filtering duplicate revisions (#5013) ... set 'revisions' to '"unique"' to have it ignore duplicate revisions	2024-01-26 14:44:15 +01:00
Mike Fährmann	c28475d325	[kemonoparty] fix deleting 'name' in orginal objects (#5103) ... ... when computing 'revision_hash' regression caused by 3d68eda4 dict.copy() only creates a shallow copy I know that and still managed to get I wrong ...	2024-01-25 23:46:19 +01:00
Mike Fährmann	beacfa7436	[bunkr] update domain to 'bunkr.sk' (#5114)	2024-01-25 23:45:41 +01:00
Mike Fährmann	67c99b1366	[patreon] prevent HttpError for stream.mux.com URLs	2024-01-21 22:50:40 +01:00
Mike Fährmann	f3ad91b44f	[bunkr] update domain (#5088)	2024-01-21 03:00:57 +01:00
Mike Fährmann	c7a42880ab	[wikimedia] support fandom wikis (#1443, #2677, #3378) ... Wikis hosted on fandom.com are just wikimedia instances and support its API.	2024-01-21 00:52:02 +01:00
Mike Fährmann	5bf156f0b1	merge #5094: [webtoons] fix extracting comic and episode name with commas	2024-01-21 00:47:26 +01:00
blankie	df718887c2	[webtoons] fix extracting comic and episode name with commas	2024-01-21 09:50:27 +11:00
Wiiplay123	6eb62f2140	Combine lh*(-**).googleusercontent.com URL regex into one line. ... Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>	2024-01-20 15:53:11 -06:00
Wiiplay123	a6fed628dd	[blogger] Fix lh*.googleusercontent.com forward slash bug, add support for lh*-**.googleusercontent.com ... Some URLs use "lh(number)-(locale).googleusercontent.com" format, so I added support for those. Also, "lh(number).googleusercontent.com" formats were broken because the regex was looking for a second forward slash. Examples: lh7.googleusercontent.com lh7-us.googleusercontent.com	2024-01-20 15:07:52 -06:00
Mike Fährmann	6f8592eaff	[hbrowse] remove from modules list	2024-01-20 18:25:38 +01:00
Mike Fährmann	acc94ac187	[realbooru] fix extraction ... revert ac97aca99c	2024-01-20 17:56:07 +01:00
Mike Fährmann	9599151118	[issuu] fix extraction	2024-01-20 16:44:48 +01:00
Mike Fährmann	9ca6117c67	[hbrowse] remove module ... website gone	2024-01-20 02:53:44 +01:00
Mike Fährmann	375eefb886	[chevereto] remove 'pixl.li' ... "Pixl is closing down" "All images will be deleted January 1st."	2024-01-20 02:21:40 +01:00
Mike Fährmann	321861af7e	[erome] fix 'count' metadata	2024-01-20 00:26:41 +01:00
Mike Fährmann	b41d9bf616	[paheal] fix 'source' metadata	2024-01-19 22:24:39 +01:00
Mike Fährmann	b0a441f1e3	[nitter] remove 'nitter.lacontrevoie.fr' ... "Fermeture de Nitter / Closing down Nitter"	2024-01-19 19:34:16 +01:00
Mike Fährmann	a1c1e80f67	[giantessbooru] update domain	2024-01-19 14:21:56 +01:00
Mike Fährmann	2007cb2f59	[tests] check extractor category values	2024-01-19 14:21:09 +01:00
Mike Fährmann	fc4e737f67	[wikimedia] include 'sha1' in default filenames	2024-01-19 03:08:43 +01:00
Mike Fährmann	44f2c15a04	[wikimedia] handle 'File:' paths	2024-01-19 03:05:45 +01:00
Mike Fährmann	93b4120e77	[gelbooru] support 'all' and empty tag (#5076)	2024-01-18 21:49:33 +01:00
Mike Fährmann	a416d4c3d5	[sankaku] support post URLs with alphanumeric IDs (#5073)	2024-01-18 16:23:14 +01:00
Mike Fährmann	ea553a1d55	[wikimedia] generalize (#1443) ... - support mediawiki.org - support mariowiki.com (#3660) - combine code into a single extractor (use prefix as subcategory) - handle non-wiki instances - unescape titles	2024-01-18 15:36:16 +01:00
Mike Fährmann	89066844f4	add 'config_instance' method ... to allow for a more streamlined access to BaseExtractor instance options	2024-01-18 03:20:36 +01:00
Mike Fährmann	c3c1635ef3	[wikimedia] update ... - rewrite using BaseExtractor - support most Wiki* domains - update docs/supportedsites - add tests	2024-01-17 00:08:06 +01:00
Ailothaen	221f54309c	[wikimedia] Improved archive identifiers	2024-01-16 02:32:32 +01:00
Ailothaen	e33056adcd	[wikimedia] Add Wikipedia/Wikimedia extractor	2024-01-16 02:32:25 +01:00
Mike Fährmann	3d68eda4ab	[kemonoparty] add 'revision_hash' metadata (#4706, #4727, #5013) ... A SHA1 hexdigest of other relevant metadata fields like title, content, file and attachment URLs. This value does NOT reflect which revisions are listed on the website. Neither does 'edited' or any other metadata field (combinations).	2024-01-16 00:38:10 +01:00
Mike Fährmann	799a8206ad	merge #5061: [webtoons] extract more metadata ... - author_name - comic_name - episode_name - username	2024-01-15 18:27:12 +01:00
Mike Fährmann	8ffa0cd3c8	[webtoons] small optimization ... don't extract the entire 'author_area' and avoid creating a second 'text.extract_from()' object	2024-01-15 18:24:47 +01:00
Mike Fährmann	59cf4b3884	merge #4444: [2ch] add 'thread' and 'board' extractors (#1009, #3540)	2024-01-15 17:50:34 +01:00
Mike Fährmann	90b382304a	[deviantart] fix KeyError: 'premium_folder_data' (#5063)	2024-01-15 17:30:03 +01:00
Mike Fährmann	4cedf378d5	[deviantart] fix AttributeError for URLs without username (#5065) ... caused by 4f367145	2024-01-15 16:28:57 +01:00
Mike Fährmann	68196589c4	[2ch] update ... - simplify extractor code - more metadata - add tests	2024-01-15 04:09:05 +01:00
hunter-gatherer8	6c4abc982e	[2ch] add 'thread' and 'board' extractors ... - [2ch] add thread extractor - [2ch] add board extractor - [2ch] add new entry to supported sites	2024-01-15 03:51:03 +01:00
blankie	bb446b1598	[webtoons] extract more metadata	2024-01-14 19:26:49 +11:00
Mike Fährmann	355b909f46	merge #5041: [steamgriddb] add support (#5033)	2024-01-13 00:59:15 +01:00
Mike Fährmann	71e2c3e5a2	merge #5037: [hatenablog] add support (#5036)	2024-01-13 00:57:21 +01:00
blankie	9f53daabb8	[hatenablog] implement additional suggestion	2024-01-13 10:43:25 +11:00
blankie	293f1559df	[hatenablog] implement suggestions	2024-01-13 10:42:22 +11:00
blankie	65f42442f5	[steamgriddb] implement another suggestion	2024-01-13 10:12:15 +11:00
blankie	8995fd5f01	[steamgriddb] implement suggestions	2024-01-13 09:55:39 +11:00
Mike Fährmann	2dcfb012ea	[patreon] download 'm3u8' manifests with ytdl	2024-01-12 02:33:27 +01:00
Mike Fährmann	1c68b7df01	[patreon] fix KeyError (#5048)	2024-01-11 17:56:47 +01:00
Mike Fährmann	2191e29e14	[nijie] fix image URL for single image posts (#5049)	2024-01-11 05:07:38 +01:00
Mike Fährmann	bbf96753e2	[gelbooru] only log "Incomplete API response" for favorites (#5045)	2024-01-10 17:27:46 +01:00
Mike Fährmann	39904c9e4e	[deviantart:avatar] add 'formats' option (#4995)	2024-01-10 17:13:34 +01:00
Mike Fährmann	5c43098a1a	[twitter] revert to using 'media' timeline by default (#4953) ... This reverts commit a94f944148.	2024-01-09 23:19:39 +01:00
Mike Fährmann	5f9a98cf0f	[deviantart:avatar] fix exception when 'comments' are enabled (#4995)	2024-01-09 20:04:46 +01:00
Mike Fährmann	887ade30a5	[batoto] support more mirror domains (#5042)	2024-01-09 18:02:49 +01:00
Mike Fährmann	0a382a5092	[batoto] improve 'manga_id' extraction (#5042)	2024-01-09 17:25:04 +01:00
blankie	100966b122	[steamgriddb] fix linting error	2024-01-09 17:18:31 +11:00
blankie	2ccb7d3bd3	[steamgriddb] add support	2024-01-09 17:12:56 +11:00
Mike Fährmann	ec958a26bc	[fuskator] make metadata extraction non-fatal (#5039) ... - prevent KeyErrors - prevent HTTP redirect - return file URLs as list	2024-01-08 19:18:12 +01:00
blankie	2cfe788f93	[hatenablog] fix extractor naming errors	2024-01-09 01:42:57 +11:00
blankie	be6949c55d	[hatenablog] fix linting error	2024-01-09 01:36:52 +11:00
blankie	61f3b2f820	[hatenablog] add support	2024-01-09 01:29:47 +11:00
Mike Fährmann	657ed93a22	[batoto] improve v2 manga URL pattern ... and add tests	2024-01-07 22:23:30 +01:00
Mike Fährmann	50eef1b5cc	merge #5029: [pixiv] update App API headers	2024-01-07 16:27:13 +01:00
Mike Fährmann	33f228756a	[mangadex] add 'list' extractor (#5025) ... supports listing manga and chapters from list feed	2024-01-07 02:59:35 +01:00
Mike Fährmann	db8de13537	[vk] transform image URLs to non-blurred versions (#5017) ... apply the same filter from before d85e66bc	2024-01-07 00:19:10 +01:00
Se AKi	d0d199414f	modify useragent of pixiv	2024-01-06 23:15:15 +09:00
Mike Fährmann	cbfb7bfdf1	[gelbooru] display error for invalid API responses (#4903)	2024-01-06 14:28:35 +01:00
Mike Fährmann	c25bdbae91	[komikcast] fix 'manga' extractor (#5027)	2024-01-06 14:19:44 +01:00
Mike Fährmann	8e1a2b5446	[komikcast] update domain to 'komikcast.lol' (#5027)	2024-01-06 02:16:43 +01:00
Mike Fährmann	a441249ea2	merge #4979: [batoto] add 'chapter' and 'manga' extractors (#1434, #2111)	2024-01-06 01:53:26 +01:00
Mike Fährmann	b11c352d66	[bato] rename to 'batoto' ... to use the same category name as the previous bato.to site	2024-01-06 01:49:34 +01:00
Mike Fährmann	3aa24c3744	[bato] simplify and update	2024-01-06 01:10:04 +01:00
Mike Fährmann	11150a7d72	[nudecollect] remove module	2024-01-05 21:32:04 +01:00
Mike Fährmann	c158927c38	merge #5016: [zzup] add 'gallery' extractor (#4517, #4604, #4659, #4863)	2024-01-05 21:25:46 +01:00
Mike Fährmann	e61f016465	[szurubooru] support 'snootbooru.com' (#5023)	2024-01-05 17:56:39 +01:00
Mike Fährmann	b4bcf40278	[weibo] fix AttributeError in 'user' extractor (#5022) ... yet another bug caused by a383eca7	2024-01-05 17:18:33 +01:00
Mike Fährmann	0ab0a10d2d	[jpgfish] update domain	2024-01-05 02:27:20 +01:00
enduser420	0f30136109	[zzup] add 'gallery' extractor	2024-01-04 21:38:59 +05:30
Mike Fährmann	a86775f617	[gelbooru] fix 'favorite' extractor (#4903) ... lots of +1/-1 and </<= mistakes	2024-01-04 15:17:49 +01:00
Mike Fährmann	7eaf648f2e	[fanbox] add 'metadata' option (#4921) ... extracts 'plan' and extended 'user' metadata	2024-01-04 15:01:33 +01:00
Mike Fährmann	4f3671458e	[deviantart] add 'avatar' and 'background' extractors (#4995)	2024-01-03 00:07:55 +01:00
Mike Fährmann	9fa4f54c24	[twitter] raise error for invalid 'strategy' values (#4953)	2024-01-01 22:58:42 +01:00
Mike Fährmann	516c69297d	[manganelo] fix extraction & recognize '.to' TLDs (#5005)	2024-01-01 22:05:21 +01:00
Mike Fährmann	63f649cd92	[idolcomplex] fix extraction & update URL patterns (#5002)	2024-01-01 17:38:32 +01:00
Mike Fährmann	b6903a4c90	[nijie] add 'count' metadata field ... https://github.com/mikf/gallery-dl/issues/146#issuecomment-1812849102	2023-12-30 22:25:59 +01:00
Mike Fährmann	b93b351db9	merge #4962: [poringa] add support (#4675)	2023-12-30 20:39:35 +01:00
Mike Fährmann	9f21c839ad	[poringa] improvements and fixes ... - add 'num' and 'count' metadata fields - prevent crash for "private" posts - prevent crash when there's no 'main-info' - update tests	2023-12-30 20:37:09 +01:00
Mike Fährmann	00d83d9588	[rule34us] add fallback for 'video-cdn1' videos (#4985)	2023-12-30 18:33:46 +01:00
Mike Fährmann	085411f3f1	[rule34] recognize URLs with 'www' subdomain (#4984)	2023-12-30 16:07:56 +01:00
Mike Fährmann	9f5051e4ed	merge #4981: [pinterest] add 'count' metadata field	2023-12-30 15:53:35 +01:00
bug-assassin	f6ce870885	Better variable names	2023-12-28 17:25:15 -05:00
bug-assassin	3553025584	Removed f-strings	2023-12-28 17:07:41 -05:00
Mike Fährmann	f36dafad06	improve 'include' handling (#4982) ... - remove spaces when given as string - warn about invalid vales	2023-12-28 19:07:04 +01:00
blankie	375f2db4c2	[pinterest] add count metadata field	2023-12-28 01:07:04 +11:00
Antonio	e348da7a06	[poringa] add support	2023-12-27 00:07:23 -06:00
bug-assassin	2c3f171d65	Fix python 3.5 linting issue	2023-12-26 23:55:47 -05:00
bug-assassin	06ff1d3a3c	Replace text.extract with extr	2023-12-26 23:47:30 -05:00
bug-assassin	9c1ce28f68	[bato] Added mangatoto alias	2023-12-26 23:44:27 -05:00
bug-assassin	663b8d789a	Fix linting	2023-12-26 23:41:37 -05:00
bug-assassin	74c225f94e	[bato] add support	2023-12-26 22:33:33 -05:00
Mike Fährmann	f9544194c0	[paheal] restore 'extension' metadata (#4976)	2023-12-26 16:09:26 +01:00
Mike Fährmann	77d46e6f0c	[lynxchan] update 'bbw-chan' domain (#4970)	2023-12-25 15:29:05 +01:00
Mike Fährmann	766316e436	[imagechest] fix loading more than 10 images in a gallery (#4469)	2023-12-25 15:14:43 +01:00
Mike Fährmann	108c978073	merge #4919: [postmill] add support (#4917)	2023-12-23 15:23:56 +01:00
blankie	8a42ea736a	[postmill] implement suggestions	2023-12-23 13:28:36 +11:00
Mike Fährmann	c184454efb	[shimmie2] small optimizations ... - unroll/remove loop - avoid copy	2023-12-22 20:29:22 +01:00
Mike Fährmann	7cd0211cc9	[shimmie2] autodetect single or double quotes	2023-12-22 15:25:28 +01:00
Mike Fährmann	2a60645095	[deviantart] set 'is_original' for intermediary URLs to 'false'	2023-12-22 14:49:10 +01:00
Mike Fährmann	01bb75f6cb	merge #4945: {shimmie2[ support 'rule34hentai.net' (#861, #4789)	2023-12-22 00:10:26 +01:00
Mike Fährmann	79e4606893	[rule34hentai] cleanup ... - fix using 'self._posts_rule34hentai' - fix 'file_url' for posts - update docs/supportedsites - add tests	2023-12-22 00:01:36 +01:00
bun-dev	ef370df41d	[shimmie2] support 'rule34hentai.net' ... - Add files via upload - Update shimmie2.py - Update shimme2.py - Delete gallery_dl/extractor/shimme2.py - spacefix - Update shimmie2.py - Update shimmie2.py - flask warnings1 - Update shimmie2.py - Update shimmie2.py	2023-12-21 23:00:58 +01:00
Mike Fährmann	627ed794a2	[danbooru] provide 'tags' as list (#4942) ... keep the old 'tag_string' values around, similar to sankaku a lot of repeat code ... would be a lot less bad if "".split(" ") returned an empty list	2023-12-21 14:39:38 +01:00
Mike Fährmann	fbebc58189	[deviantart] add 'intermediary' option (#4955)	2023-12-21 02:23:22 +01:00
Mike Fährmann	75fa1a5553	[pinterest] remove login code ... this has been broken since forever and is still "protected" by an invisible recaptcha check	2023-12-20 20:59:18 +01:00
Mike Fährmann	92ff99c8e5	[twitter] remove 'syndication' option (#3889)	2023-12-20 14:38:36 +01:00
Mike Fährmann	a75f85a2c2	[twitter] remove 'date_liked' (#3850, #4108, #4657) ... Twitter's 'sortIndex' can't be used to calculate the timestamp of when a Tweet was liked anymore.	2023-12-20 14:12:49 +01:00
Mike Fährmann	a94f944148	[twitter] default to 'tweets' timeline when 'replies' are enabled (#4953)	2023-12-20 01:57:18 +01:00
Mike Fährmann	a30a3e44d5	[nijie] move 'username required' out of _login_impl	2023-12-18 23:57:44 +01:00
Mike Fährmann	57fc6fcf83	replace '24*3600' with '86400' ... and generalize cache maxage values	2023-12-18 23:57:22 +01:00
Mike Fährmann	1f9b16a70b	replace static 'sleep-request' defaults with dynamic ones	2023-12-18 22:06:26 +01:00
Mike Fährmann	b127321b5c	[exhentai] only show 'using e-hentai.org' warning for exh domains	2023-12-18 15:29:54 +01:00
Mike Fährmann	e097aaf64a	[exhentai] output continuation URL when interrupted (#4782)	2023-12-17 23:25:47 +01:00
Mike Fährmann	99aa923322	[inkbunny] improve '/submissionsviewall.php' patterns (#4934) ... allow 'mode=…' to be in any position don't require it to be somewhere in the middle	2023-12-16 19:21:20 +01:00
Mike Fährmann	3f9c113d78	[mastodon] Support non-numeric status IDs (#4936)	2023-12-16 01:52:31 +01:00
Mike Fährmann	2852404e49	[inkbunny] add 'unread' extractor (#4934)	2023-12-15 21:20:12 +01:00
Mike Fährmann	8b87a5330d	[inkbunny] stop pagination on empty results	2023-12-15 21:15:02 +01:00
Mike Fährmann	6cd5e6adad	[patreon] fix bootstrap data extraction (#4904)	2023-12-15 18:24:52 +01:00
Mike Fährmann	aac8bb4eae	[deviantart] simplify 9951c112	2023-12-15 17:39:42 +01:00
Mike Fährmann	9951c112f8	[deviantart] workaround for integer client_id values (#4924)	2023-12-14 16:25:06 +01:00
Mike Fährmann	a37b7759bc	[myhentaigallery] recognize '/g/' URLs (#4920)	2023-12-12 20:02:28 +01:00
Mike Fährmann	da76e13e3b	[tumblr] fix exception after waiting for rate limit (#4916) ... use a loop instead of recursive function calls	2023-12-12 19:14:06 +01:00
blankie	fbe14a2745	[postmill] add support	2023-12-12 21:36:52 +11:00
Mike Fährmann	d59d4ebff4	[tumblr] support infinite 'fallback-retries'	2023-12-11 23:40:13 +01:00
Mike Fährmann	2d5cda2b92	[exhentai] fix TypeError for infinite 'fallback-retries' (#4911)	2023-12-11 23:39:56 +01:00
Mike Fährmann	92fbf09643	remove single quotes in some logging messages (#4908) ... ('FileNotFoundError: [Errno 2] No such file or directory: ''') -> (FileNotFoundError: [Errno 2] No such file or directory: '')	2023-12-11 19:13:45 +01:00
Luc Ritchie	7dd79eee93	save cookies to tempfile, then rename ... avoids wiping the cookies file if the disk is full	2023-12-11 00:47:42 -05:00
Mike Fährmann	9a001fa6e4	merge #4906: [patreon] fix bootstrap data extraction (#4904)	2023-12-10 16:53:54 +01:00
Tobi823	66cbe9da41	- fix style check failure "line to long"	2023-12-10 16:24:00 +01:00
Tobi823	244444b194	- adapt code to current code style	2023-12-10 16:22:32 +01:00
Tobi823	fd06255f93	- reformat and refactor to pass tests	2023-12-10 16:17:34 +01:00
Tobi823	5ff7106d4f	- add code for the situation when Patreon is using window.patreon = wrapInProxy({"bootstrap":' to store metadata ... - refactor code to make it more readable - output page content when the HTML structure is unknown (to make debugging easier)	2023-12-10 16:10:46 +01:00
Mike Fährmann	ac22bbe80c	[twitter] retry API requests only for Timeout errors (#4811)	2023-12-09 22:30:48 +01:00
Mike Fährmann	c55955db03	[twitter] quick and dirty fix for /media changes (#4898)	2023-12-09 15:38:42 +01:00
Mike Fährmann	9a8dc6b02b	[exhentai] add 'fallback-retries' option (#4792)	2023-12-09 02:01:59 +01:00
Mike Fährmann	bf74eb5c46	merge #4886: [urlgalleries] add 'gallery' extractor (#919, #1184, #2905)	2023-12-08 22:55:58 +01:00
Mike Fährmann	c29ae9af08	[urlgalleries] simplify + resolve redirects	2023-12-08 22:43:56 +01:00
Mike Fährmann	e256434c9e	use custom HTTPBasicAuth class ... to support LazyPrompt as password and to generate the Authorization header only once instead of for every request	2023-12-07 01:40:31 +01:00
Mike Fährmann	6a4218aa23	handle 'json' parameter in Extractor.request() manually ... Mainly to allow passing custom classes like util.LazyPrompt, but also to simplify and streamline how requests handles it.	2023-12-06 22:13:13 +01:00
Mike Fährmann	9dd5cb8c8a	interactively prompt for passwords on login when none is provided	2023-12-06 22:12:59 +01:00
Mike Fährmann	4eb3590103	[nijie] fix image URLs of multi-image posts (#4876)	2023-12-05 17:48:50 +01:00
Mike Fährmann	a4e6ea667b	[twitter] retry API calls when their response contains errors (#4811)	2023-12-05 15:57:26 +01:00
Mike Fährmann	cf5702c843	[twitter] generalize "Login Required" error (#4734, #4324)	2023-12-05 15:13:58 +01:00
jsouthgb	ecaa0feb5d	[urlgalleries] add support	2023-12-05 07:08:11 -05:00
jsouthgb	1770c31e63	[urlgalleries] add support	2023-12-05 07:07:06 -05:00
Mike Fährmann	da0da0faaa	[exhentai] store more cookies when logging in (#4881) ... include 'igneous', 'hath_perks', etc and not just 'ipb_member_id' and 'ipb_pass_hash' like before	2023-12-04 23:24:11 +01:00
Mike Fährmann	c83fbe6c2d	merge #4855: [nitter] fix video extraction (#4853)	2023-11-27 18:39:05 +01:00
Mike Fährmann	013ca21543	[idolcomplex] update to site layout changes	2023-11-27 18:27:08 +01:00
enduser420	1e9bacd169	[nitter] fix video extraction	2023-11-27 21:58:06 +05:30
Mike Fährmann	9f3368c46f	[pornhub] fix 'user' metadata for gifs	2023-11-26 23:52:24 +01:00
Mike Fährmann	bdb3ce7217	[foolslide] remove 'powermanga.org'	2023-11-26 23:19:05 +01:00
Mike Fährmann	d9734ce008	[cyberdrop] update to site layout changes	2023-11-26 18:03:13 +01:00
Mike Fährmann	8ac68ffba2	[hentaicosplays] force 'https://' for download URLs	2023-11-26 02:08:12 +01:00
Mike Fährmann	fc1101779c	[hiperdex] fix 'manga' metadata	2023-11-26 01:24:42 +01:00
Mike Fährmann	d119507037	[imagefap] fix single image resolution ... Downloading from a single image page like https://www.imagefap.com/photo/123456789/ returned only the thumbnail URL.	2023-11-26 00:30:52 +01:00
Mike Fährmann	311ec1d9ef	[mangaread] fix extraction	2023-11-25 23:53:27 +01:00
Mike Fährmann	7608201a44	[tumblr] fix 'day' extractor ... another bug caused by a383eca7	2023-11-25 00:51:14 +01:00
Mike Fährmann	c8c744a7c0	[webtoons] fix pagination when receiving an HTTP redirect	2023-11-24 22:17:34 +01:00
Mike Fährmann	23cd17997d	[wallpapercave] fix extraction	2023-11-24 21:54:21 +01:00
Mike Fährmann	5b979b5706	[xvideos] fix metadata extraction	2023-11-24 21:27:19 +01:00
Mike Fährmann	adc3aa0b77	[zerochan] fix metadata extraction ... author, path, tags	2023-11-24 21:21:14 +01:00
Mike Fährmann	f9dac43be9	[warosu] fix file URLs	2023-11-24 02:44:55 +01:00
Mike Fährmann	645b4627ef	[sankaku] update URL patterns	2023-11-24 02:41:52 +01:00
Mike Fährmann	1ae43d8123	merge #4841: [fapello] support '.su' TLD (#4840)	2023-11-22 20:18:32 +01:00
Mike Fährmann	b43be67206	[exhentai] add 'gp' option (#4576)	2023-11-22 20:15:00 +01:00
Mike Fährmann	cb9a1176e6	[pixeldrain] add 'api-key' option (#4839)	2023-11-22 19:13:51 +01:00
Mike Fährmann	e1404827a6	[pixeldrain] add 'file' and 'album' extractors (#4839)	2023-11-22 19:01:19 +01:00
enduser420	2402162e8a	[fapello] support '.su' TLD	2023-11-22 19:35:43 +05:30
Mike Fährmann	725c8dd55a	[tmohentai] 'categories' -> 'genres' ... quite likely that the site meant 'genres' by "Genders"	2023-11-21 22:11:43 +01:00
Mike Fährmann	ce7c4cb544	merge #4832: [tmohentai] add 'gallery' extractor (#4808)	2023-11-21 20:25:49 +01:00
Mike Fährmann	c4a201ed42	[tmohentai] simplify + tests	2023-11-21 20:24:07 +01:00
Mike Fährmann	e17a48fe56	[blogger] inherit from BaseExtractor ... - support www.micmicidol.club (#4759)	2023-11-21 16:52:25 +01:00
jsouthgb	714b1a7089	[tmohentai] simplify url matching	2023-11-21 10:46:48 -05:00
jsouthgb	31963fa947	[tmohentai] inherit from GalleryExtractor. refactor metadata.	2023-11-20 21:35:32 -05:00
Mike Fährmann	0fa85360a0	merge #4812: [erome] add 'count' metadata field	2023-11-20 22:42:02 +01:00
Mike Fährmann	a43cf78bb7	[erome] tests	2023-11-20 22:41:12 +01:00
Mike Fährmann	34a387b6e2	support 'metadata-*' names for '*-metadata' options ... For example, instead of 'url-metadata' it is now also possible to use 'metadata-url' as option name. - metadata-url - metadata-path - metadata-http - metadata-version - metadata-parent	2023-11-18 23:52:10 +01:00
Mike Fährmann	e97d7b1c85	[exhentai] fix empty api_url with '"source": "hitomi"' (#4829)	2023-11-18 21:06:41 +01:00
jsouthgb	ed965eecbb	[tmohentai] refactor to str.format for backwards compatibility	2023-11-18 14:39:17 -05:00
jsouthgb	dad7ba1d58	[tmohentai] fix edge cases. updated archive_fmt and filename_fmt	2023-11-17 21:08:34 -05:00
jsouthgb	286d0cb098	[tmohentai] add support	2023-11-17 19:34:34 -05:00
Mike Fährmann	07cb584231	[behance] add 'modules' option (#4799)	2023-11-17 22:54:38 +01:00
Mike Fährmann	6a753d9ff3	[behance] support 'text' modules (#4799)	2023-11-17 22:54:38 +01:00
jsouthgb	c6ad9bcd9b	[erome] add "count" for albums	2023-11-15 07:08:52 -05:00
Mike Fährmann	17e710c4bf	[oauth] warn when cache is enabled but not writeable (#4771)	2023-11-13 16:24:54 +01:00
Mike Fährmann	2e4bf54644	[hentaifoundry] check for and update expired sessions (#4694)	2023-11-13 00:27:22 +01:00
Mike Fährmann	0435c6e603	[exhentai] handle 'Downloading … requires GP' errors (#4576, #4763)	2023-11-12 22:47:40 +01:00
Mike Fährmann	4288cea94a	[mastodon] fix reblogs (#4580)	2023-11-11 00:34:49 +01:00
Mike Fährmann	7a0f145cbe	[twitter] ignore promoted Tweets (#4790, #3894) ... add 'ads' option in case someone actually wants to download promoted content for whatever reason	2023-11-10 23:46:46 +01:00
Mike Fährmann	e8b5e59a08	[weibo] detect redirects to login page (#4773)	2023-11-10 19:35:29 +01:00
Mike Fährmann	5e58d2b455	[instagram] fix exception on empty 'video_versions' (#4795)	2023-11-10 14:49:29 +01:00
Mike Fährmann	61d6558322	[exhentai] try to avoid 'DH_KEY_TOO_SMALL' errors (#1021, #4593)	2023-11-04 17:30:27 +01:00
Mike Fährmann	69b931b9bb	[exhentai] provide fallback URLs (#1021, #4745)	2023-11-04 17:06:46 +01:00
Mike Fährmann	007c433677	[patreon] support 'id:<campaign_id>' in place of a user name ... https://patreon.com/id:12345 … and remove 'campaign-id' config option	2023-11-04 00:17:41 +01:00
Mike Fährmann	3984a49abf	[nijie] set 1-2s delay between requests to avoid 429 errors	2023-11-03 23:44:47 +01:00