2489 Commits

Author	SHA1	Message	Date
Mike Fährmann	d8d9502e1e	[reactor] inherit from BaseExtractor	2021-11-26 18:58:07 +01:00
Mike Fährmann	f4ea216c95	[shopify] support loungeunderwear.com (closes #2053)	2021-11-26 18:58:06 +01:00
Mike Fährmann	93cef78450	[gelbooru] workaround pagination limits ... Gelbooru only allows to retrieve the latest 20k posts for a tag search. Add 'id:<N' to the search tags to work around that limitation, where N is the ID of the last retrieved post. http://gelbooru.me/index.php?page=forum&s=view&id=1467	2021-11-26 18:56:31 +01:00
Mike Fährmann	f2ae179713	[exhentai] fix extraction for disowned galleries (closes #2055)	2021-11-24 21:26:16 +01:00
Alice	612850438e	[skeb] add 'thumbnails' option (#2047) (#2051)	2021-11-23 21:16:42 +01:00
Mike Fährmann	11a3d96d13	[mangadex] load additional metadata using includes[] directives ... - always provide 'artist', 'author', and 'group' metadata fields (#2049) - remove 'metadata' option	2021-11-22 01:16:33 +01:00
Mike Fährmann	19e00f1322	[dynastyscans] provide 'date' as proper datetime object (#2050)	2021-11-21 22:50:52 +01:00
Mike Fährmann	af6424f398	allow testing metadata in list elements	2021-11-21 22:46:34 +01:00
Mike Fährmann	c67756e187	[kemonoparty] add 'dms' option (#2008)	2021-11-20 23:36:16 +01:00
Mike Fährmann	3a7a19c7b9	[dynastyscans] add 'manga' extractor (closes #2035)	2021-11-19 22:51:26 +01:00
Mike Fährmann	9bc83af3a6	[kemonoparty] 'postfile' -> 'file' (#1991) ... to stay consistent with the existing file types for kemono	2021-11-19 01:50:48 +01:00
Mike Fährmann	522782c09d	[subscribestar] emit metadata for posts without media (#1569)	2021-11-18 23:42:17 +01:00
Mike Fährmann	1c8aaf9318	[subscribestar] add 'num' enumeration index (closes #2040)	2021-11-18 23:38:41 +01:00
Mike Fährmann	d433735750	[kemonoparty] skip duplicate files (#2032, #1991, #1899) ... Extract the SHA-256 file hash from URLs and skip files with the same hash in the same post. - provide a 'hash' metadata field (empty string if not available) - remove 'patreon-skip-file' option	2021-11-17 22:44:15 +01:00
Mike Fährmann	d4ec245554	[kemonoparty] implement a 'files' option (#1991) ... similar to 8d676151	2021-11-17 22:43:41 +01:00
Mike Fährmann	ab8eea1a24	[twitter] fix extractor for direct image links (fixes #2030)	2021-11-16 22:57:46 +01:00
Mike Fährmann	2076d40681	[ytdl] improve error handling (#1680)	2021-11-15 22:56:42 +01:00
Mike Fährmann	2aaac3c997	[instagram] include user metadata for 'tagged' downloads (#2024) ... Adds - tagged_owner_id - tagged_full_name - tagged_username containing the values for the user profile the URL originated from, e.g. 'instagram' for https://www.instagram.com/instagram/tagged/.	2021-11-15 21:21:59 +01:00
Mike Fährmann	cfa4876848	[philomena] support furbooru.org (closes #1995)	2021-11-15 20:57:51 +01:00
Mike Fährmann	4377f1c284	[twitter] distinguish between fatal & nonfatal errors (#2020) ... only show a warning for nonfatal errors and do not raise a StopExtraction exception	2021-11-13 22:46:40 +01:00
Kyle Anthony Williams	a14b72be21	[webtoons] Use swebtoon-phinf.pstatic.net instead of webtoon-phinf.pstatic.net (#2005) ... * [webtoons] Use swebtoon-phinf.pstatic.net instead of webtoon-phinf.pstatic.net This trick to avoid having to set a Referer header comes from Webtoon's RSS feeds. The two URLs below are equivalent in content: https://webtoon-phinf.pstatic.net/20210929_153/1632867980912DmcGK_JPEG/16328679808882705182.jpg?type=q90 https://swebtoon-phinf.pstatic.net/20210929_153/1632867980912DmcGK_JPEG/16328679808882705182.jpg?type=q90 The URL with the domain "webtoon-phinf.pstatic.net" needs a Referer header, and the domain "swebtoon-phinf.pstatic.net" does not. This is because of the environment "swebtoon" images live in, one without explicit network control: RSS feeds on sites such as Feedly. This change should make it easier for gallery-dl developers to embed Webtoon comics without worrying about headers.	2021-11-11 20:03:34 +01:00
Mike Fährmann	6e3658ef52	[kemonoparty] provide 'date' metadata for gumroad (#2007) ... Not the 'published' or 'edited' values since they are 'null', but still better then nothing at all.	2021-11-11 19:38:10 +01:00
Mike Fährmann	37c9dedee1	[seisoparty] remove module	2021-11-09 22:41:04 +01:00
Mike Fährmann	efa178cc91	[ytdl] implement parsing ytdl command-line options (#1680) ... - adds 'config-file' and 'cmdline-args' options for both ytdl downloader and extractor - create 'ytdl' helper module, which combines YoutubeDL creation and option parsing. - most likely a buggy mess due to incompatibilities between the original youtube-dl and yt-dlp.	2021-11-07 02:44:11 +01:00
Mike Fährmann	7cb303d745	[redgifs] improve URL extraction ... Fields inside 'urls' can be None, which would have caused an exception with the old method.	2021-11-05 20:02:43 +01:00
Mike Fährmann	2befed1a96	[redgifs] update search URL pattern (#1984)	2021-11-05 20:00:06 +01:00
Mike Fährmann	b315a0ecef	[redgifs] update to API v2 (#1984)	2021-11-04 21:31:20 +01:00
Mike Fährmann	f0fc3b0ba1	[kemonoparty] add 'comments' option (#1980)	2021-11-03 23:02:13 +01:00
Mike Fährmann	1fac74b14d	[reddit] prevent crash for galleries with no 'media_metadata' ... (fixes #2001)	2021-11-03 17:55:40 +01:00
Mike Fährmann	211de95dd0	update extractor test results	2021-11-01 02:58:53 +01:00
Mike Fährmann	8bea02c38c	[deviantart] fix 'index' values for stashed deviations	2021-11-01 01:08:24 +01:00
Mike Fährmann	dd88a7d980	{cyberdrop] restore video extraction (fixes #1993) ... fixes a regression introduced in f33c2ef7	2021-10-31 04:34:01 +01:00
Mike Fährmann	fa5646eadc	[mangoxo] fix login and extraction	2021-10-31 02:16:13 +01:00
Mike Fährmann	4c49174579	[mangakakalot] update domain and fix extraction	2021-10-31 02:16:13 +01:00
YongChan Cho	14852f7050	[hitomi] fix image path (#1988)	2021-10-30 21:45:01 +02:00
Mike Fährmann	dad2875a3e	fix calculating retry sleep times (fixes #1990)	2021-10-29 23:53:48 +02:00
Mike Fährmann	9156e90f1f	[twitter] add 'pinned' option	2021-10-29 22:10:58 +02:00
Mike Fährmann	06b414c9a3	[redgifs] 'gfyId' -> 'id' (#1984)	2021-10-29 02:05:39 +02:00
Ryu juheon	d4614e5ba4	[hitomi] fix image URLs (#1982)	2021-10-28 19:29:48 +02:00
Mike Fährmann	6434ccf9e8	[redgifs] split from 'gfycat' (#1984) ... Update API endpoints and metadata names - mostly 'gfycat' -> 'gif' - and remove some obsolete checks.	2021-10-28 19:22:41 +02:00
Mike Fährmann	e4696b40ba	[instagram] update query hashes	2021-10-27 21:37:31 +02:00
Alice	bfd7401b1e	[skeb] add 'user' and 'post' extractors (#1031) (#1971) ... * Create skeb.py * Update __init__.py * Update supportedsites.py * Update supportedsites.md * Update supportedsites.py * Update skeb.py	2021-10-26 20:00:41 +02:00
Ryu juheon	6b6d92d51c	[hitomi]: fix image URLs (#1975)	2021-10-26 19:35:01 +02:00
Mike Fährmann	dcb201ff19	[gfycat] show warning when there are no available formats	2021-10-26 19:26:50 +02:00
Mike Fährmann	e436a2607b	[gfycat] consistent 'userName' values for 'user' downloads (#1962) ... by using the name from the input URL and not relying on possibly faulty or incomplete API results. 'userData[username]', if available, will still have the original name.	2021-10-26 19:15:30 +02:00
Mike Fährmann	f1487a3cfa	[kemonoparty:discord] improve 'inline' extraction (#1940) ... - extract media.discordapp.*NET* URLs - rewrite media.discordapp.net to cdn.discordapp.com - use a more restricted set of characters for the URL path	2021-10-24 21:15:21 +02:00
Mike Fährmann	02a247f4e5	[deviantart] full resolution for non-downloadable images (#293) ... Many thanks to @Ironchest337 for discovering this method and providing a well-documented implementation.	2021-10-24 21:11:12 +02:00
Mike Fährmann	a7ddb5f5fa	[deviantart] update 'search' argument handling (fixes #1911) ... - use 'alltime' by default - support newer 'order' values (most-recent, this-week, etc)	2021-10-23 21:48:02 +02:00
Mike Fährmann	c19e762fdf	[vk] add 'album' extractor (#474, fixes #1952) ... todo: better metadata for albums	2021-10-23 00:46:20 +02:00
Mike Fährmann	8bb442f20d	[redgifs][gfycat] provide fallback URLs (fixes #1962) ... and extend the 'format' option	2021-10-22 22:47:29 +02:00