3427 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Mike Fährmann	f1b142e993	{kemonoparty[ change default 'files' order to attachments,file,inline ... (#1991)	2021-11-29 04:41:30 +01:00
Mike Fährmann	0054ad434e	[output] write directly to sys.stdout	2021-11-29 04:41:29 +01:00
Mike Fährmann	da14b3fe9f	[output] write download progress indicator to stderr	2021-11-29 04:41:17 +01:00
Mike Fährmann	604d5b8bb2	release version 1.19.3	2021-11-27 23:13:35 +01:00
Mike Fährmann	275543b2d2	update extractor test results	2021-11-27 19:26:44 +01:00
Mike Fährmann	e7ea4f2567	[mangoxo] fix metadata extraction	2021-11-27 18:19:51 +01:00
adamus1red	1efef60350	Add py3.10 to tested python versions (#2056)	2021-11-26 20:30:49 +01:00
Mike Fährmann	e298882acc	[kemonoparty] match URLs with www subdomain	2021-11-26 18:58:26 +01:00
Mike Fährmann	addb72e1bb	[reactor] support thatpervert.com (closes #2029)	2021-11-26 18:58:07 +01:00
Mike Fährmann	d8d9502e1e	[reactor] inherit from BaseExtractor	2021-11-26 18:58:07 +01:00
Mike Fährmann	f4ea216c95	[shopify] support loungeunderwear.com (closes #2053)	2021-11-26 18:58:06 +01:00
Mike Fährmann	93cef78450	[gelbooru] workaround pagination limits ... Gelbooru only allows to retrieve the latest 20k posts for a tag search. Add 'id:<N' to the search tags to work around that limitation, where N is the ID of the last retrieved post. http://gelbooru.me/index.php?page=forum&s=view&id=1467	2021-11-26 18:56:31 +01:00
Mike Fährmann	f2ae179713	[exhentai] fix extraction for disowned galleries (closes #2055)	2021-11-24 21:26:16 +01:00
Alice	612850438e	[skeb] add 'thumbnails' option (#2047) (#2051)	2021-11-23 21:16:42 +01:00
Mike Fährmann	010d65dcec	extend blacklist/whitelist syntax (#2025) ... Each entry in such a list can now also include a subcategory '<category>:<subcategory>' and it is possible to use '*' or an empty string as placeholder '*:<subcategory>', ':<subcategory>', '<category>:*' For example "blacklist": "imgur,*:tag,gfycat:user" or "blacklist": ["imgur", "*:tag", "gfycat:user"] will filter all 'imgur' extractors, all extractors with a 'tag' subcategory (e.g. https://danbooru.donmai.us/posts?tags=bonocho), and all 'gfycat' user extractors.	2021-11-23 20:31:43 +01:00
Mike Fährmann	11a3d96d13	[mangadex] load additional metadata using includes[] directives ... - always provide 'artist', 'author', and 'group' metadata fields (#2049) - remove 'metadata' option	2021-11-22 01:16:33 +01:00
Mike Fährmann	19e00f1322	[dynastyscans] provide 'date' as proper datetime object (#2050)	2021-11-21 22:50:52 +01:00
Mike Fährmann	af6424f398	allow testing metadata in list elements	2021-11-21 22:46:34 +01:00
Mike Fährmann	c67756e187	[kemonoparty] add 'dms' option (#2008)	2021-11-20 23:36:16 +01:00
Mike Fährmann	3842cdcd8f	[formatter] implement 'D' format specifier ... To be able to parse any string into a 'datetime' object and format it as necessary. Example: {created_at:D%Y-%m-%dT%H:%M:%S%z} -> "2010-01-01 00:00:00" {created_at:D%Y-%m-%dT%H:%M:%S%z/%b %d %Y %I:%M %p} -> "Jan 01 2010 12:00 AM" with 'created_at' == "2010-01-01T01:00:00+0100"	2021-11-20 23:04:34 +01:00
Mike Fährmann	3a7a19c7b9	[dynastyscans] add 'manga' extractor (closes #2035)	2021-11-19 22:51:26 +01:00
Mike Fährmann	9bc83af3a6	[kemonoparty] 'postfile' -> 'file' (#1991) ... to stay consistent with the existing file types for kemono	2021-11-19 01:50:48 +01:00
Mike Fährmann	522782c09d	[subscribestar] emit metadata for posts without media (#1569)	2021-11-18 23:42:17 +01:00
Mike Fährmann	1c8aaf9318	[subscribestar] add 'num' enumeration index (closes #2040)	2021-11-18 23:38:41 +01:00
Mike Fährmann	d433735750	[kemonoparty] skip duplicate files (#2032, #1991, #1899) ... Extract the SHA-256 file hash from URLs and skip files with the same hash in the same post. - provide a 'hash' metadata field (empty string if not available) - remove 'patreon-skip-file' option	2021-11-17 22:44:15 +01:00
Mike Fährmann	d4ec245554	[kemonoparty] implement a 'files' option (#1991) ... similar to 8d676151	2021-11-17 22:43:41 +01:00
Mike Fährmann	ab8eea1a24	[twitter] fix extractor for direct image links (fixes #2030)	2021-11-16 22:57:46 +01:00
Mike Fährmann	2076d40681	[ytdl] improve error handling (#1680)	2021-11-15 22:56:42 +01:00
Mike Fährmann	8eaedb0bd3	[ytdl] fix some compatibility issues (#1680)	2021-11-15 22:37:03 +01:00
Mike Fährmann	2aaac3c997	[instagram] include user metadata for 'tagged' downloads (#2024) ... Adds - tagged_owner_id - tagged_full_name - tagged_username containing the values for the user profile the URL originated from, e.g. 'instagram' for https://www.instagram.com/instagram/tagged/.	2021-11-15 21:21:59 +01:00
Mike Fährmann	cfa4876848	[philomena] support furbooru.org (closes #1995)	2021-11-15 20:57:51 +01:00
Mike Fährmann	4377f1c284	[twitter] distinguish between fatal & nonfatal errors (#2020) ... only show a warning for nonfatal errors and do not raise a StopExtraction exception	2021-11-13 22:46:40 +01:00
Mike Fährmann	7c50f473e4	[ytdl] document 'cmdline-args' and 'config-file' options (#1680)	2021-11-12 18:54:04 +01:00
Mike Fährmann	19403a7fff	[downloader:ytdl] prevent crash in '_progress_hook()' (#1680) ... 'speed' is not guaranteed to be defined or convertible to 'int'	2021-11-12 18:54:04 +01:00
Mike Fährmann	01b28f3674	[ytdl] fix syntax for Python 3.4	2021-11-12 18:54:04 +01:00
Kyle Anthony Williams	a14b72be21	[webtoons] Use swebtoon-phinf.pstatic.net instead of webtoon-phinf.pstatic.net (#2005) ... * [webtoons] Use swebtoon-phinf.pstatic.net instead of webtoon-phinf.pstatic.net This trick to avoid having to set a Referer header comes from Webtoon's RSS feeds. The two URLs below are equivalent in content: https://webtoon-phinf.pstatic.net/20210929_153/1632867980912DmcGK_JPEG/16328679808882705182.jpg?type=q90 https://swebtoon-phinf.pstatic.net/20210929_153/1632867980912DmcGK_JPEG/16328679808882705182.jpg?type=q90 The URL with the domain "webtoon-phinf.pstatic.net" needs a Referer header, and the domain "swebtoon-phinf.pstatic.net" does not. This is because of the environment "swebtoon" images live in, one without explicit network control: RSS feeds on sites such as Feedly. This change should make it easier for gallery-dl developers to embed Webtoon comics without worrying about headers.	2021-11-11 20:03:34 +01:00
Mike Fährmann	6e3658ef52	[kemonoparty] provide 'date' metadata for gumroad (#2007) ... Not the 'published' or 'edited' values since they are 'null', but still better then nothing at all.	2021-11-11 19:38:10 +01:00
Mike Fährmann	f4d201f626	[ytdl] fix syntax for Python 3.4	2021-11-09 23:05:12 +01:00
Mike Fährmann	37c9dedee1	[seisoparty] remove module	2021-11-09 22:41:04 +01:00
Mike Fährmann	efa178cc91	[ytdl] implement parsing ytdl command-line options (#1680) ... - adds 'config-file' and 'cmdline-args' options for both ytdl downloader and extractor - create 'ytdl' helper module, which combines YoutubeDL creation and option parsing. - most likely a buggy mess due to incompatibilities between the original youtube-dl and yt-dlp.	2021-11-07 02:44:11 +01:00
Mike Fährmann	a881305357	release version 1.19.2	2021-11-05 20:47:45 +01:00
Mike Fährmann	7cb303d745	[redgifs] improve URL extraction ... Fields inside 'urls' can be None, which would have caused an exception with the old method.	2021-11-05 20:02:43 +01:00
Mike Fährmann	2befed1a96	[redgifs] update search URL pattern (#1984)	2021-11-05 20:00:06 +01:00
Mike Fährmann	b315a0ecef	[redgifs] update to API v2 (#1984)	2021-11-04 21:31:20 +01:00
Mike Fährmann	f0fc3b0ba1	[kemonoparty] add 'comments' option (#1980)	2021-11-03 23:02:13 +01:00
Mike Fährmann	1fac74b14d	[reddit] prevent crash for galleries with no 'media_metadata' ... (fixes #2001)	2021-11-03 17:55:40 +01:00
Mike Fährmann	2ab190ce08	add tests for special format strings	2021-11-01 23:26:18 +01:00
Mike Fährmann	211de95dd0	update extractor test results	2021-11-01 02:58:53 +01:00
Mike Fährmann	8bea02c38c	[deviantart] fix 'index' values for stashed deviations	2021-11-01 01:08:24 +01:00
Mike Fährmann	dd88a7d980	{cyberdrop] restore video extraction (fixes #1993) ... fixes a regression introduced in f33c2ef7	2021-10-31 04:34:01 +01:00