256 Commits

Author	SHA1	Message	Date
thatfuckingbird	224b883ff4	[danbooru] add option for extended metadata extraction (#1458) ... * [danbooru] add option for extended metadata extraction * appease linter * [danbooru] update docs/configuration.rst * [danbooru] rename extended-metadata -> metadata	2021-04-13 23:41:30 +02:00
thatfuckingbird	dff03a6605	[booru] add an option to extract notes (only gelbooru for now) (#1457) ... * [booru] add an option to extract notes (currently implemented only for gelbooru) * appease linter * [gelbooru] rename "text" to "body" in note extraction * add a code comment about reusing return value of _extended_tags	2021-04-13 23:40:24 +02:00
Mike Fährmann	a86ffb04bb	add 'output.fallback' option ... to enable/disable fallback URLs for -g/--get-urls	2021-04-12 02:00:41 +02:00
Mike Fährmann	d5aad999dc	[tapas] implement login with username & password (#692)	2021-03-30 01:45:28 +02:00
Mike Fährmann	e9ec91c811	[exhentai] improve image limits check ... - check if current image is the '509 Bandwidth Exceeded' notification (https://ehgt.org/g/509.gif or https://exhentai.org/img/509.gif) - remove 'limits' option	2021-03-29 19:01:13 +02:00
Mike Fährmann	2846235669	[twitter] allow specifying a custom format for user results ... (#1337)	2021-03-21 22:26:26 +01:00
Mike Fährmann	b0438c8f99	Revert "[deviantart] extend 'extra' option" ... This reverts commit 5ad2b9c82b, 5c32a7bf58, and 83f465faca. (#1387, #1356)	2021-03-19 16:24:23 +01:00
Mike Fährmann	3378b39719	[twitter] implement 'users' option (#1337)	2021-03-16 00:51:05 +01:00
Mike Fährmann	df94182e11	implement 'parent-metadata' option (#1364) ... experimental, might not work as expected, etc.	2021-03-11 01:10:34 +01:00
Mike Fährmann	83f465faca	[deviantart] refactor 'extra' (#1356) ... - change its expected type to string - let users specify a list of sources (stash, posts) or 'all'	2021-03-07 22:23:52 +01:00
Mike Fährmann	5c32a7bf58	[deviantart] allow selecting source for 'extra' (#1356) ... Setting 'extra' to "stash" or "deviations" will only download embedded sta.sh content or deviations. 'true' still downloads both.	2021-03-06 21:31:28 +01:00
Mike Fährmann	a677123abb	[instagram] recognize 'reels' as option for 'include' (#1329)	2021-03-06 18:50:19 +01:00
Mike Fährmann	7440d1f112	[pixiv] add 'translated-tags' option (closes #1354) ... (a lot more straight forward than I thought ...)	2021-03-05 17:18:51 +01:00
Mike Fährmann	6a28b7045c	update 'docs/gallery-dl.conf'	2021-03-03 03:11:48 +01:00
Mike Fährmann	bae874f370	replace 'wait-min/-max' with 'sleep-request' ... on exhentai, idolcomplex, reactor	2021-03-02 22:55:45 +01:00
Mike Fährmann	1a38fae785	add option to use different youtube-dl modules (fixes #1330) ... by setting the 'downloader.ytdl.module' value. For example { "downloader": { "ytdl": { "module": "yt_dlp" } } } or '-o module=yt_dlp'	2021-03-01 03:10:42 +01:00
Mike Fährmann	e58039358d	[mangadex] use 'api.mangadex.org' as default API server ... The caching issues seem to be gone. (#1290, #1310)	2021-02-28 01:31:50 +01:00
Mike Fährmann	29ea54dc41	[patreon] use '"browser": "firefox"' by default (#1117)	2021-02-27 16:26:42 +01:00
Mike Fährmann	61fbbd2dae	[exhentai] rename metadata fields to match API results (#1325) ... - gallery_id -> gid - gallery_token -> token - title_jp -> title_jpn - visible -> expunged - gallery_size -> filesize - count -> filecount Also changes the function of the 'metadata' option. It is now boolean and causes extra data fields from the API to be added instead of completely replacing the data from HTML when activated.	2021-02-26 18:21:04 +01:00
Mike Fährmann	5d69e437d0	[twitter] add option to download all media from a conversation ... (fixes #1319)	2021-02-26 13:50:46 +01:00
Mike Fährmann	cf5fa75d4c	add 'browser' option (#1117) ... - change default user agent to Firefox ESR 78 on Windows 10 - remove 'ciphers' option	2021-02-26 13:41:27 +01:00
Mike Fährmann	e5735361ed	[exhentai] add 'metadata' option (#1325) ... to select between gallery metadata from 'api' or 'html'	2021-02-23 00:23:42 +01:00
Mike Fährmann	5ad2b9c82b	[deviantart] extend 'extra' option ... also download from embedded DeviantArt posts	2021-02-21 19:27:04 +01:00
Mike Fährmann	560277394e	[downloader:http] add 'headers' option (#1322)	2021-02-21 19:13:39 +01:00
Mike Fährmann	6b0ecbf6bc	[hentainexus] add 'orignal' option (#1322)	2021-02-21 19:06:12 +01:00
Mike Fährmann	8974f0361c	[pixiv] update (#1304) ... - remove login with username & password - require a refresh token - add 'oauth:pixiv' functionality See also: - https://github.com/upbit/pixivpy/issues/158 - https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362	2021-02-12 18:07:16 +01:00
Mike Fährmann	cdb0b02e30	[pillowfort] add 'reblogs' option (#846)	2021-01-25 15:03:59 +01:00
Mike Fährmann	7a096c443f	[unsplash] add 'format' option (#1197)	2021-01-22 17:23:46 +01:00
Mike Fährmann	89a2bcbb2d	[furaffinity] add 'descriptions' option (#1231)	2021-01-19 19:09:29 +01:00
Mike Fährmann	36f281330a	[newgrounds] fix flash file extraction (closes #1257) ... … and add a 'flash' option to choose between flash and video formats.	2021-01-19 17:48:14 +01:00
Mike Fährmann	c008cb5100	[pixiv] add 'related' option (#1237)	2021-01-17 22:48:32 +01:00
Mike Fährmann	01b9ccd4de	[derpibooru] use the "Everything" filter by default (#1243) ... when neither 'api-key' nor 'filter' are set	2021-01-14 18:29:09 +01:00
Mike Fährmann	5d4494b15f	add "ascii" as a special 'path-restrict' value	2021-01-09 02:41:20 +01:00
Mike Fährmann	fa8ee6eac4	[derpibooru] add search and gallery extractors (#862)	2021-01-07 18:05:32 +01:00
Mike Fährmann	371e9ca6df	[pinterest] implement video support (closes #1189)	2020-12-21 16:09:06 +01:00
Mike Fährmann	bf629a2818	[instagram] add 'include' option (closes #1180) ... Split the functionality of the old 'user' extractor into separate 'posts' and 'highlights' extractors, which respond to virtual URLs ('/<user>/posts' and '/<user>/highlights')	2020-12-21 02:06:28 +01:00
Mike Fährmann	465015f75a	[sankaku] reimplement login support (#1176, #1182)	2020-12-17 16:12:59 +01:00
Mike Fährmann	b2c55f0a72	[sankaku] remove login support ... The old login method for 'https://chan.sankakucomplex.com/user/login' and the cookies it produces have no effect on the results from 'beta.sankakucomplex.com'.	2020-12-08 21:05:47 +01:00
Mike Fährmann	0bc492c0fa	add docs for 'event' and 'filename' options ... from 9c3568c3 and ca59bd69	2020-11-25 12:12:41 +01:00
Mike Fährmann	9b1bd09454	change 'extension-map' default ... Replace all JPEG filename extensions with 'jpg'.	2020-11-14 22:40:31 +01:00
Mike Fährmann	bc7b1d91bc	fix rST markup in configuration.rst ... [ci skip]	2020-11-02 15:32:29 +01:00
Mike Fährmann	e3480bc8de	implement 'extension-map' option (#318)	2020-11-02 15:27:07 +01:00
Mike Fährmann	98a4d86a01	[sankakucomplex] extract videos and embeds (closes #308)	2020-10-30 01:21:11 +01:00
Mike Fährmann	d83b95fd28	[postprocessor:metadata] accept a string-list for 'content-format' ... (closes #1080)	2020-10-27 20:09:58 +01:00
Mike Fährmann	5bcf28de93	add a 'extractor.modules' option	2020-10-25 03:05:10 +01:00
Mike Fährmann	1686dc1757	[twitter] support media from Cards (#1005, #937) ... Can be enabled with 'extractor.twitter.cards', but for now disabled by default because cards can redirect to rather large videos from YouTube or Twitch.	2020-10-22 21:33:53 +02:00
Mike Fährmann	bac8af69e8	update configuration.rst ... - add some lines to better explain post processor usage - syntax highlighting for JSON blocks and other smaller stuff	2020-10-19 21:57:26 +02:00
Mike Fährmann	05d7009cc6	rename 'Authentication' entries in supportedsites.rst ... - change 'Optional' to 'Supported' - use 'OAuth' and 'Cookies' in their own - add link to weasyl API key option	2020-10-19 20:16:17 +02:00
Mike Fährmann	d1c6d78477	fix rST markup in configuration.rst	2020-10-15 15:17:19 +02:00
Mike Fährmann	b8daabc3ca	[pinterest] implement login support (closes #1055) ... being logged allows access to secret/protected boards	2020-10-15 15:14:18 +02:00