322 Commits

Author	SHA1	Message	Date
Mike Fährmann	be52069cbc	update CHANGELOG and docs/supportedsites	2018-11-28 14:53:27 +01:00
Mike Fährmann	c47482b110	smaller changes, missing docs, etc. ... - make 'netrc' extractor-specific - rename 'downloader.enable' to 'enabled' - document 'downloader.ytdl.format' - consistent newlines in configuration.rst	2018-11-16 18:18:07 +01:00
Mike Fährmann	8ef84a6823	add option to enable/disable specific downloader modules ... ... and write URLs with no (active) downloader to unsupported-file	2018-11-13 18:06:36 +01:00
Mike Fährmann	2fa28a2609	update default user-agent string (closes #122)	2018-11-11 10:07:10 +01:00
Mike Fährmann	655549df7c	[downloader:ytdl] add several options ... The "default" downloader options (rate, retries, timeout, verify) are mapped to corresponding youtube-dl options. downloader.ytdl.logging tells the downloader to pass youtube-dl's output to a Logger object. downloader.ytdl.raw-options allows to pass arbitrary options to the YoutubeDL constructor.	2018-10-20 18:26:49 +02:00
Mike Fährmann	6ed629f2b6	allow specifying number of skips before abort/exit (closes #115) ... In addition to 'abort' and 'exit', it is now possible to specify 'abort:N' and 'exit:N' (where N is any integer) as value for 'skip' to abort/exit after consecutively skipping N downloads.	2018-10-13 17:21:55 +02:00
Mike Fährmann	d8492df51b	[deviantart] extend functionality of 'original' option	2018-10-11 22:11:49 +02:00
Mike Fährmann	392a081657	update 'docs/configuration.rst'	2018-10-08 23:09:01 +02:00
Mike Fährmann	f8b3b00249	[twitter] add experimental 'videos' option (#99) ... Enabling this option will detect videos in tweets and output them as "unsupported" URLs, so that these can then be downloaded with youtube-dl There are a lot of improvements to be made to the current implementation, but it works and does what it is supposed to, even if inefficient as can be ...	2018-09-30 21:52:23 +02:00
Mike Fährmann	2cf3f53839	[yuki] add thread extractor (closes #111)	2018-09-28 12:47:32 +02:00
Mike Fährmann	17611bfec0	update build_supportedsites.py script	2018-09-28 12:43:19 +02:00
Mike Fährmann	09d2f3e5e7	[postprocessor:ugoira] improve libx264 detection	2018-09-24 13:12:54 +02:00
Mike Fährmann	c402cc4047	[hentaifoundry] add 'popular' and 'recent' extractors ... for "Popular Pictures" and "Recent Pictures" listings	2018-09-24 13:11:18 +02:00
Mike Fährmann	a5fc311dfa	[hentaifoundry] add 'favorite' extractor	2018-09-22 21:23:29 +02:00
Mike Fährmann	55f5c87160	[postprocessor:ugoira] add 'libx264-prevent-odd' option ... A rather crude workaround for "width/height not divisible by 2" errors when using libx264.	2018-09-21 20:01:14 +02:00
Mike Fährmann	006f75b538	[hentaifoundry] rewrite + more metadata ... - extract width, height, artist per image - improve pattern regex - better extensibility for other listings	2018-09-21 11:23:51 +02:00
Mike Fährmann	6ea9a78588	[wallhaven] add login capabilities ... Being logged in is required to access NSFW wallpapers.	2018-09-19 21:04:01 +02:00
Mike Fährmann	c9290d8212	[wallhaven] add wallpaper and search extractors ... todo: - login support to gain access to NSFW wallpapers - extractors for tag-, similar-, latest-listings - skip() support	2018-09-17 21:26:13 +02:00
Mike Fährmann	22ab509a70	[bobx] rename "model" to "idol" extractor	2018-09-14 18:11:36 +02:00
Mike Fährmann	69fd61ea86	[bobx] add gallery and model extractors	2018-09-13 20:13:12 +02:00
Mike Fährmann	7742cf8601	[tumblr] change 'reblogs' option (#103) ... - rename "deleted" to "same-blog" - change test for deleted original post to test if original post owner has the same UUID (full blog name) as the one being downloaded from - add 'blog[uuid]' metadata to allow comparison with 'reblogged_from_uuid'	2018-09-10 15:40:25 +02:00
Mike Fährmann	f1695567e8	adjust values in template config file ... (#104)	2018-09-09 14:10:55 +02:00
Mike Fährmann	a666ddd16b	[tumblr] extend 'reblogs' functionality (#103) ... Setting 'reblogs' to "deleted" will check if the parent post of a reblog has been deleted and download its media content if that is the case, otherwise it will be skipped. This is a rather costly operation (1 API request per reblogged post) and should therefore be used with care.	2018-09-07 19:13:52 +02:00
Mike Fährmann	c9b8e6aefc	[reddit] fix submission-ID parsing (#104) ... Uppercase characters caused a ValueError exception	2018-09-07 18:27:54 +02:00
Mike Fährmann	3ecea4cf36	[hentaicafe] add chapter and manga extractors (#101)	2018-09-05 21:08:40 +02:00
Mike Fährmann	75862715ac	[behance] add user extractor	2018-08-31 17:42:09 +02:00
Mike Fährmann	6ecb36d88c	[postprocessor:ugoira] add 'ffmpeg-output' option	2018-08-31 17:37:35 +02:00
Mike Fährmann	ab2bfaeb46	[ngomik] add replacement for 'subapics' ... http://subapics.com/ got discontinued and replaced by http://ngomik.in/. ngomik.in is still displaying a link to the "old site" showing a big "Account Suspended" sign.	2018-08-23 15:29:53 +02:00
Mike Fährmann	e9dd2eff1d	[twitter] add extractor for media-tweet timelines (#96) ... For example "https://twitter.com/PicturesEarth/media". They are different from normal timelines in that they do not contain any (re)tweets from other users and feature all media the user ever posted, including responses to other tweets.	2018-08-19 20:46:12 +02:00
Mike Fährmann	9b1c39032c	[twitter] changes and improvements ... - rename User- to TimelineExtractor - rename 'userid' to 'user_id' to conform to the other ..._id values - adjust archive_fmt to deal with retweets - emulate browser behavior for API calls	2018-08-18 23:04:45 +02:00
Mike Fährmann	10365394d7	[twitter] add support for user-timelines (closes #96) ... also adds a 'retweets' option to filter retweeted content	2018-08-17 20:04:11 +02:00
Mike Fährmann	63fa0b2006	[pinterest] add extractors for related pins ... Related pins can not be accessed by adding a "#related" fragment to the end of a Pinterest URL, for example: - https://www.pinterest.com/pin/858146903966145189/#related - https://www.pinterest.com/g1952849/test-/#related There are no explicit real URLs for related pins, using an option to enable them results in "clunky" code, and a custom "related:<URL>" scheme doesn't feel right either.	2018-08-15 21:49:45 +02:00
Mike Fährmann	3c90df6635	[piczel] add user, folder and image extractors	2018-08-08 10:53:01 +02:00
Mike Fährmann	b164231bca	[sankaku] increase default values for 'wait-min/-max'	2018-08-03 17:06:51 +02:00
Mike Fährmann	df082e923c	[behance] add gallery extractor (#95)	2018-08-01 21:46:55 +02:00
Mike Fährmann	5f27cfeff6	[deviantart] remove prefer-public option ... All API requests now always use a public token and only switch to a private token for pagination results if `refresh-token` is set and less deviations than requested were returned.	2018-07-26 19:43:46 +02:00
Mike Fährmann	212130b048	[deviantart] improve public-private token switching ... - rename option to `prefer-public` - now also works for galleries with less than 24 items	2018-07-25 12:52:36 +02:00
Mike Fährmann	886d662582	[deviantart] add option to minimize refresh-token usage ... Always trying with a public token first and repeating the API request with a private token if deviations are missing doesn't quite work for galleries and folders with less than 25 items, so its an option and not the default.	2018-07-24 21:44:57 +02:00
Mike Fährmann	54a0d72dc8	[postprocessor:ugoira] improve frame rate handling ... By default FFmpeg assumes a 25 FPS input frame rate, leading to dropped frames if the source requires a higher frame rate than that. This commit adds a `framerate` option (default "auto"), which allows to automatically assign a (more or less) fitting frame rate based on delays between ugoira frames and avoids dropped frames.	2018-07-23 21:33:20 +02:00
Mike Fährmann	84854fcad7	[myportfolio] add user and gallery extractors (#95)	2018-07-19 18:56:45 +02:00
Mike Fährmann	c9f70e0a19	[paheal] use HTTPS	2018-07-17 21:25:03 +02:00
Mike Fährmann	ff436692bf	["deviantart] add 'journals' option	2018-07-16 18:14:41 +02:00
Mike Fährmann	00032b828c	[deviantart] add 'wait-min' option	2018-07-14 11:52:21 +02:00
Mike Fährmann	269dc2bbd5	[sankaku] add 'tags' option (#94)	2018-07-14 09:53:01 +02:00
Mike Fährmann	1d43cbbf52	[gelbooru] tag-splitting for non-api mode	2018-07-06 15:24:19 +02:00
Mike Fährmann	c20c0a4820	[safebooru] add pool extractor	2018-07-04 12:24:57 +02:00
Mike Fährmann	4a57509392	generalize tag-splitting option (#92) ... - extend functionality to other booru sites: - http://behoimi.org/ - https://konachan.com/ - https://e621.net/ - https://rule34.xxx/ - https://safebooru.org/ - https://yande.re/	2018-07-04 12:21:16 +02:00
Mike Fährmann	188e956c4e	[imagefap] use HTTPS + update test results	2018-06-30 19:40:46 +02:00
Mike Fährmann	87853538b4	[yandere] add option to split tags by type (#92)	2018-06-29 19:38:53 +02:00
Mike Fährmann	4eb94aca17	[postprocessor:ugoira] pass '-f' if not present	2018-06-22 13:26:17 +02:00
Mike Fährmann	0c1c4557dd	[postprocessor:ugoira] add option for two-pass encoding	2018-06-20 18:48:10 +02:00
Mike Fährmann	eb7a1f3b98	[pixiv] rework ugoira handling ... Frame information now gets attached to the ZIP file's keyword dict instead of being written to a separate text file.	2018-06-18 17:57:57 +02:00
Mike Fährmann	b84e71da91	add postprocessor documentation to configuration.rst	2018-06-16 15:46:41 +02:00
Mike Fährmann	6ac403c5d3	add postprocessor config example	2018-06-08 18:31:59 +02:00
Mike Fährmann	cdcc3427a0	[simplyhentai] add video extractor (#89) ... All videos hosted on their own servers seem be to dead, but myhentai.tv embeds, which are most of the videos, work fine.	2018-05-30 11:25:23 +02:00
Mike Fährmann	12797e3b1f	update configuration.rst ... ... again - some more 'Path' references - fixed some inconsistencies and errors - added note about logging config for files	2018-05-28 22:14:38 +02:00
Mike Fährmann	c43f02245f	update configuration.rst ... - fix default values for 'log' and 'unsupportedfile' [ci skip]	2018-05-27 17:12:57 +02:00
Mike Fährmann	dacda69c9e	update configuration.rst ... - document logging options - add a section for "custom types" [ci skip]	2018-05-27 16:50:35 +02:00
Mike Fährmann	55b0913412	[simplyhentai] add gallery extractor (#89)	2018-05-27 15:25:04 +02:00
Mike Fährmann	53f36176fd	update configuration.rst ... - update the API Tokens & IDs section - mention redirect URIs for deviantart - include api-secret for tumblr - add instructions for smugmug - [ci skip]	2018-05-26 11:26:50 +02:00
Mike Fährmann	ecdc3475b8	[pixhost] support .to TLDs	2018-05-23 18:32:34 +02:00
Mike Fährmann	909d105ae6	[pixiv] add extractor for illusts from followed users	2018-05-15 13:05:15 +02:00
Mike Fährmann	f3483a2b7c	[smugmug] add OAuth support	2018-05-10 18:58:05 +02:00
Mike Fährmann	69a5e6ddb3	Merge branch 'master' into 1.4-dev	2018-05-04 10:19:02 +02:00
Mike Fährmann	82c50fa609	release version 1.3.5	2018-05-04 10:03:20 +02:00
Mike Fährmann	8bf3cdd82b	implement logging options ... Standard logging to stderr, logfiles, and unsupported URL files (which are now handled through the logging module) can now be configured by setting their respective option keys (log, logfile, unsupportedfile) to a dict and specifying the following options; - format: format string for logging messages available keys: see [1] default: "[{name}][{levelname}] {message}" - format-date: format string for {asctime} fields in logging messages available keys: see [2] default: "%Y-%m-%d %H:%M:%S" - level: the lowercase levelname until which the logger should activate; available levels are debug, info, warning, error, exception default: "info" - path: path of the file to be written to - mode: 'mode' argument when opening the specified file can be either "w" to truncate the file or "a" to append to it (see [3]) If 'output.log', '.logfile', or '.unsupportedfile' is a string, it will be interpreted, as it has been, as the filepath (or as format string for .log) [1] https://docs.python.org/3/library/logging.html#logrecord-attributes [2] https://docs.python.org/3/library/time.html#time.strftime [3] https://docs.python.org/3/library/functions.html#open	2018-05-01 17:54:52 +02:00
Mike Fährmann	16e014baaa	[smugmug] added image and album extractor ... just some initial code that still requires a lot of work ... TODO: - folders - old-style albums (which are nearly all of them ...) - images from users - OAuth It could also happen that the API credentials used will become invalid whenever my 14 day trial period ends (7 days remaining), but that would just require users to supply their own.	2018-04-29 21:27:25 +02:00
Mike Fährmann	d96b3474e5	[puremashiro] remove module ... site has been unreachable for a couple of weeks and now the DNS record is gone as well	2018-04-28 14:24:20 +02:00
Mike Fährmann	b44a296404	[gomanga] remove module ... site has been unreachable for a couple of weeks and the cloudflare status page shows host errors	2018-04-28 14:24:21 +02:00
Mike Fährmann	0f1e07f627	[pinterest] scrap OAuth implementation; code improvements ... OAuth authentication isn't needed anymore and other tools like Postman are better suited for this job anyway.	2018-04-25 16:04:30 +02:00
Mike Fährmann	d10579edb5	[pinterest] improve PinterestAPI code; remove OAuth mentions ... on another note: access_tokens have been set to only allow for 10 requests per hour (from 200 yesterday)	2018-04-17 17:12:42 +02:00
Mike Fährmann	4bd182c107	[pinterest] implement oauth:pinterest (#83) ... Pinterest access tokens are rate limited at 200 requests per hour (or maybe per 2 or 3 hours?) so having just one access token for all users isn't going to work in the long run.	2018-04-16 20:03:28 +02:00
Mike Fährmann	dd49127408	[spectrumnexus] remove module ... Site stopped hosting manga scans (http://view.thespectrum.net/)	2018-04-16 09:45:07 +02:00
Mike Fährmann	a1fa4b43b0	Revert "[tumblr] add option to sort photosets by upload order" ... This reverts commit 4a26ae32df.	2018-04-09 16:08:08 +02:00
Mike Fährmann	564e12ca8f	replace 'imgyt' with 'imxto' ... https://img.yt/ wasn't available for a couple of days, but has now re-emerged as https://imx.to/ with a new web-interface. Links to older images still work (see tests).	2018-04-09 15:53:20 +02:00
Mike Fährmann	4a26ae32df	[tumblr] add option to sort photosets by upload order	2018-04-07 15:57:55 +02:00
Mike Fährmann	7b562907c3	[nijie] add favorites extractor ... adds support for 'https://nijie.info/user_like_illust_view.php?id=...'	2018-03-31 18:54:25 +02:00
Mike Fährmann	f39153b6e9	[nhentai] add extractor for search results	2018-03-28 17:21:44 +02:00
Mike Fährmann	52d41c41e7	[exhentai] add extractor for favorited galleries	2018-03-27 18:58:42 +02:00
Mike Fährmann	a993d0ea90	release version 1.3.2	2018-03-23 15:27:56 +01:00
Mike Fährmann	e7525b1b0e	[artstation] add challenge extractor (#80)	2018-03-23 15:06:09 +01:00
Mike Fährmann	b25ae30a95	reorder options in gallery-dl.conf	2018-03-22 10:05:53 +01:00
Mike Fährmann	40ca562d7b	[artstation] add album extractor (#80)	2018-03-17 17:36:31 +01:00
HRXN	241eda8c3d	Update supportedsites.rst (#81) ... Add ArtStation to Supported Sites.	2018-03-16 13:40:59 +01:00
Mike Fährmann	f0e3861ec7	update configuration.rst ... - move `base-directory` entry to extractor options (it should've been there since v1.0.0 / commit e3bfb83) - rename "General" to "Miscellaneous" and move it down a bit, since it has only two options left - mention the ability to use a list for tumblr.posts - replace any appearance of bato.to	2018-03-16 12:00:55 +01:00
Mike Fährmann	723cc66bb1	[artstation] add user-, image- and likes-extractors	2018-03-14 14:05:14 +01:00
Mike Fährmann	d6ef52897c	[imgchili] remove module ... All previously hosted images yield a 404 and the main page is just a logo.	2018-03-13 11:02:58 +01:00
Mike Fährmann	7847ab1d5a	[imagehosts] remove even more dead sites ... All removed sites either - reject all incoming connections or - display a message from their domain registrar	2018-03-12 21:25:13 +01:00
Mike Fährmann	74c8f83056	update gallery-dl.conf and add gallery-dl-example.conf ... - gallery-dl.conf contains all config options set to their default values, with the exception of 'filename', 'directory', and 'cache.file' - gallery-dl-example.conf contains a potential real-world configuration file and demonstrates some more involved options	2018-03-12 18:15:30 +01:00
Mike Fährmann	1400868f53	[mangadex] general improvements ... - support >100 chapter entries per manga - custom archive ID format - detect non-existing chapters	2018-03-06 14:15:15 +01:00
Mike Fährmann	749fbbfa6c	[mangadex] add chapter- and manga-extractor	2018-03-05 18:37:21 +01:00
Mike Fährmann	6e38cf5aab	[mangareader] use 'https://' ... The site now redirects from http://mangareader.net/ to https://mangareader.net/	2018-03-02 17:19:17 +01:00
Mike Fährmann	8cdce21dcb	make archive keys user-configurable	2018-02-25 21:57:01 +01:00
Mike Fährmann	8f338347b6	[imagehosts] cleanup ... removed - chronos.to - unable to resolve hostname - coreimg.net - same - imgmaid.net - same - hosturimage.com - everything returns 404 - imageontime.org - redirects to some shady site - imgupload.yt - cloudflare error 522, host down - img4ever.net - read timeout	2018-02-23 01:05:42 +01:00
Mike Fährmann	e1e0668ca8	add option to set default replacement field value ... Missing or undefined keywords will now be replaced with the value set for 'keywords-default'. The default is Python's 'None', which is equivalent to setting this option to JSON's 'null'.	2018-02-23 00:59:20 +01:00
Mike Fährmann	8704d850bf	add explicit proxy support (#76) ... - '--proxy' as command-line argument - 'extractor.*.proxy' as config option	2018-02-19 18:45:06 +01:00
Mike Fährmann	3cec533c28	Merge branch 'archive'	2018-02-12 18:07:58 +01:00
Mike Fährmann	7f7c16ae37	add option to specify additional key-value pairs	2018-02-08 23:10:58 +01:00
Mike Fährmann	54c36a8a34	[subapics] add chapter- and manga-extractor (#70)	2018-02-04 22:02:10 +01:00
Mike Fährmann	2dd3aeeeae	[komikcast] add chapter- and manga-extractor (#70)	2018-02-04 22:02:10 +01:00
Mike Fährmann	c0dd922c13	add '--download-archive' cmdline option ... … as well as a config file equivalent	2018-02-01 22:00:44 +01:00
Mike Fährmann	d951f13e37	add config option for unsupported-URL file ... for consistency's sake	2018-01-28 18:42:10 +01:00
Mike Fährmann	97f4f15ec0	add option to write logging output to a file ... - '--write-log FILE' as cmdline argument - 'output.logfile' as config file option	2018-01-26 18:51:51 +01:00
Mike Fährmann	1e93955170	[batoto] remove module ... Site officially shut down on 2018.01.18	2018-01-23 21:37:32 +01:00
Mike Fährmann	40d35c87bc	[paheal] add tag- and post-extractors (closes #69)	2018-01-15 16:39:05 +01:00
Mike Fährmann	cc0c2cca57	[reddit] add extractor for reddit-hosted images (closes #68)	2018-01-14 18:55:42 +01:00
Mike Fährmann	35e09869d1	[mangapark] fix image URLs and use HTTPS	2018-01-12 14:59:49 +01:00
Mike Fährmann	29d75fc3fa	[tumblr] add support for OAuth authentication (#65)	2018-01-11 14:11:37 +01:00
Mike Fährmann	b33efc99a4	[idolcomplex] add support for idol.sankakucomplex.com	2018-01-09 17:54:37 +01:00
Mike Fährmann	5b094328b5	[puremashiro] add chapter- and manga-extractor (closes #66) ... Also adds support for region subtags in language codes (e.g. en-us)	2018-01-07 21:50:43 +01:00
Mike Fährmann	03b8a548cb	[tumblr] change reblogs default value to true (#61)	2018-01-06 15:52:08 +01:00
Mike Fährmann	d235f68f59	[tumblr] add option to filter reblogged posts (#61) ... Reblogs are ignored by default, but can be included by setting 'extractor.tumblr.reblogs' to 'true'.	2018-01-05 13:05:57 +01:00
Mike Fährmann	d0886f411e	[gelbooru] re-enable API use (closes #56) ... Gelbooru's API allows access to all images and is not restricted to the first 20000. This also adds an option to select between API use and manual information extraction in case their API gets disabled again.	2017-12-21 21:42:40 +01:00
Mike Fährmann	fc7d165c97	[deviantart] add support for OAuth2 authentication ... Some user galleries [*] require you to be either logged in or authenticated via OAuth2 to access their deviations. [*] e.g. https://polinaegorussia.deviantart.com/gallery/ -------------- known issue: A deviantart 'refresh_token' can only be used once and gets updated whenever it is used to request a new 'access_token', so storing its initial value in a config file and reusing it again and again is not possible.	2017-12-18 01:16:46 +01:00
Mike Fährmann	0a9a07a6e1	[slideshare] improve metadata; flake8 ... - added 'views' and 'published' keywords - fixed longer titles and descriptions	2017-12-13 21:16:49 +01:00
Mike Fährmann	19a6ae57b2	[sankaku] add pool extractor	2017-12-12 19:45:10 +01:00
Mike Fährmann	4fb6803fa6	add option to sleep before each download	2017-12-04 17:33:10 +01:00
Mike Fährmann	300346ecdf	[mangazuki] remove extractors ... This site has been in "rebuild"-mode for a fairly long time and the current extractor code isn't going to work for the new version either.	2017-12-04 13:36:04 +01:00
Mike Fährmann	8f518e03f8	add options to set maximum download rate ... - -r/--limit-rate as cmdline option - downloader.http.rate as config option This implementation very roughly uses the idea of the token bucket algorithm [1] and mostly uses Wget's approach [2] as inspiration. [1] https://en.wikipedia.org/wiki/Token_bucket [2] http://git.savannah.gnu.org/cgit/wget.git/tree/src/retr.c?h=v1.19.2&id=ba6b44f6745b14dce414761a8e4b35d31b176bba#n111	2017-12-02 01:47:26 +01:00
Mike Fährmann	214972bc9a	[gelbooru] use manual extraction ... ... to compensate for their disabled API. (https://gelbooru.com/index.php?page=forum&s=view&id=3875) This also adds an extractor for image-pools.	2017-11-29 20:48:17 +01:00
Mike Fährmann	12de658937	[tumblr] add options to control extraction behavior (#48) ... - posts : list of post-types to inspect - inline : scan post bodies for inline images - external: follow external links	2017-11-23 15:32:54 +01:00
Mike Fährmann	e6814aebe2	add 'extractor.*.user-agent' config option	2017-11-15 14:01:33 +01:00
Mike Fährmann	2d0cfb33e1	[xvideos] add user profile extractor (#45)	2017-11-02 17:28:35 +01:00
Mike Fährmann	707b15b586	create missing directories for 'part-directory' ... also some code improvements regarding downloader config values	2017-10-27 12:22:45 +02:00
Mike Fährmann	caf26412dd	add option to set alternate location of .part files (#29) ... Note: The path set for 'downloader.*.part-directory' needs to point to an already existing directory.	2017-10-26 00:16:48 +02:00
Mike Fährmann	963670d73b	add options to control usage of .part files (#29) ... - '--no-part' command line option to disable them - 'downloader.http.part' and 'downloader.text.part' config options Disabling .part files restores the behaviour of the old downloader implementation.	2017-10-24 23:33:44 +02:00
Mike Fährmann	5fa42336a2	[sankaku] add warning for unauthenticated users ... also improve URL pattern and add missing options to default config file	2017-10-16 21:21:08 +02:00
Mike Fährmann	6af921a952	[sankaku] rewrite/improve (fixes #44) ... - add wait-time between HTTP requests similar to exhentai - add 'wait-min' and 'wait-max' options - increase retry-count for HTTP requests to 10 - implement user authentication (non-authenticated users can only view images up to page 25) - implement 'skip()' functionality (only works up to page 50) - implement image-retrieval for pages >= 51 - fix issue with multiple tags	2017-10-14 23:01:33 +02:00
Mike Fährmann	abfd2924a9	update configuration.rst ... - add instructions on how to register your own applications - combine descriptions of a few config keys	2017-10-12 23:37:28 +02:00
Mike Fährmann	f3fbaa5c3e	[reddit] allow users to override the API User-Agent ... Only overriding the Client-ID is not enough if you want to follow Reddit's API access rules [1]. [1] https://github.com/reddit/reddit/wiki/API#rules	2017-10-10 17:29:46 +02:00
Mike Fährmann	2ef3c35c98	smaller textual changes ... - swapped doc for deviantart.mature and .original - updated gallery-dl.conf - "transferred" -> "delegated"	2017-10-09 23:23:19 +02:00
Mike Fährmann	75d3a1f72f	[deviantart] always download original images ... Deviation-objects returned by the DeviantArt API don't always contain the URL and metadata of the original image ([1]). Getting this information requires an additional API call [2], which is indicated by the 'is_downloadable' and 'download_filesize' metadata within a deviation-object. [1] https://myria-moon.deviantart.com/art/Aime-Moi-part-en-vadrouille-261986576 [2] https://www.deviantart.com/developers/http/v1/20160316/deviation_download/bed6982b88949bdb08b52cd6763fcafd	2017-10-07 13:07:34 +02:00
Mike Fährmann	cebf800a7f	[foolfuuka] add support for more sites (#18) ... - https://arch.b4k.co - https://archive.whatisthisimnotgoodwithcomputers.com - https://archive.yeet.net Notes: - The name "whatisthisimnotgoodwithcomputers" is way too long ... - archive.yeet.net is out of date and also blocked by 4chan servers - newest threads are 2 weeks old - using "https://archive.yeet.net" as Referer header results in "403 Forbidden" when accessing 4chan	2017-09-16 21:36:16 +02:00
Mike Fährmann	470bbe9d8c	fix smaller stuff ... - change filename option in example config file - adapt default filename format for mangafox - remove unnecessary newline [skip ci]	2017-09-11 17:07:29 +02:00
Mike Fährmann	81877bb5f6	add '-K' as shortcut for '--list-keywords'	2017-09-09 18:48:28 +02:00
Mike Fährmann	54c0715135	allow users to set their own API access_tokens/client_ids	2017-09-09 17:50:19 +02:00
Mike Fährmann	49c7e70c10	[acidimg] add image extractor	2017-09-09 15:19:18 +02:00
Mike Fährmann	b8862ff15e	add 'downloader.http.verify' option ... (also: change the default 'timeout' from None to 30)	2017-08-31 15:21:08 +02:00
Mike Fährmann	18e6ed1c7e	[booru] add extractors for "Popular" images	2017-08-24 21:29:22 +02:00
Mike Fährmann	4ea82ea556	[warosu] add thread extractor	2017-08-18 19:54:07 +02:00
Mike Fährmann	3c9f190757	extend output of --list-keywords	2017-08-10 17:36:21 +02:00
Mike Fährmann	abd7c559cd	[yonkouprod] remove module ... Every manga chapter on this site has been removed.	2017-08-07 18:32:14 +02:00
Mike Fährmann	f08af03845	Merge branch 'cookies'	2017-07-25 14:04:53 +02:00
Mike Fährmann	d7cb3c668a	update supportedsites.rst	2017-07-24 10:50:40 +02:00
Mike Fährmann	00a23fe208	update configuration.rst	2017-07-21 18:32:56 +02:00
Mike Fährmann	4a60f6068a	[mangazuki] add manga extractor	2017-07-20 16:02:09 +02:00
Mike Fährmann	1cb1d2e0a3	[mangazuki] add chapter extractor	2017-07-19 17:20:03 +02:00
Mike Fährmann	74a2b5af06	update supportedsites.rst	2017-07-15 15:01:30 +02:00
Mike Fährmann	3460dc8950	update gallery-dl.conf	2017-07-14 08:23:11 +02:00
Mike Fährmann	9be8f7e106	[deviantart] add "extractor.deviantart.flat" option ... Setting this to 'false' downloads images into individual subdirectories for each gallery-folder or favourite-collection, otherwise it is just creating a flat list of images.	2017-07-12 17:05:31 +02:00
Mike Fährmann	9edbd6ff01	update configuration.rst	2017-07-04 19:34:34 +02:00
Mike Fährmann	06c4cae05b	extend the output of '--list-extractors' ... It now includes category and subcategory values for each extractor class.	2017-06-28 18:51:47 +02:00
Mike Fährmann	1dac76fd1c	update extractor docstrings	2017-06-28 17:39:07 +02:00
H R X N	b757687606	Update configuration.rst (#24) ... Fixed minor typo.	2017-06-28 13:55:39 +02:00
Mike Fährmann	598e3a8d88	update configuration.rst	2017-06-27 18:56:24 +02:00
Mike Fährmann	86561d7c8b	update configuration.rst	2017-06-27 17:44:02 +02:00
Mike Fährmann	d3b04076f7	add .netrc support (#22) ... Use the '--netrc' cmdline option or set the 'netrc' config option to 'true' to enable the use of .netrc authentication data. The 'machine' names for the .netrc info are the lowercase extractor names (or categories): batoto, exhentai, nijie, pixiv, seiga.	2017-06-24 12:17:26 +02:00
Mike Fährmann	fbe8c519e2	update README ... - mention and link to docs/configuration.rst - add OAuth section	2017-06-23 16:14:51 +02:00
Mike Fährmann	ccb7973b57	update configuration.rst	2017-06-20 16:20:28 +02:00
H R X N	b206b831ee	Update configuration.rst ... Fix heading.	2017-06-19 14:12:43 +02:00
Mike Fährmann	48b444e276	add a list of all configuration options ... (the format might be subject to change; putting everything in one single table is even worse)	2017-06-18 22:16:26 +02:00
Mike Fährmann	fb1904dd59	add 'Authentication' column to supportedsites.rst	2017-06-15 21:06:20 +02:00
Mike Fährmann	5f55c854b9	[flickr] replace getPublic... API call with regular ones	2017-06-12 16:37:06 +02:00
Mike Fährmann	d5a70f2580	add simple progress indicator for multiple URLs (#19) ... The output can be configured via the 'output.progress' config value. Possible values: - true: Show the default progress indicator "[{current}/{total}] {url}" (default) - false: Never show the progress indicator - <string>: Show the progress indicator using this as a custom format string(1). Possible replacement keys are: - current: current URL index - total : total number of URLs - url : current URL (1) https://docs.python.org/3/library/string.html#formatstrings	2017-06-09 20:12:15 +02:00
Mike Fährmann	090e11b35d	[reddit] enable user authentication with OAuth2 (#15) ... Call '$ gallery-dl oauth:reddit' to get a refresh_token for your account.	2017-06-08 16:17:13 +02:00
Mike Fährmann	4e80e0c884	[flickr] add user extractor (#16)	2017-06-02 17:15:05 +02:00
Mike Fährmann	c921b4f32a	code cleanup and fixing tests	2017-06-02 09:10:58 +02:00
Mike Fährmann	67791e1b36	[imgur] improve and add image extractor	2017-05-26 22:30:09 +02:00
Mike Fährmann	832a4a8ee9	[fallenangels] add manga extractor	2017-05-21 10:37:38 +02:00
Mike Fährmann	2974d782a3	[yomanga] remove module ... site has been shut down	2017-05-20 11:18:44 +02:00
Mike Fährmann	677c8ced11	[deviantart] add "journal" extractor ... (#14)	2017-05-10 17:21:33 +02:00
Mike Fährmann	1e416e5227	update README	2017-05-09 15:43:55 +02:00