580 Commits

Author	SHA1	Message	Date
Mike Fährmann	3e0848a482	[instagram] disable login with username&password (#756)	2020-05-29 23:29:40 +02:00
Mike Fährmann	da22ea8ced	use %APPDATA%\gallery-dl for config/cache on Windows	2020-05-28 02:56:38 +02:00
Mike Fährmann	45baa13615	update extractor test results ... - don't run Instagram tests on Travis anymore - replace Twitter test because timeline was made private - update Hiperdex domain to '.com' (again ...)	2020-05-28 02:18:06 +02:00
Mike Fährmann	b489f4de12	fix internal links in configuration.rst	2020-05-28 01:51:16 +02:00
Mike Fährmann	dfcf2a2c91	write OAuth token to cache by default (#616)	2020-05-25 22:35:45 +02:00
Mike Fährmann	ddc253cf9a	implement a 'path-replace' option (#662, #755)	2020-05-25 22:21:58 +02:00
Mike Fährmann	7003e61d1d	add a few more examples to gallery-dl-example.conf ... - include 'igneous' and 'hath_perks' in Exhentai cookies - add an example of how to write DeviantArt description to file - add a 'path-restrict' mapping from invalid characters in Windows paths to Unicode alternatives (taken from #662)	2020-05-25 22:21:57 +02:00
Mike Fährmann	bc53302ad6	extend 'path-restrict' option ... Allow its value to be a JSON object / Python dict that specifies a mapping from invalid/unwanted input characters to specific output characters. For example {"/": "-", "*": "+"} will transform "foo / ***bar***" into "foo - +++bar+++" (closes #662, #755)	2020-05-25 22:21:56 +02:00
Mike Fährmann	e19f665a44	[danbooru] change default for 'ugoira' to 'false' ... Downloading the pre-rendered versions should be a better default than .zip files with individual frames.	2020-05-20 19:57:28 +02:00
Mike Fährmann	0378d079a5	[webtoons] fixes and simplifications (#593, #761) ... - fix episode listings for french comics - allow input URLs without explicit scheme - add 'lang'/'language' metadata - use str.format() instead of '+' to assemble URLs	2020-05-18 20:20:03 +02:00
Mike Fährmann	ab11b1c896	[imagechest] simplify code (#750)	2020-05-18 19:11:26 +02:00
Mike Fährmann	dba87ca99e	[downloader:ytdl] change 'forward-cookies' default to 'false' ... There are currently no situations where forwarding gallery-dl's cookies to youtube-dl is necessary, and it only causes problems when forcing youtube-dl for Twitter video downloads while logged in.	2020-05-12 20:17:08 +02:00
Mike Fährmann	2d6724180b	[hiperdex] update domain to hiperdex.info	2020-05-12 17:00:51 +02:00
Mike Fährmann	be16d77073	simplify and improve documentation markup (#742) ... - use | to get a more "compact" representation - link directly to example config files	2020-05-12 17:00:34 +02:00
Vrihub	fe224416bf	Some documentation improvements (#742) ... * Link configuration examples in the intro, see #712 * Clarify overriding conf files by options, see #678	2020-05-10 23:47:20 +02:00
Mike Fährmann	f557cac074	[redgifs] add image extractor (#724)	2020-05-10 00:31:42 +02:00
Mike Fährmann	ba42ec321c	release version 1.13.6	2020-05-02 22:18:56 +02:00
Mike Fährmann	ff47641b13	test whether default/example config files contain valid JSON	2020-04-30 00:00:41 +02:00
Mike Fährmann	5e2974d699	[weibo] add 'videos' option	2020-04-30 00:00:30 +02:00
Mike Fährmann	9f638c2e01	[twitter] add 'replies' option (closes #705)	2020-04-29 23:20:06 +02:00
Mike Fährmann	a4fd620a25	[hiperdex] revert domain back to hiperdex.com	2020-04-27 20:42:31 +02:00
Mike Fährmann	a3de234e70	[hitomi] add extractor for tag searches (closes #697)	2020-04-20 21:55:19 +02:00
Mike Fährmann	d5273f9b0c	[hiperdex] update domain to hiperdex.net	2020-04-16 20:39:56 +02:00
Mike Fährmann	a6286bb551	[hiperdex] add 'artist' extractor (#606)	2020-04-12 02:32:37 +02:00
Mike Fährmann	6143050980	[aryion] add gallery and post extractors (#390, #673)	2020-04-08 21:52:51 +02:00
Mike Fährmann	2587296deb	[mastodon] add access tokens for mastodon.social and baraag.net ... (closes #665)	2020-04-02 22:34:32 +02:00
Mike Fährmann	4203dc0bdc	[mangapark] fix metadata extraction	2020-03-28 03:00:26 +01:00
Mike Fährmann	88ebbadc58	remove dashes from subcategory names in supportedsites.rst	2020-03-24 02:49:00 +01:00
Mike Fährmann	09f2271528	[35photo] add 'tag' extractor	2020-03-24 02:49:00 +01:00
Mike Fährmann	90d15e3682	[instagram] use 'itertools.chain()'	2020-03-17 15:52:44 +01:00
Mike Fährmann	5b676ea59d	[e621] document username & password support (#640)	2020-03-14 01:15:22 +01:00
Mike Fährmann	fdd2dd5136	[kabeuchi] add 'user' extractor (closes #561)	2020-03-13 16:45:42 +01:00
Mike Fährmann	59edcdc822	[hitomi] restore metadata fields from before f33b13a ... ... and add a 'metadata' option to disable visiting the gallery page and extracting data from it if this is not needed.	2020-03-12 23:43:41 +01:00
Mike Fährmann	823fbeaae6	[newgrounds] add 'favorite' extractor (#394)	2020-03-10 01:07:09 +01:00
Mike Fährmann	a45fbc38ea	[pixiv] implement 'avatar' option (#595, #623)	2020-03-09 21:18:16 +01:00
Mike Fährmann	32df8d06fe	[twitter] add 'bookmark' extractor (closes #625)	2020-03-06 01:20:04 +01:00
Mike Fährmann	cc5079c844	[hiperdex] add chapter and manga extractors (closes #606)	2020-02-22 03:09:29 +01:00
Mike Fährmann	6e759fe2f5	reword note for 'extractor.deviantart.refresh-token' (#612)	2020-02-19 23:39:27 +01:00
Mike Fährmann	765b2a0527	[hentaihand] add extractors (closes #605)	2020-02-19 21:55:47 +01:00
Mike Fährmann	67ea80222c	link to post processor example in 'pixiv.ugoira' docs	2020-02-14 01:22:16 +01:00
Mike Fährmann	74e684e828	[twitter] change default value for 'videos' to 'true' ... Every other 'videos' option defaulted to 'true', except Twitter.	2020-02-14 01:03:42 +01:00
Mike Fährmann	ab13f81fd7	add example to 'cookies' docs	2020-02-13 23:44:38 +01:00
Mike Fährmann	c7cf9dd111	[furaffinity] support classic layout (#284)	2020-02-12 21:39:43 +01:00
Mike Fährmann	138135c190	[furaffinity] add extractors (#284)	2020-02-11 19:51:24 +01:00
Mike Fährmann	da0d5f6092	[oauth] add 'port' option (#604)	2020-02-09 13:45:44 +01:00
Mike Fährmann	719b63d0ca	[bcy] add user and post extractors (#592)	2020-02-09 02:37:14 +01:00
Mike Fährmann	091f9a107d	[docs] add examples of how to filter manga chapters by language ... (closes #599)	2020-02-07 23:21:44 +01:00
Mike Fährmann	1f4c9c5f9d	[8kun] add thread and board extractors (closes #582)	2020-02-04 22:50:31 +01:00
Mike Fährmann	dff33b260c	[reddit] add 'videos' option	2020-01-31 23:45:02 +01:00
Mike Fährmann	cf7a67d67f	[yaplog] remove module ... Yaplog! ended its service on 2020-01-31	2020-01-31 12:56:54 +01:00
Mike Fährmann	56f1c96168	implement 'parent-directory' option (#551)	2020-01-29 18:32:37 +01:00
Mike Fährmann	6703b8a86b	[blogger] implement video extraction (closes #587)	2020-01-24 23:37:23 +01:00
Mike Fährmann	37247dbaff	miscellaneous fixes	2020-01-19 22:53:06 +01:00
Mike Fährmann	25d5ec4ff3	[twitter] add option to extract TwitPic embeds (#579)	2020-01-18 21:31:29 +01:00
Mike Fährmann	254f7c3999	implement a post-processor module to compare file versions ... (#530)	2020-01-18 21:08:25 +01:00
Mike Fährmann	a6b98d3bec	fix auto-generation of supportedsites.rst	2020-01-14 09:48:32 +01:00
Mike Fährmann	5cac79c3d9	[erolord] remove extractor	2020-01-10 14:40:30 +01:00
Mike Fährmann	87c8b89ddd	[postprocessor:metadata] add 'directory' option (#520)	2020-01-02 20:58:10 +01:00
Mike Fährmann	48e42e73fb	[reddit] change default value for 'comments' to '0'	2019-12-20 16:54:59 +01:00
Mike Fährmann	bf658fd84b	[vsco] implement 'videos' option	2019-12-19 18:34:19 +01:00
Mike Fährmann	95c90722ee	[instagram] implement 'videos' option (closes #521)	2019-12-19 18:34:06 +01:00
The Oddball	9a4ce20b8e	[realbooru] Add Realbooru extractor (#514)	2019-12-16 15:49:30 +01:00
Mike Fährmann	15f9bb3d14	add option to disable pyOpenSSL usage (#508) ... (pyOpenSSL is now disabled by default)	2019-12-08 21:21:00 +01:00
Mike Fährmann	93b43f74f4	update docs/configuration.rst ... - fix description of "skip": "enumerate" - use line breaks for lists of multiple examples - improve formatting of auto-generated man pages	2019-12-06 01:19:23 +01:00
Mike Fährmann	e17907ee2a	change default value of 'cookies-update' to 'true'	2019-12-05 23:43:49 +01:00
Mike Fährmann	26d2334550	[postprocessor:metadata] rename 'format' to 'content-format' ... Just to be consistent with the other 'extension-format' option name, and only 'format' is also still accepted.	2019-11-30 17:27:49 +01:00
Mike Fährmann	a412531451	[postprocessor:metadata] implement 'extension-format' option ... closes #477	2019-11-30 17:26:17 +01:00
Mike Fährmann	d45fabb79d	match user profile handling on deviantart and newgrounds	2019-11-22 23:20:21 +01:00
Mike Fährmann	3ece3976ae	[newgrounds] implement login support (#394)	2019-11-16 23:45:32 +01:00
Mike Fährmann	94a94f3b86	miscellaneous stuff	2019-11-08 20:58:53 +01:00
Mike Fährmann	dd5d2b2eac	[deviantart] add user profile extractor (#377, #419)	2019-11-07 18:29:49 +01:00
Mike Fährmann	9e88e7a344	[postprocessor:exec] improve (#421, #413) ... - add 'final' option - include job status in pp finalization - improve and extend documentation	2019-11-03 21:45:45 +01:00
Mike Fährmann	557e2c018b	[8chan] remove module	2019-11-02 20:06:47 +01:00
Mike Fährmann	c01ff78467	[twitter] extend 'videos' option to force extraction with ytdl ... (closes #459)	2019-11-01 22:06:07 +01:00
Mike Fährmann	702f2fbd1f	[issuu] add publication and user extractors (#413)	2019-10-31 16:52:57 +01:00
Mike Fährmann	109718a5e3	[blogger] add blog and post extractors (closes #364)	2019-10-26 14:15:55 +02:00
Mike Fährmann	8f38a35b91	[imgur] use API with "public" client_id (#446) ... Using the API endpoints makes it possible to access NSFW content without logging in.	2019-10-23 21:43:55 +02:00
Mike Fährmann	b23c822b23	[luscious] use GraphQL	2019-10-22 21:17:08 +02:00
Mike Fährmann	2057c6ba29	[naver] add blog and post extractors (closes #447)	2019-10-21 16:59:15 +02:00
Mike Fährmann	a5e6252137	update deviantart client-id/-secret instructions (#451, #424)	2019-10-21 16:57:45 +02:00
Mike Fährmann	389d2d7e38	implement 'cookies-update' option (#445)	2019-10-19 15:23:55 +02:00
Mike Fährmann	5882b00f2f	[imgur] implement login support (#446)	2019-10-15 22:00:22 +02:00
Mike Fährmann	91643ca54b	[nozomi] add search extractor (#388)	2019-10-14 23:49:46 +02:00
Mike Fährmann	1adafdd3d0	document cache file requirement for DeviantArt refresh tokens	2019-10-13 23:01:57 +02:00
Mike Fährmann	6779512fc7	[nozomi] add post and tag extractors (#388)	2019-10-13 22:16:03 +02:00
Mike Fährmann	8af59a4bba	fix & update docs ... - update Requests links - add example for --exec - set '-dev' version	2019-10-11 18:36:25 +02:00
Mike Fährmann	46ba173ded	[reddit] fix documentation inconsistencies (closes #429) ... - Require 'reddit.comments' to be a number and convert it to an integer to be extra sure - Link to the README's OAuth section were appropriate	2019-09-27 17:34:10 +02:00
Mike Fährmann	5a1a0f5325	change text representation of user extractors to "User Profiles"	2019-09-22 22:21:48 +02:00
Mike Fährmann	ee8b654464	[instagram] implement 'highlights' option (closes #329)	2019-09-21 23:38:20 +02:00
Mike Fährmann	4330133114	[imgur] add 'favorite' extractor (closes #420) ... … and use a newer site-internal API endpoint for user posts	2019-09-19 15:54:26 +02:00
Mike Fährmann	b1cddce865	Revert "[simplyhentai] fix extraction; remove image+video extractors" ... This reverts commit d1db5180ab.	2019-09-07 14:48:31 +02:00
Mike Fährmann	6fe9a134bf	[lineblog] add blog and post extractors (closes #404)	2019-09-06 22:16:42 +02:00
Mike Fährmann	6f3df3999a	[fuskator] add gallery and search extractor (closes #407)	2019-09-02 21:20:02 +02:00
Mike Fährmann	f02a768b5c	[danbooru] add 'ugoira' option (#406) ... to choose between ZIP archives or converted video files for Ugoira posts	2019-08-31 21:51:52 +02:00
Mike Fährmann	c6c5cb1898	improve 'deviantart.quality' description	2019-08-30 18:41:18 +02:00
Mike Fährmann	7c09545f70	[downloader:ytdl] add 'outtmpl' option (#395)	2019-08-24 22:47:59 +02:00
Mike Fährmann	d1db5180ab	[simplyhentai] fix extraction; remove image+video extractors	2019-08-22 23:56:41 +02:00
Mike Fährmann	b1728f512d	[patreon] support multi image posts and post URLs (#383)	2019-08-17 23:24:46 +02:00
Mike Fährmann	5a210991b6	Remove control characters from filesystem paths ... - add 'path-remove' option to specify the set of characters that should be removed - rename 'restrict-filenames' to 'path-restrict' - #348, #380	2019-08-16 23:12:16 +02:00
Mike Fährmann	e661a1ce19	update links to Request's documentation	2019-08-15 13:25:06 +02:00
Mike Fährmann	829b1ccf04	[imgur] distinguish album and gallery URLs (#380) ... A gallery can be either an album or a single image.	2019-08-14 21:40:14 +02:00
leezhi0513	42bb0cdb9b	Update gallery-dl.conf (#378)	2019-08-11 14:44:50 +02:00
Mike Fährmann	2ccf6a9e35	[instagram] make extractor tests happy (#373)	2019-08-08 18:50:26 +02:00
Mike Fährmann	8dc42bb178	implement 'enumerate' for 'extractor.skip' (#306) ... [ci skip]	2019-08-08 18:37:54 +02:00
Mike Fährmann	b7fb93e2b2	[downloader:http] add 'adjust-extensions' option	2019-08-08 16:54:20 +02:00
Mike Fährmann	189acbeac9	[imgbb] add extractor for individual images (closes #363)	2019-08-05 22:52:08 +02:00
Mike Fährmann	f687052daf	have e-hentai and exhentai on supportedsites.rst (#365)	2019-08-03 11:42:28 +02:00
Mike Fährmann	3957d27d79	[deviantart] add 'quality' option (#369)	2019-08-03 11:40:35 +02:00
Mike Fährmann	0609afd1e4	update default cache directory ... again ... Use a 'gallery-dl' subdirectory in ~/.cache to adhere to how other programs store their cached data, and call os.makedirs() so it also works without an existing ~/.cache directory.	2019-08-01 22:11:00 +02:00
Mike Fährmann	2c839f3760	[imgbb] add user extractor + login support (#361)	2019-08-01 21:39:20 +02:00
Mike Fährmann	a8b60b2bd9	change default cache directory for unix systems ... Use either $XDG_CACHE_HOME or ~/.cache (if the former isn't set) and store potentially sensitive cookies and tokens in a user's home directory and not in the world-readable /tmp.	2019-07-31 22:56:14 +02:00
Mike Fährmann	2153206093	[imgbb] add album extractor (#361)	2019-07-30 23:11:19 +02:00
Mark Henrick	923e1bb714	[docs] Fix inconsistency about which sites have optional authentication (#359) ... * [docs] Fix inconsistency about which sites have optional authentication * update authentication docs	2019-07-29 18:22:31 +02:00
Mike Fährmann	a90280f4e7	[postprocessor:zip] add 'mode' option (#355)	2019-07-29 16:51:26 +02:00
Mike Fährmann	dc73d02d87	[exhentai] always use e-hentai.org as domain + set nw cookie	2019-07-28 10:54:17 +02:00
Mike Fährmann	279db2c5b2	[vsco] add collection & image extractor + video support (#331)	2019-07-26 19:06:15 +02:00
Mike Fährmann	547ea71463	[downloader.ytdl] add 'forward-cookies' option (#352) ... The "long" name is necessary because just calling it 'cookies' would clash with how the lookup for '--cookies' is implemented.	2019-07-24 21:19:11 +02:00
Mike Fährmann	b1bea8aaeb	add 'restrict-filenames' option (#348)	2019-07-23 17:41:24 +02:00
Mike Fährmann	60cf40380a	[vsco] add user extractor (#331)	2019-07-23 16:23:11 +02:00
Mike Fährmann	3fe5ccdfa6	[adultempire] add gallery extractor (closes #340)	2019-07-21 22:29:57 +02:00
Mike Fährmann	21991acc49	add 'ciphers' option; update default User-Agent	2019-07-19 17:14:40 +02:00
Mike Fährmann	feb98cf196	[twitter] improve 'content' formatting; add option (#338) ... - include emoticons - leave newlines intact - remove pic.twitter.com/ links at the end	2019-07-17 16:02:51 +02:00
Mike Fährmann	8d1ae9b715	[tumblr] enable date-min/-max/-format options (#337)	2019-07-17 14:36:41 +02:00
Mike Fährmann	fb875d1ab8	add warning about NSFW sites in supportedsites.rst (#335)	2019-07-15 21:44:34 +02:00
Mike Fährmann	16c582aaf9	implement 'mtime' post-processor (#332) ... This can set a file's modification time according to a UNIX timestamp or a datetime object from its metadata.	2019-07-14 22:39:17 +02:00
Mike Fährmann	62097284fe	add 'download' option (#220)	2019-07-14 18:48:18 +02:00
Mike Fährmann	2ff73873f0	[erolord] add gallery extractor (closes #326)	2019-07-04 20:28:04 +02:00
Mike Fährmann	b4da8c5a97	[sexcom] add extractor for related pins (#325)	2019-07-03 21:04:23 +02:00
Mike Fährmann	69205df68d	allow '-1' for infinite retries (#300)	2019-06-30 23:10:47 +02:00
Mike Fährmann	f7b5c4c3e7	use values of 'retries' options correctly ... The RE-tries option now specifies exactly that: the maximum number a failed HTTP request is re-tried. For example a value of 2 will now correctly stop after 3 attempts: the initial one + 2 re-tries. The maximum wait-time now also caps at 30min and increases exponentially for both extractor.request() and downloader.http.download().	2019-06-30 23:10:18 +02:00
Mike Fährmann	f2000a69aa	implement 'image-unique' and 'chapter-unique' options (#303) ... The default value for both is 'false', i.e. duplicate URLs are NOT ignored. The previous behavior was to always ignore duplicate URLs to make '--abort-on-skip' work properly when new images where added to the beginning of a collection while gallery-dl is running.	2019-06-29 22:50:17 +02:00
Mike Fährmann	40da44b17f	Merge branch 'v1.9.0'	2019-06-29 15:39:52 +02:00
Mike Fährmann	a9c89085fb	[instagram] implement login support (#195)	2019-06-26 23:58:47 +02:00
Mike Fährmann	18a1f8c6cd	[vanillarock] add post and tag extractors (closes #254)	2019-06-23 22:45:36 +02:00
Mike Fährmann	f0c5093812	[nsfwalbum] add album extractor (closes #287)	2019-06-23 22:45:07 +02:00
Mike Fährmann	15e4ddf46d	implement custom logging formatter ... supports custom log message formats for each loglevel and, by extension, custom ANSI codes and colors for errors and warnings (#304)	2019-06-21 20:17:58 +02:00
Mike Fährmann	db3f52881a	add 'mtime' option	2019-06-20 17:19:44 +02:00
Mike Fährmann	ec3e8601f1	[slickpic] add user extractor (#249)	2019-06-14 18:55:56 +02:00
Mike Fährmann	d997c10320	[8muses] add album extractor (#305)	2019-06-10 22:17:46 +02:00
Mike Fährmann	e05a96db5e	[deviantart] rename 'stash' to 'extra' (#302) ... 'stash' is already used as a name for the StashExtractor and therefore expected to be a dictionary.	2019-06-10 21:05:25 +02:00
Mike Fährmann	2184e3a86b	[slickpic] add album extractor (#249)	2019-06-09 21:59:22 +02:00
Mike Fährmann	c23bf263fe	[deviantart] rename 'external' to 'stash' (#302) ... restrict extracted URLs to ones from https://sta.sh/...	2019-06-09 11:16:02 +02:00
Mike Fährmann	c73c2cda50	[pornhub] add gallery & user extractor (#282)	2019-06-07 16:31:20 +02:00
Mike Fährmann	035b850e82	update postprocessor entries in example config ... - use whitelists - add ugoira example (#299)	2019-06-07 13:47:02 +02:00
Mike Fährmann	2fb85178da	[deviantart] add 'external' option (#302) ... If a description is available, this will extract URLs from the description text and try to find Extractors for them.	2019-06-06 18:53:50 +02:00
Mike Fährmann	096009367b	[xhamster] add gallery & user extractor (#281)	2019-06-05 11:11:51 +02:00
Mike Fährmann	d33f5a7423	[wallhaven] rewrite ... - use API - remove login support, add 'api-key' option - remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric IDs that can't be translated to the new ID system - support direct links to wallpapers	2019-05-31 14:53:02 +02:00
Mike Fährmann	f1893b2b5b	[deviantart] add 'folders' option (#276)	2019-05-30 17:28:12 +02:00
Mike Fährmann	c849574def	[keenspot] add comic extractor (#223) ... Doesn't work for - http://brawlinthefamily.keenspot.com/ - http://flipside.keenspot.com/ - http://lastblood.keenspot.com/ - http://mysticrevolution.keenspot.com/ - http://porcelain.keenspot.com/ - http://twokinds.keenspot.com/ yet, because of custom layouts.	2019-05-28 21:34:38 +02:00
Mike Fährmann	bcd1801aa8	[sankakucomplex] add 'tag' extractor (#258)	2019-05-27 23:57:44 +02:00
Mike Fährmann	74c2415138	[sankakucomplex] move article extractor to its own module (#258)	2019-05-27 23:49:23 +02:00
Mike Fährmann	4465a3ea68	[kissmanga][readcomiconline] add 'captcha' option (#279) ... to configure how to handle CAPTCHA page redirects: - either interactively wait for the user to solve the CAPTCHA - or raise StopExtraction like before	2019-05-27 22:24:48 +02:00
Mike Fährmann	1e3e15c4f3	[sankaku] add article extractor (#258)	2019-05-26 17:42:36 +02:00
Mike Fährmann	e82cadac61	[patreon] add extractors (#226)	2019-05-17 10:47:22 +02:00
Mike Fährmann	4891f4a328	[hentainexus] add search extractor (#256)	2019-05-16 23:55:30 +02:00
Mike Fährmann	9890bfdf23	[flickr] improve code and metadata ... - simplify pagination - add more metadata and slightly change its structure - convert suitable values to int or list - move keys from ["photo"] to the base level - proper video support (#246) - rename method and variable names to better fit with other extractors	2019-05-14 22:10:50 +02:00
Mike Fährmann	ba8eb1ffec	[hentainexus] add gallery extractor (#256)	2019-05-12 23:59:41 +02:00
Mike Fährmann	0318c610dc	[sexcom] add extractor for search results (#147)	2019-04-24 22:10:01 +02:00
Mike Fährmann	a247c94c34	[sexcom] add pin and board extractors (#147)	2019-04-24 22:09:19 +02:00
Mike Fährmann	70be494161	[plurk] add a 'comments' options (#212)	2019-04-14 22:12:46 +02:00
Mike Fährmann	0b2ff406f6	[plurk] add timeline- and post-extractors (#212)	2019-04-14 21:48:38 +02:00
Mike Fährmann	e7cd5510d5	[pixnet] add extractors (closes #177) ... for: - users/blogs: http://albertayu773.pixnet.net/ - folders: https://albertayu773.pixnet.net/album/folder/1405768 - sets : https://albertayu773.pixnet.net/album/set/15078995 - photos : https://albertayu773.pixnet.net/album/photo/159443828	2019-04-11 19:27:02 +02:00
Mike Fährmann	d9b94a585d	[mangoxo] add login support (#184) ... A very recent change: It is now only possible to see more than the first 5 images of an album if you are logged in.	2019-04-10 18:55:25 +02:00
Mike Fährmann	e730fc9045	[twitter] add login support (#214)	2019-04-09 09:27:49 +02:00
Mike Fährmann	35919a9bb8	[livedoor] add blog- and post-extractors (#190)	2019-04-06 16:27:48 +02:00
Mike Fährmann	2ff043edfa	[yaplog] add user- and post-extractors (#190)	2019-04-04 17:56:56 +02:00
Mike Fährmann	790f15a56f	[photobucket] use HTTPS	2019-04-03 18:30:45 +02:00
Mike Fährmann	6da665f32e	[mangoxo] add album- and channel-extractors (closes #184)	2019-04-03 07:55:51 +02:00
Mike Fährmann	c70b21248d	[wikiart] add extractors (#179) ... for - artists: https://www.wikiart.org/en/thomas-cole - artist-listings: https://www.wikiart.org/en/artists-by-century/12 - artwork-listings: https://www.wikiart.org/en/paintings-by-media/grisaille	2019-04-02 17:34:57 +02:00
Mike Fährmann	bc2020e86c	release version 1.8.1	2019-03-29 17:37:11 +01:00
Mike Fährmann	369eb66125	consistently use '*' for rst lists	2019-03-28 16:21:41 +01:00
Mike Fährmann	c7a6b0ed90	[deviantart] add 'metadata' option (#189)	2019-03-21 14:49:42 +01:00
Mike Fährmann	8d96a8ce4c	[500px] add user-, gallery-, and image-extractors (#185)	2019-03-20 17:32:36 +01:00
Mike Fährmann	6277a739e4	[35photo] add user-, genre-, and image-extractors (#162)	2019-03-18 01:11:30 +01:00
Mike Fährmann	12482553bd	update links to youtube-dl	2019-03-13 22:03:02 +01:00
Mike Fährmann	6f57d44ec2	[seaotterscans] remove extractor ... http://seaotterscans.com/ now redirects to their MangaDex profile	2019-03-13 22:02:45 +01:00
Mike Fährmann	25aaf55514	[smugmug] improve format selection (closes #183) ... - use original image if available - support video formats - remove user info for ImageExtractor (it is no longer possible to get image owner information for a single image)	2019-03-10 15:20:35 +01:00
Mike Fährmann	976ccb267f	[myportfolio] combine gallery and user extractors ... An URL alone isn't good enough to distinguish between a gallery or a gallery-listing, so the new extractor decides what to do based on the page's content.	2019-03-06 19:45:01 +01:00
Mike Fährmann	9c0e2f294b	[shopify] add generic collection and product extractors (#175) ... with fashionnova.com as a default domain	2019-03-05 22:33:37 +01:00
Mike Fährmann	13e0f2a78f	[deviantart] add 'scraps' extractor (closes #168)	2019-03-01 14:13:34 +01:00
Mike Fährmann	280531c8ff	[pururin] add gallery extractor (closes #174)	2019-02-25 14:54:57 +01:00
Mike Fährmann	3159dd79d5	[seiga] use HTTPS	2019-02-21 22:51:11 +01:00
Mike Fährmann	ea6fbfdd39	simplify build_supportedsites.py	2019-02-21 14:29:19 +01:00
Mike Fährmann	22d3a2fcc8	[artstation] add extractor for artwork listings (#80) ... like https://www.artstation.com/artwork?sorting=latest or https://www.artstation.com/artwork?sorting=picks	2019-02-18 12:45:44 +01:00
Mike Fährmann	937a802b49	[dynastyscans] add extractors for images and image searches ... (closes #163)	2019-02-18 12:25:52 +01:00
Mike Fährmann	19860655a3	[weibo] add 'user' and 'status' extractors	2019-02-17 18:18:31 +01:00
Mike Fährmann	ae353ed3b0	provide "extractor" and "job" keys for logging output ... This allows for stuff like "{extractor.url}" and "{extractor.category}" in logging format strings. Accessing 'extractor' and 'job' in any way will return "None" if those fields aren't defined, i.e. in general logging messages.	2019-02-14 11:09:58 +01:00
Mike Fährmann	53c2fd4664	add mastodon/foolslide/foolfuuka examples to example config	2019-02-05 16:17:25 +01:00
Mike Fährmann	d36ec51e5a	[tsumino] add extractor for search results (#161)	2019-02-02 14:56:46 +01:00
Mike Fährmann	bfbbac4495	[tsumino] add login capabilities (#161)	2019-01-30 17:58:48 +01:00
Mike Fährmann	6126615698	update URLs for supportedsites.rst	2019-01-30 16:18:22 +01:00
Mike Fährmann	80a75a1ecf	[tsumino] add gallery extractor (#161)	2019-01-29 17:28:48 +01:00
Mike Fährmann	0c32dc5858	[hentaifox] add extractor for search results (#160)	2019-01-28 22:38:32 +01:00
Mike Fährmann	0156189468	[hentaifox] add chapter extractor (#160)	2019-01-28 18:00:32 +01:00
Mike Fährmann	e4171d6baf	[luscious] add login capabilities (closes #159)	2019-01-28 17:14:15 +01:00
Mike Fährmann	134487ffb0	[exhentai] stop extraction if image limit is exceeded (#141) ... can be turned off with the `exhentai.limits' option	2019-01-26 18:40:39 +01:00
Mike Fährmann	b50b30f1c9	[photobucket] download subalbums (#117)	2019-01-22 14:05:18 +01:00
Mike Fährmann	277b52101a	add 'category-transfer' option ... [ci skip]	2019-01-19 20:28:19 +01:00
Mike Fährmann	4b441c162e	release version 1.6.3	2019-01-18 12:31:04 +01:00
Mike Fährmann	8aba2bdebf	[postprocessor:metadata] add 'tags' and 'custom' modes (#135)	2019-01-17 21:18:12 +01:00
Mike Fährmann	e53cdfd6a8	update build_supportedsites.py	2019-01-09 14:58:35 +01:00
Mike Fährmann	1e4d351ad3	[danbooru] add authentication support (closes #151) ... ... via HTTP Basic Auth with username and "password". The password value in this case is not the account password itself, but the"api_key" found in your user profile.	2019-01-09 14:19:07 +01:00
Mike Fährmann	06cbf5f9c4	implement 'chapter-reverse' option (#149) ... Setting it to `true` will start with the latest chapter instead of the first one.	2019-01-07 18:22:33 +01:00
Mike Fährmann	e95b24f056	[reactor] add wait-min & -max options (#148)	2019-01-07 18:04:47 +01:00
Mike Fährmann	a36f52a730	[joyreactor] add extractor for search results (#148)	2019-01-03 16:25:56 +01:00
Mike Fährmann	6c71e9cf5d	[deviantart] add separate 'sta.sh' extractor (#113) ... - supports multiple stashed deviations per page - explicitly mentions sta.sh support on supportedsites.rst	2018-12-26 18:56:57 +01:00
Mike Fährmann	28f9539551	[tumblr] change default values for post types and inline media	2018-12-26 18:55:59 +01:00
Mike Fährmann	5be95034ba	[tumblr] add option to download avatars (#137)	2018-12-26 14:29:30 +01:00
Mike Fährmann	be52069cbc	update CHANGELOG and docs/supportedsites	2018-11-28 14:53:27 +01:00
Mike Fährmann	c47482b110	smaller changes, missing docs, etc. ... - make 'netrc' extractor-specific - rename 'downloader.enable' to 'enabled' - document 'downloader.ytdl.format' - consistent newlines in configuration.rst	2018-11-16 18:18:07 +01:00
Mike Fährmann	8ef84a6823	add option to enable/disable specific downloader modules ... ... and write URLs with no (active) downloader to unsupported-file	2018-11-13 18:06:36 +01:00
Mike Fährmann	2fa28a2609	update default user-agent string (closes #122)	2018-11-11 10:07:10 +01:00
Mike Fährmann	655549df7c	[downloader:ytdl] add several options ... The "default" downloader options (rate, retries, timeout, verify) are mapped to corresponding youtube-dl options. downloader.ytdl.logging tells the downloader to pass youtube-dl's output to a Logger object. downloader.ytdl.raw-options allows to pass arbitrary options to the YoutubeDL constructor.	2018-10-20 18:26:49 +02:00
Mike Fährmann	6ed629f2b6	allow specifying number of skips before abort/exit (closes #115) ... In addition to 'abort' and 'exit', it is now possible to specify 'abort:N' and 'exit:N' (where N is any integer) as value for 'skip' to abort/exit after consecutively skipping N downloads.	2018-10-13 17:21:55 +02:00
Mike Fährmann	d8492df51b	[deviantart] extend functionality of 'original' option	2018-10-11 22:11:49 +02:00
Mike Fährmann	392a081657	update 'docs/configuration.rst'	2018-10-08 23:09:01 +02:00
Mike Fährmann	f8b3b00249	[twitter] add experimental 'videos' option (#99) ... Enabling this option will detect videos in tweets and output them as "unsupported" URLs, so that these can then be downloaded with youtube-dl There are a lot of improvements to be made to the current implementation, but it works and does what it is supposed to, even if inefficient as can be ...	2018-09-30 21:52:23 +02:00
Mike Fährmann	2cf3f53839	[yuki] add thread extractor (closes #111)	2018-09-28 12:47:32 +02:00
Mike Fährmann	17611bfec0	update build_supportedsites.py script	2018-09-28 12:43:19 +02:00
Mike Fährmann	09d2f3e5e7	[postprocessor:ugoira] improve libx264 detection	2018-09-24 13:12:54 +02:00
Mike Fährmann	c402cc4047	[hentaifoundry] add 'popular' and 'recent' extractors ... for "Popular Pictures" and "Recent Pictures" listings	2018-09-24 13:11:18 +02:00
Mike Fährmann	a5fc311dfa	[hentaifoundry] add 'favorite' extractor	2018-09-22 21:23:29 +02:00
Mike Fährmann	55f5c87160	[postprocessor:ugoira] add 'libx264-prevent-odd' option ... A rather crude workaround for "width/height not divisible by 2" errors when using libx264.	2018-09-21 20:01:14 +02:00
Mike Fährmann	006f75b538	[hentaifoundry] rewrite + more metadata ... - extract width, height, artist per image - improve pattern regex - better extensibility for other listings	2018-09-21 11:23:51 +02:00
Mike Fährmann	6ea9a78588	[wallhaven] add login capabilities ... Being logged in is required to access NSFW wallpapers.	2018-09-19 21:04:01 +02:00
Mike Fährmann	c9290d8212	[wallhaven] add wallpaper and search extractors ... todo: - login support to gain access to NSFW wallpapers - extractors for tag-, similar-, latest-listings - skip() support	2018-09-17 21:26:13 +02:00
Mike Fährmann	22ab509a70	[bobx] rename "model" to "idol" extractor	2018-09-14 18:11:36 +02:00
Mike Fährmann	69fd61ea86	[bobx] add gallery and model extractors	2018-09-13 20:13:12 +02:00
Mike Fährmann	7742cf8601	[tumblr] change 'reblogs' option (#103) ... - rename "deleted" to "same-blog" - change test for deleted original post to test if original post owner has the same UUID (full blog name) as the one being downloaded from - add 'blog[uuid]' metadata to allow comparison with 'reblogged_from_uuid'	2018-09-10 15:40:25 +02:00
Mike Fährmann	f1695567e8	adjust values in template config file ... (#104)	2018-09-09 14:10:55 +02:00
Mike Fährmann	a666ddd16b	[tumblr] extend 'reblogs' functionality (#103) ... Setting 'reblogs' to "deleted" will check if the parent post of a reblog has been deleted and download its media content if that is the case, otherwise it will be skipped. This is a rather costly operation (1 API request per reblogged post) and should therefore be used with care.	2018-09-07 19:13:52 +02:00
Mike Fährmann	c9b8e6aefc	[reddit] fix submission-ID parsing (#104) ... Uppercase characters caused a ValueError exception	2018-09-07 18:27:54 +02:00
Mike Fährmann	3ecea4cf36	[hentaicafe] add chapter and manga extractors (#101)	2018-09-05 21:08:40 +02:00
Mike Fährmann	75862715ac	[behance] add user extractor	2018-08-31 17:42:09 +02:00
Mike Fährmann	6ecb36d88c	[postprocessor:ugoira] add 'ffmpeg-output' option	2018-08-31 17:37:35 +02:00
Mike Fährmann	ab2bfaeb46	[ngomik] add replacement for 'subapics' ... http://subapics.com/ got discontinued and replaced by http://ngomik.in/. ngomik.in is still displaying a link to the "old site" showing a big "Account Suspended" sign.	2018-08-23 15:29:53 +02:00
Mike Fährmann	e9dd2eff1d	[twitter] add extractor for media-tweet timelines (#96) ... For example "https://twitter.com/PicturesEarth/media". They are different from normal timelines in that they do not contain any (re)tweets from other users and feature all media the user ever posted, including responses to other tweets.	2018-08-19 20:46:12 +02:00
Mike Fährmann	9b1c39032c	[twitter] changes and improvements ... - rename User- to TimelineExtractor - rename 'userid' to 'user_id' to conform to the other ..._id values - adjust archive_fmt to deal with retweets - emulate browser behavior for API calls	2018-08-18 23:04:45 +02:00
Mike Fährmann	10365394d7	[twitter] add support for user-timelines (closes #96) ... also adds a 'retweets' option to filter retweeted content	2018-08-17 20:04:11 +02:00
Mike Fährmann	63fa0b2006	[pinterest] add extractors for related pins ... Related pins can not be accessed by adding a "#related" fragment to the end of a Pinterest URL, for example: - https://www.pinterest.com/pin/858146903966145189/#related - https://www.pinterest.com/g1952849/test-/#related There are no explicit real URLs for related pins, using an option to enable them results in "clunky" code, and a custom "related:<URL>" scheme doesn't feel right either.	2018-08-15 21:49:45 +02:00
Mike Fährmann	3c90df6635	[piczel] add user, folder and image extractors	2018-08-08 10:53:01 +02:00
Mike Fährmann	b164231bca	[sankaku] increase default values for 'wait-min/-max'	2018-08-03 17:06:51 +02:00
Mike Fährmann	df082e923c	[behance] add gallery extractor (#95)	2018-08-01 21:46:55 +02:00
Mike Fährmann	5f27cfeff6	[deviantart] remove prefer-public option ... All API requests now always use a public token and only switch to a private token for pagination results if `refresh-token` is set and less deviations than requested were returned.	2018-07-26 19:43:46 +02:00
Mike Fährmann	212130b048	[deviantart] improve public-private token switching ... - rename option to `prefer-public` - now also works for galleries with less than 24 items	2018-07-25 12:52:36 +02:00
Mike Fährmann	886d662582	[deviantart] add option to minimize refresh-token usage ... Always trying with a public token first and repeating the API request with a private token if deviations are missing doesn't quite work for galleries and folders with less than 25 items, so its an option and not the default.	2018-07-24 21:44:57 +02:00
Mike Fährmann	54a0d72dc8	[postprocessor:ugoira] improve frame rate handling ... By default FFmpeg assumes a 25 FPS input frame rate, leading to dropped frames if the source requires a higher frame rate than that. This commit adds a `framerate` option (default "auto"), which allows to automatically assign a (more or less) fitting frame rate based on delays between ugoira frames and avoids dropped frames.	2018-07-23 21:33:20 +02:00
Mike Fährmann	84854fcad7	[myportfolio] add user and gallery extractors (#95)	2018-07-19 18:56:45 +02:00
Mike Fährmann	c9f70e0a19	[paheal] use HTTPS	2018-07-17 21:25:03 +02:00
Mike Fährmann	ff436692bf	["deviantart] add 'journals' option	2018-07-16 18:14:41 +02:00