2108 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Mike Fährmann	ab17ea9632	[deviantart] only print warning if 'original' is enabled	2019-12-11 21:58:43 +01:00
Mike Fährmann	2188db6284	[gelbooru] fix non-API tag extraction	2019-12-10 21:31:55 +01:00
Mike Fährmann	c4702ec9b6	simplify some logging calls	2019-12-10 21:30:08 +01:00
Mike Fährmann	c9ef1b21c3	[patreon] get partial user info without /api/user/<id> (#507) ... It's a lot less data, but doesn't invoke any additional HTTP requests with potential Cloudflare CAPTCHAs.	2019-12-09 20:38:05 +01:00
Mike Fährmann	0ab9bb1721	[4chan] add extractor for entire boards (closes #510)	2019-12-09 20:36:05 +01:00
Mike Fährmann	c59b98c81b	[downloader:http] improve rate limit handling ... - Move the download "logic" with rate limit checks into its own method that only gets used if a rate limit should be enforced - Fix an issue where suspending gallery-dl during a download would basically ignore the rate limit for the remaining download when resuming its execution.	2019-12-09 20:34:22 +01:00
Mike Fährmann	bbbafc1c24	[downloader:http] catch both possible SSLException instances ... With pyOpenSSL installed, but disabled, the SSLError exception would be set to the one from pyOpenSSL, which could never get raised. This commit solves this problem by catching both, the native SSLError exception as well as the one from pyOpenSSL (if available.1)	2019-12-09 20:34:10 +01:00
Mike Fährmann	f451be48c3	release version 1.12.0	2019-12-08 22:47:13 +01:00
Mike Fährmann	15f9bb3d14	add option to disable pyOpenSSL usage (#508) ... (pyOpenSSL is now disabled by default)	2019-12-08 21:21:00 +01:00
Mike Fährmann	5eb89b2289	don't install pyOpenSSL for Travis tests (#508)	2019-12-07 23:40:01 +01:00
Mike Fährmann	c8e99e3b3b	[deviantart] fix crash on missing "token" field (#505)	2019-12-07 23:38:36 +01:00
Mike Fährmann	6ed2c7823c	[deviantart] disable original downloads if no cookies set ... For 'deviation' and 'scraps' extractors only, since original file downloads for those two will always fail with a 404 Not Found when not logged in.	2019-12-07 22:39:30 +01:00
Mike Fährmann	50deab5265	[deviantart] fix URL generation from /extended_fetch results ... (closes #505)	2019-12-07 22:07:55 +01:00
Mike Fährmann	1f209da4c0	[pixiv] match new search URLs (closes #507)	2019-12-06 21:38:38 +01:00
Mike Fährmann	93b43f74f4	update docs/configuration.rst ... - fix description of "skip": "enumerate" - use line breaks for lists of multiple examples - improve formatting of auto-generated man pages	2019-12-06 01:19:23 +01:00
Mike Fährmann	e17907ee2a	change default value of 'cookies-update' to 'true'	2019-12-05 23:43:49 +01:00
Mike Fährmann	07dafad26d	[twitter] attempt to fix infinite loops (#499) ... (Hopefully this doesn't break anything else)	2019-12-03 22:55:29 +01:00
Mike Fährmann	71acbdabf4	[2chan] fix metadata extraction	2019-12-03 17:01:11 +01:00
Mike Fährmann	173a93454e	fix config.set calls in build_testresults_db.py	2019-12-03 16:29:09 +01:00
Mike Fährmann	c0a1241648	[livedoor] force https:// for image URLs	2019-12-03 16:14:05 +01:00
Mike Fährmann	6e23c0da09	[imgur] add extractor for subreddit links (closes #500)	2019-12-02 23:44:13 +01:00
Mike Fährmann	38c05df290	[oauth] add custom/default indicator to log messages (#501)	2019-12-02 23:41:02 +01:00
Mike Fährmann	372ffe95ee	[oauth] adjust Flickr redirect URI (fixes #503) ... Flickr now automatically forces https:// for all redirect URIs.	2019-12-02 22:24:20 +01:00
Mike Fährmann	004812258d	[hentaifox] fix extraction	2019-12-02 22:21:45 +01:00
Mike Fährmann	e2710702d4	fix Cloudflare bypss	2019-12-01 01:07:24 +01:00
Mike Fährmann	8759403f37	[plurk] add delay between comment requests	2019-12-01 01:03:31 +01:00
Mike Fährmann	a28552fd19	update test results ... - hbrowse: one tag got removed - mangoxo: gallery changed owner - photobucket: ?, but photo still downloads	2019-11-30 23:59:32 +01:00
Mike Fährmann	dcaa3d01bd	[imagefap] adapt to new image URL format	2019-11-30 23:48:02 +01:00
Mike Fährmann	e62c209ca0	[nijie] fix 'date' parsing	2019-11-30 23:08:21 +01:00
Mike Fährmann	3bba763ab9	[twitter] improve ... - update metadata structure - combine all user… entries into their own dict - let 'user' always specify the Timeline owner - add 'author' entry that specifies the original Tweet author - create directories per post (closes #491) - fix username issues with /i/web/ URLs	2019-11-30 22:30:37 +01:00
Mike Fährmann	26d2334550	[postprocessor:metadata] rename 'format' to 'content-format' ... Just to be consistent with the other 'extension-format' option name, and only 'format' is also still accepted.	2019-11-30 17:27:49 +01:00
Mike Fährmann	a412531451	[postprocessor:metadata] implement 'extension-format' option ... closes #477	2019-11-30 17:26:17 +01:00
Mike Fährmann	0f1538af78	split filename formatting into its own function	2019-11-29 22:32:07 +01:00
Mike Fährmann	db35c3b581	[directlink] separate filenames from paths ... With this, all default filename formats specify an '{extension}' and PathFormat.set_extension() reliably works for all files.	2019-11-28 23:50:00 +01:00
Mike Fährmann	41a3169c67	[foolfuuka] use '{extension}' in default filename format	2019-11-28 23:12:48 +01:00
Mike Fährmann	e9aed62c91	[imgur] unescape image titles	2019-11-28 22:13:24 +01:00
Mike Fährmann	bca2222559	add '--exec-after'	2019-11-27 19:42:46 +01:00
Mike Fährmann	ed6592ea1a	remove '--abort-on-skip'	2019-11-27 19:41:24 +01:00
Mike Fährmann	2c332edaad	[plurk] fix comment pagination	2019-11-27 19:39:56 +01:00
Mike Fährmann	a3fa45bbb1	[behance] get images from 'media_collection' modules	2019-11-27 01:04:33 +01:00
Mike Fährmann	359c3bc1c5	[deviantart] revert to getting download URLs from OAuth API ... This commit (partially) reverts 27b5b24, 94eb7c6, and a437e78. Download URLs from the 'extended_fetch' endpoint are now only usable for logged in users, while those from the respective OAuth API endpoint are working again. Everything except scraps and direct deviation links should be fixed, and those two categories will work with exported cookies. (#488) TODO: - "native" login with --username and --password - better handling of internally stored cookies	2019-11-26 23:29:46 +01:00
Mike Fährmann	42b9633c7e	update test results	2019-11-26 23:27:15 +01:00
Mike Fährmann	d4d8c1c01f	update links to standalone executables in README.rst	2019-11-26 12:05:31 +01:00
Mike Fährmann	b28bd1c73e	[bobx] set generated session cookie (closes #482) ... This reverts commit 490831f and also restores original image downloads by setting a randomly generated session cookie. No login required.	2019-11-25 20:04:11 +01:00
Mike Fährmann	ae09f87602	improve SharedConfigMixin config lookups	2019-11-25 18:31:38 +01:00
Mike Fährmann	b5c964332b	improve config.py test coverage	2019-11-25 17:20:00 +01:00
Mike Fährmann	f5604492c3	update interface of config functions	2019-11-24 00:42:28 +01:00
Mike Fährmann	4ca883c66f	[smugmug] replace test for custom URLs ... The old one (http://www.creativedogportraits.com/) is empty and/or no longer handled by SmugMug.	2019-11-22 23:25:55 +01:00
Mike Fährmann	d45fabb79d	match user profile handling on deviantart and newgrounds	2019-11-22 23:20:21 +01:00
Mike Fährmann	ea80dadd09	[deviantart] restore archive keys ... Commit 9fdc5e7 changed 'username' fields to have consistent capitalization, but that invalidated the archive keys of several extractors where 'username' was usually lowercase.	2019-11-21 17:00:08 +01:00