2129 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Mike Fährmann	1921c127a5	make OSErrors during file downloads nonfatal (closes #512) ... … except ENOSPC (No space left on device), since there is no reason to continue downloading in that case. All other errors that would prevent downloading data and writing it to disk get already raised during directory creation and are therefore not checked here.	2019-12-19 18:34:05 +01:00
Mike Fährmann	d0920e84e9	update test results	2019-12-19 18:33:48 +01:00
Mike Fährmann	8c11e81c9f	Merge commit '63e6993716db8d8bedfb7b0d445c7161493046b6'	2019-12-16 17:25:23 +01:00
Mike Fährmann	63e6993716	merge 'bypost' functionality into metadata postprocessor	2019-12-16 17:19:23 +01:00
Mike Fährmann	31a29835ff	[realbooru] simplify extractors and update tests (#514)	2019-12-16 16:06:50 +01:00
The Oddball	9a4ce20b8e	[realbooru] Add Realbooru extractor (#514)	2019-12-16 15:49:30 +01:00
Mike Fährmann	f9e74320de	retain trailing zeroes in Cloudflare challenge answers	2019-12-16 15:32:49 +01:00
Mike Fährmann	72b8fbfbad	[instagram] make post-page extraction nonfatal	2019-12-15 22:56:27 +01:00
Mike Fährmann	922b8a9595	[weibo] raise NotFoundError for unavailable/deleted statuses	2019-12-14 22:10:02 +01:00
Mike Fährmann	0cd157300e	[patreon] fix regex pattern for posts ... The previous one would match the first number in the URL slug as post ID, which would fail for posts with numbers in their title.	2019-12-14 22:08:49 +01:00
Mike Fährmann	fe19e233f3	[xvideos] improve ... - derive from GalleryExtractor - match '…-channels' URLs - "better" metadata structure	2019-12-13 19:03:56 +01:00
Mike Fährmann	d3e44e899d	raise NotFoundErrors for 404 responses in GalleryExtractors	2019-12-13 18:42:04 +01:00
Mike Fährmann	a4dd8b3dab	improve _check_cookies() ... Only loop over all cookies once instead of calling cookiejar._find() for each cookie name.	2019-12-13 15:51:20 +01:00
Mike Fährmann	76e60d10a6	[patreon] raise proper exception if creator/post doesn't exist	2019-12-12 01:14:32 +01:00
Mike Fährmann	9e63804347	[patreon] make retrieving user info nonfatal (#508) ... … and fall back to the included data if an error occurs.	2019-12-12 00:31:35 +01:00
Mike Fährmann	964dc57286	[vsco] improve image resolutions ... https://im.vsco.co/ URLs redirect to the appropriate CDN server and occasionally insert a '/1200x1600/' into the image path, limiting image dimensions. This commit constructs redirect targets out of the given im,vsco.co URLs without sending extra HTTP requests and without any "builtin" resolution restrictions.	2019-12-12 00:27:45 +01:00
Mike Fährmann	0629fe8fa4	[vsco] fix user profile extraction … again ... Given the pattern from last time, collections will also change in due time and use cursor-based pagination.	2019-12-11 23:57:56 +01:00
Mike Fährmann	ab17ea9632	[deviantart] only print warning if 'original' is enabled	2019-12-11 21:58:43 +01:00
Mike Fährmann	2188db6284	[gelbooru] fix non-API tag extraction	2019-12-10 21:31:55 +01:00
Mike Fährmann	c4702ec9b6	simplify some logging calls	2019-12-10 21:30:08 +01:00
Gio	c0b9ad678d	Separate metadata from handle_url into handle_metadata, commenting	2019-12-09 16:02:15 -06:00
Mike Fährmann	c9ef1b21c3	[patreon] get partial user info without /api/user/<id> (#507) ... It's a lot less data, but doesn't invoke any additional HTTP requests with potential Cloudflare CAPTCHAs.	2019-12-09 20:38:05 +01:00
Mike Fährmann	0ab9bb1721	[4chan] add extractor for entire boards (closes #510)	2019-12-09 20:36:05 +01:00
Mike Fährmann	c59b98c81b	[downloader:http] improve rate limit handling ... - Move the download "logic" with rate limit checks into its own method that only gets used if a rate limit should be enforced - Fix an issue where suspending gallery-dl during a download would basically ignore the rate limit for the remaining download when resuming its execution.	2019-12-09 20:34:22 +01:00
Mike Fährmann	bbbafc1c24	[downloader:http] catch both possible SSLException instances ... With pyOpenSSL installed, but disabled, the SSLError exception would be set to the one from pyOpenSSL, which could never get raised. This commit solves this problem by catching both, the native SSLError exception as well as the one from pyOpenSSL (if available.1)	2019-12-09 20:34:10 +01:00
Gio	c20bb5c338	Naming convention, as per travis.	2019-12-09 01:05:01 -06:00
Gio	6ed4fc07ff	Don't print intentional metadata skips to the console.	2019-12-09 01:02:17 -06:00
Gio	cfc70a97ab	Added an additional channel for downloading the metadata of an entire post or gallery.	2019-12-09 00:56:27 -06:00
Mike Fährmann	f451be48c3	release version 1.12.0	2019-12-08 22:47:13 +01:00
Mike Fährmann	15f9bb3d14	add option to disable pyOpenSSL usage (#508) ... (pyOpenSSL is now disabled by default)	2019-12-08 21:21:00 +01:00
Mike Fährmann	5eb89b2289	don't install pyOpenSSL for Travis tests (#508)	2019-12-07 23:40:01 +01:00
Mike Fährmann	c8e99e3b3b	[deviantart] fix crash on missing "token" field (#505)	2019-12-07 23:38:36 +01:00
Mike Fährmann	6ed2c7823c	[deviantart] disable original downloads if no cookies set ... For 'deviation' and 'scraps' extractors only, since original file downloads for those two will always fail with a 404 Not Found when not logged in.	2019-12-07 22:39:30 +01:00
Mike Fährmann	50deab5265	[deviantart] fix URL generation from /extended_fetch results ... (closes #505)	2019-12-07 22:07:55 +01:00
Mike Fährmann	1f209da4c0	[pixiv] match new search URLs (closes #507)	2019-12-06 21:38:38 +01:00
Mike Fährmann	93b43f74f4	update docs/configuration.rst ... - fix description of "skip": "enumerate" - use line breaks for lists of multiple examples - improve formatting of auto-generated man pages	2019-12-06 01:19:23 +01:00
Mike Fährmann	e17907ee2a	change default value of 'cookies-update' to 'true'	2019-12-05 23:43:49 +01:00
Mike Fährmann	07dafad26d	[twitter] attempt to fix infinite loops (#499) ... (Hopefully this doesn't break anything else)	2019-12-03 22:55:29 +01:00
Mike Fährmann	71acbdabf4	[2chan] fix metadata extraction	2019-12-03 17:01:11 +01:00
Mike Fährmann	173a93454e	fix config.set calls in build_testresults_db.py	2019-12-03 16:29:09 +01:00
Mike Fährmann	c0a1241648	[livedoor] force https:// for image URLs	2019-12-03 16:14:05 +01:00
Mike Fährmann	6e23c0da09	[imgur] add extractor for subreddit links (closes #500)	2019-12-02 23:44:13 +01:00
Mike Fährmann	38c05df290	[oauth] add custom/default indicator to log messages (#501)	2019-12-02 23:41:02 +01:00
Mike Fährmann	372ffe95ee	[oauth] adjust Flickr redirect URI (fixes #503) ... Flickr now automatically forces https:// for all redirect URIs.	2019-12-02 22:24:20 +01:00
Mike Fährmann	004812258d	[hentaifox] fix extraction	2019-12-02 22:21:45 +01:00
Mike Fährmann	e2710702d4	fix Cloudflare bypss	2019-12-01 01:07:24 +01:00
Mike Fährmann	8759403f37	[plurk] add delay between comment requests	2019-12-01 01:03:31 +01:00
Mike Fährmann	a28552fd19	update test results ... - hbrowse: one tag got removed - mangoxo: gallery changed owner - photobucket: ?, but photo still downloads	2019-11-30 23:59:32 +01:00
Mike Fährmann	dcaa3d01bd	[imagefap] adapt to new image URL format	2019-11-30 23:48:02 +01:00
Mike Fährmann	e62c209ca0	[nijie] fix 'date' parsing	2019-11-30 23:08:21 +01:00