Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-22 10:42:34 +01:00
Code Issues Releases Wiki Activity
2,134 Commits 6 Branches 164 Tags 15 MiB
48e42e73fb
Commit Graph

2134 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
48e42e73fb
[reddit] change default value for 'comments' to '0' 2019-12-20 16:54:59 +01:00
Mike Fährmann
9c0928457a
[reddit] fix errors with 't1_…' submissions 2019-12-20 16:49:44 +01:00
Mike Fährmann
58391d492d
cache archive keys generated in __contains__() (#524) 
To avoid writing a different key to the archive than what was checked
against before the file download.
 2019-12-20 16:43:08 +01:00
Mike Fährmann
bf658fd84b
[vsco] implement 'videos' option 2019-12-19 18:34:19 +01:00
Mike Fährmann
95c90722ee
[instagram] implement 'videos' option (closes #521) 2019-12-19 18:34:06 +01:00
Mike Fährmann
1921c127a5
make OSErrors during file downloads nonfatal (closes #512) 
… except ENOSPC (No space left on device), since there is no reason to
continue downloading in that case.

All other errors that would prevent downloading data and writing it to
disk get already raised during directory creation and are therefore not
checked here.
 2019-12-19 18:34:05 +01:00
Mike Fährmann
d0920e84e9
update test results 2019-12-19 18:33:48 +01:00
Mike Fährmann
8c11e81c9f
Merge commit '63e6993716db8d8bedfb7b0d445c7161493046b6' 2019-12-16 17:25:23 +01:00
Mike Fährmann
63e6993716
merge 'bypost' functionality into metadata postprocessor 2019-12-16 17:19:23 +01:00
Mike Fährmann
31a29835ff
[realbooru] simplify extractors and update tests (#514) 2019-12-16 16:06:50 +01:00
The Oddball
9a4ce20b8e [realbooru] Add Realbooru extractor (#514) 2019-12-16 15:49:30 +01:00
Mike Fährmann
f9e74320de
retain trailing zeroes in Cloudflare challenge answers 2019-12-16 15:32:49 +01:00
Mike Fährmann
72b8fbfbad
[instagram] make post-page extraction nonfatal 2019-12-15 22:56:27 +01:00
Mike Fährmann
922b8a9595
[weibo] raise NotFoundError for unavailable/deleted statuses 2019-12-14 22:10:02 +01:00
Mike Fährmann
0cd157300e
[patreon] fix regex pattern for posts 
The previous one would match the first number in the URL slug as
post ID, which would fail for posts with numbers in their title.
 2019-12-14 22:08:49 +01:00
Mike Fährmann
fe19e233f3
[xvideos] improve 
- derive from GalleryExtractor
- match '…-channels' URLs
- "better" metadata structure
 2019-12-13 19:03:56 +01:00
Mike Fährmann
d3e44e899d
raise NotFoundErrors for 404 responses in GalleryExtractors 2019-12-13 18:42:04 +01:00
Mike Fährmann
a4dd8b3dab
improve _check_cookies() 
Only loop over all cookies once instead of calling
cookiejar._find() for each cookie name.
 2019-12-13 15:51:20 +01:00
Mike Fährmann
76e60d10a6
[patreon] raise proper exception if creator/post doesn't exist 2019-12-12 01:14:32 +01:00
Mike Fährmann
9e63804347
[patreon] make retrieving user info nonfatal (#508) 
… and fall back to the included data if an error occurs.
 2019-12-12 00:31:35 +01:00
Mike Fährmann
964dc57286
[vsco] improve image resolutions 
https://im.vsco.co/ URLs redirect to the appropriate CDN server
and occasionally insert a '/1200x1600/' into the image path,
limiting image dimensions.

This commit constructs redirect targets out of the given
im,vsco.co URLs without sending extra HTTP requests
and without any "builtin" resolution restrictions.
 2019-12-12 00:27:45 +01:00
Mike Fährmann
0629fe8fa4
[vsco] fix user profile extraction … again 
Given the pattern from last time, collections will also change
in due time and use cursor-based pagination.
 2019-12-11 23:57:56 +01:00
Mike Fährmann
ab17ea9632
[deviantart] only print warning if 'original' is enabled 2019-12-11 21:58:43 +01:00
Mike Fährmann
2188db6284
[gelbooru] fix non-API tag extraction 2019-12-10 21:31:55 +01:00
Mike Fährmann
c4702ec9b6
simplify some logging calls 2019-12-10 21:30:08 +01:00
Gio
c0b9ad678d Separate metadata from handle_url into handle_metadata, commenting 2019-12-09 16:02:15 -06:00
Mike Fährmann
c9ef1b21c3
[patreon] get partial user info without /api/user/<id> (#507) 
It's a lot less data, but doesn't invoke any additional
HTTP requests with potential Cloudflare CAPTCHAs.
 2019-12-09 20:38:05 +01:00
Mike Fährmann
0ab9bb1721
[4chan] add extractor for entire boards (closes #510) 2019-12-09 20:36:05 +01:00
Mike Fährmann
c59b98c81b
[downloader:http] improve rate limit handling 
- Move the download "logic" with rate limit checks into its own
  method that only gets used if a rate limit should be enforced
- Fix an issue where suspending gallery-dl during a download would
  basically ignore the rate limit for the remaining download when
  resuming its execution.
 2019-12-09 20:34:22 +01:00
Mike Fährmann
bbbafc1c24
[downloader:http] catch both possible SSLException instances 
With pyOpenSSL installed, but disabled, the SSLError exception
would be set to the one from pyOpenSSL, which could never get raised.

This commit solves this problem by catching both, the native SSLError
exception as well as the one from pyOpenSSL (if available.1)
 2019-12-09 20:34:10 +01:00
Gio
c20bb5c338 Naming convention, as per travis. 2019-12-09 01:05:01 -06:00
Gio
6ed4fc07ff Don't print intentional metadata skips to the console. 2019-12-09 01:02:17 -06:00
Gio
cfc70a97ab Added an additional channel for downloading the metadata of an entire post or gallery. 2019-12-09 00:56:27 -06:00
Mike Fährmann
f451be48c3
release version 1.12.0 2019-12-08 22:47:13 +01:00
Mike Fährmann
15f9bb3d14
add option to disable pyOpenSSL usage (#508) 
(pyOpenSSL is now disabled by default)
 2019-12-08 21:21:00 +01:00
Mike Fährmann
5eb89b2289
don't install pyOpenSSL for Travis tests (#508) 2019-12-07 23:40:01 +01:00
Mike Fährmann
c8e99e3b3b
[deviantart] fix crash on missing "token" field (#505) 2019-12-07 23:38:36 +01:00
Mike Fährmann
6ed2c7823c
[deviantart] disable original downloads if no cookies set 
For 'deviation' and 'scraps' extractors only, since original file
downloads for those two will always fail with a 404 Not Found
when not logged in.
 2019-12-07 22:39:30 +01:00
Mike Fährmann
50deab5265
[deviantart] fix URL generation from /extended_fetch results 
(closes #505)
 2019-12-07 22:07:55 +01:00
Mike Fährmann
1f209da4c0
[pixiv] match new search URLs (closes #507) 2019-12-06 21:38:38 +01:00
Mike Fährmann
93b43f74f4
update docs/configuration.rst 
- fix description of "skip": "enumerate"
- use line breaks for lists of multiple examples
- improve formatting of auto-generated man pages
 2019-12-06 01:19:23 +01:00
Mike Fährmann
e17907ee2a
change default value of 'cookies-update' to 'true' 2019-12-05 23:43:49 +01:00
Mike Fährmann
07dafad26d
[twitter] attempt to fix infinite loops (#499) 
(Hopefully this doesn't break anything else)
 2019-12-03 22:55:29 +01:00
Mike Fährmann
71acbdabf4
[2chan] fix metadata extraction 2019-12-03 17:01:11 +01:00
Mike Fährmann
173a93454e
fix config.set calls in build_testresults_db.py 2019-12-03 16:29:09 +01:00
Mike Fährmann
c0a1241648
[livedoor] force https:// for image URLs 2019-12-03 16:14:05 +01:00
Mike Fährmann
6e23c0da09
[imgur] add extractor for subreddit links (closes #500) 2019-12-02 23:44:13 +01:00
Mike Fährmann
38c05df290
[oauth] add custom/default indicator to log messages (#501) 2019-12-02 23:41:02 +01:00
Mike Fährmann
372ffe95ee
[oauth] adjust Flickr redirect URI (fixes #503) 
Flickr now automatically forces https:// for all redirect URIs.
 2019-12-02 22:24:20 +01:00
Mike Fährmann
004812258d
[hentaifox] fix extraction 2019-12-02 22:21:45 +01:00