Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-23 19:22:32 +01:00
Code Issues Releases Wiki Activity
2,335 Commits 6 Branches 164 Tags 16 MiB
cf4cef3d63
Commit Graph

2335 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
cf4cef3d63
[aryion] adjust 'date' to UTC time 2020-04-11 02:08:05 +02:00
Mike Fährmann
a0f4c295c0
add optional 'utcoffset' argument to 'parse_datetime()' 2020-04-11 02:05:00 +02:00
Mike Fährmann
6c531be294
[aryion] fix malformed 'last-modified' headers (#390) 2020-04-10 23:08:52 +02:00
Mike Fährmann
38bc6430d3
[downloader:http] don't overwrite existing '_mtime' fields 2020-04-10 23:08:03 +02:00
Mike Fährmann
dc65f7d8dc
[aryion] use generic download URLs (#390) 
i.e. /g4/data.php?id=…

- get filename & extension from Content-Disposition header
- handle all downloadable file types (docx, swf, etc)
 2020-04-10 22:08:45 +02:00
Mike Fährmann
96b78bcf04
[aryion] include path in default directory format (#390) 2020-04-10 21:58:46 +02:00
Mike Fährmann
406449b0d6
ensure keys for mastodon instances are available during tests 
Calls to config.clear() from other tests are removing the API
credentials set when importing mastodon.py for the first time.
 2020-04-08 21:56:14 +02:00
Mike Fährmann
300264f676
read config files from PyInstaller exe directory (closes #682) 2020-04-08 21:53:50 +02:00
Mike Fährmann
6143050980
[aryion] add gallery and post extractors (#390, #673) 2020-04-08 21:52:51 +02:00
Mike Fährmann
9e7dfc0cfc
[myportfolio] fix extraction of galleries without title 2020-04-08 21:08:05 +02:00
Mike Fährmann
3b50c4f49d
add tests for "Extractors" in oauth.py (#670) 2020-04-07 20:26:12 +02:00
Mike Fährmann
04bd0472de
add tests for Extractor.wait() 2020-04-07 20:24:56 +02:00
Mike Fährmann
88fca0a172
[mastodon] update OAuth credentials for pawoo.net (#665) 2020-04-06 00:50:30 +02:00
Mike Fährmann
4ae8a25567
[mastodon] use 'combine_dict()' to combine extractor info dicts 2020-04-05 21:45:00 +02:00
Mike Fährmann
220c06b86e
[mastodon] handle rate limits 2020-04-05 21:44:00 +02:00
Mike Fährmann
d02f7c1118
improve Extractor.wait() 
- allow 'until' to be a datetime object
- do "time calculations" with UTC timestamps
- set a default 'reason'
 2020-04-05 21:23:05 +02:00
Mike Fährmann
5d7404ab58
[oauth] use the new name for 'DeviantartAPI' (fixes #670) 2020-04-04 20:34:47 +02:00
Mike Fährmann
762c758af4
[hiperdex] fix extraction 2020-04-03 21:25:25 +02:00
Mike Fährmann
f9a590f92b
[deviantart] apply HTTP request limits in more places 
"Request blocked" can also happen on sta.sh and for *any* HTTP
request directed at deviantart.com
 2020-04-03 21:21:59 +02:00
Mike Fährmann
2587296deb
[mastodon] add access tokens for mastodon.social and baraag.net 
(closes #665)
 2020-04-02 22:34:32 +02:00
Mike Fährmann
ff7c0b7eff
[deviantart] handle "Request blocked" errors (#655) 
- add a 2 second wait time between requests to deviantart.com
- catch 403 "Request blocked" errors and wait for 3 minutes until
  retrying
 2020-04-02 22:14:02 +02:00
Mike Fährmann
c874684f05
[deviantart] retrieve *all* download URLs through OAuth API 
'/extended_fetch' as well as Deviation webpages now again contain
Deviation UUIDs needed to grab Deviation info through the OAuth API,
meaning cookies are no longer necessary to grab original files.

The only instance were cookies are still needed are scraps marked as
"mature", since those entries are hidden for public users.

(#655, #657, #660)
 2020-04-02 22:10:33 +02:00
Mike Fährmann
5c27b25a8f
[deviantart] improve sta.sh extraction 
Extract all sta.sh items in a single extractor run.
Don't spawn a new StashExtractor for each individual sta.sh item to
preserve the current requests.Session and its opened TCP connections.
 2020-04-01 03:17:25 +02:00
Mike Fährmann
e2fc4eaa6f
[deviantart] detect stash folders (fixes #659) 2020-04-01 01:59:03 +02:00
Mike Fährmann
c034159701
[piczel] fix extraction for single images 2020-03-31 22:47:23 +02:00
Mike Fährmann
699036ea0c
[weibo] accept status URLs with non-numeric IDs (#664) 2020-03-31 22:46:50 +02:00
Mike Fährmann
a0111edfd2
change Travis badge URL to .com 2020-03-31 22:23:53 +02:00
Mike Fährmann
fe96f99e4b
[hentainexus] reduce line length (flake8) & update test 2020-03-31 22:08:43 +02:00
墨焓
6f81cac8fa
Add metadata to hentainexus: circle, event, title_conventional. (#661) 2020-03-31 21:59:02 +02:00
Mike Fährmann
3ed72f82dc
release version 1.13.3 2020-03-28 22:03:33 +01:00
Mike Fährmann
6f911aeb1c
[deviantart] add error message for cloudFront blocks (#655) 2020-03-28 21:18:04 +01:00
Mike Fährmann
7499d71d02
[simplyhentai] ignore certificate errors in video test 2020-03-28 21:07:30 +01:00
Mike Fährmann
4203dc0bdc
[mangapark] fix metadata extraction 2020-03-28 03:00:26 +01:00
Mike Fährmann
6ecb0a19cf
handle sys.stdin being None when using '-' as input file (#653) 2020-03-25 22:33:39 +01:00
Mike Fährmann
c8db2a87e9
fix create_test_data.py script 2020-03-25 22:06:38 +01:00
Mike Fährmann
1b82d36ab2
[deviantart] handle decode errors for extended_fetch results (#655) 
This isn't going to solve the underlying problem, but it should at
least provide the server response when those errors happen.
 2020-03-24 20:56:41 +01:00
Mike Fährmann
88ebbadc58
remove dashes from subcategory names in supportedsites.rst 2020-03-24 02:49:00 +01:00
Mike Fährmann
09f2271528
[35photo] add 'tag' extractor 2020-03-24 02:49:00 +01:00
Mike Fährmann
77fda8190c
[35photo] simplify/remove tests for the 'genre' extractor 
There is still a nice genre overview page (https://35photo.pro/genre/)
but the individual sub-pages don't list photos anymore
 2020-03-24 02:48:25 +01:00
Mike Fährmann
4bc161ca0f
prevent crash when sys.stdout and co. are None (#653) 2020-03-23 23:38:55 +01:00
Mike Fährmann
d47d0f757c
[travis] allow 'results' and 'snap' tests to fail 2020-03-23 23:06:21 +01:00
Mike Fährmann
ce73796eaa
[travis] add flake8 job 2020-03-23 22:59:01 +01:00
Mike Fährmann
fb846c9ee5
[instagram] reduce line lengths and make flake8 happy 2020-03-23 22:56:43 +01:00
Mike Fährmann
ad2efa8509
[e621] derive from Danbooru extractors (#651) 
- use extractor implementations from 'danbooru'
- use "page": "b[ID]" to paginate over results instead of
  "tags": "id:<[ID]", avoiding infinite loops with certain
  post orders
- bump User-Agent version
 2020-03-22 21:08:45 +01:00
Mike Fährmann
9b39e1cd7e
[e621] fix bug in API rate limiting (#651) 2020-03-22 14:01:23 +01:00
Mike Fährmann
b607d0ad7f
[twitter] fix typo in 'x-twitter-auth-type' header (#625) 2020-03-21 23:11:39 +01:00
Mike Fährmann
9159cb8fb3
remove trailing dots and spaces from directory names (#647) 2020-03-19 21:12:18 +01:00
Mike Fährmann
2c3b9e1450
[nozomi] support multiple images per post (#646) 
This changes the default filename format as well as archive IDs,
since those assumed that each post would only have one image.
 2020-03-19 21:07:31 +01:00
Mike Fährmann
c606d0c854
[instagram] update pattern for user profile URLs 
Allow for query parameters and fragments,
for example https://www.instagram.com/instagram/?hl=en
 2020-03-18 22:24:20 +01:00
Mike Fährmann
2530db3f4d
[mangadex] transform 'date' timestamps to datetime objects 2020-03-18 02:19:19 +01:00