Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-24 03:32:33 +01:00
Code Issues Releases Wiki Activity
3,677 Commits 6 Branches 164 Tags 16 MiB
435e9c5d2e
Commit Graph

136 Commits

Author SHA1 Message Date
Mike Fährmann
9adea93aef
[pixiv] updates to avatar/background extractors (#2495) 
- add 'date' metadata to avatar/background files when available
  and use that in default filenames / archive ids
- remove deprecation warnings as their option names clash with
  subcategory names
 2022-05-04 17:30:54 +02:00
Mike Fährmann
84756982e9
[pixiv] implement 'include' option 
- split 'user' extractor and its 'avatar' and 'background' options into
  separate extractors ('artworks', 'avatar', 'background')
- avatars can now be downloaded with
  https://www.pixiv.net/en/users/ID/avatar
  as URL and will use a proper archive key; similar for backgrounds
- options for the 'user' subcategory must be moved to 'artworks' to have
  the same effect as before
 2022-05-02 09:03:35 +02:00
Mike Fährmann
82eee72b39
[pixiv] update API interface 
- start all endpoints with '/'
- use extractor.wait() for rate limit
- retry with while loop instead of recursion
- in case of error, write entire response to debug log
 2022-05-02 09:03:34 +02:00
Mike Fährmann
9e6ff42a9d
[pixiv] implement 'background' option (#623, #1124, #2495) 2022-04-21 13:53:02 +02:00
Mike Fährmann
4bec34fc94
[pixiv] allow setting a date range for search results (#2133) 
with the 'scd' and 'ecd' query parameters
 2021-12-23 23:03:39 +01:00
Mike Fährmann
e33125ad39
[pixiv] add 'sketch' extractor (#1497) 2021-10-13 00:02:11 +02:00
Mike Fährmann
eed6ef3de0
[pixiv] fix pixivision title extraction 2021-09-02 22:34:59 +02:00
Mike Fährmann
bd08ee2859
remove most 'yield Message.Version' statements 
only leave them in oauth.py as noop results
 2021-08-16 03:10:48 +02:00
Mike Fährmann
8ecca3af58
[pixiv] add extractor for 'pixivision' articles (#1672) 2021-07-07 16:27:16 +02:00
Mike Fährmann
7273cf8536
[pixiv] support fetching privately followed users (fixes #1628) 2021-06-16 19:56:09 +02:00
thatfuckingbird
e6811c7450
[pixiv] implement 'max-posts' option (#1558) 
* implement max-rank for pixiv

* rename to max-posts and make more generic
 2021-05-24 17:49:46 +02:00
Mike Fährmann
5eeaaee01d
[pixiv] add 'metadata' option (#1551) 2021-05-14 20:30:28 +02:00
Mike Fährmann
36ed1efcfb
[pixiv] rename "noop" value for 'tags' option to "original" 
(#1507)
 2021-05-07 20:41:54 +02:00
Mike Fährmann
fa519f9202
[pixiv] change 'translated-tags' option (#1507) 
- rename to 'tags'
- use string-values: "japanese", "translated", "noop"
- remove duplicate entries for "translated" tags
 2021-04-29 19:30:43 +02:00
Mike Fährmann
221015e586
[downloader:http] disable filename extension changes for ugoira 
(#1507)
 2021-04-27 01:29:09 +02:00
thatfuckingbird
141ca4ac0a
[pixiv] also save untranslated tags when translated-tags is enabled (#1501) 2021-04-23 23:02:41 +02:00
beesdotjson
5ad615f0db
fix PixivFavoriteExtractor regex (#1405) 
* fix PixivFavoriteExtractor regex

* do not use lookbehind
 2021-03-25 14:59:33 +01:00
Mike Fährmann
7440d1f112
[pixiv] add 'translated-tags' option (closes #1354) 
(a lot more straight forward than I thought ...)
 2021-03-05 17:18:51 +01:00
Mike Fährmann
8974f0361c
[pixiv] update (#1304) 
- remove login with username & password
- require a refresh token
- add 'oauth:pixiv' functionality

See also:
- https://github.com/upbit/pixivpy/issues/158
- https://gist.github.com/ZipFile/c9ebedb224406f4f11845ab700124362
 2021-02-12 18:07:16 +01:00
Mike Fährmann
193dca2ce1
update extractor test results 2021-01-21 21:35:42 +01:00
Mike Fährmann
c008cb5100
[pixiv] add 'related' option (#1237) 2021-01-17 22:48:32 +01:00
Mike Fährmann
3bd08acc8f
[pixiv] output debug message on failed login attempt 
(#1192)
 2020-12-22 14:59:31 +01:00
Mike Fährmann
b58e605dc7
raise error when required username or password are missing 
do not try to login as 'None' (#1192)
 2020-12-22 14:40:18 +01:00
Mike Fährmann
968d3e8465
remove '&' from URL patterns 
'/?&#' -> '/?#' and '?&#' -> '?#'

According to https://www.ietf.org/rfc/rfc3986.txt, URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
 2020-10-22 23:31:25 +02:00
Mike Fährmann
844793847c
update extractor test results 2020-10-11 18:15:41 +02:00
Mike Fährmann
844502cad5
update extractor test results 2020-10-03 19:24:19 +02:00
Mike Fährmann
a45fbc38ea
[pixiv] implement 'avatar' option (#595, #623) 2020-03-09 21:18:16 +01:00
Mike Fährmann
ce26070231
[pixiv] reduce calls to '/user/detail' 2020-02-09 13:54:58 +01:00
Mike Fährmann
b7eb6cecbb
[pixiv] handle tags at the end of new bookmark URLs 2020-02-06 23:42:13 +01:00
Mike Fährmann
40fe062851
[pixiv] fix user id for bookmarks API calls (closes #596) 2020-02-01 01:48:46 +01:00
Mike Fährmann
91aaaf1a9e
[pixiv] add 'rating' metadata field (#595) 
A human-friendlier representation of 'x_restrict'
 2020-02-01 01:36:06 +01:00
Mike Fährmann
71851a6241
[pixiv] update URLs of followed users to the new format 2020-01-30 15:17:42 +01:00
Mike Fährmann
b9cbf932b4
[pixiv] update URL patterns (fixes #568) 
Pixiv now uses new URLs for
- user profiles and illustration listings:
  - https://www.pixiv.net/en/users/<ID>
  - https://www.pixiv.net/en/users/<ID>/artworks
- bookmarks:
  - https://www.pixiv.net/en/users/<ID>/bookmarks/artworks
 2020-01-10 13:38:09 +01:00
Mike Fährmann
ce54b8c04c
let extractors opt-out of cookie option usage 
useful to avoid sending unnecessary cookies when all authentication
is done through OAuth tokens
 2020-01-01 21:12:37 +01:00
Mike Fährmann
1f2a69f3c5
add '_extractor' information to redirect results 2019-12-29 23:37:34 +01:00
Mike Fährmann
a27f43dad1
[pixiv] wait and retry after rate limit error (closes #535) 2019-12-28 22:06:58 +01:00
Mike Fährmann
c0f391a4e2
[pixiv] support listing followed users (#515) 2019-12-26 23:42:42 +01:00
Mike Fährmann
1f209da4c0
[pixiv] match new search URLs (closes #507) 2019-12-06 21:38:38 +01:00
Mike Fährmann
4409d00141
embed error messages in StopExtraction exceptions 2019-10-28 16:39:49 +01:00
Mike Fährmann
22bac14452
[pixiv] match '/artworks/' URLs 2019-09-24 21:53:14 +02:00
Mike Fährmann
f9285f99e6
[pixiv] fix authentication 2019-09-02 22:38:56 +02:00
Mike Fährmann
32447d0d24
[pixiv] simplify default filename format 
(#366)
 2019-08-15 13:32:47 +02:00
Mike Fährmann
ad3ac02fbc
[pixiv] update metadata entries (#366) 
- change 'num' to a simple enumerating integer
- change default filename format
- provide content of the old 'num' field as 'suffix'
- add 'filename' for ugoira
 2019-08-05 22:41:56 +02:00
Mike Fährmann
64b2935d8e
[pixiv] provide 'filename' and change default filename format 
to '{filename}.{extension}' (closes #366)
 2019-08-02 22:35:10 +02:00
Mike Fährmann
fdec59f8e2
replace extractor.request() 'expect' argument 
with
- 'fatal': allow 4xx status codes
- 'notfound': raise NotFoundError on 404
 2019-07-05 00:42:16 +02:00
Mike Fährmann
b0e85a42e3
apply workaround from 4736912 in parse_datetime() itself 2019-05-09 21:53:17 +02:00
Mike Fährmann
4736912d4e
[pixiv] work around strptime limitations in Python < 3.7 
"%z" doesn't allow a colon separator in older Python versions:
    - "+0900" is OK
    - "+09:00" raises an exception
 2019-05-08 18:08:03 +02:00
Mike Fährmann
80fdb11508
[pixiv] add 'date' metadata field (closes #248) 2019-05-08 15:43:59 +02:00
Mike Fährmann
a2af2d2965
adjust cache maxage values 2019-03-14 22:21:49 +01:00
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00