1
0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-24 03:32:33 +01:00
Commit Graph

41 Commits

Author SHA1 Message Date
Mike Fährmann
0b4be57a10
[sankaku] fix error when no tags available (closes #259)
[ci skip]
2019-05-14 23:40:07 +02:00
Mike Fährmann
aa8e366b90
[luscious] fix tag extraction 2019-05-14 17:35:52 +02:00
Mike Fährmann
a2af2d2965
adjust cache maxage values 2019-03-14 22:21:49 +01:00
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00
Mike Fährmann
6284731107
simplify extractor constants
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
2019-02-08 13:45:40 +01:00
Mike Fährmann
4d656a81ca
replace SharedConfigExtractor class with a Mixin 2019-02-04 13:46:02 +01:00
Mike Fährmann
dd358b4564
improve cookie handling during logins 2019-01-30 17:09:32 +01:00
Mike Fährmann
2d2953a5bf
add 'text.parse_float()' + cleanup in text.py 2019-01-29 16:46:21 +01:00
Mike Fährmann
78b5f29a00
[sankaku] unescape tags 2019-01-20 16:18:13 +01:00
Mike Fährmann
2be4c9ffe3
[sankaku] small code improvements 2018-09-16 21:01:28 +02:00
Mike Fährmann
99137f1bee
[sankaku] send login info as formdata
Previously they were erroneously send as URL parameters.
2018-09-14 17:54:15 +02:00
Mike Fährmann
fa64c38d5b
[sankaku] fix pagination for user favorites (#106) 2018-09-14 17:51:46 +02:00
Mike Fährmann
b164231bca
[sankaku] increase default values for 'wait-min/-max' 2018-08-03 17:06:51 +02:00
Mike Fährmann
269dc2bbd5
[sankaku] add 'tags' option (#94) 2018-07-14 09:53:01 +02:00
Mike Fährmann
cc36f88586
rename safe_int to parse_int; move parse_* to text module 2018-04-20 14:53:21 +02:00
Mike Fährmann
5008e105ee
update archive IDs
... to behave in a more straightforward way when dealing with
bookmarks/favourites/etc.

specific IDs are now grouped by their owner, album-id, ... to
allow for duplicates when it would be expected.
2018-03-01 18:20:50 +01:00
Mike Fährmann
829ddf4ac1
[sankaku] general improvements
- simplify regex
- unquote search tags
- increase default wait-time between HTTP requests
  - downloading several hundreds of images always resulted
    in '429 Too Many Requests' eventually
- circumvent paging restrictions for unauthenticated users by only
  using the 'next' parameter
  - setting 'page' to a constant, low value (or simply omitting it)
    does the trick
2018-02-27 16:51:14 +01:00
Jad
49463f76bb support multi-page URL (#79)
* support multi-page URL

* fix

* all done.

* fix, again
2018-02-26 11:13:49 +01:00
Mike Fährmann
34873dbd90
set 'archive_fmt' values
These are going to be used to create an unique id for each image.
2018-02-01 15:30:49 +01:00
Mike Fährmann
e420a28bbc
fix cookie tests 2018-01-09 21:43:52 +01:00
Mike Fährmann
b33efc99a4
[idolcomplex] add support for idol.sankakucomplex.com 2018-01-09 17:54:37 +01:00
Mike Fährmann
19a6ae57b2
[sankaku] add pool extractor 2017-12-12 19:45:10 +01:00
Mike Fährmann
e52f0cc1ed
[sankaku] add post extractor 2017-12-12 18:20:15 +01:00
Mike Fährmann
595593a35e
[sankaku] rewrite
- better code structure and extensibility
- better metadata
2017-12-12 18:09:45 +01:00
Mike Fährmann
a3924d2072
[sankaku] fix swf extraction (closes #52) 2017-12-07 15:45:43 +01:00
Mike Fährmann
e6814aebe2
add 'extractor.*.user-agent' config option 2017-11-15 14:01:33 +01:00
Mike Fährmann
5fa42336a2
[sankaku] add warning for unauthenticated users
also improve URL pattern and add missing options to default config file
2017-10-16 21:21:08 +02:00
Mike Fährmann
6af921a952
[sankaku] rewrite/improve (fixes #44)
- add wait-time between HTTP requests similar to exhentai
- add 'wait-min' and 'wait-max' options
- increase retry-count for HTTP requests to 10
- implement user authentication (non-authenticated users can only view
  images up to page 25)
- implement 'skip()' functionality (only works up to page 50)
- implement image-retrieval for pages >= 51
- fix issue with multiple tags
2017-10-14 23:01:33 +02:00
Mike Fährmann
6f30cf4c64
change keyword names to valid Python identifiers
This commit mostly replaces all minus-signs ('-') in keyword names with
underscores ('_') to allow them to be used in filter-expressions. For
example 'gallery-id' got renamed to 'gallery_id'.

(It is theoretically possible to access any variable, regardless of its
name, with 'locals()["NAME"]', but that seems a bit too convoluted if
just 'NAME' could be enough)
2017-09-10 22:20:47 +02:00
Mike Fährmann
83cf1e1d6d
[sankaku] unescape image URLs 2017-09-04 17:41:11 +02:00
Mike Fährmann
58e95a7487
share extractor and downloader sessions
There was never any "good" reason for the strict separation
between extractors and downloaders. This change allows for
reduced resource usage (probably unnoticeable) and less lines
of code at the "cost" of tighter coupling.
2017-06-30 19:38:14 +02:00
Mike Fährmann
94e10f249a
code adjustments according to pep8 nr2 2017-02-01 00:53:19 +01:00
Mike Fährmann
56d810c896
update keyword hashes for tests 2016-09-25 17:28:46 +02:00
Mike Fährmann
19c2d4ff6f
remove explicit (sub)category keywords 2016-09-25 14:22:07 +02:00
Mike Fährmann
49a05c32ed
add missing tests 2016-09-19 16:15:27 +02:00
Mike Fährmann
d7e168799d
consistent extractor naming scheme + docstrings 2016-09-12 10:34:31 +02:00
Mike Fährmann
4d56b76aa8
update all other extractors 2015-11-21 04:26:30 +01:00
Mike Fährmann
c2f0720184
code cleanup to use nameext_from_url 2015-11-16 17:32:26 +01:00
Mike Fährmann
3510faaed5
[sankaku] always use correct file-url 2015-11-10 00:55:01 +01:00
Mike Fährmann
60833abcc6
[sankaku] re-enable extractor 2015-11-09 02:29:33 +01:00
Mike Fährmann
deef91eddc initial commit 2014-10-12 21:56:44 +02:00