Mike Fährmann
4a0c98bfc9
miscellaneous fixes and adjustments
2019-08-01 22:09:43 +02:00
Mike Fährmann
40637556fa
[ngomik] fix extraction
2019-07-28 10:53:46 +02:00
Mike Fährmann
7a14aaed7d
[luscious] fix extraction
2019-05-17 10:48:47 +02:00
Mike Fährmann
aa8e366b90
[luscious] fix tag extraction
2019-05-14 17:35:52 +02:00
Mike Fährmann
f2cf1c1d73
use 'text.extract_from()' in a few places
2019-04-21 15:19:20 +02:00
Mike Fährmann
e25ebc4bff
don't disable certificate checks anymore
...
Executables generated with PyInstaller auto-include the root certificate
file and certificate checks now work out-of-the-box.
2019-04-17 13:27:19 +02:00
Mike Fährmann
2ff043edfa
[yaplog] add user- and post-extractors ( #190 )
2019-04-04 17:56:56 +02:00
Mike Fährmann
00d604cafb
[luscious] fix SearchExtractor URL-pattern
2019-03-29 15:58:08 +01:00
Mike Fährmann
1384ebf907
[luscious] fix metadata extraction
...
- remove 'artist', 'language', and 'lang' fields
- replace 'section' with 'genre'
- provide 'tags' as list
- use GalleryExtractor as base class
2019-03-29 13:06:02 +01:00
Mike Fährmann
d0f88c35be
[komikcast] fix extraction
2019-03-18 11:12:19 +01:00
Mike Fährmann
a2af2d2965
adjust cache maxage values
2019-03-14 22:21:49 +01:00
Mike Fährmann
e687a6095e
[luscious] raise exception if album is not available
2019-02-19 13:30:39 +01:00
Mike Fährmann
61741d7333
provide type information for Queue messages
...
Child extractors are now directly constructed with Extractor.from_url()
if the extractor class is known beforehand, instead of using
extractor.find() and searching through all possible extractor classes.
2019-02-12 21:32:32 +01:00
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor
2019-02-11 13:31:10 +01:00
Mike Fährmann
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
2019-02-08 13:45:40 +01:00
Mike Fährmann
00dc37ccbf
replace AsynchronousMixin Extractor with a Mixin
2019-02-04 14:21:19 +01:00
Mike Fährmann
dd358b4564
improve cookie handling during logins
2019-01-30 17:09:32 +01:00
Mike Fährmann
0c32dc5858
[hentaifox] add extractor for search results ( #160 )
2019-01-28 22:38:32 +01:00
Mike Fährmann
e4171d6baf
[luscious] add login capabilities ( closes #159 )
2019-01-28 17:14:15 +01:00
Mike Fährmann
c9ef5ed364
[luscious] ensure URLs have a scheme
2018-12-21 17:56:51 +01:00
Mike Fährmann
a4263fb253
[luscious] add extractor for search results ( closes #127 )
2018-11-25 18:57:51 +01:00
Mike Fährmann
e1d306cc48
update unit test results
2018-10-13 16:54:30 +02:00
Mike Fährmann
38d4f43cc0
[komikcast] skip ads
2018-08-14 11:17:59 +02:00
Mike Fährmann
df7e18399e
[luscious] fix image order
2018-04-17 17:32:21 +02:00
Mike Fährmann
759ba26fb0
[luscious] proper image order for picture albums
...
... and (try) to start with the first image instead of somewhere
in the middle of an album.
2018-04-05 18:12:01 +02:00
Mike Fährmann
557cb94f81
[deviantart] use proper exponential backoff on API errors
...
... and use separate API credentials for unit tests.
2018-03-15 16:01:42 +01:00
Mike Fährmann
3cec533c28
Merge branch 'archive'
2018-02-12 18:07:58 +01:00
Mike Fährmann
34873dbd90
set 'archive_fmt' values
...
These are going to be used to create an unique id for each image.
2018-02-01 15:30:49 +01:00
Mike Fährmann
a34cebc253
[luscious] jump to first image if cover does not link to it
2018-01-30 22:39:01 +01:00
Mike Fährmann
263741d243
[luscious] update URL pattern ( closes #55 )
2017-12-14 14:15:01 +01:00
Mike Fährmann
31cd5b1c1d
[luscious] detect high-load responses
2017-09-12 15:46:21 +02:00
Mike Fährmann
6f30cf4c64
change keyword names to valid Python identifiers
...
This commit mostly replaces all minus-signs ('-') in keyword names with
underscores ('_') to allow them to be used in filter-expressions. For
example 'gallery-id' got renamed to 'gallery_id'.
(It is theoretically possible to access any variable, regardless of its
name, with 'locals()["NAME"]', but that seems a bit too convoluted if
just 'NAME' could be enough)
2017-09-10 22:20:47 +02:00
Mike Fährmann
f98e3e8002
[luscious] fix tag extraction
2017-09-01 16:29:52 +02:00
Mike Fährmann
7e936e9c06
[luscious] simplify and remove dead code
2017-08-08 19:26:13 +02:00
rachmadani haryono
dcd573806e
chg: dev: fix error ( #32 )
...
* fix: dev: error
* fix: dev: AttributeError when getting artist
* fix: dev: typo on luscious parser
2017-08-04 15:01:10 +02:00
Mike Fährmann
d443822fdb
[luacious] get correct image URLs ( fixes #33 )
...
Instead of using thumbnail URLs and modifying them the extractor now
goes through every single image-page and gets its download URL from
there.
2017-08-02 19:58:13 +02:00
Mike Fährmann
841fd50242
move code into util.py
2017-03-28 13:12:44 +02:00
Mike Fährmann
94e10f249a
code adjustments according to pep8 nr2
2017-02-01 00:53:19 +01:00
Mike Fährmann
efaba5f398
[luscious] support new album URLs + small fixes
2017-01-03 15:03:36 +01:00
Mike Fährmann
79ea457e24
[luscious] use new 'html' attribute in API responses
2017-01-03 14:02:52 +01:00
Mike Fährmann
effdf9c3be
[luscious] adapt to API changes
2016-12-13 12:53:59 +01:00
Mike Fährmann
0211ec4114
update some tests
2016-12-08 00:24:23 +01:00
Mike Fährmann
91dff9827f
[luscious] always provide URLs to the original images
2016-11-29 17:17:35 +01:00
Mike Fährmann
37ea66098a
add tests
2016-11-05 13:51:52 +01:00
Mike Fährmann
a0f56b2c54
update tests
2016-10-21 20:52:19 +02:00
Mike Fährmann
56d810c896
update keyword hashes for tests
2016-09-25 17:28:46 +02:00
Mike Fährmann
19c2d4ff6f
remove explicit (sub)category keywords
2016-09-25 14:22:07 +02:00
Mike Fährmann
d7e168799d
consistent extractor naming scheme + docstrings
2016-09-12 10:34:31 +02:00
Mike Fährmann
db1738d451
[luscious] add extractor
2016-08-01 15:36:56 +02:00
