Mike Fährmann
ebc70e87ce
[e621] update to new interface / API endpoints ( closes #635 )
2020-03-06 21:12:58 +01:00
Mike Fährmann
72122eb9b3
release version 1.13.1
2020-03-01 21:17:29 +01:00
Mike Fährmann
ce5e2a58fe
[imgbb] update test results
...
Image server domain changed from
https://image.ibb.co/ to https://i.ibb.co/
2020-03-01 20:38:25 +01:00
Mike Fährmann
1d4a369ea2
update extractor test results
2020-02-27 22:15:40 +01:00
Mike Fährmann
ec85bf90de
use context managers in cache.py & add tests
2020-02-25 23:08:47 +01:00
Mike Fährmann
4e361b3008
add tests for specific datetime values
2020-02-23 16:48:30 +01:00
Mike Fährmann
90e4c645ba
[formatter] allow multiple "special" format specifiers ( #595 )
...
It is now, for example, possible to specify multiple replacement
operations per format replacement field: {name:Ra/b/Rc/d/}
2020-02-16 21:47:08 +01:00
Mike Fährmann
219c4cc78c
[formatter] allow for numeric list and string indices
2020-02-15 22:46:22 +01:00
Mike Fährmann
7d1da614d9
[formatter] implement field name alternatives ( #525 )
...
The format string '{a|b|c}' will now try to use the value from 'a' and
fall back to 'b' and 'c' if accessing a field raises an exception or
if its value is None.
2020-02-15 17:58:21 +01:00
Mike Fährmann
c7cf9dd111
[furaffinity] support classic layout ( #284 )
2020-02-12 21:39:43 +01:00
Mike Fährmann
40fe062851
[pixiv] fix user id for bookmarks API calls ( closes #596 )
2020-02-01 01:48:46 +01:00
Mike Fährmann
2852691d78
[paheal] replace test URL
...
searching for 'k-on' doesn't yield any results anymore
2020-01-27 22:19:41 +01:00
Mike Fährmann
2a9be48511
improve util.load/save_cookiestxt() and add tests
...
- take a file object as argument instead of an filename
- accept whitespace before comments (" # comment")
- map expiration "0" to None and not the number 0
2020-01-25 23:02:15 +01:00
Mike Fährmann
b3b5754f2d
update test_cookies.py
2020-01-21 22:46:58 +01:00
Mike Fährmann
174117f827
allow multiple hashes for content tests
2020-01-18 00:19:35 +01:00
Mike Fährmann
60a43f0264
fix downloader tests
2020-01-14 11:51:06 +01:00
Mike Fährmann
e89413da22
update test results
2020-01-12 21:41:06 +01:00
Mike Fährmann
5cac79c3d9
[erolord] remove extractor
2020-01-10 14:40:30 +01:00
Mike Fährmann
988cc2ec23
[mangadex] change domain to mangadex.cc ( closes #559 )
2020-01-05 21:21:40 +01:00
Mike Fährmann
87c8b89ddd
[postprocessor:metadata] add 'directory' option ( #520 )
2020-01-02 20:58:10 +01:00
Mike Fährmann
82f7f4172a
update test results
2020-01-01 16:05:38 +01:00
Mike Fährmann
d0920e84e9
update test results
2019-12-19 18:33:48 +01:00
Mike Fährmann
9e63804347
[patreon] make retrieving user info nonfatal ( #508 )
...
… and fall back to the included data if an error occurs.
2019-12-12 00:31:35 +01:00
Mike Fährmann
15f9bb3d14
add option to disable pyOpenSSL usage ( #508 )
...
(pyOpenSSL is now disabled by default)
2019-12-08 21:21:00 +01:00
Mike Fährmann
50deab5265
[deviantart] fix URL generation from /extended_fetch results
...
(closes #505 )
2019-12-07 22:07:55 +01:00
Mike Fährmann
004812258d
[hentaifox] fix extraction
2019-12-02 22:21:45 +01:00
Mike Fährmann
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
2019-11-30 17:26:17 +01:00
Mike Fährmann
b5c964332b
improve config.py test coverage
2019-11-25 17:20:00 +01:00
Mike Fährmann
f5604492c3
update interface of config functions
2019-11-24 00:42:28 +01:00
Mike Fährmann
3fc1e12949
[postprocessor:metadata] filter private entries
...
i.e. keys starting with an underscore
2019-11-21 16:58:44 +01:00
Mike Fährmann
978cb03f81
update misc test results
...
- Livedoor now uses https:// for its image URLs
- Instagram image URLs got simplified
2019-11-20 21:45:48 +01:00
Mike Fährmann
bbbeff4c41
[downloader.http] implement file-specific HTTP headers
2019-11-19 23:50:54 +01:00
Mike Fährmann
3ece3976ae
[newgrounds] implement login support ( #394 )
2019-11-16 23:45:32 +01:00
Mike Fährmann
abfcb356fc
[flickr] support 3k, 4k, 5k, and 6k photo sizes ( closes #472 )
2019-11-10 17:52:51 +01:00
Mike Fährmann
da6789b2b0
disable unique archive id checks for some tests
...
- same image twice in a livedoor blog post
- unreliable results for related pinterest items
2019-11-10 17:04:51 +01:00
Mike Fährmann
ba083b30b2
fix snap build
...
… hopefully
2019-11-08 21:44:12 +01:00
Mike Fährmann
94a94f3b86
miscellaneous stuff
2019-11-08 20:58:53 +01:00
Mike Fährmann
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
2019-11-03 21:45:45 +01:00
Mike Fährmann
2a3bd4e3c7
rename extractor classes starting with a digit
2019-11-02 20:42:09 +01:00
Mike Fährmann
64786363be
[4chan] simplify
...
- remove 'chan.py'
- slight adjustments to directory and filenames
2019-11-02 20:11:21 +01:00
Mike Fährmann
557e2c018b
[8chan] remove module
2019-11-02 20:06:47 +01:00
Mike Fährmann
322c2e7ed4
renaming variables
...
mostly 'keyword(s)' to 'kwdict'
2019-10-29 15:46:35 +01:00
Mike Fährmann
87a87bff7e
[simplyhentai] fix image URLs
2019-10-28 21:11:06 +01:00
Mike Fährmann
d5e3910270
adjust 'util.raises()'
2019-10-28 15:06:17 +01:00
Mike Fährmann
b23c822b23
[luscious] use GraphQL
2019-10-22 21:17:08 +02:00
Mike Fährmann
1693d97bd3
update extractor class hierarchies
...
- let the GalleryExtractor class inherit directly from Extractor
- make ChapterExtractor a subclass of GalleryExtractor
- change enumeration field names of GalleryExtractors to 'num'
2019-10-16 18:15:29 +02:00
Mike Fährmann
7ebd984e8d
[imgur] print error message if no JSON data is found ( #446 )
2019-10-16 17:45:14 +02:00
Mike Fährmann
de4e2029d1
[nsfwalbum] update test album
...
the old one is no longer available
2019-09-28 20:48:15 +02:00
Mike Fährmann
913460240d
[reddit] fix 'extractor.blacklist()' arguments
...
The second argument must support 'append()'.
2019-09-24 23:01:12 +02:00
Mike Fährmann
1848788970
update test results etc
2019-09-08 11:33:35 +02:00
Mike Fährmann
d5fbb2d9de
[tumblr] ignore audio links from Spotify etc.
2019-09-07 18:18:12 +02:00
Mike Fährmann
c6c5cb1898
improve 'deviantart.quality' description
2019-08-30 18:41:18 +02:00
Mike Fährmann
c9b97dbca3
extend post processor tests
2019-08-26 23:10:12 +02:00
Mike Fährmann
49f6d7176d
[deviantart] restore filenames ( #392 )
...
<title>_by_<user>_<id> --> <title>_by_<user>-<id>
2019-08-23 22:02:03 +02:00
Mike Fährmann
e528f3cb77
adjust postprocessor test results
...
see 2495b99
2019-08-21 23:54:08 +02:00
Mike Fährmann
cb7f149974
fix mtime datetime test
...
datetime.timestamp() uses local time for a naive datetime object
2019-08-14 21:57:42 +02:00
Mike Fährmann
23251356cb
require 'extension' data for each URL ( #382 )
2019-08-14 20:03:03 +02:00
Mike Fährmann
dd72ae7164
add postprocessor tests
2019-08-12 21:46:30 +02:00
Mike Fährmann
0bb873757a
update PathFormat class
...
- change 'has_extension' from a simple flag/bool to a field that
contains the original filename extension
- rename 'keywords' to 'kwdict' and some other stuff as well
- inline 'adjust_path()'
- put enumeration index before filename extension (#306 )
2019-08-12 21:40:37 +02:00
Mike Fährmann
748e37554c
update .travis.yml
...
- install pyOpenSSL before running tests
- simplify snap tests
2019-08-11 16:03:19 +02:00
Mike Fährmann
b7fb93e2b2
[downloader:http] add 'adjust-extensions' option
2019-08-08 16:54:20 +02:00
Mike Fährmann
eb7da159e2
[imagebam] update URL test results
...
Image URLs are now using https://, but the website itself is still
served as http://.
2019-08-07 21:47:44 +02:00
Mike Fährmann
fa60109e97
[exhentai] don't use e-hentai.org for exhentai URLs
2019-08-02 21:10:09 +02:00
Mike Fährmann
4a0c98bfc9
miscellaneous fixes and adjustments
2019-08-01 22:09:43 +02:00
Mike Fährmann
40637556fa
[ngomik] fix extraction
2019-07-28 10:53:46 +02:00
Mike Fährmann
d9d44ad953
[tsumino] update test results
2019-07-24 21:17:23 +02:00
Mike Fährmann
b1bea8aaeb
add 'restrict-filenames' option ( #348 )
2019-07-23 17:41:24 +02:00
Mike Fährmann
b3851e01d9
release version 1.9.0
2019-07-19 21:55:25 +02:00
Mike Fährmann
12da6bd0c9
[simplyhentai] fix/improve extraction
2019-07-06 20:25:53 +02:00
Mike Fährmann
b89f0d8d3c
update extractor result tests
2019-07-01 20:02:47 +02:00
Mike Fährmann
40da44b17f
Merge branch 'v1.9.0'
2019-06-29 15:39:52 +02:00
Mike Fährmann
7a99e85943
[kissmanga] fix download URLs and file extensions
...
The current Blogspot image URLs hosted on Kissmanga end with an
"invalid" query parameter (/000.png&upx=...), which doesn't get
recognized by 'spliturl()' and 'parseurl()' as such and gets therefore
included in the 'extension' field from 'text.nameext_from_url()'.
2019-06-28 20:34:43 +02:00
Mike Fährmann
a9c89085fb
[instagram] implement login support ( #195 )
2019-06-26 23:58:47 +02:00
Mike Fährmann
b1985d6579
test default format strings during extractor result tests
...
A missing value or an invalid "syntax" for a format replacement field
will raise an exception.
2019-06-25 18:12:32 +02:00
Mike Fährmann
95b1e4c3c0
implement R<old>/<new>/ format option ( #318 )
2019-06-23 22:45:44 +02:00
Mike Fährmann
70713f0f28
fix extractor result tests
2019-06-20 18:12:36 +02:00
Mike Fährmann
ee4d7c3d89
update downloader.find() and related code
...
Instead of replacing 'https' with 'http' for every URL in
'get_downloader()', this now only happens once during downloader
initialization. Also unit tests.
2019-06-20 16:59:44 +02:00
Mike Fährmann
179d112083
[downloader] overhaul http and text modules
...
Get rid of the modular structure and simplify/specialize those modules.
2019-06-19 22:56:11 +02:00
Mike Fährmann
a77340c647
[keenspot] fix extraction for "TwoKinds"
2019-06-17 19:49:39 +02:00
Mike Fährmann
b171befa87
implement 'parse_unicode_escapes()'
2019-06-16 21:47:24 +02:00
Mike Fährmann
e05a96db5e
[deviantart] rename 'stash' to 'extra' ( #302 )
...
'stash' is already used as a name for the StashExtractor and therefore
expected to be a dictionary.
2019-06-10 21:05:25 +02:00
Mike Fährmann
7c6cb908f9
[xhamster] update test results
2019-06-07 16:28:49 +02:00
Mike Fährmann
62335b9015
[paheal] adjust test results
2019-06-05 11:42:01 +02:00
Mike Fährmann
6a34f4b0c1
skip tests on read timeouts; print list of skipped tests
2019-06-01 20:47:31 +02:00
Mike Fährmann
d33f5a7423
[wallhaven] rewrite
...
- use API
- remove login support, add 'api-key' option
- remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric
IDs that can't be translated to the new ID system
- support direct links to wallpapers
2019-05-31 14:53:02 +02:00
Mike Fährmann
5499934ae2
[ngomik] fix extraction
2019-05-30 20:18:36 +02:00
Mike Fährmann
2b1999476e
implement 'text.rextract()'
2019-05-28 21:03:41 +02:00
Mike Fährmann
e30ada162d
fix cookie tests
...
update _get_extractor():
- always return an Extractor instance with a _login_impl() method
- use Extractor.from_url()
2019-05-26 20:22:04 +02:00
Mike Fährmann
2316e0ed3d
fix strptime workaround from b0e85a4
...
Don't return a modified version of 'date_time' if strptime fails.
2019-05-25 23:22:26 +02:00
Mike Fährmann
6764847349
fix cookie tests
...
'cookies' is a CookieJar, not a dict,
and removing the call to '.keys()' doesn't have the same effect
2019-05-14 22:32:40 +02:00
Mike Fährmann
a5b060765d
improve code in tests
...
- use 'assertRaises' as context manager
- remove calls to .keys()
2019-05-13 11:48:20 +02:00
Mike Fährmann
b0e85a42e3
apply workaround from 4736912 in parse_datetime() itself
2019-05-09 21:53:17 +02:00
Mike Fährmann
4736912d4e
[pixiv] work around strptime limitations in Python < 3.7
...
"%z" doesn't allow a colon separator in older Python versions:
- "+0900" is OK
- "+09:00" raises an exception
2019-05-08 18:08:03 +02:00
Mike Fährmann
d09864b581
implement text.parse_datetime()
2019-05-08 15:43:59 +02:00
Mike Fährmann
5582b06ae4
fix tests with 'urllist' messages
2019-04-30 16:31:48 +02:00
Mike Fährmann
5018781898
allow type tests by name
2019-04-29 17:27:59 +02:00
Mike Fährmann
6264a46212
use 'utcfromtimestamp()'
...
'fromtimestamp()' converts its results to the local timezone and causes
problems when running tests on a different machine.
2019-04-21 16:22:53 +02:00
Mike Fährmann
d670de0344
implement 'text.parse_timestamp()'
2019-04-21 15:28:27 +02:00
Mike Fährmann
21a7e395a7
implement convenience wrapper for text.extract functionality
2019-04-19 22:30:11 +02:00
Mike Fährmann
e25ebc4bff
don't disable certificate checks anymore
...
Executables generated with PyInstaller auto-include the root certificate
file and certificate checks now work out-of-the-box.
2019-04-17 13:27:19 +02:00
