Mike Fährmann
9ae58a6b3e
[exhentai] update image limit checks
...
- adjust cost of original images
- delay limit initialization until gallery and first image page have
been requested and all cookies are available
2019-09-07 13:29:01 +02:00
Mike Fährmann
6fe9a134bf
[lineblog] add blog and post extractors ( closes #404 )
2019-09-06 22:16:42 +02:00
Mike Fährmann
4e8a548a61
[livedoor] update metadata extraction
2019-09-06 21:44:25 +02:00
Mike Fährmann
f9285f99e6
[pixiv] fix authentication
2019-09-02 22:38:56 +02:00
Mike Fährmann
6f3df3999a
[fuskator] add gallery and search extractor ( closes #407 )
2019-09-02 21:20:02 +02:00
Mike Fährmann
bc0ca66c99
[twitter] small improvements
...
- handle reply tweets (#403 )
- unset cookies in Tweet extractor to "force" the legacy interface
2019-09-01 17:37:48 +02:00
Mike Fährmann
682105b8ee
prevent crash when loading unavailable downloader ( #405 )
2019-08-31 21:58:33 +02:00
Mike Fährmann
5fcebb69c2
[postprocessor:ugoira] improve error messages ( #406 )
2019-08-31 21:55:42 +02:00
Mike Fährmann
f02a768b5c
[danbooru] add 'ugoira' option ( #406 )
...
to choose between ZIP archives or converted video files
for Ugoira posts
2019-08-31 21:51:52 +02:00
Mike Fährmann
9646ccb320
release version 1.10.3
2019-08-30 19:41:16 +02:00
Mike Fährmann
dedea3b4db
[deviantart] fix journal creation ( #400 )
2019-08-30 18:50:04 +02:00
Mike Fährmann
c6c5cb1898
improve 'deviantart.quality' description
2019-08-30 18:41:18 +02:00
Mike Fährmann
8eaae58045
[downloader:http] change log message level to 'debug'
2019-08-29 23:05:47 +02:00
Mike Fährmann
efb64ad031
[deviantart] generate filenames ( #392 , #400 )
2019-08-29 10:09:21 +02:00
Mike Fährmann
0ce98169b8
improve path generation
...
- fix 'abspath()' results for Python <3.7 (closes #402 )
- 'abspath()' in Python 3.7+ removes trailing path separators
- in Python <3.7 it doesn't
- filter empty path segments
2019-08-28 23:25:18 +02:00
Mike Fährmann
b2151f3928
[seiga] support mobile URLs ( closes #401 )
2019-08-28 22:56:43 +02:00
Mike Fährmann
20fd2d8450
[flickr] skip unavailable images/videos ( fixes #398 )
2019-08-27 23:26:49 +02:00
Mike Fährmann
c9b97dbca3
extend post processor tests
2019-08-26 23:10:12 +02:00
Mike Fährmann
60c8e090da
[postprocessor:zip] fix archive names ( closes #397 )
...
Remove the trailing path separator introduced in 3284c62
2019-08-24 23:14:26 +02:00
Mike Fährmann
7c09545f70
[downloader:ytdl] add 'outtmpl' option ( #395 )
2019-08-24 22:47:59 +02:00
Mike Fährmann
5cc7be2536
[piczel] update and improve
...
- use proper pagination (fixes #396 )
- update API host and endpoints
- "fix" double slash // in image URLs
2019-08-24 20:37:33 +02:00
Mike Fährmann
0c1c7abb4d
release version 1.10.2
2019-08-23 22:10:54 +02:00
Mike Fährmann
49f6d7176d
[deviantart] restore filenames ( #392 )
...
<title>_by_<user>_<id> --> <title>_by_<user>-<id>
2019-08-23 22:02:03 +02:00
Mike Fährmann
63daa68d67
[deviantart] improvements ( #392 )
...
- consistent 'filename' entries, at least as far as possible
- GIFs and SWFs don't have a <title>_by_<artist>_<id> anywhere in
their metadata
- Generating <id> (from 'deviationid'?) might be something that needs
to be figured out, so we can build those filenames ourselves
- better code structure etc.
- tests for videos, archives, and flash animations
2019-08-23 12:27:19 +02:00
Mike Fährmann
d1db5180ab
[simplyhentai] fix extraction; remove image+video extractors
2019-08-22 23:56:41 +02:00
Mike Fährmann
30d6e284b0
[deviantart] use NAPI for artworks and scraps ( #392 )
...
TODO:
- journal downloads
- test for all media types
2019-08-21 23:56:06 +02:00
Mike Fährmann
e528f3cb77
adjust postprocessor test results
...
see 2495b99
2019-08-21 23:54:08 +02:00
Mike Fährmann
7d6af936c5
[imgur] simplify gallery extraction
2019-08-20 20:00:43 +02:00
Mike Fährmann
3284c62f22
ensure PathFormat.directory ends with a path separator
...
... plus some other small optimizations
2019-08-20 00:25:13 +02:00
Mike Fährmann
ebabc5caf1
[downloader:http] treat 416 without downloaded data as error
...
Downloading https://pbs.twimg.com/media/EB2cGUYX4AI2Vuu.jpg:orig (NSFW)
sometimes returns a 416 status code, even though no 'Range' header was
sent and no data was downloaded prior.
This code usually means a file has already been downloaded completely
and the download method indicates success, but in this case it causes
an exception down the pipeline since no file was created.
2019-08-20 00:15:17 +02:00
Mike Fährmann
2495b99347
[postprocessor:classify] improve path generation ( fixes #138 )
...
It still doesn't work for converted ugoira animations thanks to how
those files are handled, but everything else, including files with
unknown or changing file extension, now works as it should.
2019-08-19 23:05:28 +02:00
Mike Fährmann
e77a656437
optimize directory path generation
...
- use str.join() instead of os.path.join()
(less "features", but 10x as fast)
- cache directory formatters
- detect and optimize field access for 1-element format strings
2019-08-19 15:56:20 +02:00
Mike Fährmann
51d10783fc
[patreon] include image info in API results ( #383 )
2019-08-18 23:28:47 +02:00
Mike Fährmann
7a5e78741c
[booru] build directory path for each file ( #385 )
2019-08-18 23:28:33 +02:00
Mike Fährmann
b1728f512d
[patreon] support multi image posts and post URLs ( #383 )
2019-08-17 23:24:46 +02:00
Mike Fährmann
454bf1ebf9
preserve enumeration index after 'set_extension()' ( #306 )
2019-08-16 23:12:33 +02:00
Mike Fährmann
f5039b897f
replace DownloadArchive.check() with __contains__()
...
Interestingly enough, 'a in obj' is slightly faster than
'obj.check(a)' and is also nicer to look at, I think.
2019-08-16 23:12:32 +02:00
Mike Fährmann
5a210991b6
Remove control characters from filesystem paths
...
- add 'path-remove' option to specify the set of characters that
should be removed
- rename 'restrict-filenames' to 'path-restrict'
- #348 , #380
2019-08-16 23:12:16 +02:00
Mike Fährmann
c50d60a53d
[reactor] fix image URLs
2019-08-16 14:07:22 +02:00
Mike Fährmann
32447d0d24
[pixiv] simplify default filename format
...
(#366 )
2019-08-15 13:32:47 +02:00
Mike Fährmann
5f8621b29d
improve output of active post processor modules
2019-08-15 13:31:04 +02:00
Mike Fährmann
2cbbc3dec4
add a 'whitelist' to '--ugoira-conv' ( #382 )
2019-08-15 13:27:57 +02:00
Mike Fährmann
e661a1ce19
update links to Request's documentation
2019-08-15 13:25:06 +02:00
Mike Fährmann
cb7f149974
fix mtime datetime test
...
datetime.timestamp() uses local time for a naive datetime object
2019-08-14 21:57:42 +02:00
Mike Fährmann
829b1ccf04
[imgur] distinguish album and gallery URLs ( #380 )
...
A gallery can be either an album or a single image.
2019-08-14 21:40:14 +02:00
Mike Fährmann
23251356cb
require 'extension' data for each URL ( #382 )
2019-08-14 20:03:03 +02:00
Mike Fährmann
a67413d64f
[xhamster] use input URL domain
...
Don't rewrite all URLs as 'https://xhamster.com/ ...'
2019-08-14 00:21:30 +02:00
Mike Fährmann
dd72ae7164
add postprocessor tests
2019-08-12 21:46:30 +02:00
Mike Fährmann
0bb873757a
update PathFormat class
...
- change 'has_extension' from a simple flag/bool to a field that
contains the original filename extension
- rename 'keywords' to 'kwdict' and some other stuff as well
- inline 'adjust_path()'
- put enumeration index before filename extension (#306 )
2019-08-12 21:40:37 +02:00
Mike Fährmann
423f68f585
[deviantart] fix scraps extraction ( closes #376 )
2019-08-11 16:06:15 +02:00
