Mike Fährmann
20fbba9d7c
[exhentai] add metadata to search results ( #3181 )
...
'gallery_id' and 'gallery_token'
2022-11-09 11:40:27 +01:00
Mike Fährmann
6a0c5e34f4
[exhentai] fix pagination ( #3181 )
2022-11-09 11:15:34 +01:00
Mike Fährmann
05255f5be0
add 'default' argument to 'text.extr()'
2022-11-09 11:00:32 +01:00
Mike Fährmann
e326029355
build path when skipping archived files
...
fixes bug from 8124c16a
2022-11-09 10:33:09 +01:00
Mike Fährmann
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2022-11-08 21:48:31 +01:00
Mike Fährmann
8124c16a50
split 'build_path' from 'set_filename' and 'set_extension'
...
Do not automatically build a new path
when setting file metadata or updating its extension.
2022-11-08 17:03:24 +01:00
Mike Fährmann
39d9c362e4
include 'http-metadata' in '-K' output
2022-11-07 16:33:26 +01:00
Mike Fährmann
e2401c96ee
[postprocessor:metadata] add '"mode": "jsonl"'
2022-11-07 15:46:35 +01:00
Mike Fährmann
895f36e53b
[postprocessor:metadata] add 'open' and 'encoding' options
2022-11-07 15:37:22 +01:00
Mike Fährmann
93e6bd6847
[uploadir] use utf-8 filenames ( #3162 )
2022-11-05 19:03:48 +01:00
Mike Fährmann
870e6a48a0
implement 'http-metadata' option
...
or at least attempt to.
2022-11-05 18:29:29 +01:00
Mike Fährmann
b7a83ac726
[uploadir] update ( #3162 )
...
- prevent extra HTTP request from redirects
- add 'id' metadata field
- set 'filename_fmt' and 'archive_fmt'
2022-11-05 18:28:35 +01:00
Mike Fährmann
ccb80f1b8b
[uploadir] add support for 'uploadir.com' ( #3162 )
2022-11-05 14:25:09 +01:00
Mike Fährmann
b0cb4a1b9c
replace 'text.extract()' with 'text.extr()' where possible
2022-11-05 01:14:09 +01:00
Mike Fährmann
eb33e6cf2d
add 'text.extr()'
...
a stripped-down version of text.extract() that
- always returns a string (like 'extract_from')
- only returns a string
- does not deal with 'pos' arguments
- is ~20% faster
2022-11-04 21:37:36 +01:00
Mike Fährmann
597b63d922
move git head functionality to function in util.py
2022-11-04 19:49:39 +01:00
Mike Fährmann
4fd3c893fa
[booru] adjust/match '_tags' and '_notes' code
2022-11-04 19:49:39 +01:00
Mike Fährmann
88954aa2e4
[gelbooru_v02] implement 'notes' extraction
...
same code as for 'moebooru' works here as well
2022-11-04 19:49:39 +01:00
Mike Fährmann
942bc84962
add '--chunk-size' command-line option ( #3143 )
2022-11-04 19:49:39 +01:00
Mike Fährmann
79a9fc6e45
add '--user-agent' command-line option
2022-11-04 19:49:39 +01:00
Mike Fährmann
b357c2dcf8
build executables on GitHub Actions with Python 3.11
2022-11-04 19:49:39 +01:00
ClosedPort22
4e80d3210e
[tumblr] Fallback to gifv when possible ( #3095 ) ( #3159 )
2022-11-04 19:42:36 +01:00
thatfuckingbird
9d3f86dbcd
[twitter] update URL for syndication API ( #3160 )
...
Twitter changed the URL format to access tweet data through their syndication API.
2022-11-04 17:49:45 +01:00
enduser420
c01cad599a
[lolisafe] add support for xbunkr ( #3156 )
2022-11-04 16:13:52 +01:00
Allen
9fc142d27b
[mastodon] add "remote_instance" field ( #3119 )
...
Example Usage:
If the url is "mastodon:https://mastodon.example.org/@VoteChess@botsin.space the "remote_instance" will be "botsin.space"
...
"directory": ["mastodon", "{remote_instance|instance}", "{account[username]!l}"]
...
2022-11-02 17:09:38 +01:00
Mike Fährmann
bca9f965e5
[downloader:http] add 'chunk-size' option ( #3143 )
...
and double the previous default from 16384 (2**14) to 32768 (2**15)
2022-11-02 16:50:26 +01:00
Mike Fährmann
2a1cb403ee
Revert "[Deviantart] [ #1776 ] Remove the "you need session cookies to download mature scraps" warning ( #1777 )"
...
This reverts commit 1f02878351
2022-11-01 21:23:05 +01:00
Mike Fährmann
460095adca
update downloader tests
2022-11-01 18:48:35 +01:00
Mike Fährmann
0059e2bfe7
[downloader:http] add MIME type and signature for .avif files
2022-11-01 17:25:21 +01:00
Mike Fährmann
f687e64513
[downloader:http] refactor file signature checks
...
use functions/lambdas instead of startswith()
2022-11-01 17:09:13 +01:00
Mike Fährmann
86790da2d5
update Cloudflare IUAM detection
...
again
2022-10-31 18:33:52 +01:00
Mike Fährmann
c12a97bcde
[postprocessor] add 'post-after' event ( #3117 )
2022-10-31 14:35:48 +01:00
Mike Fährmann
775895f44b
[booru] refactor 'tags' and 'notes' extraction
...
- move HTML request for post pages into its own function
- move gelbooru_v02.py notes extraction to gelbooru.py
since it only works there
- clean up some code
2022-10-31 12:01:19 +01:00
Mike Fährmann
48bbe1ccf6
update CHANGELOG and README for 1.23.5
2022-10-30 18:42:12 +01:00
Luc Ritchie
0f9dfb7e62
[instagram] Fix AttributeError on user stories extraction ( #3123 )
2022-10-30 11:47:06 +01:00
Mike Fährmann
7a9cb6d7bf
update CHANGELOG and README for 1.23.4
2022-10-29 17:35:56 +02:00
Mike Fährmann
f81dd5297a
[skeb] fix extraction ( #3112 )
...
'completed_at' is no longer included in API responses
2022-10-29 15:07:21 +02:00
Mike Fährmann
b337e51e91
run flake8 on all .py files
2022-10-28 22:22:49 +02:00
enduser420
fb2dbb04e2
[moebooru] extract 'notes' ( #3094 )
2022-10-28 18:05:54 +02:00
Mike Fährmann
4e26bf98f5
[aibooru] support 'safe' subdomain ( #3110 )
2022-10-28 12:10:18 +02:00
Mike Fährmann
f037429fa4
attempt to improve '-K' output for lists
...
- use [N] instead if [] to indicate a Number needs to be placed there
- enumerate list items
2022-10-28 12:04:58 +02:00
Mike Fährmann
e140b85342
reword error text for unsupported URLs
2022-10-28 11:49:20 +02:00
Mike Fährmann
5c31791b3c
[mastodon] support '/web/' URLs ( #3109 )
2022-10-28 11:47:00 +02:00
Mike Fährmann
9a2cfd4421
[mastodon] support cross-instance user references ( #3109 )
2022-10-27 14:26:42 +02:00
Mike Fährmann
58d97188b4
[mastodon] add 'bookmark' extractor ( #3109 )
2022-10-26 21:28:50 +02:00
Mike Fährmann
c8af1f585a
fix pypy version specifier
2022-10-26 15:55:22 +02:00
Mike Fährmann
46b64251eb
[bcy] fix extraction ( #3103 )
...
- fix regex for non-watermarked images
- fetch data from '/item/detail' pages for all other posts,
since '/apiv3/user/selfPosts' only has incomplete data
2022-10-26 15:33:07 +02:00
Mike Fährmann
77173694d5
[kemonoparty] fix 'dms' extraction ( #3106 )
2022-10-26 14:25:43 +02:00
Mike Fährmann
f168ec9572
[instagram] extract 'coauthors' metadata ( #3107 )
2022-10-26 14:11:29 +02:00
Mike Fährmann
7c6af27eb8
[tumblr] add 'fallback-*' options ( #2957 )
...
specifically 'fallback-delay' and 'fallback-retries'
and change default number of retries to 2 (down from 3)
2022-10-26 13:59:09 +02:00
