Mike Fährmann
7d874e2497
[bluesky] improve API error messages
2024-02-28 14:45:36 +01:00
Mike Fährmann
d921d860f1
automatically create directory path for logging files ( #5249 )
2024-02-28 02:11:20 +01:00
Mike Fährmann
24106d9994
exclude scripts/pyprint.py from linting for Python<3.8
2024-02-27 19:07:52 +01:00
Mike Fährmann
495c9ee126
[bluesky] add 'reposts' option ( #4438 , #5248 )
2024-02-27 18:42:29 +01:00
Mike Fährmann
c8b591303f
[paheal] cleanup
2024-02-27 02:27:20 +01:00
Mike Fährmann
ba062712ad
[tests] '__main__' -> "__main__"
2024-02-27 02:10:05 +01:00
Mike Fährmann
2501adeda0
move 'pprint()' into its own module
...
to reuse its code in create_test_data.py later
rename to 'pyprint' since 'pprint' is already used by stdlib module
2024-02-27 02:01:55 +01:00
Mike Fährmann
8a11b72253
remove extractor/test.py ( #4504 )
2024-02-27 01:37:57 +01:00
Mike Fährmann
fde9e25c9f
[tests:kemonoparty] '.party' -> '.su'
2024-02-26 22:25:04 +01:00
Mike Fährmann
311a21bfb2
[bluesky] fix '/follows' not spawning child extractors ( #5246 )
2024-02-26 15:38:31 +01:00
Mike Fährmann
d3dca68225
[xvideos] fix galleries with more than 500 images ( #5244 )
2024-02-26 15:36:41 +01:00
Mike Fährmann
13443f40a3
[xvideos] support '/channels/' URLs ( #5244 )
2024-02-26 00:08:37 +01:00
Mike Fährmann
c60ebc6519
[deviantart] improve fetching extended metadata ( #5175 )
...
use multiple metadata API calls per chunk of deviations if necessary
2024-02-25 03:36:00 +01:00
Mike Fährmann
cc6b9e4c18
[zerochan] use API by default ( #3669 )
...
add 'pagination' option
2024-02-25 00:36:14 +01:00
Mike Fährmann
efccd3d3d1
merge #5097 : update Dockerfile
...
- remove a layer and reduce image size
- update pip and apk
2024-02-24 02:06:35 +01:00
Mike Fährmann
a2b55d5dde
[skeb] retry 429 responses containing a 'request_key' cookie ( #5210 )
2024-02-24 00:54:15 +01:00
Mike Fährmann
e51ee6b132
fix HttpError.status value
...
'response' with error status code evaluates to False
2024-02-24 00:04:24 +01:00
Mike Fährmann
b4c46de4b8
merge #5224 : [artstation] update URL patterns to recognize usernames with dashes
2024-02-21 14:41:02 +01:00
blankie
962f55cc68
[artstation] fix handling usernames with dashes
2024-02-21 17:39:37 +11:00
Mike Fährmann
fe7e2281ac
[nijie] increase default delay between requests ( #5221 )
...
1-2s is not enough
2024-02-20 18:19:49 +01:00
Mike Fährmann
a34312e3ac
[instagram] make accessing 'like_count' non-fatal ( #5218 )
2024-02-19 19:24:51 +01:00
Mike Fährmann
741fd00cec
[deviantart] extend 'metadata' option ( #5175 )
...
alloe fetching extended metadata in addition to the usual
'description', 'tags', etc by setting 'metadata' to a list of
'camera', 'stats', 'submission', 'collection', and 'gallery'
for example "metadata": "stats,submission"
2024-02-18 23:14:14 +01:00
Mike Fährmann
fc46177578
release version 1.26.8
2024-02-17 21:03:19 +01:00
Mike Fährmann
8a63801311
[vsco] add 'spaces' extractor ( #5202 )
...
for spaces listed on a user page
2024-02-17 18:20:48 +01:00
Mike Fährmann
ccb413df71
[wikimedia] support 'pidgi.net' and 'bulbapedia.bulbagarden.net' ( #5205 , #5206 )
2024-02-17 17:35:10 +01:00
Mike Fährmann
7033cc14e9
[vsco] add 'space' extractor ( #5202 )
2024-02-17 01:54:05 +01:00
Mike Fährmann
770aec922d
[fapachi] ignore empty entries
2024-02-16 22:43:37 +01:00
Mike Fährmann
c9efccc959
[tests] update extractor results
2024-02-16 22:42:06 +01:00
Mike Fährmann
c413834dfc
[bluesky] extend tests
2024-02-16 16:30:02 +01:00
Mike Fährmann
ee7c054855
[bluesky] add 'search' extractor ( #4438 )
...
Both https://bsky.app/search?q=QUERY and https://bsky.app/search/QUERY
are recognized as search URLs, where QUERY gets forwarded unmodified as
'q' parameter for app.bsky.feed.searchPosts .
User searches are not supported yet.
2024-02-16 15:58:47 +01:00
Mike Fährmann
91e5c4fdfe
[bluesky] add 'avatar' and 'background' extractors ( #4438 )
2024-02-16 15:41:19 +01:00
Mike Fährmann
24c1317e0d
[batoto] fix crash when manga/chapter contains a '-' ( #5200 )
2024-02-16 00:10:08 +01:00
Mike Fährmann
0abd9723af
[bluesky] add 'metadata' option ( #4438 )
...
allow extracting 'user' metadata and
make 'facets' extraction optional
2024-02-15 23:30:16 +01:00
Mike Fährmann
7e036ea290
[bluesky] add 'depth' option ( #4438 )
...
and reduce default depth and parentHeight values
2024-02-15 22:26:05 +01:00
Mike Fährmann
42335ea880
[zerochan] fix skipping every other post
2024-02-15 02:51:01 +01:00
Mike Fährmann
c97b92cc35
[fanbox] add 'home' and 'supporting' extractors ( #5138 )
2024-02-14 23:25:39 +01:00
Mike Fährmann
04e4ffc64c
[deviantart] combine 'png' option with 'quality' ( #4846 )
...
"quality": "png" to download PNGs instead og JPEGs
2024-02-14 22:07:29 +01:00
Mike Fährmann
9cc4ec2c58
[deviantart] add 'png' option ( #4846 )
2024-02-14 01:03:15 +01:00
Mike Fährmann
966c8608e6
[deviantart] move image content extraction into separate function
2024-02-14 00:30:06 +01:00
Mike Fährmann
61a50da086
merge #5195 : [pornpics] support multiple 'channel' values
...
i.e. change 'channel' from string to list
use '{channel[0]}' to get the old behavior
2024-02-13 23:54:10 +01:00
Mike Fährmann
1d1ffe3317
[pornpics] update 'channel' extraction & add test
...
change 'channel' to a list, since extracting both 'channel' and
'channels' does not really work with text.extract_from()
2024-02-13 23:48:46 +01:00
cc1234
32472d7d6c
Add support for multi channels
2024-02-13 18:34:04 +00:00
Mike Fährmann
139ff3f6ab
[kemonoparty] add 'posts' extractor ( #5194 )
2024-02-13 15:41:34 +01:00
Mike Fährmann
814ad9321e
[deviantart] skip locked/blurred posts ( #4567 , #5193 )
2024-02-13 14:15:12 +01:00
Mike Fährmann
f7f8ef8684
[twitter] support communities ( #4913 )
2024-02-13 01:30:23 +01:00
Mike Fährmann
8f27f43d4d
[tests] implement explicitly disabling auth
2024-02-13 00:08:27 +01:00
Mike Fährmann
cae77e85f8
[twitter] update query hashes
...
... as well as 'variables' and 'features' values
also remove unused legacy API code
2024-02-12 23:19:13 +01:00
Mike Fährmann
06cb518d97
[bunkr] fix extraction ( #5088 , #5151 , #5153 )
...
- remove legacy code
- map legacy domains to bunkr.sk
- use input URL domain for newer domains
- update tests (some files got slightly modified or deleted)
2024-02-11 22:36:03 +01:00
Mike Fährmann
dcc6e3f65c
merge #5134 : [bunkr] add new bunkr domains ( #5130 )
2024-02-11 21:10:06 +01:00
Mike Fährmann
4641937ca3
[imagetwist] add 'gallery' extractor ( #5190 )
2024-02-11 18:41:02 +01:00
