Mike Fährmann
27ec653991
fix bug in test_init and update example URLs
2023-09-14 13:27:03 +02:00
Mike Fährmann
a453335a9f
remove test results in extractor modules
...
and add generic example URLs
2023-09-11 16:30:55 +02:00
Mike Fährmann
d94aa1ee02
[gelbooru] fix --range for favorites ( #3704 )
2023-03-23 22:58:13 +01:00
Mike Fährmann
1f82b00b8f
[gelbooru] fix and improve --range for pools
2023-03-23 18:22:46 +01:00
Mike Fährmann
dcb8af659a
[gelbooru] extract favorites without needing cookies ( #3704 )
...
TODO: fix --range
2023-03-15 19:21:35 +01:00
Mike Fährmann
b756dc13aa
[gelbooru] warn about missing cookies for favorites ( #3704 )
...
and add docstring so it shows up in --list-extractors
2023-03-15 14:58:55 +01:00
Mike Fährmann
b14f8d5817
[gelbooru] add 'favorite' extractor ( #3704 )
...
requires logged in cookies to work
2023-02-27 18:03:47 +01:00
Mike Fährmann
ed2d715019
fix 'keywords' in extractor tests ( #3491 )
2023-01-03 15:14:23 +01:00
Mike Fährmann
b0cb4a1b9c
replace 'text.extract()' with 'text.extr()' where possible
2022-11-05 01:14:09 +01:00
Mike Fährmann
775895f44b
[booru] refactor 'tags' and 'notes' extraction
...
- move HTML request for post pages into its own function
- move gelbooru_v02.py notes extraction to gelbooru.py
since it only works there
- clean up some code
2022-10-31 12:01:19 +01:00
KJ16609
300bc03deb
[gelbooru] allow alternate parameter order in post URLs ( #2821 )
2022-10-21 14:59:29 +02:00
Mike Fährmann
d508b2c049
[gelbooru] implement 'pool' pagination ( #2853 )
2022-08-26 17:57:17 +02:00
Mike Fährmann
f225247670
[gelbooru] add support for api_key and user_id ( #2767 )
2022-07-18 18:46:31 +02:00
Mike Fährmann
e2be199124
[gelbooru] improve and fix pagination ( #2230 , #2232 )
...
Use 'id:<POSTID' as a tag instead of going through pages with 'pid'.
Something similar was already implemented in 93cef7843085aac4
2022-01-27 17:44:47 +01:00
Mike Fährmann
cdc96e1217
[gelbooru] improve video file detection ( fixes #2188 )
...
not all files from 'https://video-cdnN.gelbooru.com ' are videos
2022-01-12 21:33:02 +01:00
Mike Fährmann
3085aac4d8
[gelbooru] handle changed API response format ( #2157 )
2022-01-03 16:42:48 +01:00
Mike Fährmann
3e4ffb0821
[gelbooru] add extractor for '/redirect.php' URLs ( #1530 )
2021-05-07 15:34:53 +02:00
thatfuckingbird
dff03a6605
[booru] add an option to extract notes (only gelbooru for now) ( #1457 )
...
* [booru] add an option to extract notes (currently implemented only for gelbooru)
* appease linter
* [gelbooru] rename "text" to "body" in note extraction
* add a code comment about reusing return value of _extended_tags
2021-04-13 23:40:24 +02:00
thatfuckingbird
918b0441fb
[gelbooru] fix tag category extraction ( #1455 )
2021-04-10 19:05:00 +02:00
Mike Fährmann
780bac4c8a
[gelbooru] update video server ( fixes #1368 )
...
from 'https://img2.gelbooru.com ' to 'https://img3.gelbooru.com '
and provide fallback URLs
2021-03-10 01:48:07 +01:00
Mike Fährmann
08d7934c6e
move extractors from booru.py into their own gelbooru_v02 module
2021-02-17 00:26:24 +01:00
Mike Fährmann
e41e2be2f9
[booru] split '_prepare_post()'
2020-12-24 01:13:54 +01:00
Mike Fährmann
a3a863fc13
[booru] add generalized extractors for *booru sites
...
similar to cc15fbe7
2020-12-08 18:34:30 +01:00
Mike Fährmann
7a0ba370d1
[gelbooru] rewrite mp4 video URLs ( fixes #1048 )
2020-10-15 15:14:18 +02:00
Mike Fährmann
fda9e296dd
[gelbooru] fix extraction without API
2020-08-28 22:33:37 +02:00
Mike Fährmann
9b4635917f
[gelbooru] simplify and fix pool extraction
...
use 'pool:<pool id>' as search tag to get pool posts
2020-05-18 19:04:51 +02:00
Mike Fährmann
2188db6284
[gelbooru] fix non-API tag extraction
2019-12-10 21:31:55 +01:00
Mike Fährmann
7a5e78741c
[booru] build directory path for each file ( #385 )
2019-08-18 23:28:33 +02:00
Mike Fährmann
17a3426845
[gelbooru] enable all content when not using API
2019-07-27 11:13:38 +02:00
Mike Fährmann
6284731107
simplify extractor constants
...
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
2019-02-08 13:45:40 +01:00
Mike Fährmann
1d43cbbf52
[gelbooru] tag-splitting for non-api mode
2018-07-06 15:24:19 +02:00
Mike Fährmann
3dbc7c5f8d
[gelbooru] restore pool functionality
2018-07-04 12:21:41 +02:00
Mike Fährmann
a2c74bc6f0
[gelbooru] inherit from BooruExtractor class
...
Breaks pool functionality when using API calls (for now),
but reduces code clutter and enables the `tags` option.
2018-07-04 12:21:41 +02:00
Mike Fährmann
cc36f88586
rename safe_int to parse_int; move parse_* to text module
2018-04-20 14:53:21 +02:00
Mike Fährmann
5008e105ee
update archive IDs
...
... to behave in a more straightforward way when dealing with
bookmarks/favourites/etc.
specific IDs are now grouped by their owner, album-id, ... to
allow for duplicates when it would be expected.
2018-03-01 18:20:50 +01:00
Mike Fährmann
34873dbd90
set 'archive_fmt' values
...
These are going to be used to create an unique id for each image.
2018-02-01 15:30:49 +01:00
Mike Fährmann
d0886f411e
[gelbooru] re-enable API use ( closes #56 )
...
Gelbooru's API allows access to all images and is not restricted
to the first 20000.
This also adds an option to select between API use and manual
information extraction in case their API gets disabled again.
2017-12-21 21:42:40 +01:00
Mike Fährmann
291369eab2
various smaller changes/additions
2017-12-06 21:45:56 +01:00
Mike Fährmann
a1980b16f3
[gelbooru] various improvements
...
- better metadata for pools
- map ratings to s/q/e like other boorus do
- skip() support
2017-12-03 01:41:30 +01:00
Mike Fährmann
214972bc9a
[gelbooru] use manual extraction
...
... to compensate for their disabled API.
(https://gelbooru.com/index.php?page=forum&s=view&id=3875 )
This also adds an extractor for image-pools.
2017-11-29 20:48:17 +01:00
Mike Fährmann
07214f4007
[booru] place subcategories into base classes
2017-08-26 22:27:55 +02:00
Mike Fährmann
4b967fa189
implement and use extractor.config() method
2017-04-25 17:12:48 +02:00
Mike Fährmann
13dc5d72bc
update some extractors to use https
2017-04-20 13:32:40 +02:00
Mike Fährmann
ff92674379
[booru] simple skip functionality
...
(#6 )
2017-03-04 23:21:55 +01:00
Mike Fährmann
94e10f249a
code adjustments according to pep8 nr2
2017-02-01 00:53:19 +01:00
Mike Fährmann
d7e168799d
consistent extractor naming scheme + docstrings
2016-09-12 10:34:31 +02:00
Mike Fährmann
59445c821b
[gelbooru] remove login/auth requirement
2016-07-20 13:14:02 +02:00
Mike Fährmann
616e0aedd6
update booru testdata
2015-12-22 03:10:52 +01:00
Mike Fährmann
5a8541afa5
remove 'unstable' tests
2015-12-15 23:45:40 +01:00
Mike Fährmann
ba99506c72
more extractor test-cases
2015-12-14 03:00:58 +01:00
