Mike Fährmann
2d41702762
[deviantart] implement '"group": "skip"' ( #4630 )
2023-10-12 22:14:20 +02:00
Mike Fährmann
992e86ec94
[deviantart] disable 'jwt' ( #4652 )
2023-10-12 21:37:10 +02:00
Mike Fährmann
64dbc58a5a
[deviantart] update Eclipse API endpoints 2 ( #4615 )
2023-10-03 14:58:52 +02:00
Mike Fährmann
bb39779e1a
[deviantart] use private tokens for 'is_mature' posts ( #4563 )
2023-09-24 19:15:35 +02:00
Mike Fährmann
0c5d8b1505
[deviantart] re-add 'quality' option and 'intermediary' transform
2023-09-24 17:36:05 +02:00
Mike Fährmann
20d1683c47
[deviantart] fix JWT replacement ( #293 , #4548 , #4563 )
...
And again, a huge thank you to @Ironchest337
for discovering this.
2023-09-24 15:06:16 +02:00
Mike Fährmann
2ad75bab05
[deviantart] add 'is_original' metadata field ( #4559 )
...
true for 'downloadable' content, journals, flash animations,
and images without '/v1/' in their URL; false otherwise
2023-09-22 17:37:11 +02:00
Mike Fährmann
9d8317d963
[deviantart] disable JWT updates ( #4548 , #4563 )
...
back to lowres images ...
2023-09-22 17:13:05 +02:00
Mike Fährmann
8064663bda
[deviantart] update Eclipse API endpoints ( #4553 )
2023-09-22 16:30:16 +02:00
Mike Fährmann
2cd801232b
fix --range causing crashes ( #4557 )
...
regression caused by a383eca7
2023-09-22 16:28:20 +02:00
Mike Fährmann
3ecb512722
send Referer headers by default
2023-09-19 00:02:04 +02:00
Mike Fährmann
a453335a9f
remove test results in extractor modules
...
and add generic example URLs
2023-09-11 16:30:55 +02:00
Mike Fährmann
1d2b5d0c60
update test comment positions
...
always put them above the test they're referring to
2023-09-06 18:16:09 +02:00
Mike Fährmann
df5c7ee03e
[deviantart] fix search ( #4384 )
...
send correct usernames instead of 'u'
2023-08-04 17:16:04 +02:00
Mike Fährmann
a383eca7f6
decouple extractor initialization
...
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().
This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
2023-07-25 22:16:16 +02:00
Mike Fährmann
d97b8c2fba
consistent cookie-related names
...
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
2023-07-22 01:20:50 +02:00
Mike Fährmann
22099422ca
[deviantart] fix shortened URLs ( #4316 )
2023-07-18 17:55:13 +02:00
Mike Fährmann
98c9fdb414
[deviantart] revert e9353c63; retry downloads with private token
2023-04-23 21:10:16 +02:00
Mike Fährmann
e9353c63d6
[deviantart] keep using private access tokens
...
for deviations returned from a private API call
also fixes a bug from 0a7eee3e
2023-04-13 14:46:06 +02:00
Mike Fährmann
0a7eee3ee0
[deviantart] add 'public' option
2023-04-08 23:04:34 +02:00
Mike Fährmann
34699fbf64
[deviantart:search] detect login redirects ( #3860 )
2023-04-03 19:37:12 +02:00
ClosedPort22
c489aecb3e
[deviantart] add support for fxdeviantart.com URLs
...
fxdeviantart.com is a service that fixes embeds on Discord, similar to
fxtwitter.com
2023-03-07 16:04:17 +08:00
Mike Fährmann
6b03506655
[deviantart] allow searching when not logged in
2023-02-25 19:04:18 +01:00
Mike Fährmann
3fa456d989
[deviantart] remove mature scraps warning ( #3691 )
...
warn about private deviations
when paginating over eclipse results
2023-02-23 22:56:37 +01:00
Mike Fährmann
51301e0c31
replace remaining time.sleep() calls
...
with Extractor.sleep() or request_interval
2023-02-23 00:35:37 +01:00
Mike Fährmann
6ed4309aba
[deviantart] add 'gallery-search' extractor ( #1695 )
2023-02-22 17:25:31 +01:00
Mike Fährmann
e1df7f73b1
[deviantart] add 'search' extractor
...
(#538 , #1264 , #2954 , #2970 , #3577 )
Requires login to fetch any results, since the API endpoint raises an
error for not logged in requests.
TODO: parse HTML search results
2023-02-20 20:54:46 +01:00
Mike Fährmann
b4899c266f
merge #3656 : [deviantart] fix crash when handling deleted deviations in status updates
2023-02-18 13:11:47 +01:00
Mike Fährmann
725baedad3
[deviantart] use '/collections/all' endpoint for favorites
...
(#3666 ,#3668)
2023-02-17 23:11:35 +01:00
ClosedPort22
f4e211356d
[deviantart] slight refactor
2023-02-16 21:23:55 +08:00
ClosedPort22
a74114ef7a
[deviantart] fix crash when handling deleted deviations
...
in status updates
2023-02-15 00:17:11 +08:00
Mike Fährmann
7e2fd2e573
merge #3560 : [deviantart] add support for /deviation/ and fav.me URLs
2023-01-23 14:16:18 +01:00
ClosedPort22
c90b4ea8d9
[deviantart] add support for fav.me URLs
2023-01-23 20:35:03 +08:00
ClosedPort22
674c719646
[deviantart] refactor base36 conversion
2023-01-23 17:59:55 +08:00
ClosedPort22
293abb8921
[deviantart] add support for /deviation/ URLs
2023-01-23 17:37:57 +08:00
ClosedPort22
597b89245e
[deviantart] misc improvements to status extractor
...
- relax regex pattern
- handle invalid 'items' field
- add a test for shared sta.sh item
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2023-01-21 13:04:27 +08:00
ClosedPort22
013733c9e9
[deviantart] fix index fields for embedded/shared images
2023-01-19 12:17:27 +08:00
ClosedPort22
c4aeca7a5a
[deviantart] improve handling of statuses
...
- recursively yield statuses
- ignore items with missing or unexpected field(s)
2023-01-19 11:48:21 +08:00
ClosedPort22
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2023-01-19 11:30:48 +08:00
Mike Fährmann
77df8d3116
[deviantart] implement username&password login for scraps ( #1029 )
...
re-login when getting prematurely logged out by dA
is missing at the moment
2023-01-03 18:42:28 +01:00
Mike Fährmann
8d7585534e
Merge pull request #3367 from the-blank-x/deviantart-view
...
[deviantart] add /view URL support
2022-12-17 20:05:35 +01:00
blankie
6614d94b08
[deviantart] add /view URL support
2022-12-17 23:27:55 +07:00
ClosedPort22
bf3fd5951a
Merge branch 'master' into da-extra-stash
2022-12-10 21:34:37 +08:00
Mike Fährmann
cd931e1139
update extractor test results
2022-12-08 18:58:29 +01:00
ClosedPort22
13d825731e
[deviantart] fix test for sta.sh URL extraction
...
Without the 'count' assertion, the test would be essentially useless.
2022-12-07 00:32:03 +08:00
ClosedPort22
6356c9be96
[deviantart] extract sta.sh URLs from 'text_content'
2022-12-06 22:57:53 +08:00
Mike Fährmann
b0cb4a1b9c
replace 'text.extract()' with 'text.extr()' where possible
2022-11-05 01:14:09 +01:00
Mike Fährmann
2a1cb403ee
Revert "[Deviantart] [ #1776 ] Remove the "you need session cookies to download mature scraps" warning ( #1777 )"
...
This reverts commit 1f02878351
2022-11-01 21:23:05 +01:00
Mike Fährmann
14717f3fc9
[deviantart] add 'group' option ( #3018 )
...
disabling this option allows to better download from deleted accounts
2022-10-08 15:41:21 +02:00
Mike Fährmann
a12ce2bb41
[deviantart] fix 'deviation' extraction ( #2981 )
2022-10-05 21:30:08 +02:00
