Mike Fährmann
492436f936
[twitter] add 'warnings' option ( #2258 )
...
disable reporting any non-fatal errors by default
2022-02-02 18:37:19 +01:00
Mike Fährmann
a5163e4c70
[twitter] restore 'logout' functionality ( #1719 )
2022-02-02 18:21:15 +01:00
Mike Fährmann
f58364f6a8
update Firefox cipher list
2022-02-01 02:33:01 +01:00
Mike Fährmann
7e6981dda6
rename 'disabletls12' to 'tls12'
...
and let config options override any default settings
2022-02-01 01:37:03 +01:00
Mike Fährmann
bb3e182562
overhaul session initialization
...
- share adapter & connection pool across sessions with the same
ssl options, ssl ciphers, and source address
- simplify browser emulation to just a list of headers and ciphers
2022-01-31 23:12:08 +01:00
Mike Fährmann
e670dc518e
[weibo] update pagination code ( fixes #2244 )
...
- send proper headers and query parameters
- use 'since_id' instead of page numbers
- set a 1-2 second delay between requests
2022-01-31 19:16:01 +01:00
Robert Pendell
4c651f6252
[patreon] Disable TLS 1.2 by default ( #2249 )
...
Disables TLS 1.2 on Patreon by default.
2022-01-30 23:30:44 +01:00
Robert Pendell
392cf079f7
Add ability to disable TLS 1.2 ( #2243 )
...
Fix for Patreon Cloudflare issues by having only TLS v1.3 or higher establish HTTPS connections
This now allows you to disable it on a per-host or global basis. Add disabletls12 as a config option either under extractor.(host) or just under extractor. Option is false by default.
Example:
"patreon":
{
"disabletls12": true,
"cookies": {
"session_id": "X"
}
}
2022-01-30 22:14:43 +01:00
Mike Fährmann
d33227fc38
[twitter] restore errors for protected timelines etc ( fixes #2237 )
2022-01-30 16:42:13 +01:00
Mike Fährmann
ebd3d5c1cc
[bunkr] fix .mp4 downloads ( closes #2239 )
2022-01-28 23:21:16 +01:00
Mike Fährmann
e2be199124
[gelbooru] improve and fix pagination ( #2230 , #2232 )
...
Use 'id:<POSTID' as a tag instead of going through pages with 'pid'.
Something similar was already implemented in 93cef7843085aac4
2022-01-27 17:44:47 +01:00
Mike Fährmann
806badbeec
release version 1.20.3
2022-01-26 01:10:44 +01:00
Mike Fährmann
8230f31800
[twitter] update query hashes
2022-01-26 00:49:46 +01:00
Mike Fährmann
c180806cec
[twitter] fix deleted/invalid retweets ( #2225 )
2022-01-25 23:57:13 +01:00
Mike Fährmann
a2eecc6aa8
[kemonoparty] fix DMs extraction ( #2008 )
2022-01-25 23:16:13 +01:00
Mike Fährmann
2bf554a896
[twitter] fix several errors ( #2212 , #2216 , #2225 )
...
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
2022-01-25 16:13:22 +01:00
Mike Fährmann
fbd17547f5
release version 1.20.2
2022-01-24 18:24:50 +01:00
Mike Fährmann
e5242b83bf
[twitter] define directory format for events ( #2109 )
2022-01-24 17:44:17 +01:00
Mike Fährmann
efb3e65a6a
[sexcom] extend URL pattern ( fixes #2220 )
2022-01-24 01:19:40 +01:00
vsyx
3f2b6335d7
[instagram] fix highlights extraction ( #2197 )
...
* [instagram] fix highlights extraction
* [instagram] improve highlights extraction
- 'yield' individual reels instead of collecting them in a list
and returning them all at once
- reduce 'chunk_size' to an even saver value
(instagram.com also uses 5)
2022-01-24 00:20:12 +01:00
Mike Fährmann
5ed26e1773
[twitter] fix pinned tweets ( #2216 )
...
caused by the changes in dffa440ede
2022-01-23 22:52:57 +01:00
Mike Fährmann
a9f78e6527
[twitter] improve error handling
...
- handle accounts without 'rest_id'
- handle timelines with empty 'instructions'
2022-01-23 18:01:05 +01:00
Mike Fährmann
729b07c1f5
[twitter] simplify
...
- use dict with common GraphQL variables
- reduce 'variables' size with custom JSON encoder instance
- centralise TwitterAPI() creation
2022-01-23 01:44:55 +01:00
Mike Fährmann
7cb29224f0
[philomena] fix search parameter escaping ( #2215 )
...
The pluses from search terms in /tags/ URLs need to be
replaced with spaces to get accepted by Philomena.
2022-01-23 01:03:37 +01:00
Mike Fährmann
9ca8bb2dc0
[twitter] improve error handling
2022-01-22 23:09:45 +01:00
Mike Fährmann
9a221494c3
[twitter] add 'event' extractor ( closes #2109 )
2022-01-22 20:55:50 +01:00
Mike Fährmann
14867dad6b
[twitter] fix unified cards from search results
2022-01-22 20:25:10 +01:00
Mike Fährmann
dffa440ede
[twitter] improve handling of deleted tweets ( #2212 )
2022-01-22 00:41:58 +01:00
Mike Fährmann
54ef874ba4
[twitter] fix retweet filter ( #2212 )
2022-01-21 23:53:59 +01:00
Mike Fährmann
cb43f7731b
[twitter] update to GraphQL API ( #2212 )
...
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.
Only Twitter's search doesn't have a GraphQL interface yet.
2022-01-21 23:34:41 +01:00
Mike Fährmann
de754590e0
add --source-address command-line option ( closes #2206 )
2022-01-21 17:07:56 +01:00
Mike Fährmann
698f35215e
[blogger] support new image domain ( fixes #2204 )
2022-01-20 23:13:07 +01:00
Mike Fährmann
c587b678d0
[mangadex] re-enable warning for external chapters ( #2193 )
2022-01-16 03:21:50 +01:00
Mike Fährmann
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
2022-01-15 22:02:57 +01:00
Mike Fährmann
2d34d8ff8b
[reddit] allow downloading from quarantined subreddits ( #2180 )
2022-01-14 21:55:59 +01:00
Mike Fährmann
17c9c47ca0
[hitomi] fix 'tag' extraction ( fixes #2189 )
2022-01-13 16:45:46 +01:00
Mike Fährmann
df2f0c09bb
[twitter] support "image_carousel_website" unified cards
2022-01-13 16:05:52 +01:00
Mike Fährmann
cdc96e1217
[gelbooru] improve video file detection ( fixes #2188 )
...
not all files from 'https://video-cdnN.gelbooru.com ' are videos
2022-01-12 21:33:02 +01:00
Mike Fährmann
4acc31bd9f
[newgrounds] set suitabilities filter before starting a search
2022-01-11 23:50:29 +01:00
Mike Fährmann
58a7921b5c
release version 1.20.1
2022-01-08 23:25:59 +01:00
Mike Fährmann
170711af7e
[mangadex] fix extraction ( closes #2177 )
2022-01-08 17:21:35 +01:00
Mike Fährmann
199e7616a7
[rule34] use https://api.rule34.xxx for API requests
2022-01-08 17:14:50 +01:00
Mike Fährmann
6e0a6c484f
apply SPECIAL_EXTRACTORS only for blacklist settings
...
as was the case before 010d65dc
2022-01-06 21:09:30 +01:00
Mike Fährmann
37beb1298e
[newgrounds] add 'search' extractor ( closes #2161 )
2022-01-06 19:32:39 +01:00
Mike Fährmann
8b910dd8ae
[hitomi] fix image URLs
...
again and again ...
2022-01-06 18:21:26 +01:00
Mike Fährmann
dcfe08838d
restore -d/--dest functionality
...
change short option for --directory from -d to -D
2022-01-03 18:30:36 +01:00
Mike Fährmann
3085aac4d8
[gelbooru] handle changed API response format ( #2157 )
2022-01-03 16:42:48 +01:00
Mike Fährmann
38e2af29d6
[hitomi] fix image URLs
...
update '_parse_gg()' yet again
2022-01-03 16:41:00 +01:00
Mike Fährmann
6f2e0c9c3d
fix cookie checks for patreon, fanbox, fantia
...
The changes in 9a255344
2022-01-01 17:55:58 +01:00
Mike Fährmann
1e0278702d
[hitomi] update '_parse_gg()'
2022-01-01 17:55:58 +01:00
