Mike Fährmann
77bdd8fe0f
[twitter] implement constant 'user' for 'from:…' searches
2022-07-17 19:14:32 +02:00
Mike Fährmann
a267a05a3f
[twitter] update 'quote_id' and 'quote_by'
...
- 'quote_id' is now non-null for quoted Tweets and has the ID of the
quoting Tweet, instead the other way round like before
- 'quote_by' is now the 'screen_name' of the quoting user
(was the same the new 'quote_id' is now)
2022-07-17 18:50:21 +02:00
Mike Fährmann
749802c7bd
[twitter] update 'user' and 'author' fields
...
- 'author' is always the user who authored a tweet
- 'user' is always the user specified in the input URL
or equal to 'author' when the former is not given
2022-07-17 17:04:24 +02:00
blankie
836402bf58
[twitter] unescape content ( #2756 ) ( #2757 )
...
Fixes #2756
2022-07-13 19:45:14 +02:00
Mike Fährmann
b590774f67
[twitter] add 'count' metadata field ( #2741 )
2022-07-10 14:37:04 +02:00
Mike Fährmann
1d14928bd9
[twitter] ignore previously seen Tweets ( #2712 )
...
occurs primarily for /with_replies results when logged in
2022-07-03 16:13:53 +02:00
Mike Fährmann
4b2a0a0eda
[twitter] implement 'strategy' option ( #2712 )
...
to be able to better control what Tweets get used an returned
for twitter.com/USER URLs.
2022-07-03 14:29:15 +02:00
Mike Fährmann
7b073bf9ef
Revert "[twitter] improve strategy for user URLs ( #2665 )"
...
'user_tweets_and_replies' was a mistake
2022-06-28 20:38:56 +02:00
Mike Fährmann
d6c6c8a4a0
[twitter] improve '"replies": "self"' ( #2665 )
...
If a username is given in the input URL,
only download from replies by that user.
2022-06-13 19:21:32 +02:00
Mike Fährmann
9c8d895d19
[twitter] implement 'csrf' option ( #2676 )
2022-06-13 18:36:39 +02:00
Mike Fährmann
08db8435f1
[twitter] fix pagination for conversion tweets
...
a relic from the switch to GraphQL API
2022-06-13 16:27:30 +02:00
Mike Fährmann
1da3ccf608
[twitter] implement 'expand' option ( #2665 )
2022-06-12 17:26:51 +02:00
Mike Fährmann
0add1fc090
[twitter] improve strategy for user URLs ( #2665 )
...
- use '/with_replies' when appropriate
- consider 'text-tweets'
- build search query as necessary
2022-06-12 17:24:53 +02:00
thatfuckingbird
da0696e1f5
recognize vxtwitter URLs ( #2621 )
2022-05-25 17:01:58 +02:00
Mike Fährmann
dcb580240d
[twitter] extract alt texts as 'description' ( closes #2617 )
2022-05-24 12:37:38 +02:00
Mike Fährmann
915dba8345
[twitter] improve results for regular user URLs
...
- continuation of 3346f58ahttps://twitter.com/USER/tweets )
although there should be no need to ever do that
2022-05-23 18:33:52 +02:00
Mike Fährmann
9df4e0f65b
[twitter] disable 'cards' by default
2022-05-21 15:39:25 +02:00
Mike Fährmann
3346f58a2a
[twitter] use twMediaDownloader strategy for user URLs
...
- use media timeline + search for default user URLs like
https://twitter.com/SCREEN_NAME
- fetches all/most media for the type of twitter URL that most users
use with gallery-dl
- can be disabled by setting 'strategy' to any truthy value,
like "timeline"
2022-05-02 09:03:35 +02:00
Mike Fährmann
ad5a4b1756
[twitter] fix various syndication issues
...
- handle retweets
- fix videos without dimensions in URL (3e942a58#2499 )
2022-04-15 20:49:26 +02:00
Mike Fährmann
3e942a58be
[twitter] improve syndication video selection ( #2354 )
...
- ignore .m3u8 manifests
- always select largest format
2022-04-11 17:06:10 +02:00
thatfuckingbird
4527a35aba
[twitter] accept fxtwitter.com URLs ( #2484 )
2022-04-08 14:32:08 +02:00
Mike Fährmann
1171911dc3
[twitter] add 'syndication' option ( #2354 )
...
to fetch age-restricted content using Twitter's syndication API
2022-04-01 16:56:47 +02:00
Mike Fährmann
2aa47e8382
[twitter] handle Tweets with "softIntervention" entries
...
or other such things where the actual Tweet data is one level deeper
than usual
2022-03-03 02:06:54 +01:00
Mike Fährmann
64bbc7969d
[twitter] warn about age-restricted Tweets ( #2354 )
2022-03-03 02:03:27 +01:00
Mike Fährmann
e778be52bc
[twitter] update query hashes
2022-03-02 23:05:31 +01:00
Mike Fährmann
4385a34e05
[twitter] fix handling of 429 responses ( fixes #2339 )
...
Twitter doesn't return a valid JSON response for 429 errors anymore.
2022-02-28 16:42:55 +01:00
Mike Fährmann
bc0e853d30
combine KeyError & IndexError to common base class LookupError
2022-02-11 00:42:49 +01:00
Mike Fährmann
0f1e7ff319
[twitter] fix extraction ( #2275 )
2022-02-07 23:18:35 +01:00
Mike Fährmann
70e6e1549e
[twitter] provide fallback URLs for card images
...
f2e8aedd74 (commitcomment-64057751)
2022-02-03 23:43:18 +01:00
Mike Fährmann
492436f936
[twitter] add 'warnings' option ( #2258 )
...
disable reporting any non-fatal errors by default
2022-02-02 18:37:19 +01:00
Mike Fährmann
a5163e4c70
[twitter] restore 'logout' functionality ( #1719 )
2022-02-02 18:21:15 +01:00
Mike Fährmann
d33227fc38
[twitter] restore errors for protected timelines etc ( fixes #2237 )
2022-01-30 16:42:13 +01:00
Mike Fährmann
8230f31800
[twitter] update query hashes
2022-01-26 00:49:46 +01:00
Mike Fährmann
c180806cec
[twitter] fix deleted/invalid retweets ( #2225 )
2022-01-25 23:57:13 +01:00
Mike Fährmann
2bf554a896
[twitter] fix several errors ( #2212 , #2216 , #2225 )
...
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
2022-01-25 16:13:22 +01:00
Mike Fährmann
e5242b83bf
[twitter] define directory format for events ( #2109 )
2022-01-24 17:44:17 +01:00
Mike Fährmann
5ed26e1773
[twitter] fix pinned tweets ( #2216 )
...
caused by the changes in dffa440ede
2022-01-23 22:52:57 +01:00
Mike Fährmann
a9f78e6527
[twitter] improve error handling
...
- handle accounts without 'rest_id'
- handle timelines with empty 'instructions'
2022-01-23 18:01:05 +01:00
Mike Fährmann
729b07c1f5
[twitter] simplify
...
- use dict with common GraphQL variables
- reduce 'variables' size with custom JSON encoder instance
- centralise TwitterAPI() creation
2022-01-23 01:44:55 +01:00
Mike Fährmann
9ca8bb2dc0
[twitter] improve error handling
2022-01-22 23:09:45 +01:00
Mike Fährmann
9a221494c3
[twitter] add 'event' extractor ( closes #2109 )
2022-01-22 20:55:50 +01:00
Mike Fährmann
14867dad6b
[twitter] fix unified cards from search results
2022-01-22 20:25:10 +01:00
Mike Fährmann
dffa440ede
[twitter] improve handling of deleted tweets ( #2212 )
2022-01-22 00:41:58 +01:00
Mike Fährmann
54ef874ba4
[twitter] fix retweet filter ( #2212 )
2022-01-21 23:53:59 +01:00
Mike Fährmann
cb43f7731b
[twitter] update to GraphQL API ( #2212 )
...
The old REST API endpoints, which were not used by Twitter since
summer 2021, are going to finally be phased out it seems, with
'/2/timeline/profile/USERID.json' being the first one.
Only Twitter's search doesn't have a GraphQL interface yet.
2022-01-21 23:34:41 +01:00
Mike Fährmann
f2e8aedd74
[twitter] changes to 'cards' option
...
- change default value to 'true'
- only invoke youtube-dl for cards unsupported by gallery
when 'cards' is set to "ytdl"
"cards": true --> only download card images
"cards": "ytdl" --> download card images and
use youtube_dl on otherwise unsupported cards
2022-01-15 22:02:57 +01:00
Mike Fährmann
df2f0c09bb
[twitter] support "image_carousel_website" unified cards
2022-01-13 16:05:52 +01:00
Mike Fährmann
f587458a3c
[twitter] include '4096x4096' as a default image fallback
...
(closes #2107 , closes #1881 )
2021-12-15 23:19:30 +01:00
Mike Fährmann
ab8eea1a24
[twitter] fix extractor for direct image links ( fixes #2030 )
2021-11-16 22:57:46 +01:00
Mike Fährmann
4377f1c284
[twitter] distinguish between fatal & nonfatal errors ( #2020 )
...
only show a warning for nonfatal errors
and do not raise a StopExtraction exception
2021-11-13 22:46:40 +01:00
