1
0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-23 19:22:32 +01:00
Commit Graph

30 Commits

Author SHA1 Message Date
Mike Fährmann
d26da3b9e5
add pre-generated 'pattern' for supported BaseExtractor sites 2022-05-09 22:20:09 +02:00
Mike Fährmann
9377543162
[mastodon] add 'following' extractor (#1891) 2021-09-26 00:12:34 +02:00
Mike Fährmann
2c2932973c
[mastodon] support specifying accounts by ID
Same as a3b473bd for Twitter

Instead of just
https://instance.tld/@user

it is now also possible to refer to that account with
https://instance.tld/users/user
https://instance.tld/@id:12345
https://instance.tld/users/id:12345
2021-09-25 20:28:16 +02:00
Mike Fährmann
312a28e78a
[mastodon] add 'replies' option (#1669) 2021-07-07 00:59:02 +02:00
Mike Fährmann
513c491cea
[mastodon] reset 'params' after first pagination iteration
otherwise query parameters in 'params' get specified twice the second
time around - once from the 'links["next"]' URL and once from 'params'
itself.
2021-07-07 00:07:18 +02:00
Mike Fährmann
a1f5b78039
[mastodon] add 'reblogs' option (#1669) 2021-07-06 23:27:32 +02:00
Mike Fährmann
93d356712c
[mastodon] implement 'text-posts' option (#1569)
similar to Twitter's 'text-tweets'
2021-07-02 22:12:41 +02:00
Mike Fährmann
1d145a6186
[mastodon] use cache for OAuth tokens (#616) 2021-01-31 01:38:23 +01:00
Mike Fährmann
fa33f13453
[mastodon] update
- inherit from BaseExtractor
- remove custom generate_extractors() and config()
- improve layout of MastodonAPI internals
2021-01-27 23:49:01 +01:00
Mike Fährmann
968d3e8465
remove '&' from URL patterns
'/?&#' -> '/?#' and '?&#' -> '?#'

According to https://www.ietf.org/rfc/rfc3986.txt, URLs are
"organized hierarchically" by using "the slash ("/"), question
mark ("?"), and number sign ("#") characters to delimit components"
2020-10-22 23:31:25 +02:00
Mike Fährmann
c51fbd72ba
update extractor test results 2020-07-13 22:57:48 +02:00
Mike Fährmann
53cc498d9c
improve config lookup when there are multiple possible locations
This specifically applies to all Mastodon extractors and all
extractors with a 'basecategory', i.e. 'booru', 'foolslide', etc.

Values inside those general config locations wouldn't be recognized
when a value with the same was set on the 'extractor' level.

For example 'extractor.mastodon.directory' should be used over
'extractor.directory' when both are set, but this was impossible
with the previous implementation.

(fixes #843)
2020-06-21 00:07:10 +02:00
Mike Fährmann
8c857052d7
[mastodon] ignore toots without media attachments 2020-06-17 20:21:28 +02:00
Mike Fährmann
de045d39b2
[mastodon] add 'date' metadata field (#839) 2020-06-17 19:22:28 +02:00
Mike Fährmann
3201fe3521
add global SENTINEL object 2020-05-19 22:32:53 +02:00
Mike Fährmann
12ff23b6cc
[mastodon] improve account searches (fixes #704)
Searching for just the username ("@NAME") can produce multiple
unrelated results, so we now search for username + mastodon instance
("@NAME@INSTANCE")
2020-04-23 20:23:10 +02:00
Mike Fährmann
d6facdee7b
[mastodon] add tests (#701) 2020-04-22 21:10:34 +02:00
Mike Fährmann
e749402191
[mastodon] fix pagination (#701) 2020-04-22 17:58:55 +02:00
Mike Fährmann
88fca0a172
[mastodon] update OAuth credentials for pawoo.net (#665) 2020-04-06 00:50:30 +02:00
Mike Fährmann
4ae8a25567
[mastodon] use 'combine_dict()' to combine extractor info dicts 2020-04-05 21:45:00 +02:00
Mike Fährmann
220c06b86e
[mastodon] handle rate limits 2020-04-05 21:44:00 +02:00
Mike Fährmann
2587296deb
[mastodon] add access tokens for mastodon.social and baraag.net
(closes #665)
2020-04-02 22:34:32 +02:00
Mike Fährmann
ce54b8c04c
let extractors opt-out of cookie option usage
useful to avoid sending unnecessary cookies when all authentication
is done through OAuth tokens
2020-01-01 21:12:37 +01:00
Mike Fährmann
f5604492c3
update interface of config functions 2019-11-24 00:42:28 +01:00
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00
Mike Fährmann
6284731107
simplify extractor constants
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
2019-02-08 13:45:40 +01:00
Mike Fährmann
347398f692
fix various tests 2019-02-04 14:40:21 +01:00
Mike Fährmann
ccb95d0ba4
[mastodon] changes/improvements based on foolfuuka/-slide 2019-02-04 13:13:58 +01:00
Mike Fährmann
4f49fdf065
[mastodon] various improvements and fixes (#144)
- allow instances to specify their own 'category'
- improve config lookup:
  - first look into extractor.<category>.*
  - and afterwards look into extractor.mastodon.<instance>.*
- add a default entry for pawoo.net in a way that actually works
- add an 'instance' keyword and turn 'tags' into a usable list
2019-01-27 20:43:33 +01:00
Mike Fährmann
b8fed34548
add generalized extractors for Mastodon instances (#144)
Extractors for Mastodon instances can now be dynamically generated,
based on the instance names in the 'extractor.mastodon.*' config path.

Example:
{
    "extractor": {
        "mastodon": {
            "pawoo.net": { ... },
            "mastodon.xyz": { ... },
            "tabletop.social": { ... },
            ...
        }
    }
}

Each entry requires an 'access-token' value, which can be generated with
'gallery-dl oauth:mastodon:<instance URL>'.
An 'access-token' (as well as a 'client-id' and 'client-secret') for
pawoo.net is always available, but can be overwritten as necessary.
2019-01-19 14:28:59 +01:00