Mike Fährmann
c1cce4a80b
[twitter] extend 'conversations' option ( #4211 )
2023-06-24 21:34:34 +02:00
Mike Fährmann
f50b81d3e9
[docs] add novel-related keys to 'pixiv.include' ( #4111 )
2023-06-19 15:01:53 +02:00
Mike Fährmann
deff3b434d
[vipergirls] implement login support ( #4166 )
2023-06-13 21:05:09 +02:00
Mike Fährmann
54cf1fa3e7
[twitter] use GraphQL search endpoint ( #3942 )
...
for guest users; selectable with 'search-endpoint' option.
adapted from 9c7b888ffa
2023-06-01 21:37:31 +02:00
Mike Fährmann
0cf7282fa0
[pixiv] add 'full-series' option for novels ( #4111 )
2023-06-01 13:07:20 +02:00
Mike Fährmann
28f6487c64
[instagram] add 'metadata' option ( #3107 )
2023-05-27 15:51:13 +02:00
Mike Fährmann
f3cca50b9e
[mangadex] update links to API docs
2023-05-24 12:32:13 +02:00
Mike Fährmann
65a9f4b124
merge #3950 : [misskey] add 'favorite' extractor
2023-05-23 22:18:32 +02:00
Mike Fährmann
c76f0f3a1b
[misskey] update
...
- rename to 'MisskeyFavoriteExtractor'
- add 'access-token' option to docs
- add test URLs for other instances
- simplify 'pattern'
2023-05-23 22:17:17 +02:00
Mike Fährmann
3fca455b82
[pixiv] add 'embeds' option ( #1241 )
2023-05-23 12:14:06 +02:00
Mike Fährmann
e6f55d1555
[imagechest] add API support and 'access-token' option ( #4065 )
2023-05-20 17:00:27 +02:00
Mike Fährmann
77abcf5ab3
[gofile] automatically fetch 'website-token' by default
...
the hardcoded token changed yet again
2023-05-20 17:00:27 +02:00
Mike Fährmann
a83983c651
[instagram] add 'order-posts' option ( #4017 , #3993 )
2023-05-18 22:50:12 +02:00
Mike Fährmann
d680623db3
[instagram] add 'order-files' option ( #4017 , #3993 )
2023-05-18 22:34:33 +02:00
Mike Fährmann
5037013e2b
[gofile] update 'website-token' ( #4056 )
2023-05-15 14:59:44 +02:00
Mike Fährmann
a14b63d941
support selecting a domain for '--cookies-from-browser'
...
for example 'gallery-dl --cookies-from-browser firefox/twitter.com'
2023-05-06 15:09:49 +02:00
Mike Fährmann
3ca5dac8b6
extend 'cookies-update' functionality
...
Allow writing cookies to a different file than a given cookies.txt,
making it possible to export cookies imported with --cookies-from-browser
To convert browser cookies to cookies.txt format:
gallery-dl --cookies-fr chromium \
-o cookies-update=cookies.txt \
--no-download \
http://example.org/file.jpg
2023-05-04 15:10:47 +02:00
Mike Fährmann
79f47f98dd
[nana] remove module
...
permanently gone since 2023-03-13
2023-05-03 18:05:53 +02:00
Mike Fährmann
c182094ebf
merge #3748 : [downloader:http] add 'consume-content' option
2023-04-26 23:03:18 +02:00
Mike Fährmann
b81ce381f0
fix backticks for 'extractor.redgifs.format' docs
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1518331927
2023-04-21 23:12:01 +02:00
Mike Fährmann
aaf58a1259
[imgur] document 'client-id' option ( #3937 )
2023-04-21 15:08:50 +02:00
Mike Fährmann
0a7eee3ee0
[deviantart] add 'public' option
2023-04-08 23:04:34 +02:00
Mike Fährmann
75666cf9c3
[danbooru] reduce API requests for fetching extended 'metadata'
...
Instead of using one additional API request per post object (N+1),
this requires only one request per 200-post batch.
2023-04-02 20:11:52 +02:00
Mike Fährmann
00f0233b28
[postprocessor:metadata] add 'skip' option ( #3786 )
2023-03-17 23:30:11 +01:00
Mike Fährmann
a1ca2404f9
add 'globals' instead of overwriting the default ( #3773 )
2023-03-16 18:37:00 +01:00
ClosedPort22
fcaeaf539c
[downloader:http] handle exceptions while consuming content
2023-03-11 21:36:37 +08:00
ClosedPort22
df77271438
[downloader:http] add 'consume-content' option
...
* fix connection not being released when the response is neither
successful nor retried
* add the ability to consume the HTTP response body instead of closing
the connection
reference:
https://docs.python-requests.org/en/latest/user/advanced/#body-content-workflow
2023-03-09 21:07:10 +08:00
Mike Fährmann
da9840a39d
[reddit] update 'videos' option ( #3712 )
...
- add 'dash' to directly extract DASH manifest URLs
(was default behavior since a7c79531a7c79531
2023-03-06 12:18:25 +01:00
enduser420
e1867cf5eb
[misskey] add 'renotes' and 'replies' options
2023-03-02 19:56:19 +05:30
Mike Fährmann
0d142e403c
[szurubooru] add 'tag' and 'post' extractors ( #3583 , #3713 )
2023-03-01 18:47:10 +01:00
Mike Fährmann
d788e6c60c
implement 'globals' option
2023-02-28 18:18:55 +01:00
Mike Fährmann
e480a93337
add 'output.stdout', '.stdin', and '.stderr' options
...
(#1621 , #2152 , #2529 )
Allow setting custom input/output encodings and options
without having to rely on Python's defaults.
2023-02-26 14:56:19 +01:00
Mike Fährmann
b7337d810e
[postprocessor:metadata] add 'sort' and 'separators' options
2023-02-07 18:28:14 +01:00
Mike Fährmann
762a68996b
implement 'archive-pragma' option
2023-02-05 17:00:31 +01:00
Mike Fährmann
bbf0911a46
[e621] implement 'notes' and 'pools' metadata extraction
...
(#3425 )
2023-02-04 21:20:38 +01:00
Mike Fährmann
925b467496
split e621 from danbooru module ( #3425 )
2023-02-03 19:24:31 +01:00
Mike Fährmann
1ae48a54f8
[twitter] add 'transform' option
2023-02-02 22:01:36 +01:00
Mike Fährmann
78d3960a31
[postprocessor:exec] implement archive options ( #3584 )
2023-02-01 13:57:16 +01:00
Mike Fährmann
4572c122d8
document 'extractor.module-sources'
2023-01-30 23:32:36 +01:00
Mike Fährmann
f58215705a
add '-O/--postprocessor-option' command-line option ( #3565 )
2023-01-26 14:59:24 +01:00
ClosedPort22
3b32671fbd
[deviantart] add extractor for status updates
...
extract user status updates using the '/user/statuses/' endpoint
2023-01-19 11:30:48 +08:00
Mike Fährmann
4e86aaa4db
document 'extractor.headers' and 'extractor.ciphers'
2023-01-16 14:49:56 +01:00
Mike Fährmann
c881548a27
add 'extractor.retry-codes' option ( #3313 )
...
do not retry 429 and 430 by default
2023-01-14 17:25:30 +01:00
Mike Fährmann
512abeb4ae
[booru] add 'url' option
2023-01-13 16:32:32 +01:00
Mike Fährmann
c87bd1a752
[danbooru] extend 'metadata' option
...
make it possible to specify a custom list of metadata includes
2023-01-13 16:20:01 +01:00
Mike Fährmann
c0d7d2be35
[downloader:http] add 'validate' option
2023-01-11 15:37:40 +01:00
Mike Fährmann
cb9d100bac
update Authentication support in docs
...
https://github.com/mikf/gallery-dl/pull/3075#issuecomment-1374723412
2023-01-08 16:57:31 +01:00
Mike Fährmann
362cd6991b
[pixiv] implement 'metadata-bookmark' option ( #3417 )
2023-01-07 23:19:43 +01:00
Mike Fährmann
0895e6afee
merge #3462 : [docs] Update links and fix field typo
2023-01-07 15:22:03 +01:00
Mike Fährmann
4b9bb632f2
[docs] update links
2023-01-07 15:21:40 +01:00
Mike Fährmann
ac78712618
merge #3496 : [docs] update OAuth guide in README.rst
2023-01-06 13:36:23 +01:00
Mike Fährmann
2f31d21509
merge #3455 : [twitter] apply tweet type checks before uniqueness check
2023-01-06 13:32:50 +01:00
Mike Fährmann
64435cc37e
[twitter] update docs
2023-01-06 13:32:08 +01:00
Mike Fährmann
b125fcafc5
update docs/configuration.rst
...
- document metadata.indent
- add concrete mappings to object types
- fix lists for code examples
- fix typos
2023-01-06 13:07:33 +01:00
ClosedPort22
c94692f39b
[docs] document 'access-token' for Mastodon instances
2023-01-06 13:20:41 +08:00
Mike Fährmann
4fd6da474f
merge #3473 : [twitter] fix crash when using 'expand' and 'syndication'
2023-01-05 14:19:47 +01:00
Mike Fährmann
818f6fa71f
merge #3453 : [twitter] document #1396 and #2712
2023-01-05 14:18:21 +01:00
ClosedPort22
455481ad76
[twitter] improve documentation
...
This includes:
- The automatic expansion of conversations (#2712 )
- How to retrieve age-restricted content from search results (#1396 )
2023-01-05 12:14:42 +08:00
Mike Fährmann
6933727b45
merge #3483 : [twitter] implement 'syndication=extended'
2023-01-04 17:36:17 +01:00
Mike Fährmann
9116398c1c
[pinterest] add 'domain' option ( #3484 )
...
use input URL domain by default
2023-01-04 17:20:14 +01:00
ClosedPort22
6853b14be3
[twitter] apply suggestions from code review
...
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
2023-01-02 21:03:01 +08:00
Mike Fährmann
4611237f8c
merge #3457 : [danbooru] extract uploader metadata (if option is set)
2023-01-01 21:20:39 +01:00
Mike Fährmann
0ed5986062
document custom 'output.mode' ( #2529 )
2022-12-30 17:14:42 +01:00
ClosedPort22
7c8eab8d52
[twitter] implement 'syndication=extended'
...
to be able to fetch extended user metadata
2022-12-30 20:48:41 +08:00
ClosedPort22
be3286206a
[twitter] assume 'conversation_id' when using syndication
...
not possible to expand replies at the momemt
2022-12-30 13:57:37 +08:00
ClosedPort22
ce8dbb1ccc
[twitter] fix crash when using 'expand' and 'syndication'
...
caused by KeyError: 'conversation_id_str'
2022-12-30 12:45:44 +08:00
Mike Fährmann
d651d45239
implement specifying ranges in slice notation ( #918 , #2865 )
...
e.g.
- '1:101' or ':101' or ':101:' for files 1 to 100
- '1::2' or '::2' for every second file
- '1:101:5' or ':101:5' for files 1, 6, 11, ..., 91, 96
(the second argument specifies the first index NOT included)
2022-12-27 18:21:12 +01:00
pi_allen
64902f518e
[docs] Update links and fix field typo
2022-12-26 07:46:41 +01:00
ClosedPort22
5d5ea58887
[twitter] update docs for 'quoted' & 'unique'
2022-12-25 15:59:26 +08:00
blankie
f82ee93676
[danbooru] extract uploader metadata (if metadata is set)
2022-12-25 11:00:59 +07:00
Mike Fährmann
1800bd7d14
allow '*-filter' options to be a list of expressions
2022-12-23 22:20:21 +01:00
Mike Fährmann
73ab5d84c0
update docs/configuration.rst
2022-12-20 17:30:46 +01:00
ClosedPort22
dd4a4a3fa6
[e621] softcode the pagination threshold
2022-12-17 21:06:47 +08:00
ClosedPort22
d0ad6d0e67
[e621] implement manual pagination mode
2022-12-16 16:15:36 +08:00
Mike Fährmann
6b6f886dcf
[bunkr] update domain ( #3391 )
...
and improve bunkr/app.bunkr handling
2022-12-11 23:18:01 +01:00
Mike Fährmann
dfe7b23579
support Firefox containers for --cookies-from-browser ( #3346 )
2022-12-09 19:43:55 +01:00
Mike Fährmann
80102fa367
[downloader:http] add 'retry-codes' option ( #3313 )
2022-12-01 11:08:23 +01:00
ClosedPort22
b14b33f19e
Implement version-metadata option ( #3201 )
2022-11-27 16:09:42 +01:00
Mike Fährmann
6cb12f513b
[nitter] support quoted Tweets
...
- distinguish between regular and quoted Tweets and media
- add 'quoted' option and metadata field
2022-11-26 11:23:03 +01:00
Mike Fährmann
a41d093bb1
[nitter] add 'retweets' option ( #3278 )
2022-11-25 19:53:28 +01:00
Mike Fährmann
e31d12139c
[nitter] add 'videos' option ( #3279 )
...
with the same semantics as for twitter
2022-11-24 22:56:01 +01:00
Mike Fährmann
501d9bccfe
[artstation] add 'max-posts' option ( #3270 )
2022-11-23 22:00:18 +01:00
Mike Fährmann
b1ad6f2289
[artstation] add 'pro-first' option ( #3273 )
2022-11-23 21:45:20 +01:00
Mike Fährmann
6b97dcf2e0
[postprocessor:metadata] add 'private' option
2022-11-20 15:27:36 +01:00
Mike Fährmann
6379157543
[instagram] use REST API by default
...
regardless of logged in status
2022-11-17 17:15:38 +01:00
enduser420
7897f68225
[wallhaven] update 'user' extractor ( #3226 )
...
* [wallhaven] update 'user' extractor
* [wallhaven] update 'configuration.rst'
add 'extractor.wallhaven.include' entry
* [wallhaven] add 'wallhaven.include' in gallery-dl.conf
2022-11-15 17:35:52 +01:00
Mike Fährmann
9f06e79868
implement '"user-agent": "browser"' ( #2636 )
2022-11-13 19:17:39 +01:00
Mike Fährmann
8d543b040d
update docs
...
- document '"mode": "jsonl"' for metadata pp
- remove 'username' and 'password' from 'instagram'
- add 'ytdl' example (#3180 )
2022-11-09 22:17:08 +01:00
Mike Fährmann
171262c1b6
[instagram] remove login support
...
broken feature that I cannot get to work anymore
2022-11-08 21:48:31 +01:00
Mike Fährmann
895f36e53b
[postprocessor:metadata] add 'open' and 'encoding' options
2022-11-07 15:37:22 +01:00
Mike Fährmann
870e6a48a0
implement 'http-metadata' option
...
or at least attempt to.
2022-11-05 18:29:29 +01:00
Mike Fährmann
bca9f965e5
[downloader:http] add 'chunk-size' option ( #3143 )
...
and double the previous default from 16384 (2**14) to 32768 (2**15)
2022-11-02 16:50:26 +01:00
Mike Fährmann
c12a97bcde
[postprocessor] add 'post-after' event ( #3117 )
2022-10-31 14:35:48 +01:00
Mike Fährmann
7c6af27eb8
[tumblr] add 'fallback-*' options ( #2957 )
...
specifically 'fallback-delay' and 'fallback-retries'
and change default number of retries to 2 (down from 3)
2022-10-26 13:59:09 +02:00
Mike Fährmann
cf86f68864
[instagram] add 'avatar' extractor ( #929 , #1097 , #2992 )
2022-10-19 10:58:42 +02:00
Mike Fährmann
52d1eb928d
[pixiv] extend 'metadata' option ( #3057 )
...
make it usable for all 'pixiv' extractors
2022-10-16 15:32:31 +02:00
Mike Fährmann
0714274f1f
[instagram] remove 'channel' extractor
2022-10-15 21:13:58 +02:00
Mike Fährmann
f1f89b2436
[tumblr] add 'offset' option
2022-10-11 10:54:23 +02:00
Mike Fährmann
15a4d7696f
[docs] update 'adjust-extensions' explanation ( #3031 )
2022-10-11 10:48:28 +02:00
Mike Fährmann
7f30a0d7a7
add 'path-extended' option ( #3021 )
2022-10-08 15:41:39 +02:00
