Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-22 18:53:21 +01:00
Code Issues Releases Wiki Activity
3,133 Commits 6 Branches 164 Tags 15 MiB
a751afdfb3
Commit Graph

282 Commits

Author SHA1 Message Date
Mike Fährmann
a751afdfb3
[twitter] change some defaults 
- 'retweets' option: true -> false
- 'quoted' option  : true -> false

  i.e. disable downloading tweets from other user's timelines by default

- search directory:
    '["{category}", "Search", "{search}"]' ->
    '["{category}", "{user[name]}"]'

  i.e. change it to the same as other twitter extractors (#1308)
 2021-06-11 21:26:11 +02:00
Mike Fährmann
84d2e64024
combine conditional filenames into filename option (#1394) 2021-06-08 18:00:06 +02:00
Mike Fährmann
07c8adbd8b
[mangadex] implement login with username & password (#1535) 2021-06-08 02:12:57 +02:00
Mike Fährmann
3cbbefd4ed
support 'filter' option for post processors (#1460) 2021-06-04 18:23:32 +02:00
Mike Fährmann
4cf40434d7
initial support for conditional filenames (#1394) 2021-06-04 16:45:32 +02:00
Mike Fährmann
f3d692a378
improve docs for danbooru and e621 passwords (#1593, #1596) 2021-06-02 23:02:23 +02:00
Mike Fährmann
c80b18a477
[weibo] extend 'retweets' option (closes #1542) 
Setting 'retweets' to "original" will use metadata from the
original posts, and not from the retweeted ones.
 2021-05-27 23:09:42 +02:00
Mike Fährmann
74d97e6c6b
[postprocessor:ugoira] add 'repeat-last-frame' option (#1550) 2021-05-26 15:13:11 +02:00
Mike Fährmann
be9547a5f3
[postprocessor:ugoira] use 'image2' demuxer (#1550) 
Ensures exact frame timecodes with no duplicate frames.

Possible issues are the duration the last frame in an Ugoira with variable
frame durations is shown and insufficient timestamp precision of the
underlying file system (e.g. FAT32, ext3; works on ext4, tmpfs, NTFS).
 2021-05-26 14:59:51 +02:00
Mike Fährmann
b8de0671e5
update description for 'instagram.include' 
- add double quotes for "reels"
- add "tagged"
 2021-05-26 14:59:34 +02:00
thatfuckingbird
e6811c7450
[pixiv] implement 'max-posts' option (#1558) 
* implement max-rank for pixiv

* rename to max-posts and make more generic
 2021-05-24 17:49:46 +02:00
Mike Fährmann
b5affc62aa
[twitter] rename 'text-only' to 'text-tweets' (#570) 2021-05-22 21:41:12 +02:00
Mike Fährmann
724ca61f36
[twitter] add 'text-only' option (#570) 2021-05-22 17:01:49 +02:00
Mike Fährmann
1eabfa5c7a
[pillowfort] implement login with username & password (#846) 2021-05-19 02:59:16 +02:00
Mike Fährmann
a7e4917ee1
[pillowfort] add 'inline' option (#846) 
to support images present in a post's 'content',
but not listed in 'media'.

also separates the file hash present at the beginning
of each 'filename' into its own field.
 2021-05-17 03:03:58 +02:00
Mike Fährmann
efa6cc8ec3
[pillowfort] add 'external' option (#846) 
for links to external Twitter posts etc.
 2021-05-17 01:46:42 +02:00
Mike Fährmann
5eeaaee01d
[pixiv] add 'metadata' option (#1551) 2021-05-14 20:30:28 +02:00
Mike Fährmann
0717456b4e
[kemonoparty] add 'metadata' option (closes #1548) 
to fetch creator names with an additional HTTP request
 2021-05-14 19:56:49 +02:00
Mike Fährmann
7ab8374385
add 'parent-skip' option (#1399) 2021-05-13 16:40:04 +02:00
Mike Fährmann
c693db5b1a
add '"skip": "terminate"' option 
Stops not only the current extractor/job,
but all parent extractors/jobs as well.
 2021-05-12 02:22:28 +02:00
Mike Fährmann
36ed1efcfb
[pixiv] rename "noop" value for 'tags' option to "original" 
(#1507)
 2021-05-07 20:41:54 +02:00
Mike Fährmann
e300da1424
add 'output.skip' option 2021-05-04 19:27:18 +02:00
Mike Fährmann
fa519f9202
[pixiv] change 'translated-tags' option (#1507) 
- rename to 'tags'
- use string-values: "japanese", "translated", "noop"
- remove duplicate entries for "translated" tags
 2021-04-29 19:30:43 +02:00
thatfuckingbird
e47952ac14
add extractors for fantia and fanbox (#1459) 
* add extractors for fantia and fanbox

* appease linter

* make docstrings unique

* [fantia] refactor post extraction

* [fantia] capitalize

* [fantia] improve regex pattern

* code style

* capitalize

* [fanbox] use BASE_PATTERN for url regexes

* [fanbox] refactor metadata and post extraction

* [fanbox] improve url base pattern

* [fanbox] accept creator page links ending with /posts

* [fanbox] more tests

* [fantia] improved pagination

* [fanbox] misc. code logic improvements

* [fantia] finish restructuring pagination code

* [fanbox] avoid making a request for each individual post when processing a creator page

* [fanbox] support embedded videos

* [fanbox] fix errors

* [fanbox] document extractor.fanbox.videos

* [fanbox] handle "article" and "entry" post types, all embeds

* [fanbox] fix downloading of embedded fanbox posts
 2021-04-25 19:39:13 +02:00
Mike Fährmann
9514cb8c12
[exhentai] update 'limits' check (#1487) 
Only use 'limits' to set a custom upper bound.
Checking if the actual maximum gets exceeded is not necessary.
 2021-04-23 23:20:45 +02:00
Renan Vedovato Traba
9322c5e43b
[exhentai] restore limit config (#1487) 
This partially reverts commit e9ec91c8
 2021-04-22 21:21:41 +02:00
thatfuckingbird
224b883ff4
[danbooru] add option for extended metadata extraction (#1458) 
* [danbooru] add option for extended metadata extraction

* appease linter

* [danbooru] update docs/configuration.rst

* [danbooru] rename extended-metadata -> metadata
 2021-04-13 23:41:30 +02:00
thatfuckingbird
dff03a6605
[booru] add an option to extract notes (only gelbooru for now) (#1457) 
* [booru] add an option to extract notes (currently implemented only for gelbooru)

* appease linter

* [gelbooru] rename "text" to "body" in note extraction

* add a code comment about reusing return value of _extended_tags
 2021-04-13 23:40:24 +02:00
Mike Fährmann
a86ffb04bb
add 'output.fallback' option 
to enable/disable fallback URLs for -g/--get-urls
 2021-04-12 02:00:41 +02:00
Mike Fährmann
d5aad999dc
[tapas] implement login with username & password (#692) 2021-03-30 01:45:28 +02:00
Mike Fährmann
e9ec91c811
[exhentai] improve image limits check 
- check if current image is the '509 Bandwidth Exceeded' notification
  (https://ehgt.org/g/509.gif or https://exhentai.org/img/509.gif)
- remove 'limits' option
 2021-03-29 19:01:13 +02:00
Mike Fährmann
2846235669
[twitter] allow specifying a custom format for user results 
(#1337)
 2021-03-21 22:26:26 +01:00
Mike Fährmann
b0438c8f99
Revert "[deviantart] extend 'extra' option" 
This reverts commit
5ad2b9c82b,
5c32a7bf58, and
83f465faca.

(#1387, #1356)
 2021-03-19 16:24:23 +01:00
Mike Fährmann
3378b39719
[twitter] implement 'users' option (#1337) 2021-03-16 00:51:05 +01:00
Mike Fährmann
df94182e11
implement 'parent-metadata' option (#1364) 
experimental, might not work as expected, etc.
 2021-03-11 01:10:34 +01:00
Mike Fährmann
83f465faca
[deviantart] refactor 'extra' (#1356) 
- change its expected type to string
- let users specify a list of sources (stash, posts) or 'all'
 2021-03-07 22:23:52 +01:00
Mike Fährmann
5c32a7bf58
[deviantart] allow selecting source for 'extra' (#1356) 
Setting 'extra' to "stash" or "deviations" will only download embedded
sta.sh content or deviations. 'true' still downloads both.
 2021-03-06 21:31:28 +01:00
Mike Fährmann
a677123abb
[instagram] recognize 'reels' as option for 'include' (#1329) 2021-03-06 18:50:19 +01:00
Mike Fährmann
7440d1f112
[pixiv] add 'translated-tags' option (closes #1354) 
(a lot more straight forward than I thought ...)
 2021-03-05 17:18:51 +01:00
Mike Fährmann
6a28b7045c
update 'docs/gallery-dl.conf' 2021-03-03 03:11:48 +01:00
Mike Fährmann
bae874f370
replace 'wait-min/-max' with 'sleep-request' 
on exhentai, idolcomplex, reactor
 2021-03-02 22:55:45 +01:00
Mike Fährmann
1a38fae785
add option to use different youtube-dl modules (fixes #1330) 
by setting the 'downloader.ytdl.module' value. For example

{
    "downloader": {
        "ytdl": {
            "module": "yt_dlp"
        }
    }
}

or '-o module=yt_dlp'
 2021-03-01 03:10:42 +01:00
Mike Fährmann
e58039358d
[mangadex] use 'api.mangadex.org' as default API server 
The caching issues seem to be gone.
(#1290, #1310)
 2021-02-28 01:31:50 +01:00
Mike Fährmann
29ea54dc41
[patreon] use '"browser": "firefox"' by default (#1117) 2021-02-27 16:26:42 +01:00
Mike Fährmann
61fbbd2dae
[exhentai] rename metadata fields to match API results (#1325) 
- gallery_id    -> gid
- gallery_token -> token
- title_jp      -> title_jpn
- visible       -> expunged
- gallery_size  -> filesize
- count         -> filecount

Also changes the function of the 'metadata' option.
It is now boolean and causes extra data fields from the API to be added
instead of completely replacing the data from HTML when activated.
 2021-02-26 18:21:04 +01:00
Mike Fährmann
5d69e437d0
[twitter] add option to download all media from a conversation 
(fixes #1319)
 2021-02-26 13:50:46 +01:00
Mike Fährmann
cf5fa75d4c
add 'browser' option (#1117) 
- change default user agent to Firefox ESR 78 on Windows 10
- remove 'ciphers' option
 2021-02-26 13:41:27 +01:00
Mike Fährmann
e5735361ed
[exhentai] add 'metadata' option (#1325) 
to select between gallery metadata from 'api' or 'html'
 2021-02-23 00:23:42 +01:00
Mike Fährmann
5ad2b9c82b
[deviantart] extend 'extra' option 
also download from embedded DeviantArt posts
 2021-02-21 19:27:04 +01:00
Mike Fährmann
560277394e
[downloader:http] add 'headers' option (#1322) 2021-02-21 19:13:39 +01:00