1
0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-23 11:12:40 +01:00
Commit Graph

172 Commits

Author SHA1 Message Date
Mike Fährmann
be52069cbc
update CHANGELOG and docs/supportedsites 2018-11-28 14:53:27 +01:00
Mike Fährmann
c47482b110
smaller changes, missing docs, etc.
- make 'netrc' extractor-specific
- rename 'downloader.enable' to 'enabled'
- document 'downloader.ytdl.format'
- consistent newlines in configuration.rst
2018-11-16 18:18:07 +01:00
Mike Fährmann
8ef84a6823
add option to enable/disable specific downloader modules
... and write URLs with no (active) downloader to unsupported-file
2018-11-13 18:06:36 +01:00
Mike Fährmann
2fa28a2609
update default user-agent string (closes #122) 2018-11-11 10:07:10 +01:00
Mike Fährmann
655549df7c
[downloader:ytdl] add several options
The "default" downloader options (rate, retries, timeout, verify) are
mapped to corresponding youtube-dl options.

downloader.ytdl.logging tells the downloader to pass youtube-dl's output
to a Logger object.

downloader.ytdl.raw-options allows to pass arbitrary options to the
YoutubeDL constructor.
2018-10-20 18:26:49 +02:00
Mike Fährmann
6ed629f2b6
allow specifying number of skips before abort/exit (closes #115)
In addition to 'abort' and 'exit', it is now possible to specify
'abort:N' and 'exit:N' (where N is any integer) as value for 'skip'
to abort/exit after consecutively skipping N downloads.
2018-10-13 17:21:55 +02:00
Mike Fährmann
d8492df51b
[deviantart] extend functionality of 'original' option 2018-10-11 22:11:49 +02:00
Mike Fährmann
392a081657
update 'docs/configuration.rst' 2018-10-08 23:09:01 +02:00
Mike Fährmann
f8b3b00249
[twitter] add experimental 'videos' option (#99)
Enabling this option will detect videos in tweets and output them as
"unsupported" URLs, so that these can then be downloaded with youtube-dl

There are a lot of improvements to be made to the current
implementation, but it works and does what it is supposed to, even if
inefficient as can be ...
2018-09-30 21:52:23 +02:00
Mike Fährmann
2cf3f53839
[yuki] add thread extractor (closes #111) 2018-09-28 12:47:32 +02:00
Mike Fährmann
17611bfec0
update build_supportedsites.py script 2018-09-28 12:43:19 +02:00
Mike Fährmann
09d2f3e5e7
[postprocessor:ugoira] improve libx264 detection 2018-09-24 13:12:54 +02:00
Mike Fährmann
c402cc4047
[hentaifoundry] add 'popular' and 'recent' extractors
for "Popular Pictures" and "Recent Pictures" listings
2018-09-24 13:11:18 +02:00
Mike Fährmann
a5fc311dfa
[hentaifoundry] add 'favorite' extractor 2018-09-22 21:23:29 +02:00
Mike Fährmann
55f5c87160
[postprocessor:ugoira] add 'libx264-prevent-odd' option
A rather crude workaround for "width/height not divisible by 2"
errors when using libx264.
2018-09-21 20:01:14 +02:00
Mike Fährmann
006f75b538
[hentaifoundry] rewrite + more metadata
- extract width, height, artist per image
- improve pattern regex
- better extensibility for other listings
2018-09-21 11:23:51 +02:00
Mike Fährmann
6ea9a78588
[wallhaven] add login capabilities
Being logged in is required to access NSFW wallpapers.
2018-09-19 21:04:01 +02:00
Mike Fährmann
c9290d8212
[wallhaven] add wallpaper and search extractors
todo:
- login support to gain access to NSFW wallpapers
- extractors for tag-, similar-, latest-listings
- skip() support
2018-09-17 21:26:13 +02:00
Mike Fährmann
22ab509a70
[bobx] rename "model" to "idol" extractor 2018-09-14 18:11:36 +02:00
Mike Fährmann
69fd61ea86
[bobx] add gallery and model extractors 2018-09-13 20:13:12 +02:00
Mike Fährmann
7742cf8601
[tumblr] change 'reblogs' option (#103)
- rename "deleted" to "same-blog"
- change test for deleted original post to test if
  original post owner has the same UUID (full blog name) as the one
  being downloaded from
- add 'blog[uuid]' metadata to allow comparison with
  'reblogged_from_uuid'
2018-09-10 15:40:25 +02:00
Mike Fährmann
f1695567e8
adjust values in template config file
(#104)
2018-09-09 14:10:55 +02:00
Mike Fährmann
a666ddd16b
[tumblr] extend 'reblogs' functionality (#103)
Setting 'reblogs' to "deleted" will check if the parent post of a
reblog has been deleted and download its media content if that is the
case, otherwise it will be skipped.

This is a rather costly operation (1 API request per reblogged post)
and should therefore be used with care.
2018-09-07 19:13:52 +02:00
Mike Fährmann
c9b8e6aefc
[reddit] fix submission-ID parsing (#104)
Uppercase characters caused a ValueError exception
2018-09-07 18:27:54 +02:00
Mike Fährmann
3ecea4cf36
[hentaicafe] add chapter and manga extractors (#101) 2018-09-05 21:08:40 +02:00
Mike Fährmann
75862715ac
[behance] add user extractor 2018-08-31 17:42:09 +02:00
Mike Fährmann
6ecb36d88c
[postprocessor:ugoira] add 'ffmpeg-output' option 2018-08-31 17:37:35 +02:00
Mike Fährmann
ab2bfaeb46
[ngomik] add replacement for 'subapics'
http://subapics.com/ got discontinued and replaced by http://ngomik.in/.

ngomik.in is still displaying a link to the "old site" showing a big
"Account Suspended" sign.
2018-08-23 15:29:53 +02:00
Mike Fährmann
e9dd2eff1d
[twitter] add extractor for media-tweet timelines (#96)
For example "https://twitter.com/PicturesEarth/media".
They are different from normal timelines in that they do not contain
any (re)tweets from other users and feature all media the user ever
posted, including responses to other tweets.
2018-08-19 20:46:12 +02:00
Mike Fährmann
9b1c39032c
[twitter] changes and improvements
- rename User- to TimelineExtractor
- rename 'userid' to 'user_id' to conform to the other ..._id values
- adjust archive_fmt to deal with retweets
- emulate browser behavior for API calls
2018-08-18 23:04:45 +02:00
Mike Fährmann
10365394d7
[twitter] add support for user-timelines (closes #96)
also adds a 'retweets' option to filter retweeted content
2018-08-17 20:04:11 +02:00
Mike Fährmann
63fa0b2006
[pinterest] add extractors for related pins
Related pins can not be accessed by adding a "#related" fragment
to the end of a Pinterest URL, for example:
- https://www.pinterest.com/pin/858146903966145189/#related
- https://www.pinterest.com/g1952849/test-/#related

There are no explicit real URLs for related pins,
using an option to enable them results in "clunky" code,
and a custom "related:<URL>" scheme doesn't feel right either.
2018-08-15 21:49:45 +02:00
Mike Fährmann
3c90df6635
[piczel] add user, folder and image extractors 2018-08-08 10:53:01 +02:00
Mike Fährmann
b164231bca
[sankaku] increase default values for 'wait-min/-max' 2018-08-03 17:06:51 +02:00
Mike Fährmann
df082e923c
[behance] add gallery extractor (#95) 2018-08-01 21:46:55 +02:00
Mike Fährmann
5f27cfeff6
[deviantart] remove prefer-public option
All API requests now always use a public token and only switch to
a private token for pagination results if `refresh-token` is set
and less deviations than requested were returned.
2018-07-26 19:43:46 +02:00
Mike Fährmann
212130b048
[deviantart] improve public-private token switching
- rename option to `prefer-public`
- now also works for galleries with less than 24 items
2018-07-25 12:52:36 +02:00
Mike Fährmann
886d662582
[deviantart] add option to minimize refresh-token usage
Always trying with a public token first and repeating the API request
with a private token if deviations are missing doesn't quite work for
galleries and folders with less than 25 items, so its an option and
not the default.
2018-07-24 21:44:57 +02:00
Mike Fährmann
54a0d72dc8
[postprocessor:ugoira] improve frame rate handling
By default FFmpeg assumes a 25 FPS input frame rate, leading to dropped
frames if the source requires a higher frame rate than that.

This commit adds a `framerate` option (default "auto"), which allows to
automatically assign a (more or less) fitting frame rate based on
delays between ugoira frames and avoids dropped frames.
2018-07-23 21:33:20 +02:00
Mike Fährmann
84854fcad7
[myportfolio] add user and gallery extractors (#95) 2018-07-19 18:56:45 +02:00
Mike Fährmann
c9f70e0a19
[paheal] use HTTPS 2018-07-17 21:25:03 +02:00
Mike Fährmann
ff436692bf
["deviantart] add 'journals' option 2018-07-16 18:14:41 +02:00
Mike Fährmann
00032b828c
[deviantart] add 'wait-min' option 2018-07-14 11:52:21 +02:00
Mike Fährmann
269dc2bbd5
[sankaku] add 'tags' option (#94) 2018-07-14 09:53:01 +02:00
Mike Fährmann
1d43cbbf52
[gelbooru] tag-splitting for non-api mode 2018-07-06 15:24:19 +02:00
Mike Fährmann
c20c0a4820
[safebooru] add pool extractor 2018-07-04 12:24:57 +02:00
Mike Fährmann
4a57509392
generalize tag-splitting option (#92)
- extend functionality to other booru sites:
  - http://behoimi.org/
  - https://konachan.com/
  - https://e621.net/
  - https://rule34.xxx/
  - https://safebooru.org/
  - https://yande.re/
2018-07-04 12:21:16 +02:00
Mike Fährmann
188e956c4e
[imagefap] use HTTPS + update test results 2018-06-30 19:40:46 +02:00
Mike Fährmann
87853538b4
[yandere] add option to split tags by type (#92) 2018-06-29 19:38:53 +02:00
Mike Fährmann
4eb94aca17
[postprocessor:ugoira] pass '-f' if not present 2018-06-22 13:26:17 +02:00