Mark Henrick
923e1bb714
[docs] Fix inconsistency about which sites have optional authentication ( #359 )
...
* [docs] Fix inconsistency about which sites have optional authentication
* update authentication docs
2019-07-29 18:22:31 +02:00
Mike Fährmann
a90280f4e7
[postprocessor:zip] add 'mode' option ( #355 )
2019-07-29 16:51:26 +02:00
Mike Fährmann
dc73d02d87
[exhentai] always use e-hentai.org as domain + set nw cookie
2019-07-28 10:54:17 +02:00
Mike Fährmann
279db2c5b2
[vsco] add collection & image extractor + video support ( #331 )
2019-07-26 19:06:15 +02:00
Mike Fährmann
547ea71463
[downloader.ytdl] add 'forward-cookies' option ( #352 )
...
The "long" name is necessary because just calling it 'cookies' would
clash with how the lookup for '--cookies' is implemented.
2019-07-24 21:19:11 +02:00
Mike Fährmann
b1bea8aaeb
add 'restrict-filenames' option ( #348 )
2019-07-23 17:41:24 +02:00
Mike Fährmann
60cf40380a
[vsco] add user extractor ( #331 )
2019-07-23 16:23:11 +02:00
Mike Fährmann
3fe5ccdfa6
[adultempire] add gallery extractor ( closes #340 )
2019-07-21 22:29:57 +02:00
Mike Fährmann
21991acc49
add 'ciphers' option; update default User-Agent
2019-07-19 17:14:40 +02:00
Mike Fährmann
feb98cf196
[twitter] improve 'content' formatting; add option ( #338 )
...
- include emoticons
- leave newlines intact
- remove pic.twitter.com/ links at the end
2019-07-17 16:02:51 +02:00
Mike Fährmann
8d1ae9b715
[tumblr] enable date-min/-max/-format options ( #337 )
2019-07-17 14:36:41 +02:00
Mike Fährmann
fb875d1ab8
add warning about NSFW sites in supportedsites.rst ( #335 )
2019-07-15 21:44:34 +02:00
Mike Fährmann
16c582aaf9
implement 'mtime' post-processor ( #332 )
...
This can set a file's modification time according to a UNIX timestamp
or a datetime object from its metadata.
2019-07-14 22:39:17 +02:00
Mike Fährmann
62097284fe
add 'download' option ( #220 )
2019-07-14 18:48:18 +02:00
Mike Fährmann
2ff73873f0
[erolord] add gallery extractor ( closes #326 )
2019-07-04 20:28:04 +02:00
Mike Fährmann
b4da8c5a97
[sexcom] add extractor for related pins ( #325 )
2019-07-03 21:04:23 +02:00
Mike Fährmann
69205df68d
allow '-1' for infinite retries ( #300 )
2019-06-30 23:10:47 +02:00
Mike Fährmann
f7b5c4c3e7
use values of 'retries' options correctly
...
The RE-tries option now specifies exactly that: the maximum number a
failed HTTP request is re-tried. For example a value of 2 will now
correctly stop after 3 attempts: the initial one + 2 re-tries.
The maximum wait-time now also caps at 30min and increases exponentially
for both extractor.request() and downloader.http.download().
2019-06-30 23:10:18 +02:00
Mike Fährmann
f2000a69aa
implement 'image-unique' and 'chapter-unique' options ( #303 )
...
The default value for both is 'false', i.e. duplicate URLs are NOT
ignored.
The previous behavior was to always ignore duplicate URLs to make
'--abort-on-skip' work properly when new images where added to the
beginning of a collection while gallery-dl is running.
2019-06-29 22:50:17 +02:00
Mike Fährmann
40da44b17f
Merge branch 'v1.9.0'
2019-06-29 15:39:52 +02:00
Mike Fährmann
a9c89085fb
[instagram] implement login support ( #195 )
2019-06-26 23:58:47 +02:00
Mike Fährmann
18a1f8c6cd
[vanillarock] add post and tag extractors ( closes #254 )
2019-06-23 22:45:36 +02:00
Mike Fährmann
f0c5093812
[nsfwalbum] add album extractor ( closes #287 )
2019-06-23 22:45:07 +02:00
Mike Fährmann
15e4ddf46d
implement custom logging formatter
...
supports custom log message formats for each loglevel and, by
extension, custom ANSI codes and colors for errors and warnings
(#304 )
2019-06-21 20:17:58 +02:00
Mike Fährmann
db3f52881a
add 'mtime' option
2019-06-20 17:19:44 +02:00
Mike Fährmann
ec3e8601f1
[slickpic] add user extractor ( #249 )
2019-06-14 18:55:56 +02:00
Mike Fährmann
d997c10320
[8muses] add album extractor ( #305 )
2019-06-10 22:17:46 +02:00
Mike Fährmann
e05a96db5e
[deviantart] rename 'stash' to 'extra' ( #302 )
...
'stash' is already used as a name for the StashExtractor and therefore
expected to be a dictionary.
2019-06-10 21:05:25 +02:00
Mike Fährmann
2184e3a86b
[slickpic] add album extractor ( #249 )
2019-06-09 21:59:22 +02:00
Mike Fährmann
c23bf263fe
[deviantart] rename 'external' to 'stash' ( #302 )
...
restrict extracted URLs to ones from https://sta.sh/ ...
2019-06-09 11:16:02 +02:00
Mike Fährmann
c73c2cda50
[pornhub] add gallery & user extractor ( #282 )
2019-06-07 16:31:20 +02:00
Mike Fährmann
035b850e82
update postprocessor entries in example config
...
- use whitelists
- add ugoira example (#299 )
2019-06-07 13:47:02 +02:00
Mike Fährmann
2fb85178da
[deviantart] add 'external' option ( #302 )
...
If a description is available, this will extract URLs from the
description text and try to find Extractors for them.
2019-06-06 18:53:50 +02:00
Mike Fährmann
096009367b
[xhamster] add gallery & user extractor ( #281 )
2019-06-05 11:11:51 +02:00
Mike Fährmann
d33f5a7423
[wallhaven] rewrite
...
- use API
- remove login support, add 'api-key' option
- remove support for "alpha" subdomain - alpha.wallhaven.cc used numeric
IDs that can't be translated to the new ID system
- support direct links to wallpapers
2019-05-31 14:53:02 +02:00
Mike Fährmann
f1893b2b5b
[deviantart] add 'folders' option ( #276 )
2019-05-30 17:28:12 +02:00
Mike Fährmann
c849574def
[keenspot] add comic extractor ( #223 )
...
Doesn't work for
- http://brawlinthefamily.keenspot.com/
- http://flipside.keenspot.com/
- http://lastblood.keenspot.com/
- http://mysticrevolution.keenspot.com/
- http://porcelain.keenspot.com/
- http://twokinds.keenspot.com/
yet, because of custom layouts.
2019-05-28 21:34:38 +02:00
Mike Fährmann
bcd1801aa8
[sankakucomplex] add 'tag' extractor ( #258 )
2019-05-27 23:57:44 +02:00
Mike Fährmann
74c2415138
[sankakucomplex] move article extractor to its own module ( #258 )
2019-05-27 23:49:23 +02:00
Mike Fährmann
4465a3ea68
[kissmanga][readcomiconline] add 'captcha' option ( #279 )
...
to configure how to handle CAPTCHA page redirects:
- either interactively wait for the user to solve the CAPTCHA
- or raise StopExtraction like before
2019-05-27 22:24:48 +02:00
Mike Fährmann
1e3e15c4f3
[sankaku] add article extractor ( #258 )
2019-05-26 17:42:36 +02:00
Mike Fährmann
e82cadac61
[patreon] add extractors ( #226 )
2019-05-17 10:47:22 +02:00
Mike Fährmann
4891f4a328
[hentainexus] add search extractor ( #256 )
2019-05-16 23:55:30 +02:00
Mike Fährmann
9890bfdf23
[flickr] improve code and metadata
...
- simplify pagination
- add more metadata and slightly change its structure
- convert suitable values to int or list
- move keys from ["photo"] to the base level
- proper video support (#246 )
- rename method and variable names to better fit with other extractors
2019-05-14 22:10:50 +02:00
Mike Fährmann
ba8eb1ffec
[hentainexus] add gallery extractor ( #256 )
2019-05-12 23:59:41 +02:00
Mike Fährmann
0318c610dc
[sexcom] add extractor for search results ( #147 )
2019-04-24 22:10:01 +02:00
Mike Fährmann
a247c94c34
[sexcom] add pin and board extractors ( #147 )
2019-04-24 22:09:19 +02:00
Mike Fährmann
70be494161
[plurk] add a 'comments' options ( #212 )
2019-04-14 22:12:46 +02:00
Mike Fährmann
0b2ff406f6
[plurk] add timeline- and post-extractors ( #212 )
2019-04-14 21:48:38 +02:00
Mike Fährmann
e7cd5510d5
[pixnet] add extractors ( closes #177 )
...
for:
- users/blogs: http://albertayu773.pixnet.net/
- folders: https://albertayu773.pixnet.net/album/folder/1405768
- sets : https://albertayu773.pixnet.net/album/set/15078995
- photos : https://albertayu773.pixnet.net/album/photo/159443828
2019-04-11 19:27:02 +02:00
Mike Fährmann
d9b94a585d
[mangoxo] add login support ( #184 )
...
A very recent change: It is now only possible to see more
than the first 5 images of an album if you are logged in.
2019-04-10 18:55:25 +02:00
Mike Fährmann
e730fc9045
[twitter] add login support ( #214 )
2019-04-09 09:27:49 +02:00
Mike Fährmann
35919a9bb8
[livedoor] add blog- and post-extractors ( #190 )
2019-04-06 16:27:48 +02:00
Mike Fährmann
2ff043edfa
[yaplog] add user- and post-extractors ( #190 )
2019-04-04 17:56:56 +02:00
Mike Fährmann
790f15a56f
[photobucket] use HTTPS
2019-04-03 18:30:45 +02:00
Mike Fährmann
6da665f32e
[mangoxo] add album- and channel-extractors ( closes #184 )
2019-04-03 07:55:51 +02:00
Mike Fährmann
c70b21248d
[wikiart] add extractors ( #179 )
...
for
- artists: https://www.wikiart.org/en/thomas-cole
- artist-listings: https://www.wikiart.org/en/artists-by-century/12
- artwork-listings: https://www.wikiart.org/en/paintings-by-media/grisaille
2019-04-02 17:34:57 +02:00
Mike Fährmann
bc2020e86c
release version 1.8.1
2019-03-29 17:37:11 +01:00
Mike Fährmann
369eb66125
consistently use '*' for rst lists
2019-03-28 16:21:41 +01:00
Mike Fährmann
c7a6b0ed90
[deviantart] add 'metadata' option ( #189 )
2019-03-21 14:49:42 +01:00
Mike Fährmann
8d96a8ce4c
[500px] add user-, gallery-, and image-extractors ( #185 )
2019-03-20 17:32:36 +01:00
Mike Fährmann
6277a739e4
[35photo] add user-, genre-, and image-extractors ( #162 )
2019-03-18 01:11:30 +01:00
Mike Fährmann
12482553bd
update links to youtube-dl
2019-03-13 22:03:02 +01:00
Mike Fährmann
6f57d44ec2
[seaotterscans] remove extractor
...
http://seaotterscans.com/ now redirects to their MangaDex profile
2019-03-13 22:02:45 +01:00
Mike Fährmann
25aaf55514
[smugmug] improve format selection ( closes #183 )
...
- use original image if available
- support video formats
- remove user info for ImageExtractor (it is no longer possible to get
image owner information for a single image)
2019-03-10 15:20:35 +01:00
Mike Fährmann
976ccb267f
[myportfolio] combine gallery and user extractors
...
An URL alone isn't good enough to distinguish between a gallery or a
gallery-listing, so the new extractor decides what to do based on the
page's content.
2019-03-06 19:45:01 +01:00
Mike Fährmann
9c0e2f294b
[shopify] add generic collection and product extractors ( #175 )
...
with fashionnova.com as a default domain
2019-03-05 22:33:37 +01:00
Mike Fährmann
13e0f2a78f
[deviantart] add 'scraps' extractor ( closes #168 )
2019-03-01 14:13:34 +01:00
Mike Fährmann
280531c8ff
[pururin] add gallery extractor ( closes #174 )
2019-02-25 14:54:57 +01:00
Mike Fährmann
3159dd79d5
[seiga] use HTTPS
2019-02-21 22:51:11 +01:00
Mike Fährmann
ea6fbfdd39
simplify build_supportedsites.py
2019-02-21 14:29:19 +01:00
Mike Fährmann
22d3a2fcc8
[artstation] add extractor for artwork listings ( #80 )
...
like https://www.artstation.com/artwork?sorting=latest
or https://www.artstation.com/artwork?sorting=picks
2019-02-18 12:45:44 +01:00
Mike Fährmann
937a802b49
[dynastyscans] add extractors for images and image searches
...
(closes #163 )
2019-02-18 12:25:52 +01:00
Mike Fährmann
19860655a3
[weibo] add 'user' and 'status' extractors
2019-02-17 18:18:31 +01:00
Mike Fährmann
ae353ed3b0
provide "extractor" and "job" keys for logging output
...
This allows for stuff like "{extractor.url}" and "{extractor.category}"
in logging format strings.
Accessing 'extractor' and 'job' in any way will return "None" if those
fields aren't defined, i.e. in general logging messages.
2019-02-14 11:09:58 +01:00
Mike Fährmann
53c2fd4664
add mastodon/foolslide/foolfuuka examples to example config
2019-02-05 16:17:25 +01:00
Mike Fährmann
d36ec51e5a
[tsumino] add extractor for search results ( #161 )
2019-02-02 14:56:46 +01:00
Mike Fährmann
bfbbac4495
[tsumino] add login capabilities ( #161 )
2019-01-30 17:58:48 +01:00
Mike Fährmann
6126615698
update URLs for supportedsites.rst
2019-01-30 16:18:22 +01:00
Mike Fährmann
80a75a1ecf
[tsumino] add gallery extractor ( #161 )
2019-01-29 17:28:48 +01:00
Mike Fährmann
0c32dc5858
[hentaifox] add extractor for search results ( #160 )
2019-01-28 22:38:32 +01:00
Mike Fährmann
0156189468
[hentaifox] add chapter extractor ( #160 )
2019-01-28 18:00:32 +01:00
Mike Fährmann
e4171d6baf
[luscious] add login capabilities ( closes #159 )
2019-01-28 17:14:15 +01:00
Mike Fährmann
134487ffb0
[exhentai] stop extraction if image limit is exceeded ( #141 )
...
can be turned off with the `exhentai.limits' option
2019-01-26 18:40:39 +01:00
Mike Fährmann
b50b30f1c9
[photobucket] download subalbums ( #117 )
2019-01-22 14:05:18 +01:00
Mike Fährmann
277b52101a
add 'category-transfer' option
...
[ci skip]
2019-01-19 20:28:19 +01:00
Mike Fährmann
4b441c162e
release version 1.6.3
2019-01-18 12:31:04 +01:00
Mike Fährmann
8aba2bdebf
[postprocessor:metadata] add 'tags' and 'custom' modes ( #135 )
2019-01-17 21:18:12 +01:00
Mike Fährmann
e53cdfd6a8
update build_supportedsites.py
2019-01-09 14:58:35 +01:00
Mike Fährmann
1e4d351ad3
[danbooru] add authentication support ( closes #151 )
...
... via HTTP Basic Auth with username and "password".
The password value in this case is not the account password itself,
but the"api_key" found in your user profile.
2019-01-09 14:19:07 +01:00
Mike Fährmann
06cbf5f9c4
implement 'chapter-reverse' option ( #149 )
...
Setting it to `true` will start with the latest chapter instead of the
first one.
2019-01-07 18:22:33 +01:00
Mike Fährmann
e95b24f056
[reactor] add wait-min & -max options ( #148 )
2019-01-07 18:04:47 +01:00
Mike Fährmann
a36f52a730
[joyreactor] add extractor for search results ( #148 )
2019-01-03 16:25:56 +01:00
Mike Fährmann
6c71e9cf5d
[deviantart] add separate 'sta.sh' extractor ( #113 )
...
- supports multiple stashed deviations per page
- explicitly mentions sta.sh support on supportedsites.rst
2018-12-26 18:56:57 +01:00
Mike Fährmann
28f9539551
[tumblr] change default values for post types and inline media
2018-12-26 18:55:59 +01:00
Mike Fährmann
5be95034ba
[tumblr] add option to download avatars ( #137 )
2018-12-26 14:29:30 +01:00
Mike Fährmann
be52069cbc
update CHANGELOG and docs/supportedsites
2018-11-28 14:53:27 +01:00
Mike Fährmann
c47482b110
smaller changes, missing docs, etc.
...
- make 'netrc' extractor-specific
- rename 'downloader.enable' to 'enabled'
- document 'downloader.ytdl.format'
- consistent newlines in configuration.rst
2018-11-16 18:18:07 +01:00
Mike Fährmann
8ef84a6823
add option to enable/disable specific downloader modules
...
... and write URLs with no (active) downloader to unsupported-file
2018-11-13 18:06:36 +01:00
Mike Fährmann
2fa28a2609
update default user-agent string ( closes #122 )
2018-11-11 10:07:10 +01:00
