Mike Fährmann
b2c55f0a72
[sankaku] remove login support
...
The old login method for 'https://chan.sankakucomplex.com/user/login '
and the cookies it produces have no effect on the results from
'beta.sankakucomplex.com'.
2020-12-08 21:05:47 +01:00
Mike Fährmann
0bc492c0fa
add docs for 'event' and 'filename' options
...
from 9c3568c3ca59bd69
2020-11-25 12:12:41 +01:00
Mike Fährmann
9b1bd09454
change 'extension-map' default
...
Replace all JPEG filename extensions with 'jpg'.
2020-11-14 22:40:31 +01:00
Mike Fährmann
bc7b1d91bc
fix rST markup in configuration.rst
...
[ci skip]
2020-11-02 15:32:29 +01:00
Mike Fährmann
e3480bc8de
implement 'extension-map' option ( #318 )
2020-11-02 15:27:07 +01:00
Mike Fährmann
98a4d86a01
[sankakucomplex] extract videos and embeds ( closes #308 )
2020-10-30 01:21:11 +01:00
Mike Fährmann
d83b95fd28
[postprocessor:metadata] accept a string-list for 'content-format'
...
(closes #1080 )
2020-10-27 20:09:58 +01:00
Mike Fährmann
5bcf28de93
add a 'extractor.modules' option
2020-10-25 03:05:10 +01:00
Mike Fährmann
1686dc1757
[twitter] support media from Cards ( #1005 , #937 )
...
Can be enabled with 'extractor.twitter.cards', but for now disabled by
default because cards can redirect to rather large videos from YouTube
or Twitch.
2020-10-22 21:33:53 +02:00
Mike Fährmann
bac8af69e8
update configuration.rst
...
- add some lines to better explain post processor usage
- syntax highlighting for JSON blocks and other smaller stuff
2020-10-19 21:57:26 +02:00
Mike Fährmann
05d7009cc6
rename 'Authentication' entries in supportedsites.rst
...
- change 'Optional' to 'Supported'
- use 'OAuth' and 'Cookies' in their own
- add link to weasyl API key option
2020-10-19 20:16:17 +02:00
Mike Fährmann
d1c6d78477
fix rST markup in configuration.rst
2020-10-15 15:17:19 +02:00
Mike Fährmann
b8daabc3ca
[pinterest] implement login support ( closes #1055 )
...
being logged allows access to secret/protected boards
2020-10-15 15:14:18 +02:00
Zanny
3ec60e894a
[weasyl] api-key authentication ( #1057 )
...
* [weasyl] support api keys
* [weasyl] document api-key authentication
* [weasyl] usernames can contain ~
2020-10-15 15:12:09 +02:00
Mike Fährmann
35056a07d1
release version 1.15.1
2020-10-11 18:44:46 +02:00
Mike Fährmann
c874071f5a
[kissmanga] remove module
2020-10-04 22:46:41 +02:00
Mike Fährmann
430b6d6e2e
[twitter] extend 'retweets' option ( closes #1026 )
...
Setting 'retweets' to '"original"' will use metadata from the
original retweeted Tweets, and not from the Retweet entry.
2020-09-28 23:03:35 +02:00
Mike Fährmann
b9bdd2c564
[hentaifoundry] add support for stories ( closes #734 )
2020-09-27 02:27:40 +02:00
Mike Fährmann
117582d798
use definition lists instead of tables in docs/configuration.rst
...
and strip some clutter from generated man pages
2020-09-26 13:33:46 +02:00
Mike Fährmann
6514312126
[nijie] add 'include' option ( closes #1018 )
2020-09-25 18:18:35 +02:00
Mike Fährmann
0d43456323
[hentaifoundry] add 'include' option
2020-09-25 18:18:03 +02:00
Mike Fährmann
1e313d5b84
implement 'sleep-request' option
2020-09-20 20:28:17 +02:00
Mike Fährmann
6a5f6396d4
add clearing cache to dA client-id instructions ( #996 )
2020-09-16 01:24:18 +02:00
Mike Fährmann
3afd362e2e
add 'sleep-extractor' option ( closes #964 )
...
(would have been nice if this were possible without code duplication)
2020-09-12 21:04:47 +02:00
Mike Fährmann
3918b69677
remove 'extractor.blacklist' context manager
2020-09-11 13:17:35 +02:00
Mike Fährmann
c78aa17506
add general 'blacklist' and 'whitelist' options ( #492 , #844 )
2020-09-11 13:17:12 +02:00
Mike Fährmann
5912727b88
support format string replacement fields in archive paths
...
(closes #985 )
2020-09-10 22:09:30 +02:00
Mike Fährmann
f6fd449b59
reduce wait time growth rate from exponential to linear
...
Waiting for 2**N seconds after each error grows too fast.
Simply waiting N seconds seems far more reasonable.
2020-09-06 22:38:25 +02:00
Mike Fährmann
ac3036ef56
add 'filesize-min' and 'filesize-max' options ( closes #780 )
2020-09-03 18:21:04 +02:00
ArtaxIsSleeping
0e941553ec
[aryion] Add username/password support ( #960 )
...
* Add username/password support to aryion extractor
* Update docs to match
* Fix code style
2020-08-27 22:45:30 +02:00
Mike Fährmann
dbbbb21180
[exhentai] add ability to specify custom image limit ( #940 )
2020-08-17 22:29:20 +02:00
Mike Fährmann
901a897e6f
[inkbunny] mention login support
2020-07-26 23:52:52 +02:00
Mike Fährmann
b921fee24d
[inkbunny] fix submission order ( #283 )
...
Getting detailed submission info via /api_submissions.php reordered the
input submissions and sorted them by ID. InkbunnyAPI.detail() now sorts
them back and ensures they are returned in their original order.
This commit also removes the 'metadata' option and always requests
submission descriptions.
2020-07-26 23:12:45 +02:00
Mike Fährmann
c4ed9f4faa
[inkbunny] add 'metadata' option ( #283 )
2020-07-24 18:05:53 +02:00
Mike Fährmann
493cadb1e7
[inkbunny] add 'orderby' option ( #283 )
2020-07-24 17:50:32 +02:00
Mike Fährmann
d5fcffcced
[subscribestar] add login capabilities ( #852 )
2020-07-17 22:18:01 +02:00
Mike Fährmann
ecaecc4064
[exhentai] add 'domain' option ( #897 )
2020-07-17 22:17:46 +02:00
Mike Fährmann
cb0132e441
[khinsider] add 'format' option ( closes #840 )
2020-07-13 17:17:58 +02:00
Mike Fährmann
f1ddbff0b5
[aryion] add 'recursive' option ( fixes #832 )
...
This is enabled by default and will recursively go through all
(sub)folders in an artist's gallery.
The old method of using "Latest Updates" lists can be restored by
disabling this option.
2020-06-26 23:36:50 +02:00
Mike Fährmann
9da2bc67f8
[twitter] add option to filter media from quoted tweets ( #854 )
2020-06-25 18:59:25 +02:00
Mike Fährmann
5ba90f72ca
[pinterest] add support for sections ( closes #835 )
2020-06-16 14:41:05 +02:00
Mike Fährmann
c184cce876
update configuration.rst
...
- fix anonymous links
- update description of 'extractor.twitter.videos'
- document 'extractor.redgifs.format' (#724 )
2020-06-12 18:25:17 +02:00
Mike Fährmann
41d03160ff
[deviantart] also search journals for sta.sh links ( #712 )
...
when 'extra' is enabled
2020-06-04 14:47:08 +02:00
Mike Fährmann
0138e9c1d8
[twitter] remove 'content' option
...
The text content of each tweet is always available as 'full_text'
2020-06-04 01:24:21 +02:00
Mike Fährmann
0f459f340b
[instagram] fix and re-enable login with username&password
...
This reverts commit 3e0848a482#756 , #771 , #797 , #803 )
https://github.com/althonos/InsaLooter/issues/287#issuecomment-630456522
2020-05-31 00:29:09 +02:00
Mike Fährmann
3e0848a482
[instagram] disable login with username&password ( #756 )
2020-05-29 23:29:40 +02:00
Mike Fährmann
da22ea8ced
use %APPDATA%\gallery-dl for config/cache on Windows
2020-05-28 02:56:38 +02:00
Mike Fährmann
b489f4de12
fix internal links in configuration.rst
2020-05-28 01:51:16 +02:00
Mike Fährmann
dfcf2a2c91
write OAuth token to cache by default ( #616 )
2020-05-25 22:35:45 +02:00
Mike Fährmann
ddc253cf9a
implement a 'path-replace' option ( #662 , #755 )
2020-05-25 22:21:58 +02:00
Mike Fährmann
bc53302ad6
extend 'path-restrict' option
...
Allow its value to be a JSON object / Python dict that specifies
a mapping from invalid/unwanted input characters to specific
output characters.
For example {"/": "-", "*": "+"} will transform
"foo / ***bar***" into "foo - +++bar+++"
(closes #662 , #755 )
2020-05-25 22:21:56 +02:00
Mike Fährmann
e19f665a44
[danbooru] change default for 'ugoira' to 'false'
...
Downloading the pre-rendered versions should be a better default
than .zip files with individual frames.
2020-05-20 19:57:28 +02:00
Mike Fährmann
dba87ca99e
[downloader:ytdl] change 'forward-cookies' default to 'false'
...
There are currently no situations where forwarding gallery-dl's cookies
to youtube-dl is necessary, and it only causes problems when forcing
youtube-dl for Twitter video downloads while logged in.
2020-05-12 20:17:08 +02:00
Mike Fährmann
be16d77073
simplify and improve documentation markup ( #742 )
...
- use | to get a more "compact" representation
- link directly to example config files
2020-05-12 17:00:34 +02:00
Vrihub
fe224416bf
Some documentation improvements ( #742 )
...
* Link configuration examples in the intro, see #712
* Clarify overriding conf files by options, see #678
2020-05-10 23:47:20 +02:00
Mike Fährmann
5e2974d699
[weibo] add 'videos' option
2020-04-30 00:00:30 +02:00
Mike Fährmann
9f638c2e01
[twitter] add 'replies' option ( closes #705 )
2020-04-29 23:20:06 +02:00
Mike Fährmann
5b676ea59d
[e621] document username & password support ( #640 )
2020-03-14 01:15:22 +01:00
Mike Fährmann
59edcdc822
[hitomi] restore metadata fields from before f33b13a
...
... and add a 'metadata' option to disable
visiting the gallery page and extracting data from it
if this is not needed.
2020-03-12 23:43:41 +01:00
Mike Fährmann
a45fbc38ea
[pixiv] implement 'avatar' option ( #595 , #623 )
2020-03-09 21:18:16 +01:00
Mike Fährmann
6e759fe2f5
reword note for 'extractor.deviantart.refresh-token' ( #612 )
2020-02-19 23:39:27 +01:00
Mike Fährmann
67ea80222c
link to post processor example in 'pixiv.ugoira' docs
2020-02-14 01:22:16 +01:00
Mike Fährmann
74e684e828
[twitter] change default value for 'videos' to 'true'
...
Every other 'videos' option defaulted to 'true', except Twitter.
2020-02-14 01:03:42 +01:00
Mike Fährmann
ab13f81fd7
add example to 'cookies' docs
2020-02-13 23:44:38 +01:00
Mike Fährmann
c7cf9dd111
[furaffinity] support classic layout ( #284 )
2020-02-12 21:39:43 +01:00
Mike Fährmann
da0d5f6092
[oauth] add 'port' option ( #604 )
2020-02-09 13:45:44 +01:00
Mike Fährmann
091f9a107d
[docs] add examples of how to filter manga chapters by language
...
(closes #599 )
2020-02-07 23:21:44 +01:00
Mike Fährmann
dff33b260c
[reddit] add 'videos' option
2020-01-31 23:45:02 +01:00
Mike Fährmann
56f1c96168
implement 'parent-directory' option ( #551 )
2020-01-29 18:32:37 +01:00
Mike Fährmann
6703b8a86b
[blogger] implement video extraction ( closes #587 )
2020-01-24 23:37:23 +01:00
Mike Fährmann
37247dbaff
miscellaneous fixes
2020-01-19 22:53:06 +01:00
Mike Fährmann
25d5ec4ff3
[twitter] add option to extract TwitPic embeds ( #579 )
2020-01-18 21:31:29 +01:00
Mike Fährmann
254f7c3999
implement a post-processor module to compare file versions
...
(#530 )
2020-01-18 21:08:25 +01:00
Mike Fährmann
87c8b89ddd
[postprocessor:metadata] add 'directory' option ( #520 )
2020-01-02 20:58:10 +01:00
Mike Fährmann
48e42e73fb
[reddit] change default value for 'comments' to '0'
2019-12-20 16:54:59 +01:00
Mike Fährmann
bf658fd84b
[vsco] implement 'videos' option
2019-12-19 18:34:19 +01:00
Mike Fährmann
95c90722ee
[instagram] implement 'videos' option ( closes #521 )
2019-12-19 18:34:06 +01:00
Mike Fährmann
15f9bb3d14
add option to disable pyOpenSSL usage ( #508 )
...
(pyOpenSSL is now disabled by default)
2019-12-08 21:21:00 +01:00
Mike Fährmann
93b43f74f4
update docs/configuration.rst
...
- fix description of "skip": "enumerate"
- use line breaks for lists of multiple examples
- improve formatting of auto-generated man pages
2019-12-06 01:19:23 +01:00
Mike Fährmann
e17907ee2a
change default value of 'cookies-update' to 'true'
2019-12-05 23:43:49 +01:00
Mike Fährmann
26d2334550
[postprocessor:metadata] rename 'format' to 'content-format'
...
Just to be consistent with the other 'extension-format' option name,
and only 'format' is also still accepted.
2019-11-30 17:27:49 +01:00
Mike Fährmann
a412531451
[postprocessor:metadata] implement 'extension-format' option
...
closes #477
2019-11-30 17:26:17 +01:00
Mike Fährmann
d45fabb79d
match user profile handling on deviantart and newgrounds
2019-11-22 23:20:21 +01:00
Mike Fährmann
dd5d2b2eac
[deviantart] add user profile extractor ( #377 , #419 )
2019-11-07 18:29:49 +01:00
Mike Fährmann
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
2019-11-03 21:45:45 +01:00
Mike Fährmann
c01ff78467
[twitter] extend 'videos' option to force extraction with ytdl
...
(closes #459 )
2019-11-01 22:06:07 +01:00
Mike Fährmann
a5e6252137
update deviantart client-id/-secret instructions ( #451 , #424 )
2019-10-21 16:57:45 +02:00
Mike Fährmann
389d2d7e38
implement 'cookies-update' option ( #445 )
2019-10-19 15:23:55 +02:00
Mike Fährmann
1adafdd3d0
document cache file requirement for DeviantArt refresh tokens
2019-10-13 23:01:57 +02:00
Mike Fährmann
8af59a4bba
fix & update docs
...
- update Requests links
- add example for --exec
- set '-dev' version
2019-10-11 18:36:25 +02:00
Mike Fährmann
46ba173ded
[reddit] fix documentation inconsistencies ( closes #429 )
...
- Require 'reddit.comments' to be a number and convert it to an
integer to be extra sure
- Link to the README's OAuth section were appropriate
2019-09-27 17:34:10 +02:00
Mike Fährmann
ee8b654464
[instagram] implement 'highlights' option ( closes #329 )
2019-09-21 23:38:20 +02:00
Mike Fährmann
f02a768b5c
[danbooru] add 'ugoira' option ( #406 )
...
to choose between ZIP archives or converted video files
for Ugoira posts
2019-08-31 21:51:52 +02:00
Mike Fährmann
c6c5cb1898
improve 'deviantart.quality' description
2019-08-30 18:41:18 +02:00
Mike Fährmann
7c09545f70
[downloader:ytdl] add 'outtmpl' option ( #395 )
2019-08-24 22:47:59 +02:00
Mike Fährmann
5a210991b6
Remove control characters from filesystem paths
...
- add 'path-remove' option to specify the set of characters that
should be removed
- rename 'restrict-filenames' to 'path-restrict'
- #348 , #380
2019-08-16 23:12:16 +02:00
Mike Fährmann
e661a1ce19
update links to Request's documentation
2019-08-15 13:25:06 +02:00
Mike Fährmann
8dc42bb178
implement 'enumerate' for 'extractor.skip' ( #306 )
...
[ci skip]
2019-08-08 18:37:54 +02:00
Mike Fährmann
b7fb93e2b2
[downloader:http] add 'adjust-extensions' option
2019-08-08 16:54:20 +02:00
Mike Fährmann
3957d27d79
[deviantart] add 'quality' option ( #369 )
2019-08-03 11:40:35 +02:00
