1
0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-26 12:42:29 +01:00
Commit Graph

2012 Commits

Author SHA1 Message Date
Mike Fährmann
b7fb93e2b2
[downloader:http] add 'adjust-extensions' option 2019-08-08 16:54:20 +02:00
Mike Fährmann
eb7da159e2
[imagebam] update URL test results
Image URLs are now using https://, but the website itself is still
served as http://.
2019-08-07 21:47:44 +02:00
Mike Fährmann
189acbeac9
[imgbb] add extractor for individual images (closes #363) 2019-08-05 22:52:08 +02:00
Mike Fährmann
ad3ac02fbc
[pixiv] update metadata entries (#366)
- change 'num' to a simple enumerating integer
- change default filename format
- provide content of the old 'num' field as 'suffix'
- add 'filename' for ugoira
2019-08-05 22:41:56 +02:00
Mike Fährmann
1ff4c4ec03
[adultempire] consistent artist order 2019-08-05 22:06:11 +02:00
Leonardo Taccari
2df050e627 [instagram] Add support for stories (#371)
* [instagram] Add support for stories

Add support for Instagram user's stories
(https://www.instagram.com/stories/<username>/).

First the shared_data in instagram.com/stories/<username> is fetched in
order to retrieve the user_id that is then passed to fetch the stories
via the corresponding graphql query.

Please note that fetching stories is supported only when authentication
is enabled and the corresponding <username> is followed.

* [instagram] Add an only-matching test for stories

* [instagram] Simplify InstagramExtractor.items() and _extract_stories()

Simplify handling of typename in InstagramExtractor.items() and multi-line
string in _extract_stories().  NFCI.
2019-08-05 22:04:34 +02:00
Mike Fährmann
f4bc75e854
fix rate limit handling for OAuth APIs (#368) 2019-08-03 13:43:00 +02:00
Mike Fährmann
f687052daf
have e-hentai and exhentai on supportedsites.rst (#365) 2019-08-03 11:42:28 +02:00
Mike Fährmann
3957d27d79
[deviantart] add 'quality' option (#369) 2019-08-03 11:40:35 +02:00
Mike Fährmann
64b2935d8e
[pixiv] provide 'filename' and change default filename format
to '{filename}.{extension}' (closes #366)
2019-08-02 22:35:10 +02:00
Mike Fährmann
2f33bac030
release version 1.10.1 2019-08-02 21:23:06 +02:00
Mike Fährmann
fa60109e97
[exhentai] don't use e-hentai.org for exhentai URLs 2019-08-02 21:10:09 +02:00
Mike Fährmann
dfe552421b
release version 1.10.0 2019-08-01 23:22:58 +02:00
Mike Fährmann
0609afd1e4
update default cache directory ... again
Use a 'gallery-dl' subdirectory in ~/.cache to adhere to how other
programs store their cached data, and call os.makedirs() so it also
works without an existing ~/.cache directory.
2019-08-01 22:11:00 +02:00
Mike Fährmann
4a0c98bfc9
miscellaneous fixes and adjustments 2019-08-01 22:09:43 +02:00
Mike Fährmann
2c839f3760
[imgbb] add user extractor + login support (#361) 2019-08-01 21:39:20 +02:00
Mike Fährmann
a8b60b2bd9
change default cache directory for unix systems
Use either $XDG_CACHE_HOME or ~/.cache (if the former isn't set)
and store potentially sensitive cookies and tokens in a user's
home directory and not in the world-readable /tmp.
2019-07-31 22:56:14 +02:00
Mike Fährmann
4b6edfbfd2
restrict permissions without importing 'pathlib'
and only on non-Windows systems.

1. On Windows the 'mode' argument for os.open() has no (visible) effect
   on access permissions for new files.
2. The default location for 'cache.file' on Windows is in
   %USERPROFILE%\AppData\Local\Temp which can only be accessed by the
   owner himself (or an admin).
2019-07-31 21:48:09 +02:00
Leonardo Taccari
afce1ee1eb
Avoid possible sensitive information disclosure via cache.file
Previously cache.file could be created world readable leading to
possible sensitive information disclosure on multi-user systems.
Restrict permissions only to the owner by creating an empty file.

Please note that cache.file created before this commit may need a
`chmod 600' or similar!
2019-07-31 15:05:26 +02:00
Mike Fährmann
2153206093
[imgbb] add album extractor (#361) 2019-07-30 23:11:19 +02:00
Mike Fährmann
beb4fab2e6
[exhentai] improve limit and error handling (#360)
- check image limit before opening the first gallery or image page
- prevent any further exhentai extractors from running after the image
  limit has been reached
2019-07-30 22:58:35 +02:00
Mark Henrick
923e1bb714 [docs] Fix inconsistency about which sites have optional authentication (#359)
* [docs] Fix inconsistency about which sites have optional authentication

* update authentication docs
2019-07-29 18:22:31 +02:00
Mike Fährmann
81b35ed3cb
[exhentai] catch more error states (#356, #360)
- warn on MPV-enabled galleries
- catch parsing errors for gallery pages and image info
- write page content to debug output
2019-07-29 16:54:31 +02:00
Mike Fährmann
a90280f4e7
[postprocessor:zip] add 'mode' option (#355) 2019-07-29 16:51:26 +02:00
Mike Fährmann
6ce22f606b
[exhentai] update login procedure and tests
Logging in now follows the natural login flow that also happens in a
browser more closely and collects more cookies than just ipb_member_id
and ipb_pass_hash.

Test URLs have been updated and now point to the e-hentai.org domain.
2019-07-28 16:51:05 +02:00
Mike Fährmann
dc73d02d87
[exhentai] always use e-hentai.org as domain + set nw cookie 2019-07-28 10:54:17 +02:00
Mike Fährmann
40637556fa
[ngomik] fix extraction 2019-07-28 10:53:46 +02:00
Mike Fährmann
3969f9cbbd
[behance] fix collection extraction 2019-07-27 14:26:40 +02:00
Mike Fährmann
20f7b07312
ensure postproc finalize() is called during C-c or crash (#355) 2019-07-27 11:14:52 +02:00
Mike Fährmann
17a3426845
[gelbooru] enable all content when not using API 2019-07-27 11:13:38 +02:00
Mike Fährmann
279db2c5b2
[vsco] add collection & image extractor + video support (#331) 2019-07-26 19:06:15 +02:00
Mike Fährmann
547ea71463
[downloader.ytdl] add 'forward-cookies' option (#352)
The "long" name is necessary because just calling it 'cookies' would
clash with how the lookup for '--cookies' is implemented.
2019-07-24 21:19:11 +02:00
Mike Fährmann
d9d44ad953
[tsumino] update test results 2019-07-24 21:17:23 +02:00
Mike Fährmann
b1bea8aaeb
add 'restrict-filenames' option (#348) 2019-07-23 17:41:24 +02:00
Mike Fährmann
60cf40380a
[vsco] add user extractor (#331) 2019-07-23 16:23:11 +02:00
Mike Fährmann
3fe5ccdfa6
[adultempire] add gallery extractor (closes #340) 2019-07-21 22:29:57 +02:00
Mike Fährmann
b3851e01d9
release version 1.9.0 2019-07-19 21:55:25 +02:00
Mike Fährmann
5d968412ca
[deviantart] case-insensitive folder name matching (fixes #343) 2019-07-19 18:05:31 +02:00
Mike Fährmann
a3c736fedc
[500px] fix extraction
Maximum available image dimensions have been reduced to 4096px
on the longest edge. (from 5000px)
A few (unimportant) metadata fields are no longer available or have
been changed to 'null'.
2019-07-19 17:23:03 +02:00
Mike Fährmann
1133b7fcbd
[smugmug] update unit tests
The account used for tests before has been deleted.
2019-07-19 17:16:24 +02:00
Mike Fährmann
21991acc49
add 'ciphers' option; update default User-Agent 2019-07-19 17:14:40 +02:00
Mike Fährmann
84f4d3bc0b
replace urllib3's default cipher list with Firefox's (#342)
Avoids Cloudflare CAPTCHAs on both Linux in Windows without
pyOpenSSL installed.
2019-07-18 19:42:13 +02:00
Mike Fährmann
feb98cf196
[twitter] improve 'content' formatting; add option (#338)
- include emoticons
- leave newlines intact
- remove pic.twitter.com/ links at the end
2019-07-17 16:02:51 +02:00
Mike Fährmann
1740086d8a
add 'repl' and 'sep' arguments to text.replace_html() 2019-07-17 14:48:24 +02:00
Mike Fährmann
8d1ae9b715
[tumblr] enable date-min/-max/-format options (#337) 2019-07-17 14:36:41 +02:00
Mike Fährmann
09f37fde39
[reddit] move date-min/-max handling into Extractor class 2019-07-16 22:54:39 +02:00
Mike Fährmann
fb875d1ab8
add warning about NSFW sites in supportedsites.rst (#335) 2019-07-15 21:44:34 +02:00
Mike Fährmann
7b77ecc35a
fix paths for files without extension (#220) 2019-07-15 16:39:03 +02:00
Mike Fährmann
c41ff9441e
improve find() for downloaders and postprocessors 2019-07-15 16:33:03 +02:00
Mike Fährmann
0151e250f5
[twitter] extract 'content' metadata (closes #333) 2019-07-15 16:25:22 +02:00