Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-26 04:32:51 +01:00
Code Issues Releases Wiki Activity
901 Commits 6 Branches 164 Tags 15 MiB
a4bc5a3491
Commit Graph

901 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Mike Fährmann
a4bc5a3491
update setup.py and README.rst 2017-10-27 16:08:57 +02:00
Mike Fährmann
707b15b586
create missing directories for 'part-directory' 
also some code improvements regarding downloader config values
 2017-10-27 12:22:45 +02:00
Mike Fährmann
035ef655f1
[imagefap] update unit tests 
old gallery/image has been deleted
 2017-10-27 12:22:16 +02:00
Mike Fährmann
caf26412dd
add option to set alternate location of .part files (#29) 
Note: The path set for 'downloader.*.part-directory' needs to point to an
already existing directory.
 2017-10-26 00:16:48 +02:00
Mike Fährmann
ea8ca4cfa4
add 'util.expand_path()' 2017-10-26 00:04:28 +02:00
Mike Fährmann
9a41002b77
fix partial downloads for 'text:' URLs 
Using a filesize in bytes as offset into a Python string is not
a good idea if said file contains non-ASCII characters.
 2017-10-25 15:04:45 +02:00
Mike Fährmann
239d7afea7
[hosturimage] fix extraction of larger images 2017-10-25 12:56:16 +02:00
Mike Fährmann
27c026543f
re-enable download unit tests 2017-10-25 12:55:36 +02:00
Mike Fährmann
963670d73b
add options to control usage of .part files (#29) 
- '--no-part' command line option to disable them
- 'downloader.http.part' and 'downloader.text.part' config options

Disabling .part files restores the behaviour of the old downloader
implementation.
 2017-10-24 23:33:44 +02:00
Mike Fährmann
158e60ee89
[3dbooru] enable download continuation 
behoimi.org doesn't respect 'Range' headers and doesn't report
'Content-Length' for compressed content encodings.
 2017-10-24 13:05:31 +02:00
Mike Fährmann
b0353aa02d
rewrite download modules (#29) 
- use '.part' files during file-download
- implement continuation of incomplete downloads
- check if file size matches the one reported by server
 2017-10-24 12:53:03 +02:00
Mike Fährmann
c4fcdf2691
Revert "[senmanga] fix extraction and download" 
This reverts commit 2ace5c7b3c.
 2017-10-24 00:22:05 +02:00
Mike Fährmann
81a7788b40
replace space characters in unit test URLs 2017-10-23 17:00:53 +02:00
Mike Fährmann
bf82181359
[jaiminisbox] fix extraction 2017-10-22 13:26:09 +02:00
Mike Fährmann
2e982f56af
use 'Content-Length' to determine incomplete downloads (#29) 2017-10-20 18:56:18 +02:00
Mike Fährmann
16783e327f
[common] fix UnboundLocalError in Extractor.request() 2017-10-20 18:51:06 +02:00
Mike Fährmann
2ace5c7b3c
[senmanga] fix extraction and download 2017-10-19 18:25:31 +02:00
Mike Fährmann
4d8387f93b
[pixiv] support mobile URLs (https://touch.pixiv.net/) 2017-10-17 16:49:42 +02:00
Mike Fährmann
ab2bf0b0dd
[deviantart] replace collection unittest 2017-10-17 15:58:16 +02:00
Mike Fährmann
289d6b65d2
[danbooru] extend and improve URL regex 
- add support for danbooru mirrors:
  - hijiribe.donmai.us
  - sonohara.donmai.us
  - todo: actually use these domains instead of redirecting everything
          to danbooru itself
- improve handling of query string parameters
 2017-10-16 21:21:19 +02:00
Mike Fährmann
5fa42336a2
[sankaku] add warning for unauthenticated users 
also improve URL pattern and add missing options to default config file
 2017-10-16 21:21:08 +02:00
Mike Fährmann
6af921a952
[sankaku] rewrite/improve (fixes #44) 
- add wait-time between HTTP requests similar to exhentai
- add 'wait-min' and 'wait-max' options
- increase retry-count for HTTP requests to 10
- implement user authentication (non-authenticated users can only view
  images up to page 25)
- implement 'skip()' functionality (only works up to page 50)
- implement image-retrieval for pages >= 51
- fix issue with multiple tags
 2017-10-14 23:01:33 +02:00
Mike Fährmann
9aecc67841
[common] explicitly handle HTTP status code 429 2017-10-14 21:37:59 +02:00
Mike Fährmann
abfd2924a9
update configuration.rst 
- add instructions on how to register your own applications
- combine descriptions of a few config keys
 2017-10-12 23:37:28 +02:00
Mike Fährmann
d68a24aa70
[kissmanga] fix extraction 
site changed '\n' to '\r\n' for newlines
 2017-10-12 14:49:27 +02:00
Mike Fährmann
864a63ed33
fix typo 
[skip ci]
 2017-10-10 17:42:06 +02:00
Mike Fährmann
f3fbaa5c3e
[reddit] allow users to override the API User-Agent 
Only overriding the Client-ID is not enough if you want to follow
Reddit's API access rules [1].

[1] https://github.com/reddit/reddit/wiki/API#rules
 2017-10-10 17:29:46 +02:00
Mike Fährmann
31ea6001e8
[dynastyscans] improve metadata and filename formats 2017-10-10 17:14:39 +02:00
Mike Fährmann
2ef3c35c98
smaller textual changes 
- swapped doc for deviantart.mature and .original
- updated gallery-dl.conf
- "transferred" -> "delegated"
 2017-10-09 23:23:19 +02:00
Mike Fährmann
68a0a7579c
fix/improve some regular expressions 2017-10-09 22:37:50 +02:00
Mike Fährmann
832b8b76ac
[util] extend global namespace for filter expressions 2017-10-09 22:12:58 +02:00
Mike Fährmann
393755ee94
[tumblr] update tests 2017-10-09 00:10:37 +02:00
Mike Fährmann
75d3a1f72f
[deviantart] always download original images 
Deviation-objects returned by the DeviantArt API don't always contain
the URL and metadata of the original image ([1]). Getting this
information requires an additional API call [2], which is indicated by
the 'is_downloadable' and 'download_filesize' metadata within a
deviation-object.

[1] https://myria-moon.deviantart.com/art/Aime-Moi-part-en-vadrouille-261986576
[2] https://www.deviantart.com/developers/http/v1/20160316/deviation_download/bed6982b88949bdb08b52cd6763fcafd
 2017-10-07 13:07:34 +02:00
Mike Fährmann
8e6a767109
[util] restructure formatter for better exception propagation 2017-10-06 17:10:35 +02:00
Mike Fährmann
0386503c80
fix (sub)category-transfer for DownloadJob instances (#41) 
... and extend "parent" parameters to TestJob- and DataJob-classes
as well.
 2017-10-06 15:38:35 +02:00
Mike Fährmann
a1c8b21cfd
[senmanga] improve metadata 2017-10-04 18:54:39 +02:00
Mike Fährmann
8df023e144
[util:filter] re-enable builtins 
Trying to restrict access to Python's builtin functions (exec,
print, __import__, ...) can easily be circumvented and is
therefore completely pointless.

This also adds 'safe_int()' and the 'datetime' module to the global
namespace used when evaluating filter expressions.
 2017-10-04 16:00:12 +02:00
Mike Fährmann
994b2fc1e7
[deviantart] replace 'author[urlname]' keyword 
author[urlname] has always only been the lowercase version of
author[username], which can now be directly converted to lowercase
using the 'l' conversion: '{author[username]!l}'
 2017-10-04 15:59:05 +02:00
Mike Fährmann
633b376f35
improve/adjust default filename formats for manga sites 2017-10-02 19:06:24 +02:00
Mike Fährmann
41adb99e9c
[pawoo] fix extraction 
- changed access_token
- use account-search instead of general search
 2017-10-02 18:33:52 +02:00
Mike Fährmann
b319f4bab3
smaller code and text changes 2017-10-01 18:23:40 +02:00
Mike Fährmann
ad4580800c
[pixiv] add support for more URL patterns 
- https://www.pixiv.net/mypage.php#id=USERID
- https://www.pixiv.net/#id=USERID
 2017-09-30 18:07:20 +02:00
Mike Fährmann
82ea6c0cd3
adjust format strings with optional titles 
... except for anything manga/comic related
 2017-09-28 18:00:19 +02:00
Mike Fährmann
c1f0afe4c6
add custom string formatter class 2017-09-28 17:12:39 +02:00
Mike Fährmann
85a2b2ae59
[khinsider] fix extraction 2017-09-28 11:47:26 +02:00
Mike Fährmann
26a866e7d8
implement (sub)category-transfer between extractors (#41) 
ImageFap- and all Manga-Extractors will transfer their (sub)category
values to other extractors instantiated by them, which will in turn
allow those to use options set for their parents.

Example:
ImagefapGalleryExtractors will use options set under
extractor.imagefap.user, if (and only if) they have been instantiated by
a ImagefapUserExtractor; and options from extractor.imagefap.gallery
otherwise.
 2017-09-26 21:05:11 +02:00
Mike Fährmann
1ab4c7986f
[mangahere] fix extraction 
would switch to HTTPS, but there seem to be certificate issues
 2017-09-26 21:05:11 +02:00
Mike Fährmann
8e14714c2b
[imgspice] fix extraction 2017-09-26 21:04:48 +02:00
Mike Fährmann
9c138dfc1f
[common] detect empty HTTP response bodies 2017-09-26 16:49:58 +02:00
Mike Fährmann
c51616f8d8
[foolslide] fix minor chapter number 2017-09-26 12:49:50 +02:00