Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-23 03:02:50 +01:00
Code Issues Releases Wiki Activity
2,793 Commits 6 Branches 164 Tags 16 MiB
1c9435e0df
Commit Graph

59 Commits

Author SHA1 Message Date
Mike Fährmann
1c9435e0df
add '-G' command-line option (#1217) 
A "stronger" version of '-g', resolving all intermediate URLs.
 2021-01-07 19:07:05 +01:00
Mike Fährmann
9c3568c397
[postprocessor:exec] add 'event' option 
and remove 'final' option -- use '"event": "finalize"' instead.
 2020-11-19 02:30:48 +01:00
Mike Fährmann
abda352a5b
add '--no-skip' command-line option (closes #986) 2020-09-11 01:23:39 +02:00
Mike Fährmann
ac3036ef56
add 'filesize-min' and 'filesize-max' options (closes #780) 2020-09-03 18:21:04 +02:00
Unit 193
b7ebf51229
Fix typo: defaut → default. (#754) 2020-05-14 23:37:15 +02:00
Mike Fährmann
f8f95e68a7
improve '--write-pages' (#737) 
- move code into its own function
- add enumeration index to filenames
- dump responses regardless of status code
 2020-05-12 20:40:25 +02:00
Vrihub
4cc761c730
Implement --write-pages option (#736) 
* Implement --write-pages option

* Fix long lines

* Fix file mode to binary

* Fix pattern for Windows compatibility
 2020-05-12 14:25:21 +02:00
Mike Fährmann
bca2222559
add '--exec-after' 2019-11-27 19:42:46 +01:00
Mike Fährmann
ed6592ea1a
remove '--abort-on-skip' 2019-11-27 19:41:24 +01:00
Mike Fährmann
f5604492c3
update interface of config functions 2019-11-24 00:42:28 +01:00
Mike Fährmann
244d396b0b
add '--ugoira-conv-lossless' command-line option (#432) 
and cleanup the arguments for the regular '--ugoira-conv':
- remove '-an'
- enable two-pass encoding
 2019-10-26 00:32:19 +02:00
Mike Fährmann
8af59a4bba
fix & update docs 
- update Requests links
- add example for --exec
- set '-dev' version
 2019-10-11 18:36:25 +02:00
Mike Fährmann
b06c372e4d
[postprocessor:exec] improve; add command-line option (#421) 2019-10-05 23:46:55 +02:00
Mike Fährmann
2cbbc3dec4
add a 'whitelist' to '--ugoira-conv' (#382) 2019-08-15 13:27:57 +02:00
Mike Fährmann
16c582aaf9
implement 'mtime' post-processor (#332) 
This can set a file's modification time according to a UNIX timestamp
or a datetime object from its metadata.
 2019-07-14 22:39:17 +02:00
Mike Fährmann
62097284fe
add 'download' option (#220) 2019-07-14 18:48:18 +02:00
Mike Fährmann
69205df68d
allow '-1' for infinite retries (#300) 2019-06-30 23:10:47 +02:00
Mike Fährmann
f7b5c4c3e7
use values of 'retries' options correctly 
The RE-tries option now specifies exactly that: the maximum number a
failed HTTP request is re-tried. For example a value of 2 will now
correctly stop after 3 attempts: the initial one + 2 re-tries.

The maximum wait-time now also caps at 30min and increases exponentially
for both extractor.request() and downloader.http.download().
 2019-06-30 23:10:18 +02:00
Mike Fährmann
6393b47db2
add '-A/--abort'; deprecate '--abort-on-skip' 2019-06-30 14:28:28 +02:00
Mike Fährmann
db3f52881a
add 'mtime' option 2019-06-20 17:19:44 +02:00
Mike Fährmann
bd9cb3d191
improve job class selection code 
+ consistent argument order for add_argument() calls
 2019-05-10 22:05:57 +02:00
Mike Fährmann
e64773ffdd
allow multiple post-processor command-line options (#253) 
... without overwriting any previous ones
 2019-05-10 15:32:23 +02:00
Mike Fährmann
bc26fc2439
implement '--clear-cache' 
Effectively clears all cached values from the cache database by
executing "DELETE FROM data" without any further user input.
 2019-04-25 21:31:01 +02:00
Mike Fährmann
e7d0d98c88
improve FFmpeg arguments for --ugoira-conv 2019-03-23 09:50:39 +01:00
Mike Fährmann
9fba48fbd7
[postprocessor:metadata] add '--write-tags' flag (#135) 2019-01-23 21:56:03 +01:00
Mike Fährmann
38500ad697
[postprocessor:metadata] first implementation (#135) 2019-01-05 16:39:05 +01:00
Mike Fährmann
af3f81c7d9
add '--no-check-certificate' command-line option 2018-10-07 21:46:00 +02:00
Mike Fährmann
0514d6a0ae
make --filter and --range config-file options 
The functionality of --(chapter-)filter and --(chapter-)range are now
also exposed as the following config-file options:

- extractor.*.image-filter
- extractor.*.image-range
- extractor.*.chapter-filter
- extractor.*.chapter-range

TODO: update configuration.rst
 2018-10-07 21:39:56 +02:00
Mike Fährmann
91340d9d27
[pixiv] fix ugoira test 2018-06-18 19:22:54 +02:00
Mike Fährmann
709c5d466d
add '--zip' and '--ugoira-conv' command-line options 2018-06-18 18:14:38 +02:00
Mike Fährmann
2df1a15fb8
add '-s/--simulate' to run data extraction without download 
Useful for quick testing (even though -g and -j kind of do the same)
and to fill a download archive without actually downloading the files.

-s does the same as the default behaviour, except downloading stuff.
Maybe it should get a more fitting name, as it does actually write to
disk (cache, archive)?
 2018-05-25 16:07:18 +02:00
Mike Fährmann
8704d850bf
add explicit proxy support (#76) 
- '--proxy' as command-line argument
- 'extractor.*.proxy' as config option
 2018-02-19 18:45:06 +01:00
Mike Fährmann
c0dd922c13
add '--download-archive' cmdline option 
… as well as a config file equivalent
 2018-02-01 22:00:44 +01:00
Mike Fährmann
d951f13e37
add config option for unsupported-URL file 
for consistency's sake
 2018-01-28 18:42:10 +01:00
Mike Fährmann
364e335440
smaller adjustments and improvements 
- requests and urllib3 version on 1 line
- close input file after reading from it
- use expand_path for unsupported-urls file
- remove unnecessary logging from options.py
 2018-01-27 01:05:17 +01:00
Mike Fährmann
97f4f15ec0
add option to write logging output to a file 
- '--write-log FILE' as cmdline argument
- 'output.logfile' as config file option
 2018-01-26 18:51:51 +01:00
Mike Fährmann
291369eab2
various smaller changes/additions 2017-12-06 21:45:56 +01:00
Mike Fährmann
4fb6803fa6
add option to sleep before each download 2017-12-04 17:33:10 +01:00
Mike Fährmann
0e5057b15d
remove deprecated options 2017-12-02 15:31:57 +01:00
Mike Fährmann
8f518e03f8
add options to set maximum download rate 
- -r/--limit-rate as cmdline option
- downloader.http.rate as config option

This implementation very roughly uses the idea of the token bucket
algorithm [1] and mostly uses Wget's approach [2] as inspiration.

[1] https://en.wikipedia.org/wiki/Token_bucket
[2] http://git.savannah.gnu.org/cgit/wget.git/tree/src/retr.c?h=v1.19.2&id=ba6b44f6745b14dce414761a8e4b35d31b176bba#n111
 2017-12-02 01:47:26 +01:00
Mike Fährmann
963670d73b
add options to control usage of .part files (#29) 
- '--no-part' command line option to disable them
- 'downloader.http.part' and 'downloader.text.part' config options

Disabling .part files restores the behaviour of the old downloader
implementation.
 2017-10-24 23:33:44 +02:00
Mike Fährmann
2ef3c35c98
smaller textual changes 
- swapped doc for deviantart.mature and .original
- updated gallery-dl.conf
- "transferred" -> "delegated"
 2017-10-09 23:23:19 +02:00
Mike Fährmann
b319f4bab3
smaller code and text changes 2017-10-01 18:23:40 +02:00
Mike Fährmann
8a97bd0433
rename '--images' and '--chapters' 
... to '--range' and '--chapter-range' to be consistent with
'--filter' and '--chapter-filter'
 2017-09-23 17:31:40 +02:00
Mike Fährmann
0dedbe759c
enable '--chapter-filter' 
The same filter infrastructure that can be applied to image URLS now
also works for manga chapters and other delegated URLs.

TODO: actually provide any metadata (currently supported is only
deviantart and imagefap).
 2017-09-12 16:19:00 +02:00
Mike Fährmann
81877bb5f6
add '-K' as shortcut for '--list-keywords' 2017-09-09 18:48:28 +02:00
Mike Fährmann
9b21d3f13c
add '--filter' command-line option 
This allows for image filtering via Python expressions by the same
metadata that is also used to build filenames (--list-keywords).

The usually shunned eval() function is used to evaluate
filter-expressions, but it seemed quite appropriate in this case and
shouldn't introduce any new security issues, as any attacker that could do
> gallery-dl --filter "delete-everything()" ...
could as well do
> python -c "delete-everything()"
 2017-09-08 17:52:00 +02:00
Mike Fährmann
6078ec5908
restructure the output of --help 
Using argument groups is a definite improvement over how things looked
previously, but general group membership of individual items might be
a thing to reconsider.
 2017-08-16 19:56:50 +02:00
Mike Fährmann
a804a42e23
add '--cookies' command-line option 2017-07-03 15:02:19 +02:00
Mike Fährmann
06c4cae05b
extend the output of '--list-extractors' 
It now includes category and subcategory values for
each extractor class.
 2017-06-28 18:51:47 +02:00