mirror of
https://github.com/mikf/gallery-dl.git
synced 2024-11-22 18:53:21 +01:00
5a210991b6
- add 'path-remove' option to specify the set of characters that should be removed - rename 'restrict-filenames' to 'path-restrict' - #348, #380
1825 lines
54 KiB
ReStructuredText
1825 lines
54 KiB
ReStructuredText
Configuration
|
||
#############
|
||
|
||
Contents
|
||
========
|
||
|
||
1) `Extractor Options`_
|
||
2) `Extractor-specific Options`_
|
||
3) `Downloader Options`_
|
||
4) `Output Options`_
|
||
5) `Postprocessor Options`_
|
||
6) `Miscellaneous Options`_
|
||
7) `API Tokens & IDs`_
|
||
|
||
|
||
|
||
Extractor Options
|
||
=================
|
||
|
||
|
||
Each extractor is identified by its ``category`` and ``subcategory``.
|
||
The ``category`` is the lowercase site name without any spaces or special
|
||
characters, which is usually just the module name
|
||
(``pixiv``, ``danbooru``, ...).
|
||
The ``subcategory`` is a lowercase word describing the general functionality
|
||
of that extractor (``user``, ``favorite``, ``manga``, ...).
|
||
|
||
Each one of the following options can be specified on multiple levels of the
|
||
configuration tree:
|
||
|
||
================== =====
|
||
Base level: ``extractor.<option-name>``
|
||
Category level: ``extractor.<category>.<option-name>``
|
||
Subcategory level: ``extractor.<category>.<subcategory>.<option-name>``
|
||
================== =====
|
||
|
||
A value in a "deeper" level hereby overrides a value of the same name on a
|
||
lower level. Setting the ``extractor.pixiv.filename`` value, for example, lets
|
||
you specify a general filename pattern for all the different pixiv extractors.
|
||
Using the ``extractor.pixiv.user.filename`` value lets you override this
|
||
general pattern specifically for ``PixivUserExtractor`` instances.
|
||
|
||
The ``category`` and ``subcategory`` of all extractors are included in the
|
||
output of ``gallery-dl --list-extractors``. For a specific URL these values
|
||
can also be determined by using the ``-K``/``--list-keywords`` command-line
|
||
option (see the example below).
|
||
|
||
extractor.*.filename
|
||
--------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Example ``"{manga}_c{chapter}_{page:>03}.{extension}"``
|
||
Description A `format string`_ to build the resulting filename
|
||
for a downloaded file.
|
||
|
||
The available replacement keys depend on the extractor used. A list
|
||
of keys for a specific one can be acquired by calling *gallery-dl*
|
||
with the ``-K``/``--list-keywords`` command-line option.
|
||
For example:
|
||
|
||
.. code::
|
||
|
||
$ gallery-dl -K http://seiga.nicovideo.jp/seiga/im5977527
|
||
Keywords for directory names:
|
||
-----------------------------
|
||
category
|
||
seiga
|
||
subcategory
|
||
image
|
||
|
||
Keywords for filenames:
|
||
-----------------------
|
||
category
|
||
seiga
|
||
extension
|
||
None
|
||
image-id
|
||
5977527
|
||
subcategory
|
||
image
|
||
|
||
Note: Even if the value of the ``extension`` key is missing or
|
||
``None``, it will filled in later when the file download is
|
||
starting. This key is therefore always available to provide
|
||
a valid filename extension.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.directory
|
||
---------------------
|
||
=========== =====
|
||
Type ``list`` of ``strings``
|
||
Example ``["{category}", "{manga}", "c{chapter} - {title}"]``
|
||
Description A list of `format strings`_ for the resulting target directory.
|
||
|
||
Each individual string in such a list represents a single path
|
||
segment, which will be joined together and appended to the
|
||
base-directory_ to form the complete target directory path.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.base-directory
|
||
--------------------------
|
||
=========== =====
|
||
Type |Path|_
|
||
Default ``"./gallery-dl/"``
|
||
Description Directory path used as the base for all download destinations.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.path-restrict
|
||
-------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"auto"``
|
||
Example ``"/!? (){}"``
|
||
Description Set of characters to replace with underscores (``_``)
|
||
in generated path segment names.
|
||
|
||
Special values:
|
||
|
||
* ``"auto"``: Use characters from ``"unix"`` or ``"windows"``
|
||
depending on the local operating system
|
||
* ``"unix"``: ``"/"``
|
||
* ``"windows"``: ``"\\\\|/<>:\"?*"``
|
||
|
||
Note: In a set with 2 or more characters, ``[]^-\`` need to be
|
||
escaped with backslashes, e.g. ``"\\[\\]"``
|
||
=========== =====
|
||
|
||
|
||
extractor.*.path-remove
|
||
-----------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"\\u0000-\\u001f\\u007f"`` (ASCII control characters)
|
||
Description Set of characters to remove from generated path names.
|
||
|
||
Note: In a set with 2 or more characters, ``[]^-\`` need to be
|
||
escaped with backslashes, e.g. ``"\\[\\]"``
|
||
=========== =====
|
||
|
||
|
||
extractor.*.skip
|
||
----------------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default ``true``
|
||
Description Controls the behavior when downloading files that have been
|
||
downloaded before, i.e. a file with the same filename already
|
||
exists or its ID is in a `download archive`__.
|
||
|
||
__ `extractor.*.archive`_
|
||
|
||
* ``true``: Skip downloads
|
||
* ``false``: Overwrite already existing files
|
||
|
||
* ``"abort"``: Abort the current extractor run
|
||
* ``"abort:N"``: Skip downloads and abort extractor run
|
||
after ``N`` consecutive skips
|
||
|
||
* ``"exit"``: Exit the program altogether
|
||
* ``"exit:N"``: Skip downloads and exit the program
|
||
after ``N`` consecutive skips
|
||
|
||
* ``"enumerate"``: Append a numeric suffix to the end of the
|
||
original filename (``file.ext.1``, ``file.ext.2``, etc)
|
||
=========== =====
|
||
|
||
|
||
extractor.*.sleep
|
||
-----------------
|
||
=========== =====
|
||
Type ``float``
|
||
Default ``0``
|
||
Description Number of seconds to sleep before each download.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.username & .password
|
||
--------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``null``
|
||
Description The username and password to use when attempting to log in to
|
||
another site.
|
||
|
||
Specifying username and password is required for the
|
||
``pixiv``, ``nijie``, and ``seiga``
|
||
modules and optional (but strongly recommended) for
|
||
``danbooru``, ``exhentai``, ``idolcomplex``, ``instagram``,
|
||
``luscious``, ``sankaku``, ``tsumino``, and ``twitter``.
|
||
|
||
These values can also be set via the ``-u/--username`` and
|
||
``-p/--password`` command-line options or by using a |.netrc|_ file.
|
||
(see Authentication_)
|
||
|
||
Note: The password for ``danbooru`` is the API key found in your
|
||
user profile, not the password for your account.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.netrc
|
||
-----------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Enable the use of |.netrc|_ authentication data.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.cookies
|
||
-------------------
|
||
=========== =====
|
||
Type |Path|_ or ``object``
|
||
Default ``null``
|
||
Description Source to read additional cookies from.
|
||
|
||
* If this is a |Path|_, it specifies a
|
||
Mozilla/Netscape format cookies.txt file.
|
||
* If this is an ``object``, its key-value pairs, which should both
|
||
be ``strings``, will be used as cookie-names and -values.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.proxy
|
||
-----------------
|
||
=========== =====
|
||
Type ``string`` or ``object``
|
||
Default ``null``
|
||
Description Proxy (or proxies) to be used for remote connections.
|
||
|
||
* If this is a ``string``, it is the proxy URL for all
|
||
outgoing requests.
|
||
* If this is an ``object``, it is a scheme-to-proxy mapping to
|
||
specify different proxy URLs for each scheme.
|
||
It is also possible to set a proxy for a specific host by using
|
||
``scheme://host`` as key.
|
||
See `Requests' proxy documentation`_ for more details.
|
||
|
||
Example:
|
||
|
||
.. code::
|
||
|
||
{
|
||
"http": "http://10.10.1.10:3128",
|
||
"https": "http://10.10.1.10:1080",
|
||
"http://10.20.1.128": "http://10.10.1.10:5323"
|
||
}
|
||
|
||
Note: All proxy URLs should include a scheme,
|
||
otherwise ``http://`` is assumed.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.user-agent
|
||
----------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"``
|
||
Description User-Agent header value to be used for HTTP requests.
|
||
|
||
Note: This option has no effect on `pixiv` and
|
||
`readcomiconline` extractors, as these need specific values to
|
||
function correctly.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.keywords
|
||
--------------------
|
||
=========== =====
|
||
Type ``object``
|
||
Example ``{"type": "Pixel Art", "type_id": 123}``
|
||
Description Additional key-value pairs to be added to each metadata dictionary.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.keywords-default
|
||
----------------------------
|
||
=========== =====
|
||
Type any
|
||
Default ``"None"``
|
||
Description Default value used for missing or undefined keyword names in
|
||
format strings.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.category-transfer
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default Extractor-specific
|
||
Description Transfer an extractor's (sub)category values to all child
|
||
extractors spawned by it, to let them inherit their parent's
|
||
config options.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.archive
|
||
-------------------
|
||
=========== =====
|
||
Type |Path|_
|
||
Default ``null``
|
||
Description File to store IDs of downloaded files in. Downloads of files
|
||
already recorded in this archive file will be skipped_.
|
||
|
||
The resulting archive file is not a plain text file but an SQLite3
|
||
database, as either lookup operations are significantly faster or
|
||
memory requirements are significantly lower when the
|
||
amount of stored IDs gets reasonably large.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.archive-format
|
||
--------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Example ``"{id}_{offset}"``
|
||
Description An alternative `format string`_ to build archive IDs with.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.postprocessors
|
||
--------------------------
|
||
=========== =====
|
||
Type ``list`` of |Postprocessor Configuration|_ objects
|
||
Example .. code::
|
||
|
||
[
|
||
{"name": "zip", "compression": "zip"},
|
||
{"name": "exec", "command": ["/home/foobar/script", "{category}", "{image_id}"]}
|
||
]
|
||
|
||
Description A list of post-processors to be applied to each downloaded file
|
||
in the same order as they are specified.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.retries
|
||
-------------------
|
||
=========== =====
|
||
Type ``integer``
|
||
Default ``4``
|
||
Description Maximum number of times a failed HTTP request is retried before
|
||
giving up or ``-1`` for infinite retries.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.timeout
|
||
-------------------
|
||
=========== =====
|
||
Type ``float`` or ``null``
|
||
Default ``30``
|
||
Description Amount of time (in seconds) to wait for a successful connection
|
||
and response from a remote server.
|
||
|
||
This value gets internally used as the |timeout|_ parameter for the
|
||
|requests.request()|_ method.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.verify
|
||
------------------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default ``true``
|
||
Description Controls whether to verify SSL/TLS certificates for HTTPS requests.
|
||
|
||
If this is a ``string``, it must be the path to a CA bundle to use
|
||
instead of the default certificates.
|
||
|
||
This value gets internally used as the |verify|_ parameter for the
|
||
|requests.request()|_ method.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.download
|
||
--------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Controls whether to download media files.
|
||
|
||
Setting this to ``false`` won't download any files, but all other
|
||
functions (postprocessors_, `download archive`_, etc.)
|
||
will be executed as normal.
|
||
=========== =====
|
||
|
||
.. _postprocessors: `extractor.*.postprocessors`_
|
||
.. _download archive: `extractor.*.archive`_
|
||
|
||
|
||
extractor.*.image-range
|
||
-----------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Example | ``"10-20"``,
|
||
| ``"-5, 10, 30-50, 100-"``
|
||
Description Index-range(s) specifying which images to download.
|
||
|
||
Note: The index of the first image is ``1``.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.chapter-range
|
||
-------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Description Like `image-range`__, but applies to delegated URLs
|
||
like manga-chapters, etc.
|
||
=========== =====
|
||
|
||
__ `extractor.*.image-range`_
|
||
|
||
|
||
extractor.*.image-filter
|
||
------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Example | ``"width >= 1200 and width/height > 1.2"``,
|
||
| ``"re.search(r'foo(bar)+', description)"``
|
||
Description | Python expression controlling which images to download.
|
||
| Files for which the expression evaluates to ``False``
|
||
are ignored.
|
||
| Available keys are the filename-specific ones listed
|
||
by ``-K`` or ``-j``.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.chapter-filter
|
||
--------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Description Like `image-filter`__, but applies to delegated URLs
|
||
like manga-chapters, etc.
|
||
=========== =====
|
||
|
||
__ `extractor.*.image-filter`_
|
||
|
||
|
||
extractor.*.image-unique
|
||
------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Ignore image URLs that have been encountered before during the
|
||
current extractor run.
|
||
=========== =====
|
||
|
||
|
||
extractor.*.chapter-unique
|
||
--------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Like `image-unique`__, but applies to delegated URLs
|
||
like manga-chapters, etc.
|
||
=========== =====
|
||
|
||
__ `extractor.*.image-unique`_
|
||
|
||
|
||
extractor.*.date-format
|
||
----------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"%Y-%m-%dT%H:%M:%S"``
|
||
Description Format string used to parse ``string`` values of
|
||
`date-min` and `date-max`.
|
||
|
||
See |strptime|_ for a list of formatting directives.
|
||
=========== =====
|
||
|
||
|
||
|
||
Extractor-specific Options
|
||
==========================
|
||
|
||
|
||
extractor.artstation.external
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Try to follow external URLs of embedded players.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.extra
|
||
--------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Download extra Sta.sh resources from description texts.
|
||
|
||
Note: Enabling this option also enables deviantart.metadata_.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.flat
|
||
-------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Select the directory structure created by the Gallery- and
|
||
Favorite-Extractors.
|
||
|
||
* ``true``: Use a flat directory structure.
|
||
* ``false``: Collect a list of all gallery-folders or
|
||
favorites-collections and transfer any further work to other
|
||
extractors (``folder`` or ``collection``), which will then
|
||
create individual subdirectories for each of them.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.folders
|
||
----------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Provide a ``folders`` metadata field that contains the names of all
|
||
folders a deviation is present in.
|
||
|
||
Note: Gathering this information requires a lot of API calls.
|
||
Use with caution.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.journals
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"html"``
|
||
Description Selects the output format of journal entries.
|
||
|
||
* ``"html"``: HTML with (roughly) the same layout as on DeviantArt.
|
||
* ``"text"``: Plain text with image references and HTML tags removed.
|
||
* ``"none"``: Don't download journals.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.mature
|
||
---------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Enable mature content.
|
||
|
||
This option simply sets the |mature_content|_ parameter for API
|
||
calls to either ``"true"`` or ``"false"`` and does not do any other
|
||
form of content filtering.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.metadata
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Request extended metadata for deviation objects to additionally
|
||
provide ``description``, ``tags``, ``license`` and ``is_watching``
|
||
fields.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.original
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default ``true``
|
||
Description Download original files if available.
|
||
|
||
Setting this option to ``"images"`` only downloads original
|
||
files if they are images and falls back to preview versions for
|
||
everything else (archives, etc.).
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.quality
|
||
----------------------------
|
||
=========== =====
|
||
Type ``integer``
|
||
Default ``100``
|
||
Description JPEG compression quality for newer images hosted on wixmp servers.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.refresh-token
|
||
----------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``null``
|
||
Description The ``refresh_token`` value you get from linking your
|
||
DeviantArt account to *gallery-dl*.
|
||
|
||
Using a ``refresh_token`` allows you to access private or otherwise
|
||
not publicly available deviations.
|
||
=========== =====
|
||
|
||
|
||
extractor.deviantart.wait-min
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``integer``
|
||
Default ``0``
|
||
Description Minimum wait time in seconds before API requests.
|
||
|
||
Note: This value will internally be rounded up
|
||
to the next power of 2.
|
||
=========== =====
|
||
|
||
|
||
extractor.exhentai.limits
|
||
-------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Check image download limits
|
||
and stop extraction when they are exceeded.
|
||
=========== =====
|
||
|
||
|
||
extractor.exhentai.original
|
||
---------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Download full-sized original images if available.
|
||
=========== =====
|
||
|
||
|
||
extractor.exhentai.wait-min & .wait-max
|
||
---------------------------------------
|
||
=========== =====
|
||
Type ``float``
|
||
Default ``3.0`` and ``6.0``
|
||
Description Minimum and maximum wait time in seconds between each image
|
||
|
||
ExHentai detects and blocks automated downloaders.
|
||
*gallery-dl* waits a randomly selected number of
|
||
seconds between ``wait-min`` and ``wait-max`` after
|
||
each image to prevent getting blocked.
|
||
=========== =====
|
||
|
||
|
||
extractor.flickr.access-token & .access-token-secret
|
||
----------------------------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``null``
|
||
Description The ``access_token`` and ``access_token_secret`` values you get
|
||
from linking your Flickr account to *gallery-dl*.
|
||
=========== =====
|
||
|
||
|
||
extractor.flickr.videos
|
||
-----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Extract and download videos.
|
||
=========== =====
|
||
|
||
|
||
extractor.flickr.size-max
|
||
--------------------------
|
||
=========== =====
|
||
Type ``integer`` or ``string``
|
||
Default ``null``
|
||
Description Sets the maximum allowed size for downloaded images.
|
||
|
||
* If this is an ``integer``, it specifies the maximum image dimension
|
||
(width and height) in pixels.
|
||
* If this is a ``string``, it should be one of Flickr's format specifiers
|
||
(``"Original"``, ``"Large"``, ... or ``"o"``, ``"k"``, ``"h"``,
|
||
``"l"``, ...) to use as an upper limit.
|
||
=========== =====
|
||
|
||
|
||
extractor.gelbooru.api
|
||
----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Enable use of Gelbooru's API.
|
||
|
||
Set this value to `false` if the API has been disabled to switch
|
||
to manual information extraction.
|
||
=========== =====
|
||
|
||
|
||
extractor.gfycat.format
|
||
-----------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"mp4"``
|
||
Description The name of the preferred animation format, which can be one of
|
||
``"mp4"``, ``"webm"``, ``"gif"``, ``"webp"`` or ``"mjpg"``.
|
||
|
||
If the selected format is not available, ``"mp4"``, ``"webm"``
|
||
and ``"gif"`` (in that order) will be tried instead, until an
|
||
available format is found.
|
||
=========== =====
|
||
|
||
|
||
extractor.imgur.mp4
|
||
-------------------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default ``true``
|
||
Description Controls whether to choose the GIF or MP4 version of an animation.
|
||
|
||
* ``true``: Follow Imgur's advice and choose MP4 if the
|
||
``prefer_video`` flag in an image's metadata is set.
|
||
* ``false``: Always choose GIF.
|
||
* ``"always"``: Always choose MP4.
|
||
=========== =====
|
||
|
||
|
||
extractor.kissmanga.captcha
|
||
---------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"stop"``
|
||
Description Controls how to handle redirects to CAPTCHA pages.
|
||
|
||
* ``"stop``: Stop the current extractor run.
|
||
* ``"wait``: Ask the user to solve the CAPTCHA and wait.
|
||
=========== =====
|
||
|
||
|
||
extractor.oauth.browser
|
||
-----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Controls how a user is directed to an OAuth authorization site.
|
||
|
||
* ``true``: Use Python's |webbrowser.open()|_ method to automatically
|
||
open the URL in the user's browser.
|
||
* ``false``: Ask the user to copy & paste an URL from the terminal.
|
||
=========== =====
|
||
|
||
|
||
extractor.photobucket.subalbums
|
||
-------------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Download subalbums.
|
||
=========== =====
|
||
|
||
|
||
extractor.pixiv.ugoira
|
||
----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Download Pixiv's Ugoira animations or ignore them.
|
||
|
||
These animations come as a ``.zip`` file containing all the single
|
||
animation frames in JPEG format.
|
||
=========== =====
|
||
|
||
|
||
extractor.plurk.comments
|
||
------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Also search Plurk comments for URLs.
|
||
=========== =====
|
||
|
||
|
||
extractor.reactor.wait-min & .wait-max
|
||
--------------------------------------
|
||
=========== =====
|
||
Type ``float``
|
||
Default ``3.0`` and ``6.0``
|
||
Description Minimum and maximum wait time in seconds between HTTP requests
|
||
during the extraction process.
|
||
=========== =====
|
||
|
||
|
||
extractor.readcomiconline.captcha
|
||
---------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"stop"``
|
||
Description Controls how to handle redirects to CAPTCHA pages.
|
||
|
||
* ``"stop``: Stop the current extractor run.
|
||
* ``"wait``: Ask the user to solve the CAPTCHA and wait.
|
||
=========== =====
|
||
|
||
|
||
extractor.recursive.blacklist
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``list`` of ``strings``
|
||
Default ``["directlink", "oauth", "recursive", "test"]``
|
||
Description A list of extractor categories which should be ignored when using
|
||
the ``recursive`` extractor.
|
||
=========== =====
|
||
|
||
|
||
extractor.reddit.comments
|
||
-------------------------
|
||
=========== =====
|
||
Type ``integer`` or ``string``
|
||
Default ``500``
|
||
Description The value of the ``limit`` parameter when loading
|
||
a submission and its comments.
|
||
This number (roughly) specifies the total amount of comments
|
||
being retrieved with the first API call.
|
||
|
||
Reddit's internal default and maximum values for this parameter
|
||
appear to be 200 and 500 respectively.
|
||
|
||
The value `0` ignores all comments and significantly reduces the
|
||
time required when scanning a subreddit.
|
||
=========== =====
|
||
|
||
|
||
extractor.reddit.morecomments
|
||
-----------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Retrieve additional comments by resolving the ``more`` comment
|
||
stubs in the base comment tree.
|
||
|
||
This requires 1 additional API call for every 100 extra comments.
|
||
=========== =====
|
||
|
||
|
||
extractor.reddit.date-min & .date-max
|
||
-------------------------------------
|
||
=========== =====
|
||
Type |Date|_
|
||
Default ``0`` and ``253402210800`` (timestamp of |datetime.max|_)
|
||
Description Ignore all submissions posted before/after this date.
|
||
=========== =====
|
||
|
||
|
||
extractor.reddit.id-min & .id-max
|
||
---------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Example ``"6kmzv2"``
|
||
Description Ignore all submissions posted before/after the submission with
|
||
this ID.
|
||
=========== =====
|
||
|
||
|
||
extractor.reddit.recursion
|
||
--------------------------
|
||
=========== =====
|
||
Type ``integer``
|
||
Default ``0``
|
||
Description Reddit extractors can recursively visit other submissions
|
||
linked to in the initial set of submissions.
|
||
This value sets the maximum recursion depth.
|
||
|
||
Special values:
|
||
|
||
* ``0``: Recursion is disabled
|
||
* ``-1``: Infinite recursion (don't do this)
|
||
=========== =====
|
||
|
||
|
||
extractor.reddit.refresh-token
|
||
------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``null``
|
||
Description The ``refresh_token`` value you get from linking your
|
||
Reddit account to *gallery-dl*.
|
||
|
||
Using a ``refresh_token`` allows you to access private or otherwise
|
||
not publicly available subreddits, given that your account is
|
||
authorized to do so,
|
||
but requests to the reddit API are going to be rate limited
|
||
at 600 requests every 10 minutes/600 seconds.
|
||
=========== =====
|
||
|
||
|
||
extractor.sankaku.wait-min & .wait-max
|
||
--------------------------------------
|
||
=========== =====
|
||
Type ``float``
|
||
Default ``3.0`` and ``6.0``
|
||
Description Minimum and maximum wait time in seconds between each image
|
||
|
||
Sankaku Channel responds with ``429 Too Many Requests`` if it
|
||
receives too many HTTP requests in a certain amount of time.
|
||
Waiting a few seconds between each request tries to prevent that.
|
||
=========== =====
|
||
|
||
|
||
extractor.smugmug.videos
|
||
------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Download video files.
|
||
=========== =====
|
||
|
||
|
||
extractor.tumblr.avatar
|
||
-----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Download blog avatars.
|
||
=========== =====
|
||
|
||
|
||
extractor.tumblr.date-min & .date-max
|
||
-------------------------------------
|
||
=========== =====
|
||
Type |Date|_
|
||
Default ``0`` and ``null``
|
||
Description Ignore all posts published before/after this date.
|
||
=========== =====
|
||
|
||
|
||
extractor.tumblr.external
|
||
-------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Follow external URLs (e.g. from "Link" posts) and try to extract
|
||
images from them.
|
||
=========== =====
|
||
|
||
|
||
extractor.tumblr.inline
|
||
-----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Search posts for inline images and videos.
|
||
=========== =====
|
||
|
||
|
||
extractor.tumblr.reblogs
|
||
------------------------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default ``true``
|
||
Description * ``true``: Extract media from reblogged posts
|
||
* ``false``: Skip reblogged posts
|
||
* ``"same-blog"``: Skip reblogged posts unless the original post
|
||
is from the same blog
|
||
=========== =====
|
||
|
||
|
||
extractor.tumblr.posts
|
||
----------------------
|
||
=========== =====
|
||
Type ``string`` or ``list`` of ``strings``
|
||
Default ``"all"``
|
||
Example ``"video,audio,link"`` or ``["video", "audio", "link"]``
|
||
Description A (comma-separated) list of post types to extract images, etc. from.
|
||
|
||
Possible types are ``text``, ``quote``, ``link``, ``answer``,
|
||
``video``, ``audio``, ``photo``, ``chat``.
|
||
|
||
You can use ``"all"`` instead of listing all types separately.
|
||
=========== =====
|
||
|
||
|
||
extractor.twitter.content
|
||
-------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Extract tweet text as ``content`` metadata.
|
||
=========== =====
|
||
|
||
|
||
extractor.twitter.retweets
|
||
--------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Extract images from retweets.
|
||
=========== =====
|
||
|
||
|
||
extractor.twitter.videos
|
||
------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Use `youtube-dl`_ to download from video tweets.
|
||
=========== =====
|
||
|
||
|
||
extractor.wallhaven.api-key
|
||
---------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``null``
|
||
Description Your `API Key <https://wallhaven.cc/settings/account>`__ to use
|
||
your account's browsing settings and default filters when searching.
|
||
|
||
See https://wallhaven.cc/help/api for more information.
|
||
=========== =====
|
||
|
||
|
||
extractor.[booru].tags
|
||
----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Categorize tags by their respective types
|
||
and provide them as ``tags_<type>`` metadata fields.
|
||
|
||
Note: This requires 1 additional HTTP request for each post.
|
||
=========== =====
|
||
|
||
|
||
extractor.[manga-extractor].chapter-reverse
|
||
-------------------------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Reverse the order of chapter URLs extracted from manga pages.
|
||
|
||
* ``true``: Start with the latest chapter
|
||
* ``false``: Start with the first chapter
|
||
=========== =====
|
||
|
||
|
||
|
||
Downloader Options
|
||
==================
|
||
|
||
|
||
downloader.*.enabled
|
||
--------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Enable/Disable this downloader module.
|
||
=========== =====
|
||
|
||
|
||
downloader.*.mtime
|
||
------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Use |Last-Modified|_ HTTP response headers
|
||
to set file modification times.
|
||
=========== =====
|
||
|
||
|
||
downloader.*.part
|
||
-----------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Controls the use of ``.part`` files during file downloads.
|
||
|
||
* ``true``: Write downloaded data into ``.part`` files and rename
|
||
them upon download completion. This mode additionally supports
|
||
resuming incomplete downloads.
|
||
* ``false``: Do not use ``.part`` files and write data directly
|
||
into the actual output files.
|
||
=========== =====
|
||
|
||
|
||
downloader.*.part-directory
|
||
---------------------------
|
||
=========== =====
|
||
Type |Path|_
|
||
Default ``null``
|
||
Description Alternate location for ``.part`` files.
|
||
|
||
Missing directories will be created as needed.
|
||
If this value is ``null``, ``.part`` files are going to be stored
|
||
alongside the actual output files.
|
||
=========== =====
|
||
|
||
|
||
downloader.*.rate
|
||
-----------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``null``
|
||
Examples ``"32000"``, ``"500k"``, ``"2.5M"``
|
||
Description Maximum download rate in bytes per second.
|
||
|
||
Possible values are valid integer or floating-point numbers
|
||
optionally followed by one of ``k``, ``m``. ``g``, ``t`` or ``p``.
|
||
These suffixes are case-insensitive.
|
||
=========== =====
|
||
|
||
|
||
downloader.*.retries
|
||
--------------------
|
||
=========== =====
|
||
Type ``integer``
|
||
Default `extractor.*.retries`_
|
||
Description Maximum number of retries during file downloads
|
||
or ``-1`` for infinite retries.
|
||
=========== =====
|
||
|
||
|
||
downloader.*.timeout
|
||
--------------------
|
||
=========== =====
|
||
Type ``float`` or ``null``
|
||
Default `extractor.*.timeout`_
|
||
Description Connection timeout during file downloads.
|
||
=========== =====
|
||
|
||
|
||
downloader.*.verify
|
||
-------------------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default `extractor.*.verify`_
|
||
Description Certificate validation during file downloads.
|
||
=========== =====
|
||
|
||
|
||
downloader.http.adjust-extensions
|
||
---------------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Check the file headers of ``jpg``, ``png``, and ``gif`` files
|
||
and adjust their filename extensions if they do not match.
|
||
=========== =====
|
||
|
||
|
||
downloader.ytdl.format
|
||
----------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default youtube-dl's default, currently ``"bestvideo+bestaudio/best"``
|
||
Description Video `format selection
|
||
<https://github.com/ytdl-org/youtube-dl#format-selection>`__
|
||
directly passed to youtube-dl.
|
||
=========== =====
|
||
|
||
|
||
downloader.ytdl.forward-cookies
|
||
-------------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Forward cookies to youtube-dl.
|
||
=========== =====
|
||
|
||
|
||
downloader.ytdl.logging
|
||
-----------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description | Route youtube-dl's output through gallery-dl's logging system.
|
||
| Otherwise youtube-dl will write its output directly to stdout/stderr.
|
||
|
||
Note: Set ``quiet`` and ``no_warnings`` in
|
||
`downloader.ytdl.raw-options`_ to ``true`` to suppress all output.
|
||
=========== =====
|
||
|
||
|
||
downloader.ytdl.raw-options
|
||
---------------------------
|
||
=========== =====
|
||
Type ``object``
|
||
Example .. code::
|
||
|
||
{
|
||
"quiet": true,
|
||
"writesubtitles": true,
|
||
"merge_output_format": "mkv"
|
||
}
|
||
|
||
Description | Additional options passed directly to the ``YoutubeDL`` constructor.
|
||
| All available options can be found in `youtube-dl's docstrings
|
||
<https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L138-L318>`__.
|
||
=========== =====
|
||
|
||
|
||
|
||
Output Options
|
||
==============
|
||
|
||
|
||
output.mode
|
||
-----------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"auto"``
|
||
Description Controls the output string format and status indicators.
|
||
|
||
* ``"null"``: No output
|
||
* ``"pipe"``: Suitable for piping to other processes or files
|
||
* ``"terminal"``: Suitable for the standard Windows console
|
||
* ``"color"``: Suitable for terminals that understand ANSI escape codes and colors
|
||
* ``"auto"``: Automatically choose the best suitable output mode
|
||
=========== =====
|
||
|
||
|
||
output.shorten
|
||
--------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Controls whether the output strings should be shortened to fit
|
||
on one console line.
|
||
=========== =====
|
||
|
||
|
||
output.progress
|
||
---------------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default ``true``
|
||
Description Controls the progress indicator when *gallery-dl* is run with
|
||
multiple URLs as arguments.
|
||
|
||
* ``true``: Show the default progress indicator
|
||
(``"[{current}/{total}] {url}"``)
|
||
* ``false``: Do not show any progress indicator
|
||
* Any ``string``: Show the progress indicator using this
|
||
as a custom `format string`_. Possible replacement keys are
|
||
``current``, ``total`` and ``url``.
|
||
=========== =====
|
||
|
||
|
||
output.log
|
||
----------
|
||
=========== =====
|
||
Type ``string`` or |Logging Configuration|_
|
||
Default ``"[{name}][{levelname}] {message}"``
|
||
Description Configuration for standard logging output to stderr.
|
||
|
||
If this is a simple ``string``, it specifies
|
||
the format string for logging messages.
|
||
=========== =====
|
||
|
||
|
||
output.logfile
|
||
--------------
|
||
=========== =====
|
||
Type |Path|_ or |Logging Configuration|_
|
||
Default ``null``
|
||
Description File to write logging output to.
|
||
=========== =====
|
||
|
||
|
||
output.unsupportedfile
|
||
----------------------
|
||
=========== =====
|
||
Type |Path|_ or |Logging Configuration|_
|
||
Default ``null``
|
||
Description File to write external URLs unsupported by *gallery-dl* to.
|
||
|
||
The default format string here is ``"{message}"``.
|
||
=========== =====
|
||
|
||
|
||
output.num-to-str
|
||
-----------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Convert numeric values (``integer`` or ``float``) to ``string``
|
||
before outputting them as JSON.
|
||
=========== =====
|
||
|
||
|
||
|
||
Postprocessor Options
|
||
=====================
|
||
|
||
|
||
classify
|
||
--------
|
||
|
||
Categorize files by filename extension
|
||
|
||
classify.mapping
|
||
----------------
|
||
=========== =====
|
||
Type ``object``
|
||
Default .. code::
|
||
|
||
{
|
||
"Pictures" : ["jpg", "jpeg", "png", "gif", "bmp", "svg", "webp"],
|
||
"Video" : ["flv", "ogv", "avi", "mp4", "mpg", "mpeg", "3gp", "mkv", "webm", "vob", "wmv"],
|
||
"Music" : ["mp3", "aac", "flac", "ogg", "wma", "m4a", "wav"],
|
||
"Archives" : ["zip", "rar", "7z", "tar", "gz", "bz2"]
|
||
}
|
||
|
||
Description A mapping from directory names to filename extensions that should
|
||
be stored in them.
|
||
|
||
Files with an extension not listed will be ignored and stored
|
||
in their default location.
|
||
=========== =====
|
||
|
||
|
||
exec
|
||
----
|
||
|
||
Execute external commands.
|
||
|
||
exec.async
|
||
----------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Controls whether to wait for a subprocess to finish
|
||
or to let it run asynchronously.
|
||
=========== =====
|
||
|
||
exec.command
|
||
------------
|
||
=========== =====
|
||
Type ``list`` of ``strings``
|
||
Example ``["echo", "{user[account]}", "{id}"]``
|
||
Description The command to run.
|
||
|
||
Each element of this list is treated as a `format string`_ using
|
||
the files' metadata.
|
||
=========== =====
|
||
|
||
|
||
metadata
|
||
--------
|
||
|
||
Write image metadata to separate files
|
||
|
||
metadata.mode
|
||
-------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"json"``
|
||
Description Select how to write metadata.
|
||
|
||
* ``"json"``: all metadata using `json.dump()
|
||
<https://docs.python.org/3/library/json.html#json.dump>`_
|
||
* ``"tags"``: ``tags`` separated by newlines
|
||
* ``"custom"``: result of applying `metadata.format`_ to a file's
|
||
metadata dictionary
|
||
=========== =====
|
||
|
||
metadata.extension
|
||
------------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"json"`` or ``"txt"``
|
||
Description Filename extension for metadata files.
|
||
=========== =====
|
||
|
||
metadata.format
|
||
---------------
|
||
=========== =====
|
||
Type ``string``
|
||
Example ``"tags:\n\n{tags:J\n}\n"``
|
||
Description Custom format string to build content of metadata files.
|
||
|
||
Note: Only applies for ``"mode": "custom"``.
|
||
=========== =====
|
||
|
||
|
||
mtime
|
||
-----
|
||
|
||
Set file modification time according to its metadata
|
||
|
||
mtime.key
|
||
---------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"date"``
|
||
Description Name of the metadata field whose value should be used.
|
||
|
||
This value must either be a UNIX timestamp or a
|
||
|datetime|_ object.
|
||
=========== =====
|
||
|
||
|
||
ugoira
|
||
------
|
||
|
||
Convert Pixiv Ugoira to WebM using `FFmpeg <https://www.ffmpeg.org/>`__.
|
||
|
||
ugoira.extension
|
||
----------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"webm"``
|
||
Description Filename extension for the resulting video files.
|
||
=========== =====
|
||
|
||
ugoira.ffmpeg-args
|
||
------------------
|
||
=========== =====
|
||
Type ``list`` of ``strings``
|
||
Default ``null``
|
||
Example ``["-c:v", "libvpx-vp9", "-an", "-b:v", "2M"]``
|
||
Description Additional FFmpeg command-line arguments.
|
||
=========== =====
|
||
|
||
ugoira.ffmpeg-location
|
||
----------------------
|
||
=========== =====
|
||
Type |Path|_
|
||
Default ``"ffmpeg"``
|
||
Description Location of the ``ffmpeg`` (or ``avconv``) executable to use.
|
||
=========== =====
|
||
|
||
ugoira.ffmpeg-output
|
||
--------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Show FFmpeg output.
|
||
=========== =====
|
||
|
||
ugoira.ffmpeg-twopass
|
||
---------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Enable Two-Pass encoding.
|
||
=========== =====
|
||
|
||
ugoira.framerate
|
||
----------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"auto"``
|
||
Description Controls the frame rate argument (``-r``) for FFmpeg
|
||
|
||
* ``"auto"``: Automatically assign a fitting frame rate
|
||
based on delays between frames.
|
||
* any other ``string``: Use this value as argument for ``-r``.
|
||
* ``null`` or an empty ``string``: Don't set an explicit frame rate.
|
||
=========== =====
|
||
|
||
ugoira.keep-files
|
||
-----------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Keep ZIP archives after conversion.
|
||
=========== =====
|
||
|
||
ugoira.libx264-prevent-odd
|
||
--------------------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``true``
|
||
Description Prevent ``"width/height not divisible by 2"`` errors
|
||
when using ``libx264`` or ``libx265`` encoders
|
||
by applying a simple cropping filter. See this `Stack Overflow
|
||
thread <https://stackoverflow.com/questions/20847674>`__
|
||
for more information.
|
||
|
||
This option, when ``libx264/5`` is used, automatically
|
||
adds ``["-vf", "crop=iw-mod(iw\\,2):ih-mod(ih\\,2)"]``
|
||
to the list of FFmpeg command-line arguments
|
||
to reduce an odd width/height by 1 pixel and make them even.
|
||
=========== =====
|
||
|
||
|
||
zip
|
||
---
|
||
|
||
Store files in a ZIP archive.
|
||
|
||
zip.compression
|
||
---------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"store"``
|
||
Description Compression method to use when writing the archive.
|
||
|
||
Possible values are ``"store"``, ``"zip"``, ``"bzip2"``, ``"lzma"``.
|
||
=========== =====
|
||
|
||
zip.extension
|
||
-------------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"zip"``
|
||
Description Filename extension for the created ZIP archive.
|
||
=========== =====
|
||
|
||
zip.keep-files
|
||
--------------
|
||
=========== =====
|
||
Type ``bool``
|
||
Default ``false``
|
||
Description Keep the actual files after writing them to a ZIP archive.
|
||
=========== =====
|
||
|
||
zip.mode
|
||
--------
|
||
=========== =====
|
||
Type ``string``
|
||
Default ``"default"``
|
||
Description * ``"default"``: Write the central directory file header
|
||
once after everything is done or an exception is raised.
|
||
|
||
* ``"safe"``: Update the central directory file header
|
||
each time a file is stored in a ZIP archive.
|
||
|
||
This greatly reduces the chance a ZIP archive gets corrupted in
|
||
case the Python interpreter gets shut down unexpectedly
|
||
(power outage, SIGKILL) but is also a lot slower.
|
||
=========== =====
|
||
|
||
|
||
|
||
Miscellaneous Options
|
||
=====================
|
||
|
||
|
||
cache.file
|
||
----------
|
||
=========== =====
|
||
Type |Path|_
|
||
Default * |tempfile.gettempdir()|_ + ``".gallery-dl.cache"`` on Windows
|
||
* (``$XDG_CACHE_HOME`` or ``"~/.cache"``) + ``"/gallery-dl/cache.sqlite3"`` on all other platforms
|
||
Description Path of the SQLite3 database used to cache login sessions,
|
||
cookies and API tokens across `gallery-dl` invocations.
|
||
|
||
Set this option to ``null`` or an invalid path to disable
|
||
this cache.
|
||
=========== =====
|
||
|
||
|
||
ciphers
|
||
-------
|
||
=========== =====
|
||
Type ``bool`` or ``string``
|
||
Default ``true``
|
||
Description * ``true``: Update urllib3's default cipher list
|
||
* ``false``: Leave the default cipher list as is
|
||
* Any ``string``: Replace urllib3's default ciphers with these
|
||
(See `SSLContext.set_ciphers() <https://docs.python.org/3/library/ssl.html#ssl.SSLContext.set_ciphers>`__
|
||
for details)
|
||
=========== =====
|
||
|
||
|
||
|
||
API Tokens & IDs
|
||
================
|
||
|
||
|
||
All configuration keys listed in this section have fully functional default
|
||
values embedded into *gallery-dl* itself, but if things unexpectedly break
|
||
or you want to use your own personal client credentials, you can follow these
|
||
instructions to get an alternative set of API tokens and IDs.
|
||
|
||
|
||
extractor.deviantart.client-id & .client-secret
|
||
-----------------------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
How To * login and visit DeviantArt's
|
||
`Applications & Keys <https://www.deviantart.com/developers/apps>`__
|
||
section
|
||
* click "Register your Application"
|
||
* scroll to "OAuth2 Redirect URI Whitelist (Required)"
|
||
and enter "https://mikf.github.io/gallery-dl/oauth-redirect.html"
|
||
* click "Save" (top right)
|
||
* copy ``client_id`` and ``client_secret`` of your new
|
||
application and put them in your configuration file
|
||
=========== =====
|
||
|
||
|
||
extractor.flickr.api-key & .api-secret
|
||
--------------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
How To * login and `Create an App <https://www.flickr.com/services/apps/create/apply/>`__
|
||
in Flickr's `App Garden <https://www.flickr.com/services/>`__
|
||
* click "APPLY FOR A NON-COMMERCIAL KEY"
|
||
* fill out the form with a random name and description
|
||
and click "SUBMIT"
|
||
* copy ``Key`` and ``Secret`` and put them in your configuration
|
||
file
|
||
=========== =====
|
||
|
||
|
||
extractor.pawoo.access-token
|
||
----------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
How To
|
||
=========== =====
|
||
|
||
|
||
extractor.reddit.client-id & .user-agent
|
||
----------------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
How To * login and visit the `apps <https://www.reddit.com/prefs/apps/>`__
|
||
section of your account's preferences
|
||
* click the "are you a developer? create an app..." button
|
||
* fill out the form, choose "installed app", preferably set
|
||
"http://localhost:6414/" as "redirect uri" and finally click
|
||
"create app"
|
||
* copy the client id (third line, under your application's name and
|
||
"installed app") and put it in your configuration file
|
||
* use "``Python:<application name>:v1.0 (by /u/<username>)``" as
|
||
user-agent and replace ``<application name>`` and ``<username>``
|
||
accordingly (see Reddit's
|
||
`API access rules <https://github.com/reddit/reddit/wiki/API>`__)
|
||
=========== =====
|
||
|
||
|
||
extractor.smugmug.api-key & .api-secret
|
||
---------------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
How To * login and `Apply for an API Key <https://api.smugmug.com/api/developer/apply>`__
|
||
* use a random name and description,
|
||
set "Type" to "Application", "Platform" to "All",
|
||
and "Use" to "Non-Commercial"
|
||
* fill out the two checkboxes at the bottom and click "Apply"
|
||
* copy ``API Key`` and ``API Secret``
|
||
and put them in your configuration file
|
||
=========== =====
|
||
|
||
|
||
extractor.tumblr.api-key & .api-secret
|
||
--------------------------------------
|
||
=========== =====
|
||
Type ``string``
|
||
How To * login and visit Tumblr's
|
||
`Applications <https://www.tumblr.com/oauth/apps>`__ section
|
||
* click "Register application"
|
||
* fill out the form: use a random name and description, set
|
||
https://example.org/ as "Application Website" and "Default
|
||
callback URL"
|
||
* solve Google's "I'm not a robot" challenge and click "Register"
|
||
* click "Show secret key" (below "OAuth Consumer Key")
|
||
* copy your ``OAuth Consumer Key`` and ``Secret Key``
|
||
and put them in your configuration file
|
||
=========== =====
|
||
|
||
|
||
|
||
Custom Types
|
||
============
|
||
|
||
|
||
Date
|
||
----
|
||
=========== =====
|
||
Type ``string`` or ``integer``
|
||
Examples * ``"2019-01-01T00:00:00"``
|
||
* ``"2019"`` with ``"%Y"`` as date-format_
|
||
* ``1546297200``
|
||
Description A |Date|_ value represents a specific point in time.
|
||
|
||
* If given as ``string``, it is parsed according to date-format_.
|
||
* If given as ``integer``, it is interpreted as UTC timestamp.
|
||
=========== =====
|
||
|
||
|
||
Path
|
||
----
|
||
=========== =====
|
||
Type ``string`` or ``list`` of ``strings``
|
||
Examples * ``"file.ext"``
|
||
* ``"~/path/to/file.ext"``
|
||
* ``"$HOME/path/to/file.ext"``
|
||
* ``["$HOME", "path", "to", "file.ext"]``
|
||
Description A |Path|_ is a ``string`` representing the location of a file
|
||
or directory.
|
||
|
||
Simple `tilde expansion <https://docs.python.org/3/library/os.path.html#os.path.expanduser>`__
|
||
and `environment variable expansion <https://docs.python.org/3/library/os.path.html#os.path.expandvars>`__
|
||
is supported.
|
||
|
||
In Windows environments, backslashes (``"\"``) can, in addition to
|
||
forward slashes (``"/"``), be used as path separators.
|
||
Because backslashes are JSON's escape character,
|
||
they themselves have to be escaped.
|
||
The path ``C:\path\to\file.ext`` has therefore to be written as
|
||
``"C:\\path\\to\\file.ext"`` if you want to use backslashes.
|
||
=========== =====
|
||
|
||
|
||
Logging Configuration
|
||
---------------------
|
||
=========== =====
|
||
Type ``object``
|
||
|
||
Examples .. code::
|
||
|
||
{
|
||
"format": "{asctime} {name}: {message}",
|
||
"format-date": "%H:%M:%S",
|
||
"path": "~/log.txt",
|
||
"encoding": "ascii"
|
||
}
|
||
|
||
{
|
||
"level": "debug",
|
||
"format": {
|
||
"debug" : "debug: {message}",
|
||
"info" : "[{name}] {message}",
|
||
"warning": "Warning: {message}",
|
||
"error" : "ERROR: {message}"
|
||
}
|
||
}
|
||
|
||
Description Extended logging output configuration.
|
||
|
||
* format
|
||
* General format string for logging messages
|
||
or a dictionary with format strings for each loglevel.
|
||
|
||
In addition to the default
|
||
`LogRecord attributes <https://docs.python.org/3/library/logging.html#logrecord-attributes>`__,
|
||
it is also possible to access the current
|
||
`extractor <https://github.com/mikf/gallery-dl/blob/2e516a1e3e09cb8a9e36a8f6f7e41ce8d4402f5a/gallery_dl/extractor/common.py#L24>`__
|
||
and `job <https://github.com/mikf/gallery-dl/blob/2e516a1e3e09cb8a9e36a8f6f7e41ce8d4402f5a/gallery_dl/job.py#L19>`__
|
||
objects as well as their attributes
|
||
(e.g. ``"{extractor.url}"``)
|
||
* Default: ``"[{name}][{levelname}] {message}"``
|
||
* format-date
|
||
* Format string for ``{asctime}`` fields in logging messages
|
||
(see `strftime() directives <https://docs.python.org/3/library/time.html#time.strftime>`__)
|
||
* Default: ``"%Y-%m-%d %H:%M:%S"``
|
||
* level
|
||
* Minimum logging message level
|
||
(one of ``"debug"``, ``"info"``, ``"warning"``, ``"error"``, ``"exception"``)
|
||
* Default: ``"info"``
|
||
* path
|
||
* |Path|_ to the output file
|
||
* mode
|
||
* Mode in which the file is opened;
|
||
use ``"w"`` to truncate or ``"a"`` to append
|
||
(see `open() <https://docs.python.org/3/library/functions.html#open>`__)
|
||
* Default: ``"w"``
|
||
* encoding
|
||
* File encoding
|
||
* Default: ``"utf-8"``
|
||
|
||
Note: path, mode and encoding are only applied when configuring
|
||
logging output to a file.
|
||
=========== =====
|
||
|
||
|
||
Postprocessor Configuration
|
||
---------------------------
|
||
=========== =====
|
||
Type ``object``
|
||
|
||
Example .. code::
|
||
|
||
{
|
||
"name": "zip",
|
||
"compression": "store",
|
||
"extension": "cbz",
|
||
"whitelist": ["mangadex", "exhentai", "nhentai"]
|
||
}
|
||
|
||
Description An object with the ``name`` of a post-processor and its options.
|
||
|
||
See `Postprocessor Options`_ for a list of all available
|
||
post-processors and their respective options.
|
||
|
||
You can also set a ``whitelist`` or ``blacklist`` to
|
||
only enable or disable a post-processor for the specified
|
||
extractor categories.
|
||
=========== =====
|
||
|
||
|
||
|
||
.. |.netrc| replace:: ``.netrc``
|
||
.. |tempfile.gettempdir()| replace:: ``tempfile.gettempdir()``
|
||
.. |requests.request()| replace:: ``requests.request()``
|
||
.. |timeout| replace:: ``timeout``
|
||
.. |verify| replace:: ``verify``
|
||
.. |mature_content| replace:: ``mature_content``
|
||
.. |webbrowser.open()| replace:: ``webbrowser.open()``
|
||
.. |datetime| replace:: ``datetime``
|
||
.. |datetime.max| replace:: ``datetime.max``
|
||
.. |Date| replace:: ``Date``
|
||
.. |Path| replace:: ``Path``
|
||
.. |Last-Modified| replace:: ``Last-Modified``
|
||
.. |Logging Configuration| replace:: ``Logging Configuration``
|
||
.. |Postprocessor Configuration| replace:: ``Postprocessor Configuration``
|
||
.. |strptime| replace:: strftime() and strptime() Behavior
|
||
|
||
.. _base-directory: `extractor.*.base-directory`_
|
||
.. _skipped: `extractor.*.skip`_
|
||
.. _date-format: `extractor.*.date-format`_
|
||
.. _deviantart.metadata: extractor.deviantart.metadata_
|
||
|
||
.. _.netrc: https://stackoverflow.com/tags/.netrc/info
|
||
.. _tempfile.gettempdir(): https://docs.python.org/3/library/tempfile.html#tempfile.gettempdir
|
||
.. _requests.request(): https://docs.python-requests.org/en/master/api/#requests.request
|
||
.. _timeout: https://docs.python-requests.org/en/latest/user/advanced/#timeouts
|
||
.. _verify: https://docs.python-requests.org/en/master/user/advanced/#ssl-cert-verification
|
||
.. _Last-Modified: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.29
|
||
.. _`Requests' proxy documentation`: https://2.python-requests.org/en/master/user/advanced/#proxies
|
||
.. _format string: https://docs.python.org/3/library/string.html#formatstrings
|
||
.. _format strings: https://docs.python.org/3/library/string.html#formatstrings
|
||
.. _strptime: https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
|
||
.. _mature_content: https://www.deviantart.com/developers/http/v1/20160316/object/deviation
|
||
.. _webbrowser.open(): https://docs.python.org/3/library/webbrowser.html
|
||
.. _datetime: https://docs.python.org/3/library/datetime.html#datetime-objects
|
||
.. _datetime.max: https://docs.python.org/3/library/datetime.html#datetime.datetime.max
|
||
.. _Authentication: https://github.com/mikf/gallery-dl#authentication
|
||
.. _youtube-dl: https://github.com/ytdl-org/youtube-dl
|