2017-06-18 22:16:26 +02:00
|
|
|
Configuration
|
|
|
|
#############
|
|
|
|
|
2020-05-11 00:56:41 +02:00
|
|
|
| Configuration files for *gallery-dl* use a JSON-based file format.
|
|
|
|
| For a (more or less) complete example with options set to their default values,
|
|
|
|
see `gallery-dl.conf <gallery-dl.conf>`__.
|
|
|
|
| For a configuration file example with more involved settings and options,
|
|
|
|
see `gallery-dl-example.conf <gallery-dl-example.conf>`__.
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
2020-05-10 23:47:20 +02:00
|
|
|
|
|
|
|
This file lists all available configuration options and their descriptions.
|
|
|
|
|
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
Contents
|
|
|
|
========
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
1) `Extractor Options`_
|
|
|
|
2) `Extractor-specific Options`_
|
2017-06-18 22:16:26 +02:00
|
|
|
3) `Downloader Options`_
|
2018-03-16 11:49:49 +01:00
|
|
|
4) `Output Options`_
|
2018-06-16 15:43:24 +02:00
|
|
|
5) `Postprocessor Options`_
|
|
|
|
6) `Miscellaneous Options`_
|
|
|
|
7) `API Tokens & IDs`_
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
|
|
|
Extractor Options
|
|
|
|
=================
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
|
2017-06-28 18:51:47 +02:00
|
|
|
Each extractor is identified by its ``category`` and ``subcategory``.
|
|
|
|
The ``category`` is the lowercase site name without any spaces or special
|
|
|
|
characters, which is usually just the module name
|
2018-03-16 11:49:49 +01:00
|
|
|
(``pixiv``, ``danbooru``, ...).
|
2017-06-28 18:51:47 +02:00
|
|
|
The ``subcategory`` is a lowercase word describing the general functionality
|
|
|
|
of that extractor (``user``, ``favorite``, ``manga``, ...).
|
2017-06-27 18:56:24 +02:00
|
|
|
|
2017-06-27 17:44:02 +02:00
|
|
|
Each one of the following options can be specified on multiple levels of the
|
|
|
|
configuration tree:
|
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
================== =======
|
2017-06-27 17:44:02 +02:00
|
|
|
Base level: ``extractor.<option-name>``
|
|
|
|
Category level: ``extractor.<category>.<option-name>``
|
|
|
|
Subcategory level: ``extractor.<category>.<subcategory>.<option-name>``
|
2020-09-26 13:33:46 +02:00
|
|
|
================== =======
|
2017-06-27 17:44:02 +02:00
|
|
|
|
|
|
|
A value in a "deeper" level hereby overrides a value of the same name on a
|
|
|
|
lower level. Setting the ``extractor.pixiv.filename`` value, for example, lets
|
|
|
|
you specify a general filename pattern for all the different pixiv extractors.
|
|
|
|
Using the ``extractor.pixiv.user.filename`` value lets you override this
|
|
|
|
general pattern specifically for ``PixivUserExtractor`` instances.
|
|
|
|
|
2017-06-28 18:51:47 +02:00
|
|
|
The ``category`` and ``subcategory`` of all extractors are included in the
|
|
|
|
output of ``gallery-dl --list-extractors``. For a specific URL these values
|
2017-09-09 18:48:28 +02:00
|
|
|
can also be determined by using the ``-K``/``--list-keywords`` command-line
|
|
|
|
option (see the example below).
|
2017-06-28 13:55:39 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
extractor.*.filename
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-06-08 18:00:06 +02:00
|
|
|
``string`` or ``object``
|
2020-09-26 13:33:46 +02:00
|
|
|
Example
|
2021-06-20 19:44:12 +02:00
|
|
|
* .. code:: json
|
2021-06-08 18:00:06 +02:00
|
|
|
|
|
|
|
"{manga}_c{chapter}_{page:>03}.{extension}"
|
|
|
|
|
|
|
|
* .. code:: json
|
|
|
|
|
|
|
|
{
|
|
|
|
"extension == 'mp4'": "{id}_video.{extension}",
|
|
|
|
"'nature' in title" : "{id}_{title}.{extension}",
|
|
|
|
"" : "{id}_default.{extension}"
|
|
|
|
}
|
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2021-06-08 18:00:06 +02:00
|
|
|
A `format string`_ to build filenames for downloaded files with.
|
|
|
|
|
|
|
|
If this is an ``object``, it must contain Python expressions mapping to the
|
|
|
|
filename format strings to use.
|
|
|
|
These expressions are evaluated in the order as specified in Python 3.6+
|
|
|
|
and in an undetermined order in Python 3.4 and 3.5.
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
The available replacement keys depend on the extractor used. A list
|
|
|
|
of keys for a specific one can be acquired by calling *gallery-dl*
|
|
|
|
with the ``-K``/``--list-keywords`` command-line option.
|
|
|
|
For example:
|
|
|
|
|
|
|
|
.. code::
|
|
|
|
|
|
|
|
$ gallery-dl -K http://seiga.nicovideo.jp/seiga/im5977527
|
|
|
|
Keywords for directory names:
|
|
|
|
-----------------------------
|
|
|
|
category
|
|
|
|
seiga
|
|
|
|
subcategory
|
|
|
|
image
|
|
|
|
|
|
|
|
Keywords for filenames:
|
|
|
|
-----------------------
|
|
|
|
category
|
|
|
|
seiga
|
|
|
|
extension
|
|
|
|
None
|
|
|
|
image-id
|
|
|
|
5977527
|
|
|
|
subcategory
|
|
|
|
image
|
|
|
|
|
|
|
|
Note: Even if the value of the ``extension`` key is missing or
|
|
|
|
``None``, it will be filled in later when the file download is
|
|
|
|
starting. This key is therefore always available to provide
|
|
|
|
a valid filename extension.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.directory
|
|
|
|
---------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-06-20 19:44:12 +02:00
|
|
|
``list`` of ``strings`` or ``object``
|
2020-09-26 13:33:46 +02:00
|
|
|
Example
|
2021-06-20 19:44:12 +02:00
|
|
|
* .. code:: json
|
|
|
|
|
|
|
|
["{category}", "{manga}", "c{chapter} - {title}"]
|
|
|
|
|
|
|
|
* .. code:: json
|
|
|
|
|
|
|
|
{
|
|
|
|
"'nature' in content": ["Nature Pictures"],
|
|
|
|
"retweet_id != 0" : ["{category}", "{user[name]}", "Retweets"],
|
|
|
|
"" : ["{category}", "{user[name]}"]
|
|
|
|
}
|
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2021-06-20 19:44:12 +02:00
|
|
|
A list of `format strings`_ to build target directory paths with.
|
|
|
|
|
|
|
|
If this is an ``object``, it must contain Python expressions mapping to the
|
|
|
|
list of format strings to use.
|
2017-06-27 17:44:02 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Each individual string in such a list represents a single path
|
|
|
|
segment, which will be joined together and appended to the
|
|
|
|
base-directory_ to form the complete target directory path.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
extractor.*.base-directory
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_
|
|
|
|
Default
|
|
|
|
``"./gallery-dl/"``
|
|
|
|
Description
|
|
|
|
Directory path used as base for all download destinations.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2020-01-29 18:32:37 +01:00
|
|
|
extractor.*.parent-directory
|
|
|
|
----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Use an extractor's current target directory as
|
|
|
|
`base-directory <extractor.*.base-directory_>`__
|
|
|
|
for any spawned child extractors.
|
2020-01-29 18:32:37 +01:00
|
|
|
|
|
|
|
|
2021-03-11 01:10:34 +01:00
|
|
|
extractor.*.parent-metadata
|
|
|
|
---------------------------
|
|
|
|
Type
|
2021-07-13 02:04:59 +02:00
|
|
|
``bool`` or ``string``
|
2021-03-11 01:10:34 +01:00
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
2021-07-13 02:04:59 +02:00
|
|
|
If ``true``, overwrite any metadata provided by a child extractor
|
|
|
|
with its parent's.
|
|
|
|
|
|
|
|
| If this is a ``string``, add a parent's metadata to its children's
|
|
|
|
to a field named after said string.
|
|
|
|
| For example with ``"parent-metadata": "_p_"``:
|
|
|
|
|
|
|
|
.. code:: json
|
|
|
|
|
|
|
|
{
|
|
|
|
"id": "child-id",
|
|
|
|
"_p_": {"id": "parent-id"}
|
|
|
|
}
|
2021-03-11 01:10:34 +01:00
|
|
|
|
|
|
|
|
2021-05-12 23:37:01 +02:00
|
|
|
extractor.*.parent-skip
|
|
|
|
-----------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Share number of skipped downloads between parent and child extractors.
|
|
|
|
|
|
|
|
|
2019-08-16 21:13:49 +02:00
|
|
|
extractor.*.path-restrict
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``object``
|
|
|
|
Default
|
|
|
|
``"auto"``
|
|
|
|
Example
|
|
|
|
* ``"/!? (){}"``
|
|
|
|
* ``{" ": "_", "/": "-", "|": "-", ":": "-", "*": "+"}``
|
|
|
|
Description
|
|
|
|
| A string of characters to be replaced with the value of
|
|
|
|
`path-replace <extractor.*.path-replace_>`__
|
|
|
|
| or an object mapping invalid/unwanted characters to their replacements
|
|
|
|
| for generated path segment names.
|
|
|
|
|
|
|
|
Special values:
|
|
|
|
|
|
|
|
* ``"auto"``: Use characters from ``"unix"`` or ``"windows"``
|
|
|
|
depending on the local operating system
|
|
|
|
* ``"unix"``: ``"/"``
|
|
|
|
* ``"windows"``: ``"\\\\|/<>:\"?*"``
|
2021-01-09 02:35:51 +01:00
|
|
|
* ``"ascii"``: ``"^0-9A-Za-z_."``
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
Note: In a string with 2 or more characters, ``[]^-\`` need to be
|
|
|
|
escaped with backslashes, e.g. ``"\\[\\]"``
|
2019-08-16 21:13:49 +02:00
|
|
|
|
|
|
|
|
2020-05-24 17:35:25 +02:00
|
|
|
extractor.*.path-replace
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"_"``
|
|
|
|
Description
|
|
|
|
The replacement character(s) for
|
|
|
|
`path-restrict <extractor.*.path-restrict_>`__
|
2020-05-24 17:35:25 +02:00
|
|
|
|
|
|
|
|
2019-08-16 21:13:49 +02:00
|
|
|
extractor.*.path-remove
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"\u0000-\u001f\u007f"`` (ASCII control characters)
|
|
|
|
Description
|
|
|
|
Set of characters to remove from generated path names.
|
2019-08-16 21:13:49 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: In a string with 2 or more characters, ``[]^-\`` need to be
|
|
|
|
escaped with backslashes, e.g. ``"\\[\\]"``
|
2019-07-23 17:36:07 +02:00
|
|
|
|
|
|
|
|
2021-08-24 23:23:12 +02:00
|
|
|
extractor.*.path-strip
|
|
|
|
----------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"auto"``
|
|
|
|
Description
|
|
|
|
Set of characters to remove from the end of generated path segment names
|
|
|
|
using `str.rstrip() <https://docs.python.org/3/library/stdtypes.html#str.rstrip>`_
|
|
|
|
|
|
|
|
Special values:
|
|
|
|
|
|
|
|
* ``"auto"``: Use characters from ``"unix"`` or ``"windows"``
|
|
|
|
depending on the local operating system
|
|
|
|
* ``"unix"``: ``""``
|
|
|
|
* ``"windows"``: ``". "``
|
|
|
|
|
|
|
|
|
2020-10-31 22:42:42 +01:00
|
|
|
extractor.*.extension-map
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Default
|
|
|
|
.. code:: json
|
2020-11-02 15:32:29 +01:00
|
|
|
|
2020-10-31 22:42:42 +01:00
|
|
|
{
|
|
|
|
"jpeg": "jpg",
|
|
|
|
"jpe" : "jpg",
|
|
|
|
"jfif": "jpg",
|
|
|
|
"jif" : "jpg",
|
|
|
|
"jfi" : "jpg"
|
|
|
|
}
|
|
|
|
Description
|
2020-11-14 22:40:31 +01:00
|
|
|
A JSON ``object`` mapping filename extensions to their replacements.
|
2020-10-31 22:42:42 +01:00
|
|
|
|
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
extractor.*.skip
|
|
|
|
----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls the behavior when downloading files that have been
|
|
|
|
downloaded before, i.e. a file with the same filename already
|
|
|
|
exists or its ID is in a `download archive <extractor.*.archive_>`__.
|
2019-08-08 18:34:31 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Skip downloads
|
|
|
|
* ``false``: Overwrite already existing files
|
2018-10-13 17:21:55 +02:00
|
|
|
|
2021-05-12 02:22:28 +02:00
|
|
|
* ``"abort"``: Stop the current extractor run
|
|
|
|
* ``"abort:N"``: Skip downloads and stop the current extractor run
|
2020-09-26 13:33:46 +02:00
|
|
|
after ``N`` consecutive skips
|
2018-10-13 17:21:55 +02:00
|
|
|
|
2021-05-12 02:22:28 +02:00
|
|
|
* ``"terminate"``: Stop the current extractor run, including parent extractors
|
|
|
|
* ``"terminate:N"``: Skip downloads and stop the current extractor run,
|
|
|
|
including parent extractors, after ``N`` consecutive skips
|
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``"exit"``: Exit the program altogether
|
|
|
|
* ``"exit:N"``: Skip downloads and exit the program
|
|
|
|
after ``N`` consecutive skips
|
2019-08-08 18:34:31 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``"enumerate"``: Add an enumeration index to the beginning of the
|
|
|
|
filename extension (``file.1.ext``, ``file.2.ext``, etc.)
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2017-12-04 17:06:17 +01:00
|
|
|
extractor.*.sleep
|
2017-12-18 00:12:08 +01:00
|
|
|
-----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-09-14 17:40:05 +02:00
|
|
|
|Duration|_
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
|
|
|
``0``
|
|
|
|
Description
|
|
|
|
Number of seconds to sleep before each download.
|
2020-09-12 21:04:47 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.sleep-extractor
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-09-14 17:40:05 +02:00
|
|
|
|Duration|_
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
|
|
|
``0``
|
|
|
|
Description
|
|
|
|
Number of seconds to sleep before handling an input URL,
|
|
|
|
i.e. before starting a new extractor.
|
2017-12-04 17:06:17 +01:00
|
|
|
|
|
|
|
|
2020-09-19 22:07:41 +02:00
|
|
|
extractor.*.sleep-request
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-09-14 17:40:05 +02:00
|
|
|
|Duration|_
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
|
|
|
``0``
|
|
|
|
Description
|
|
|
|
Minimal time interval in seconds between each HTTP request
|
|
|
|
during data extraction.
|
2020-09-19 22:07:41 +02:00
|
|
|
|
|
|
|
|
2017-10-12 23:37:28 +02:00
|
|
|
extractor.*.username & .password
|
|
|
|
--------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
The username and password to use when attempting to log in to
|
|
|
|
another site.
|
|
|
|
|
|
|
|
Specifying a username and password is required for
|
|
|
|
|
|
|
|
* ``nijie``
|
|
|
|
* ``seiga``
|
|
|
|
|
|
|
|
and optional for
|
|
|
|
|
|
|
|
* ``aryion``
|
2021-06-02 23:02:23 +02:00
|
|
|
* ``danbooru`` (*)
|
|
|
|
* ``e621`` (*)
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``exhentai``
|
|
|
|
* ``idolcomplex``
|
2021-03-03 03:05:25 +01:00
|
|
|
* ``imgbb``
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``inkbunny``
|
|
|
|
* ``instagram``
|
2021-09-09 01:02:59 +02:00
|
|
|
* ``kemonoparty``
|
2021-06-08 02:06:19 +02:00
|
|
|
* ``mangadex``
|
2021-03-03 03:05:25 +01:00
|
|
|
* ``mangoxo``
|
2021-05-19 02:57:36 +02:00
|
|
|
* ``pillowfort``
|
2020-12-17 16:12:59 +01:00
|
|
|
* ``sankaku``
|
2021-10-08 22:44:31 +02:00
|
|
|
* ``seisoparty``
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``subscribestar``
|
2021-03-29 23:06:47 +02:00
|
|
|
* ``tapas``
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``tsumino``
|
|
|
|
* ``twitter``
|
|
|
|
|
2020-10-15 00:51:53 +02:00
|
|
|
These values can also be specified via the
|
|
|
|
``-u/--username`` and ``-p/--password`` command-line options or
|
|
|
|
by using a |.netrc|_ file. (see Authentication_)
|
2020-09-26 13:33:46 +02:00
|
|
|
|
2021-06-02 23:02:23 +02:00
|
|
|
(*) The password value for ``danbooru`` and ``e621`` should be
|
2020-10-19 21:57:26 +02:00
|
|
|
the API key found in your user profile, not the actual account password.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
extractor.*.netrc
|
|
|
|
-----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Enable the use of |.netrc|_ authentication data.
|
2018-11-16 18:02:24 +01:00
|
|
|
|
|
|
|
|
2017-07-21 18:32:56 +02:00
|
|
|
extractor.*.cookies
|
|
|
|
-------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_ or ``object``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Source to read additional cookies from. Either as
|
2020-02-13 23:44:02 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* the |Path|_ to a Mozilla/Netscape format cookies.txt file or
|
|
|
|
* a JSON ``object`` specifying cookies as a name-to-value mapping
|
2020-02-13 23:44:02 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Example:
|
2020-02-13 23:44:02 +01:00
|
|
|
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2017-07-21 18:32:56 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
{
|
|
|
|
"cookie-name": "cookie-value",
|
|
|
|
"sessionid" : "14313336321%3AsabDFvuASDnlpb%3A31",
|
|
|
|
"isAdult" : "1"
|
|
|
|
}
|
2017-07-21 18:32:56 +02:00
|
|
|
|
|
|
|
|
2019-10-18 21:31:33 +02:00
|
|
|
extractor.*.cookies-update
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
If `extractor.*.cookies`_ specifies the |Path|_ to a cookies.txt
|
|
|
|
file and it can be opened and parsed without errors,
|
|
|
|
update its contents with cookies received during data extraction.
|
2019-10-18 21:31:33 +02:00
|
|
|
|
|
|
|
|
2018-02-19 18:24:56 +01:00
|
|
|
extractor.*.proxy
|
|
|
|
-----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``object``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Proxy (or proxies) to be used for remote connections.
|
2018-02-19 18:24:56 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* If this is a ``string``, it is the proxy URL for all
|
|
|
|
outgoing requests.
|
|
|
|
* If this is an ``object``, it is a scheme-to-proxy mapping to
|
|
|
|
specify different proxy URLs for each scheme.
|
|
|
|
It is also possible to set a proxy for a specific host by using
|
|
|
|
``scheme://host`` as key.
|
|
|
|
See `Requests' proxy documentation`_ for more details.
|
2018-02-19 18:24:56 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Example:
|
2018-02-19 18:24:56 +01:00
|
|
|
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2018-02-19 18:24:56 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
{
|
2020-10-19 21:57:26 +02:00
|
|
|
"http" : "http://10.10.1.10:3128",
|
2020-09-26 13:33:46 +02:00
|
|
|
"https": "http://10.10.1.10:1080",
|
|
|
|
"http://10.20.1.128": "http://10.10.1.10:5323"
|
|
|
|
}
|
2018-02-19 18:24:56 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: All proxy URLs should include a scheme,
|
|
|
|
otherwise ``http://`` is assumed.
|
2018-02-19 18:24:56 +01:00
|
|
|
|
|
|
|
|
2017-11-15 13:54:40 +01:00
|
|
|
extractor.*.user-agent
|
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
2021-02-25 23:39:34 +01:00
|
|
|
``"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0"``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
User-Agent header value to be used for HTTP requests.
|
2017-11-15 13:54:40 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: This option has no effect on `pixiv` extractors,
|
|
|
|
as these need specific values to function correctly.
|
2018-02-08 23:10:58 +01:00
|
|
|
|
|
|
|
|
2021-02-25 23:39:34 +01:00
|
|
|
extractor.*.browser
|
|
|
|
-------------------
|
|
|
|
Type
|
|
|
|
``string``
|
2021-02-27 16:26:42 +01:00
|
|
|
Default
|
|
|
|
``"firefox"`` for ``patreon``, ``null`` everywhere else
|
2021-02-25 23:39:34 +01:00
|
|
|
Example
|
|
|
|
* ``"chrome:macos"``
|
|
|
|
Description
|
|
|
|
Try to emulate a real browser (``firefox`` or ``chrome``)
|
|
|
|
by using their default HTTP headers and TLS ciphers for HTTP requests.
|
|
|
|
|
|
|
|
Optionally, the operating system used in the ``User-Agent`` header can be
|
|
|
|
specified after a ``:`` (``windows``, ``linux``, or ``macos``).
|
|
|
|
|
|
|
|
Note: ``requests`` and ``urllib3`` only support HTTP/1.1, while a real
|
|
|
|
browser would use HTTP/2.
|
|
|
|
|
|
|
|
|
2018-02-08 23:10:58 +01:00
|
|
|
extractor.*.keywords
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Example
|
|
|
|
``{"type": "Pixel Art", "type_id": 123}``
|
|
|
|
Description
|
|
|
|
Additional key-value pairs to be added to each metadata dictionary.
|
2017-11-15 13:54:40 +01:00
|
|
|
|
|
|
|
|
2018-02-21 23:18:21 +01:00
|
|
|
extractor.*.keywords-default
|
|
|
|
----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
any
|
|
|
|
Default
|
|
|
|
``"None"``
|
|
|
|
Description
|
|
|
|
Default value used for missing or undefined keyword names in
|
2021-06-29 19:24:18 +02:00
|
|
|
`format strings`_.
|
2018-02-21 23:18:21 +01:00
|
|
|
|
|
|
|
|
2021-08-16 01:47:59 +02:00
|
|
|
extractor.*.url-metadata
|
|
|
|
------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Insert a file's download URL into its metadata dictionary as the given name.
|
|
|
|
|
2021-09-28 22:59:37 +02:00
|
|
|
For example, setting this option to ``"gdl_file_url"`` will cause a new
|
|
|
|
metadata field with name ``gdl_file_url`` to appear, which contains the
|
|
|
|
current file's download URL.
|
|
|
|
This can then be used in `filenames <extractor.*.filename_>`_,
|
|
|
|
with a ``metadata`` post processor, etc.
|
|
|
|
|
2021-08-16 01:47:59 +02:00
|
|
|
|
2019-01-19 20:28:19 +01:00
|
|
|
extractor.*.category-transfer
|
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
Extractor-specific
|
|
|
|
Description
|
|
|
|
Transfer an extractor's (sub)category values to all child
|
|
|
|
extractors spawned by it, to let them inherit their parent's
|
|
|
|
config options.
|
2019-01-19 20:28:19 +01:00
|
|
|
|
|
|
|
|
2020-09-10 22:54:10 +02:00
|
|
|
extractor.*.blacklist & .whitelist
|
|
|
|
----------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``["oauth", "recursive", "test"]`` + current extractor category
|
2022-01-06 23:36:57 +01:00
|
|
|
Example
|
|
|
|
``["imgur", "gfycat:user", "*:image"]``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2022-01-06 23:36:57 +01:00
|
|
|
A list of extractor identifiers to ignore (or allow)
|
2020-09-26 13:33:46 +02:00
|
|
|
when spawning child extractors for unknown URLs,
|
|
|
|
e.g. from ``reddit`` or ``plurk``.
|
2020-09-10 22:54:10 +02:00
|
|
|
|
2022-01-06 23:36:57 +01:00
|
|
|
Each identifier can be
|
|
|
|
|
|
|
|
* A category or basecategory name (``"imgur"``, ``"mastodon"``)
|
|
|
|
* | A (base)category-subcategory pair, where both names are separated by a colon (``"gfycat:user"``).
|
|
|
|
| Both names can be a `*` or left empty, matching all possible names (``"*:image"``, ``":user"``).
|
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: Any ``blacklist`` setting will automatically include
|
|
|
|
``"oauth"``, ``"recursive"``, and ``"test"``.
|
2020-09-10 22:54:10 +02:00
|
|
|
|
|
|
|
|
2018-02-01 22:00:44 +01:00
|
|
|
extractor.*.archive
|
|
|
|
-------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Example
|
|
|
|
``"$HOME/.archives/{category}.sqlite3"``
|
|
|
|
Description
|
|
|
|
File to store IDs of downloaded files in. Downloads of files
|
|
|
|
already recorded in this archive file will be
|
|
|
|
`skipped <extractor.*.skip_>`__.
|
|
|
|
|
|
|
|
The resulting archive file is not a plain text file but an SQLite3
|
|
|
|
database, as either lookup operations are significantly faster or
|
|
|
|
memory requirements are significantly lower when the
|
|
|
|
amount of stored IDs gets reasonably large.
|
|
|
|
|
|
|
|
Note: archive paths support regular `format string`_ replacements,
|
|
|
|
but be aware that using external inputs for building local paths
|
|
|
|
may pose a security risk.
|
2018-02-01 22:00:44 +01:00
|
|
|
|
|
|
|
|
2018-02-24 21:21:59 +01:00
|
|
|
extractor.*.archive-format
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Example
|
|
|
|
``"{id}_{offset}"``
|
|
|
|
Description
|
2021-06-29 19:24:18 +02:00
|
|
|
An alternative `format string`__ to build archive IDs with.
|
|
|
|
|
|
|
|
.. __: https://docs.python.org/3/library/string.html#format-string-syntax
|
2018-02-24 21:21:59 +01:00
|
|
|
|
|
|
|
|
2021-07-20 19:59:42 +02:00
|
|
|
extractor.*.archive-prefix
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"{category}"``
|
|
|
|
Description
|
|
|
|
Prefix for archive IDs.
|
|
|
|
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
extractor.*.postprocessors
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``list`` of |Postprocessor Configuration|_ objects
|
|
|
|
Example
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
[
|
2020-10-19 21:57:26 +02:00
|
|
|
{
|
|
|
|
"name": "zip" ,
|
|
|
|
"compression": "store"
|
|
|
|
},
|
|
|
|
{
|
|
|
|
"name": "exec",
|
|
|
|
"command": ["/home/foobar/script", "{category}", "{image_id}"]
|
|
|
|
}
|
2020-09-26 13:33:46 +02:00
|
|
|
]
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2020-10-19 21:57:26 +02:00
|
|
|
A list of `post processors <Postprocessor Configuration_>`__
|
2020-09-26 13:33:46 +02:00
|
|
|
to be applied to each downloaded file in the specified order.
|
|
|
|
|
2020-10-19 21:57:26 +02:00
|
|
|
| Unlike other options, a |postprocessors|_ setting at a deeper level
|
|
|
|
does not override any |postprocessors|_ setting at a lower level.
|
|
|
|
| Instead, all post processors from all applicable |postprocessors|_
|
|
|
|
settings get combined into a single list.
|
|
|
|
|
|
|
|
For example
|
|
|
|
|
|
|
|
* an ``mtime`` post processor at ``extractor.postprocessors``,
|
|
|
|
* a ``zip`` post processor at ``extractor.pixiv.postprocessors``,
|
|
|
|
* and using ``--exec``
|
|
|
|
|
|
|
|
will run all three post processors - ``mtime``, ``zip``, ``exec`` -
|
|
|
|
for each downloaded ``pixiv`` file.
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
|
2018-10-08 23:08:11 +02:00
|
|
|
extractor.*.retries
|
|
|
|
-------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``4``
|
|
|
|
Description
|
|
|
|
Maximum number of times a failed HTTP request is retried before
|
2020-10-19 21:57:26 +02:00
|
|
|
giving up, or ``-1`` for infinite retries.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.timeout
|
|
|
|
-------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``float``
|
|
|
|
Default
|
|
|
|
``30.0``
|
|
|
|
Description
|
|
|
|
Amount of time (in seconds) to wait for a successful connection
|
|
|
|
and response from a remote server.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
This value gets internally used as the |timeout|_ parameter for the
|
|
|
|
|requests.request()|_ method.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.verify
|
|
|
|
------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls whether to verify SSL/TLS certificates for HTTPS requests.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
If this is a ``string``, it must be the path to a CA bundle to use
|
|
|
|
instead of the default certificates.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
This value gets internally used as the |verify|_ parameter for the
|
|
|
|
|requests.request()|_ method.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
|
|
|
|
2019-07-13 21:49:26 +02:00
|
|
|
extractor.*.download
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls whether to download media files.
|
2019-07-13 21:49:26 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Setting this to ``false`` won't download any files, but all other
|
|
|
|
functions (`postprocessors`_, `download archive`_, etc.)
|
|
|
|
will be executed as normal.
|
2019-07-13 21:49:26 +02:00
|
|
|
|
|
|
|
|
2021-08-16 01:47:59 +02:00
|
|
|
extractor.*.fallback
|
|
|
|
--------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Use fallback download URLs when a download fails.
|
|
|
|
|
|
|
|
|
2018-10-08 23:08:11 +02:00
|
|
|
extractor.*.image-range
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Example
|
|
|
|
* ``"10-20"``
|
|
|
|
* ``"-5, 10, 30-50, 100-"``
|
|
|
|
Description
|
|
|
|
Index-range(s) specifying which images to download.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: The index of the first image is ``1``.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.chapter-range
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Description
|
|
|
|
Like `image-range <extractor.*.image-range_>`__,
|
|
|
|
but applies to delegated URLs like manga-chapters, etc.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.image-filter
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Example
|
|
|
|
* ``"width >= 1200 and width/height > 1.2"``
|
|
|
|
* ``"re.search(r'foo(bar)+', description)"``
|
|
|
|
Description
|
|
|
|
Python expression controlling which files to download.
|
|
|
|
|
|
|
|
| Files for which the expression evaluates to ``False`` are ignored.
|
|
|
|
| Available keys are the filename-specific ones listed by ``-K`` or ``-j``.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.chapter-filter
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Example
|
|
|
|
* ``"lang == 'en'"``
|
|
|
|
* ``"language == 'French' and 10 <= chapter < 20"``
|
|
|
|
Description
|
|
|
|
Like `image-filter <extractor.*.image-filter_>`__,
|
|
|
|
but applies to delegated URLs like manga-chapters, etc.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2019-06-29 22:48:59 +02:00
|
|
|
extractor.*.image-unique
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Ignore image URLs that have been encountered before during the
|
|
|
|
current extractor run.
|
2019-06-29 22:48:59 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.*.chapter-unique
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Like `image-unique <extractor.*.image-unique_>`__,
|
|
|
|
but applies to delegated URLs like manga-chapters, etc.
|
2019-06-29 22:48:59 +02:00
|
|
|
|
|
|
|
|
2019-07-16 23:08:27 +02:00
|
|
|
extractor.*.date-format
|
2021-06-29 19:24:18 +02:00
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"%Y-%m-%dT%H:%M:%S"``
|
|
|
|
Description
|
|
|
|
Format string used to parse ``string`` values of
|
|
|
|
`date-min` and `date-max`.
|
2019-07-16 23:08:27 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
See |strptime|_ for a list of formatting directives.
|
2019-07-16 23:08:27 +02:00
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
Extractor-specific Options
|
|
|
|
==========================
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
|
2018-03-14 14:03:53 +01:00
|
|
|
extractor.artstation.external
|
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Try to follow external URLs of embedded players.
|
2018-03-14 14:03:53 +01:00
|
|
|
|
|
|
|
|
2020-06-26 22:05:02 +02:00
|
|
|
extractor.aryion.recursive
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls the post extraction strategy.
|
2020-06-26 22:05:02 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Start on users' main gallery pages and recursively
|
|
|
|
descend into subfolders
|
|
|
|
* ``false``: Get posts from "Latest Updates" pages
|
2020-06-26 22:05:02 +02:00
|
|
|
|
|
|
|
|
2021-07-30 01:09:32 +02:00
|
|
|
extractor.bbc.width
|
|
|
|
-------------------
|
|
|
|
Type
|
|
|
|
``int``
|
|
|
|
Default
|
|
|
|
``1920``
|
|
|
|
Description
|
|
|
|
Specifies the requested image width.
|
|
|
|
|
|
|
|
This value must be divisble by 16 and gets rounded down otherwise.
|
|
|
|
The maximum possible value appears to be ``1920``.
|
|
|
|
|
|
|
|
|
2020-01-23 22:42:56 +01:00
|
|
|
extractor.blogger.videos
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download embedded videos hosted on https://www.blogger.com/
|
2020-01-23 22:42:56 +01:00
|
|
|
|
|
|
|
|
2021-08-08 19:18:41 +02:00
|
|
|
extractor.danbooru.external
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
2021-08-08 19:18:41 +02:00
|
|
|
For unavailable or restricted posts,
|
|
|
|
follow the ``source`` and download from there if possible.
|
2019-08-31 21:46:49 +02:00
|
|
|
|
2021-06-20 22:41:41 +02:00
|
|
|
|
2021-04-13 23:41:30 +02:00
|
|
|
extractor.danbooru.metadata
|
|
|
|
---------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract additional metadata (notes, artist commentary, parent, children)
|
|
|
|
|
|
|
|
Note: This requires 1 additional HTTP request for each post.
|
2019-08-31 21:46:49 +02:00
|
|
|
|
2021-06-20 22:41:41 +02:00
|
|
|
|
2021-08-08 19:18:41 +02:00
|
|
|
extractor.danbooru.ugoira
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Controls the download target for Ugoira posts.
|
|
|
|
|
|
|
|
* ``true``: Original ZIP archives
|
|
|
|
* ``false``: Converted video files
|
|
|
|
|
|
|
|
|
2021-01-07 18:05:32 +01:00
|
|
|
extractor.derpibooru.api-key
|
|
|
|
----------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Your `Derpibooru API Key <https://derpibooru.org/registrations/edit>`__,
|
|
|
|
to use your account's browsing settings and filters.
|
|
|
|
|
|
|
|
|
|
|
|
extractor.derpibooru.filter
|
|
|
|
---------------------------
|
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``56027`` (`Everything <https://derpibooru.org/filters/56027>`_ filter)
|
|
|
|
Description
|
|
|
|
The content filter ID to use.
|
|
|
|
|
|
|
|
Setting an explicit filter ID overrides any default filters and can be used
|
|
|
|
to access 18+ content without `API Key <extractor.derpibooru.api-key_>`_.
|
|
|
|
|
|
|
|
See `Filters <https://derpibooru.org/filters>`_ for details.
|
|
|
|
|
|
|
|
|
2021-09-07 21:16:49 +02:00
|
|
|
extractor.deviantart.auto-watch
|
|
|
|
-------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Automatically watch users when encountering "Watchers-Only Deviations"
|
|
|
|
(requires a `refresh-token <extractor.deviantart.refresh-token_>`_).
|
|
|
|
|
|
|
|
|
|
|
|
extractor.deviantart.auto-unwatch
|
|
|
|
---------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
After watching a user through `auto-watch <extractor.deviantart.auto-watch_>`_,
|
|
|
|
unwatch that user at the end of the current extractor run.
|
|
|
|
|
|
|
|
|
2021-08-29 20:56:34 +02:00
|
|
|
extractor.deviantart.comments
|
|
|
|
-----------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract ``comments`` metadata.
|
|
|
|
|
|
|
|
|
2019-06-10 21:05:25 +02:00
|
|
|
extractor.deviantart.extra
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-03-19 16:24:23 +01:00
|
|
|
``bool``
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
2021-03-19 16:24:23 +01:00
|
|
|
``false``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2021-03-19 16:24:23 +01:00
|
|
|
Download extra Sta.sh resources from
|
|
|
|
description texts and journals.
|
2021-03-06 21:31:28 +01:00
|
|
|
|
2021-03-19 16:24:23 +01:00
|
|
|
Note: Enabling this option also enables deviantart.metadata_.
|
2019-06-10 21:05:25 +02:00
|
|
|
|
|
|
|
|
2017-07-12 17:05:31 +02:00
|
|
|
extractor.deviantart.flat
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Select the directory structure created by the Gallery- and
|
|
|
|
Favorite-Extractors.
|
2017-07-12 17:05:31 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Use a flat directory structure.
|
|
|
|
* ``false``: Collect a list of all gallery-folders or
|
|
|
|
favorites-collections and transfer any further work to other
|
|
|
|
extractors (``folder`` or ``collection``), which will then
|
|
|
|
create individual subdirectories for each of them.
|
|
|
|
|
|
|
|
Note: Going through all gallery folders will not be able to
|
|
|
|
fetch deviations which aren't in any folder.
|
2017-07-12 17:05:31 +02:00
|
|
|
|
|
|
|
|
2019-05-29 23:50:05 +02:00
|
|
|
extractor.deviantart.folders
|
|
|
|
----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Provide a ``folders`` metadata field that contains the names of all
|
|
|
|
folders a deviation is present in.
|
2019-05-29 23:50:05 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: Gathering this information requires a lot of API calls.
|
|
|
|
Use with caution.
|
2019-05-29 23:50:05 +02:00
|
|
|
|
|
|
|
|
2019-11-06 23:57:12 +01:00
|
|
|
extractor.deviantart.include
|
|
|
|
----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``"gallery"``
|
|
|
|
Example
|
|
|
|
``"favorite,journal,scraps"`` or ``["favorite", "journal", "scraps"]``
|
|
|
|
Description
|
|
|
|
A (comma-separated) list of subcategories to include
|
|
|
|
when processing a user profile.
|
2019-11-06 23:57:12 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are
|
|
|
|
``"gallery"``, ``"scraps"``, ``"journal"``, ``"favorite"``.
|
2019-11-06 23:57:12 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
You can use ``"all"`` instead of listing all values separately.
|
2019-11-06 23:57:12 +01:00
|
|
|
|
|
|
|
|
2018-07-16 18:14:41 +02:00
|
|
|
extractor.deviantart.journals
|
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"html"``
|
|
|
|
Description
|
|
|
|
Selects the output format of journal entries.
|
2018-07-16 18:14:41 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``"html"``: HTML with (roughly) the same layout as on DeviantArt.
|
|
|
|
* ``"text"``: Plain text with image references and HTML tags removed.
|
|
|
|
* ``"none"``: Don't download journals.
|
2018-07-16 18:14:41 +02:00
|
|
|
|
|
|
|
|
2017-10-09 23:20:17 +02:00
|
|
|
extractor.deviantart.mature
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Enable mature content.
|
2017-10-07 13:07:34 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
This option simply sets the |mature_content|_ parameter for API
|
|
|
|
calls to either ``"true"`` or ``"false"`` and does not do any other
|
|
|
|
form of content filtering.
|
2017-10-07 13:07:34 +02:00
|
|
|
|
|
|
|
|
2019-03-21 14:46:47 +01:00
|
|
|
extractor.deviantart.metadata
|
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Request extended metadata for deviation objects to additionally provide
|
|
|
|
``description``, ``tags``, ``license`` and ``is_watching`` fields.
|
2019-03-21 14:46:47 +01:00
|
|
|
|
|
|
|
|
2017-10-09 23:20:17 +02:00
|
|
|
extractor.deviantart.original
|
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download original files if available.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Setting this option to ``"images"`` only downloads original
|
|
|
|
files if they are images and falls back to preview versions for
|
|
|
|
everything else (archives, etc.).
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2018-07-25 12:52:36 +02:00
|
|
|
extractor.deviantart.refresh-token
|
|
|
|
----------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
The ``refresh-token`` value you get from
|
|
|
|
`linking your DeviantArt account to gallery-dl <OAuth_>`__.
|
2018-07-25 12:52:36 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Using a ``refresh-token`` allows you to access private or otherwise
|
|
|
|
not publicly available deviations.
|
2019-10-13 23:01:57 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: The ``refresh-token`` becomes invalid
|
|
|
|
`after 3 months <https://www.deviantart.com/developers/authentication#refresh>`__
|
|
|
|
or whenever your `cache file <cache.file_>`__ is deleted or cleared.
|
2018-07-25 12:52:36 +02:00
|
|
|
|
|
|
|
|
2018-07-14 11:52:21 +02:00
|
|
|
extractor.deviantart.wait-min
|
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``0``
|
|
|
|
Description
|
|
|
|
Minimum wait time in seconds before API requests.
|
2018-07-14 11:52:21 +02:00
|
|
|
|
|
|
|
|
2020-07-17 19:25:19 +02:00
|
|
|
extractor.exhentai.domain
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"auto"``
|
|
|
|
Description
|
|
|
|
* ``"auto"``: Use ``e-hentai.org`` or ``exhentai.org``
|
|
|
|
depending on the input URL
|
|
|
|
* ``"e-hentai.org"``: Use ``e-hentai.org`` for all URLs
|
|
|
|
* ``"exhentai.org"``: Use ``exhentai.org`` for all URLs
|
2020-07-17 19:25:19 +02:00
|
|
|
|
|
|
|
|
2021-12-16 22:29:04 +01:00
|
|
|
extractor.exhentai.limits
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Sets a custom image download limit and
|
|
|
|
stops extraction when it gets exceeded.
|
|
|
|
|
|
|
|
|
2021-02-22 22:59:51 +01:00
|
|
|
extractor.exhentai.metadata
|
|
|
|
---------------------------
|
|
|
|
Type
|
2021-02-26 17:53:27 +01:00
|
|
|
``bool``
|
2021-02-22 22:59:51 +01:00
|
|
|
Default
|
2021-02-26 17:53:27 +01:00
|
|
|
``false``
|
2021-02-22 22:59:51 +01:00
|
|
|
Description
|
2021-02-26 17:53:27 +01:00
|
|
|
Load extended gallery metadata from the
|
|
|
|
`API <https://ehwiki.org/wiki/API#Gallery_Metadata>`_.
|
2021-02-22 22:59:51 +01:00
|
|
|
|
2021-02-26 17:53:27 +01:00
|
|
|
Adds ``archiver_key``, ``posted``, and ``torrents``.
|
|
|
|
Makes ``date`` and ``filesize`` more precise.
|
2021-02-22 22:59:51 +01:00
|
|
|
|
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
extractor.exhentai.original
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download full-sized original images if available.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2021-12-16 22:29:04 +01:00
|
|
|
extractor.exhentai.source
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"gallery"``
|
|
|
|
Description
|
|
|
|
Selects an alternative source to download files from.
|
|
|
|
|
|
|
|
* ``"hitomi"``: Download the corresponding gallery from ``hitomi.la``
|
|
|
|
|
|
|
|
|
2021-04-25 19:39:13 +02:00
|
|
|
extractor.fanbox.embeds
|
|
|
|
-----------------------
|
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Control behavior on embedded content from external sites.
|
|
|
|
|
|
|
|
* ``true``: Extract embed URLs and download them if supported
|
|
|
|
(videos are not downloaded).
|
|
|
|
* ``"ytdl"``: Like ``true``, but let `youtube-dl`_ handle video
|
|
|
|
extraction and download for YouTube, Vimeo and SoundCloud embeds.
|
|
|
|
* ``false``: Ignore embeds.
|
|
|
|
|
|
|
|
|
2017-10-12 23:37:28 +02:00
|
|
|
extractor.flickr.access-token & .access-token-secret
|
|
|
|
----------------------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
The ``access_token`` and ``access_token_secret`` values you get
|
|
|
|
from `linking your Flickr account to gallery-dl <OAuth_>`__.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2019-05-14 18:12:02 +02:00
|
|
|
extractor.flickr.videos
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Extract and download videos.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2017-06-20 16:20:28 +02:00
|
|
|
extractor.flickr.size-max
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``integer`` or ``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Sets the maximum allowed size for downloaded images.
|
2017-06-20 16:20:28 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* If this is an ``integer``, it specifies the maximum image dimension
|
|
|
|
(width and height) in pixels.
|
|
|
|
* If this is a ``string``, it should be one of Flickr's format specifiers
|
|
|
|
(``"Original"``, ``"Large"``, ... or ``"o"``, ``"k"``, ``"h"``,
|
|
|
|
``"l"``, ...) to use as an upper limit.
|
2017-06-20 16:20:28 +02:00
|
|
|
|
2017-12-21 21:42:40 +01:00
|
|
|
|
2021-01-19 19:09:29 +01:00
|
|
|
extractor.furaffinity.descriptions
|
|
|
|
----------------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"text"``
|
|
|
|
Description
|
|
|
|
Controls the format of ``description`` metadata fields.
|
|
|
|
|
|
|
|
* ``"text"``: Plain text with HTML tags removed
|
|
|
|
* ``"html"``: Raw HTML content
|
|
|
|
|
|
|
|
|
2021-08-08 18:53:02 +02:00
|
|
|
extractor.furaffinity.external
|
|
|
|
------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Follow external URLs linked in descriptions.
|
|
|
|
|
|
|
|
|
2020-02-12 21:39:43 +01:00
|
|
|
extractor.furaffinity.include
|
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``"gallery"``
|
|
|
|
Example
|
|
|
|
``"scraps,favorite"`` or ``["scraps", "favorite"]``
|
|
|
|
Description
|
|
|
|
A (comma-separated) list of subcategories to include
|
|
|
|
when processing a user profile.
|
2020-02-12 21:39:43 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are
|
|
|
|
``"gallery"``, ``"scraps"``, ``"favorite"``.
|
2020-02-12 21:39:43 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
You can use ``"all"`` instead of listing all values separately.
|
2020-02-12 21:39:43 +01:00
|
|
|
|
|
|
|
|
2021-12-29 22:45:07 +01:00
|
|
|
extractor.generic.enabled
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Match **all** URLs not otherwise supported by gallery-dl,
|
|
|
|
even ones without a ``generic:`` prefix.
|
|
|
|
|
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
extractor.gfycat.format
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-10-22 22:47:29 +02:00
|
|
|
* ``list`` of ``strings``
|
|
|
|
* ``string``
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
2021-10-22 22:47:29 +02:00
|
|
|
``["mp4", "webm", "mobile", "gif"]``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2021-10-22 22:47:29 +02:00
|
|
|
List of names of the preferred animation format, which can be
|
|
|
|
``"mp4"``, ``"webm"``, ``"mobile"``, ``"gif"``, or ``"webp"``.
|
|
|
|
|
|
|
|
If a selected format is not available, the next one in the list will be
|
|
|
|
tried until an available format is found.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2021-10-22 22:47:29 +02:00
|
|
|
If the format is given as ``string``, it will be extended with
|
|
|
|
``["mp4", "webm", "mobile", "gif"]``. Use a list with one element to
|
|
|
|
restrict it to only one possible format.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2020-09-24 00:48:08 +02:00
|
|
|
extractor.hentaifoundry.include
|
|
|
|
-------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``list`` of ``strings``
|
|
|
|
Default
|
2020-10-11 18:44:46 +02:00
|
|
|
``"pictures"``
|
2020-09-26 13:33:46 +02:00
|
|
|
Example
|
2020-09-27 02:22:23 +02:00
|
|
|
``"scraps,stories"`` or ``["scraps", "stories"]``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
A (comma-separated) list of subcategories to include
|
|
|
|
when processing a user profile.
|
2020-09-24 00:48:08 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are
|
2020-10-11 18:44:46 +02:00
|
|
|
``"pictures"``, ``"scraps"``, ``"stories"``, ``"favorite"``.
|
2020-09-24 00:48:08 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
You can use ``"all"`` instead of listing all values separately.
|
2020-09-24 00:48:08 +02:00
|
|
|
|
|
|
|
|
2020-03-12 23:39:29 +01:00
|
|
|
extractor.hitomi.metadata
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
2021-12-16 22:21:07 +01:00
|
|
|
``false``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
Try to extract
|
|
|
|
``artist``, ``group``, ``parody``, and ``characters`` metadata.
|
2020-03-12 23:39:29 +01:00
|
|
|
|
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
extractor.imgur.mp4
|
|
|
|
-------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls whether to choose the GIF or MP4 version of an animation.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Follow Imgur's advice and choose MP4 if the
|
|
|
|
``prefer_video`` flag in an image's metadata is set.
|
|
|
|
* ``false``: Always choose GIF.
|
|
|
|
* ``"always"``: Always choose MP4.
|
2017-06-20 16:20:28 +02:00
|
|
|
|
|
|
|
|
2020-07-24 17:50:32 +02:00
|
|
|
extractor.inkbunny.orderby
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"create_datetime"``
|
|
|
|
Description
|
|
|
|
Value of the ``orderby`` parameter for submission searches.
|
2020-07-24 17:50:32 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
(See `API#Search <https://wiki.inkbunny.net/wiki/API#Search>`__
|
|
|
|
for details)
|
2020-07-24 17:50:32 +02:00
|
|
|
|
|
|
|
|
2020-12-20 23:20:32 +01:00
|
|
|
extractor.instagram.include
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2020-12-20 23:20:32 +01:00
|
|
|
``string`` or ``list`` of ``strings``
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
2020-12-20 23:20:32 +01:00
|
|
|
``"posts"``
|
|
|
|
Example
|
|
|
|
``"stories,highlights,posts"`` or ``["stories", "highlights", "posts"]``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2020-12-20 23:20:32 +01:00
|
|
|
A (comma-separated) list of subcategories to include
|
|
|
|
when processing a user profile.
|
|
|
|
|
|
|
|
Possible values are
|
2021-05-24 23:04:30 +02:00
|
|
|
``"posts"``, ``"reels"``, ``"channel"``, ``"tagged"``,
|
|
|
|
``"stories"``, ``"highlights"``.
|
2020-12-20 23:20:32 +01:00
|
|
|
|
|
|
|
You can use ``"all"`` instead of listing all values separately.
|
2019-09-21 23:38:20 +02:00
|
|
|
|
|
|
|
|
2019-12-19 17:15:41 +01:00
|
|
|
extractor.instagram.videos
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download video files.
|
2019-12-19 17:15:41 +01:00
|
|
|
|
|
|
|
|
2021-11-03 22:52:15 +01:00
|
|
|
extractor.kemonoparty.comments
|
2021-11-12 18:29:40 +01:00
|
|
|
------------------------------
|
2021-11-03 22:52:15 +01:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract ``comments`` metadata.
|
|
|
|
|
|
|
|
|
2021-11-20 23:36:16 +01:00
|
|
|
extractor.kemonoparty.dms
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract a user's direct messages as ``dms`` metadata.
|
|
|
|
|
|
|
|
|
2021-11-17 19:59:24 +01:00
|
|
|
extractor.kemonoparty.files
|
|
|
|
---------------------------
|
|
|
|
Type
|
|
|
|
``list`` of ``strings``
|
|
|
|
Default
|
2021-11-29 02:15:44 +01:00
|
|
|
``["attachments", "file", "inline"]``
|
2021-11-17 19:59:24 +01:00
|
|
|
Description
|
|
|
|
Determines the type and order of files to be downloaded.
|
|
|
|
|
2021-11-19 01:50:48 +01:00
|
|
|
Available types are ``file``, ``attachments``, and ``inline``.
|
2021-11-17 19:59:24 +01:00
|
|
|
|
|
|
|
|
2021-07-09 18:19:02 +02:00
|
|
|
extractor.kemonoparty.max-posts
|
|
|
|
-------------------------------
|
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Limit the number of posts to download.
|
|
|
|
|
|
|
|
|
2021-05-14 19:54:16 +02:00
|
|
|
extractor.kemonoparty.metadata
|
|
|
|
------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract ``username`` metadata
|
|
|
|
|
|
|
|
|
2020-07-12 23:06:42 +02:00
|
|
|
extractor.khinsider.format
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"mp3"``
|
|
|
|
Description
|
|
|
|
The name of the preferred file format to download.
|
2020-07-12 23:06:42 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Use ``"all"`` to download all available formats,
|
|
|
|
or a (comma-separated) list to select multiple formats.
|
2020-07-12 23:06:42 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
If the selected format is not available,
|
|
|
|
the first in the list gets chosen (usually `mp3`).
|
2020-07-12 23:06:42 +02:00
|
|
|
|
|
|
|
|
2021-08-12 15:12:42 +02:00
|
|
|
extractor.luscious.gif
|
2021-08-12 16:05:26 +02:00
|
|
|
----------------------
|
2021-08-12 15:12:42 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Format in which to download animated images.
|
|
|
|
|
2021-08-12 16:05:26 +02:00
|
|
|
Use ``true`` to download animated images as gifs and ``false``
|
2021-08-12 15:12:42 +02:00
|
|
|
to download as mp4 videos.
|
|
|
|
|
|
|
|
|
2021-02-28 01:31:50 +01:00
|
|
|
extractor.mangadex.api-server
|
|
|
|
-----------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"https://api.mangadex.org"``
|
|
|
|
Description
|
|
|
|
The server to use for API requests.
|
|
|
|
|
|
|
|
|
2021-10-05 19:46:48 +02:00
|
|
|
extractor.mangadex.api-parameters
|
|
|
|
---------------------------------
|
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Example
|
|
|
|
``{"order[updatedAt]": "desc"}``
|
|
|
|
Description
|
|
|
|
Additional query parameters to send when fetching manga chapters.
|
|
|
|
|
|
|
|
(See `/manga/{id}/feed <https://api.mangadex.org/docs.html#operation/get-manga-id-feed>`_
|
|
|
|
and `/user/follows/manga/feed <https://api.mangadex.org/docs.html#operation/get-user-follows-manga-feed>`_)
|
|
|
|
|
|
|
|
|
2021-06-20 22:41:41 +02:00
|
|
|
extractor.mangadex.lang
|
|
|
|
-----------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Example
|
|
|
|
``"en"``
|
|
|
|
Description
|
|
|
|
`ISO 639-1 <https://en.wikipedia.org/wiki/ISO_639-1>`_ language code
|
|
|
|
to filter chapters by.
|
|
|
|
|
|
|
|
|
2021-10-05 19:46:48 +02:00
|
|
|
extractor.mangadex.ratings
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``["safe", "suggestive", "erotica", "pornographic"]``
|
|
|
|
Description
|
|
|
|
List of acceptable content ratings for returned chapters.
|
|
|
|
|
|
|
|
|
2021-07-06 23:13:58 +02:00
|
|
|
extractor.mastodon.reblogs
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
2021-07-07 00:56:42 +02:00
|
|
|
Fetch media from reblogged posts.
|
|
|
|
|
|
|
|
|
|
|
|
extractor.mastodon.replies
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Fetch media from replies to other posts.
|
2021-07-06 23:13:58 +02:00
|
|
|
|
|
|
|
|
2021-07-02 22:12:41 +02:00
|
|
|
extractor.mastodon.text-posts
|
|
|
|
-----------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Also emit metadata for text-only posts without media content.
|
|
|
|
|
|
|
|
|
2021-01-19 17:43:55 +01:00
|
|
|
extractor.newgrounds.flash
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download original Adobe Flash animations instead of pre-rendered videos.
|
|
|
|
|
|
|
|
|
2021-07-29 19:11:20 +02:00
|
|
|
extractor.newgrounds.format
|
|
|
|
---------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"original"``
|
|
|
|
Example
|
|
|
|
``"720p"``
|
|
|
|
Description
|
|
|
|
Selects the preferred format for video downloads.
|
|
|
|
|
|
|
|
If the selected format is not available,
|
|
|
|
the next smaller one gets chosen.
|
|
|
|
|
|
|
|
|
2019-11-22 23:20:21 +01:00
|
|
|
extractor.newgrounds.include
|
|
|
|
----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``"art"``
|
|
|
|
Example
|
|
|
|
``"movies,audio"`` or ``["movies", "audio"]``
|
|
|
|
Description
|
|
|
|
A (comma-separated) list of subcategories to include
|
|
|
|
when processing a user profile.
|
2019-11-22 23:20:21 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are
|
|
|
|
``"art"``, ``"audio"``, ``"movies"``.
|
2019-11-22 23:20:21 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
You can use ``"all"`` instead of listing all values separately.
|
2019-11-22 23:20:21 +01:00
|
|
|
|
|
|
|
|
2020-09-24 23:26:33 +02:00
|
|
|
extractor.nijie.include
|
|
|
|
----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``"illustration,doujin"``
|
|
|
|
Description
|
|
|
|
A (comma-separated) list of subcategories to include
|
|
|
|
when processing a user profile.
|
2020-09-24 23:26:33 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are
|
|
|
|
``"illustration"``, ``"doujin"``, ``"favorite"``.
|
2020-09-24 23:26:33 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
You can use ``"all"`` instead of listing all values separately.
|
2020-09-24 23:26:33 +02:00
|
|
|
|
|
|
|
|
2017-06-20 16:20:28 +02:00
|
|
|
extractor.oauth.browser
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls how a user is directed to an OAuth authorization page.
|
2017-06-20 16:20:28 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Use Python's |webbrowser.open()|_ method to automatically
|
|
|
|
open the URL in the user's default browser.
|
|
|
|
* ``false``: Ask the user to copy & paste an URL from the terminal.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2020-05-25 22:19:58 +02:00
|
|
|
extractor.oauth.cache
|
|
|
|
---------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Store tokens received during OAuth authorizations
|
|
|
|
in `cache <cache.file_>`__.
|
2020-05-25 22:19:58 +02:00
|
|
|
|
|
|
|
|
2020-02-09 13:45:44 +01:00
|
|
|
extractor.oauth.port
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``6414``
|
|
|
|
Description
|
|
|
|
Port number to listen on during OAuth authorization.
|
2020-02-09 13:45:44 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: All redirects will go to http://localhost:6414/, regardless
|
|
|
|
of the port specified here. You'll have to manually adjust the
|
|
|
|
port number in your browser's address bar when using a different
|
|
|
|
port than the default.
|
2020-02-09 13:45:44 +01:00
|
|
|
|
|
|
|
|
2021-10-17 04:14:58 +02:00
|
|
|
extractor.patreon.files
|
|
|
|
-----------------------
|
|
|
|
Type
|
|
|
|
``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``["images", "attachments", "postfile", "content"]``
|
|
|
|
Description
|
|
|
|
Determines the type and order of files to be downloaded.
|
|
|
|
|
|
|
|
Available types are
|
|
|
|
``postfile``, ``images``, ``attachments``, and ``content``.
|
|
|
|
|
|
|
|
|
2019-01-21 19:55:05 +01:00
|
|
|
extractor.photobucket.subalbums
|
|
|
|
-------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download subalbums.
|
2019-01-21 19:55:05 +01:00
|
|
|
|
|
|
|
|
2021-05-17 01:38:00 +02:00
|
|
|
extractor.pillowfort.external
|
|
|
|
-----------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Follow links to external sites, e.g. Twitter,
|
|
|
|
|
|
|
|
|
2021-05-17 02:57:02 +02:00
|
|
|
extractor.pillowfort.inline
|
|
|
|
---------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Extract inline images.
|
|
|
|
|
|
|
|
|
2021-01-25 00:38:19 +01:00
|
|
|
extractor.pillowfort.reblogs
|
|
|
|
----------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract media from reblogged posts.
|
|
|
|
|
|
|
|
|
2020-06-16 14:41:05 +02:00
|
|
|
extractor.pinterest.sections
|
|
|
|
----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Include pins from board sections.
|
2020-06-16 14:41:05 +02:00
|
|
|
|
|
|
|
|
2020-12-21 16:09:06 +01:00
|
|
|
extractor.pinterest.videos
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download from video pins.
|
|
|
|
|
|
|
|
|
2020-03-09 21:17:16 +01:00
|
|
|
extractor.pixiv.user.avatar
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Download user avatars.
|
2020-03-09 21:17:16 +01:00
|
|
|
|
|
|
|
|
2021-05-14 20:30:28 +02:00
|
|
|
extractor.pixiv.user.metadata
|
|
|
|
-----------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Fetch extended ``user`` metadata.
|
|
|
|
|
|
|
|
|
2021-01-17 16:37:07 +01:00
|
|
|
extractor.pixiv.work.related
|
|
|
|
----------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Also download related artworks.
|
|
|
|
|
|
|
|
|
2021-04-27 23:51:37 +02:00
|
|
|
extractor.pixiv.tags
|
|
|
|
--------------------
|
2021-03-05 17:18:51 +01:00
|
|
|
Type
|
2021-04-27 23:51:37 +02:00
|
|
|
``string``
|
2021-03-05 17:18:51 +01:00
|
|
|
Default
|
2021-04-27 23:51:37 +02:00
|
|
|
``"japanese"``
|
2021-03-05 17:18:51 +01:00
|
|
|
Description
|
2021-05-07 20:41:54 +02:00
|
|
|
Controls the ``tags`` metadata field.
|
2021-04-27 23:51:37 +02:00
|
|
|
|
2021-05-07 20:41:54 +02:00
|
|
|
* `"japanese"`: List of Japanese tags
|
2021-04-27 23:51:37 +02:00
|
|
|
* `"translated"`: List of translated tags
|
2021-05-07 20:41:54 +02:00
|
|
|
* `"original"`: Unmodified list with both Japanese and translated tags
|
2021-03-05 17:18:51 +01:00
|
|
|
|
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
extractor.pixiv.ugoira
|
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download Pixiv's Ugoira animations or ignore them.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
These animations come as a ``.zip`` file containing all
|
|
|
|
animation frames in JPEG format.
|
2020-02-14 01:22:16 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Use an `ugoira` post processor to convert them
|
|
|
|
to watchable videos. (Example__)
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2020-06-12 18:25:17 +02:00
|
|
|
.. __: https://github.com/mikf/gallery-dl/blob/v1.12.3/docs/gallery-dl-example.conf#L9-L14
|
2020-02-14 01:22:16 +01:00
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2021-05-24 17:49:46 +02:00
|
|
|
extractor.pixiv.max-posts
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``0``
|
|
|
|
Description
|
|
|
|
When downloading galleries, this sets the maximum number of posts to get.
|
|
|
|
A value of ``0`` means no limit.
|
|
|
|
|
|
|
|
|
2019-04-14 21:52:23 +02:00
|
|
|
extractor.plurk.comments
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Also search Plurk comments for URLs.
|
2019-04-14 21:52:23 +02:00
|
|
|
|
|
|
|
|
2021-08-12 15:12:42 +02:00
|
|
|
extractor.reactor.gif
|
2021-08-12 16:05:26 +02:00
|
|
|
---------------------
|
2021-08-12 15:12:42 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Format in which to download animated images.
|
|
|
|
|
2021-08-12 16:05:26 +02:00
|
|
|
Use ``true`` to download animated images as gifs and ``false``
|
2021-08-12 15:12:42 +02:00
|
|
|
to download as mp4 videos.
|
|
|
|
|
|
|
|
|
2019-05-27 22:24:48 +02:00
|
|
|
extractor.readcomiconline.captcha
|
|
|
|
---------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"stop"``
|
|
|
|
Description
|
|
|
|
Controls how to handle redirects to CAPTCHA pages.
|
2019-05-27 22:24:48 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``"stop``: Stop the current extractor run.
|
|
|
|
* ``"wait``: Ask the user to solve the CAPTCHA and wait.
|
2019-05-27 22:24:48 +02:00
|
|
|
|
|
|
|
|
2017-06-18 22:16:26 +02:00
|
|
|
extractor.reddit.comments
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``0``
|
|
|
|
Description
|
|
|
|
The value of the ``limit`` parameter when loading
|
|
|
|
a submission and its comments.
|
|
|
|
This number (roughly) specifies the total amount of comments
|
|
|
|
being retrieved with the first API call.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Reddit's internal default and maximum values for this parameter
|
|
|
|
appear to be 200 and 500 respectively.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
The value ``0`` ignores all comments and significantly reduces the
|
|
|
|
time required when scanning a subreddit.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2017-06-23 16:14:51 +02:00
|
|
|
extractor.reddit.morecomments
|
2017-06-24 12:17:26 +02:00
|
|
|
-----------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Retrieve additional comments by resolving the ``more`` comment
|
|
|
|
stubs in the base comment tree.
|
2017-06-23 16:14:51 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
This requires 1 additional API call for every 100 extra comments.
|
2017-06-23 16:14:51 +02:00
|
|
|
|
|
|
|
|
2017-10-12 23:37:28 +02:00
|
|
|
extractor.reddit.date-min & .date-max
|
|
|
|
-------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Date|_
|
|
|
|
Default
|
|
|
|
``0`` and ``253402210800`` (timestamp of |datetime.max|_)
|
|
|
|
Description
|
|
|
|
Ignore all submissions posted before/after this date.
|
2017-07-04 19:34:34 +02:00
|
|
|
|
|
|
|
|
2017-10-12 23:37:28 +02:00
|
|
|
extractor.reddit.id-min & .id-max
|
|
|
|
---------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Example
|
|
|
|
``"6kmzv2"``
|
|
|
|
Description
|
|
|
|
Ignore all submissions posted before/after the submission with this ID.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.reddit.recursion
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
``0``
|
|
|
|
Description
|
|
|
|
Reddit extractors can recursively visit other submissions
|
|
|
|
linked to in the initial set of submissions.
|
|
|
|
This value sets the maximum recursion depth.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Special values:
|
2021-03-20 01:31:12 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``0``: Recursion is disabled
|
|
|
|
* ``-1``: Infinite recursion (don't do this)
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.reddit.refresh-token
|
|
|
|
------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
The ``refresh-token`` value you get from
|
|
|
|
`linking your Reddit account to gallery-dl <OAuth_>`__.
|
2017-06-23 16:14:51 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Using a ``refresh-token`` allows you to access private or otherwise
|
|
|
|
not publicly available subreddits, given that your account is
|
|
|
|
authorized to do so,
|
|
|
|
but requests to the reddit API are going to be rate limited
|
|
|
|
at 600 requests every 10 minutes/600 seconds.
|
2017-06-18 22:16:26 +02:00
|
|
|
|
|
|
|
|
2020-01-31 23:45:02 +01:00
|
|
|
extractor.reddit.videos
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Control video download behavior.
|
2020-01-31 23:45:02 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Download videos and use `youtube-dl`_ to handle
|
|
|
|
HLS and DASH manifests
|
|
|
|
* ``"ytdl"``: Download videos and let `youtube-dl`_ handle all of
|
|
|
|
video extraction and download
|
|
|
|
* ``false``: Ignore videos
|
2020-01-31 23:45:02 +01:00
|
|
|
|
|
|
|
|
2020-06-12 18:25:17 +02:00
|
|
|
extractor.redgifs.format
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-10-22 22:47:29 +02:00
|
|
|
* ``list`` of ``strings``
|
|
|
|
* ``string``
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
2021-11-04 21:31:20 +01:00
|
|
|
``["hd", "sd", "gif"]``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2021-10-22 22:47:29 +02:00
|
|
|
List of names of the preferred animation format, which can be
|
2021-11-04 21:31:20 +01:00
|
|
|
``"hd"``, ``"sd"``, `"gif"``, `"vthumbnail"``, `"thumbnail"``, or ``"poster"``.
|
2020-06-12 18:25:17 +02:00
|
|
|
|
2021-10-22 22:47:29 +02:00
|
|
|
If a selected format is not available, the next one in the list will be
|
|
|
|
tried until an available format is found.
|
|
|
|
|
|
|
|
If the format is given as ``string``, it will be extended with
|
2021-11-04 21:31:20 +01:00
|
|
|
``["hd", "sd", "gif"]``. Use a list with one element to
|
2021-10-22 22:47:29 +02:00
|
|
|
restrict it to only one possible format.
|
2020-06-12 18:25:17 +02:00
|
|
|
|
|
|
|
|
2020-10-30 00:53:11 +01:00
|
|
|
extractor.sankakucomplex.embeds
|
|
|
|
-------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Download video embeds from external sites.
|
|
|
|
|
|
|
|
|
|
|
|
extractor.sankakucomplex.videos
|
|
|
|
-------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download videos.
|
|
|
|
|
|
|
|
|
2021-11-23 21:16:42 +01:00
|
|
|
extractor.skeb.thumbnails
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Download thumbnails.
|
|
|
|
|
|
|
|
|
2019-03-10 15:20:35 +01:00
|
|
|
extractor.smugmug.videos
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download video files.
|
2019-03-10 15:20:35 +01:00
|
|
|
|
|
|
|
|
2018-12-26 14:29:30 +01:00
|
|
|
extractor.tumblr.avatar
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Download blog avatars.
|
2018-12-26 14:29:30 +01:00
|
|
|
|
|
|
|
|
2019-07-16 23:08:27 +02:00
|
|
|
extractor.tumblr.date-min & .date-max
|
|
|
|
-------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Date|_
|
|
|
|
Default
|
|
|
|
``0`` and ``null``
|
|
|
|
Description
|
|
|
|
Ignore all posts published before/after this date.
|
2019-07-16 23:08:27 +02:00
|
|
|
|
|
|
|
|
2017-11-22 23:09:08 +01:00
|
|
|
extractor.tumblr.external
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Follow external URLs (e.g. from "Link" posts) and try to extract
|
|
|
|
images from them.
|
2017-11-22 23:09:08 +01:00
|
|
|
|
|
|
|
|
|
|
|
extractor.tumblr.inline
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Search posts for inline images and videos.
|
2017-11-22 23:09:08 +01:00
|
|
|
|
|
|
|
|
2018-01-05 13:00:25 +01:00
|
|
|
extractor.tumblr.reblogs
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
* ``true``: Extract media from reblogged posts
|
|
|
|
* ``false``: Skip reblogged posts
|
|
|
|
* ``"same-blog"``: Skip reblogged posts unless the original post
|
|
|
|
is from the same blog
|
2018-01-05 13:00:25 +01:00
|
|
|
|
|
|
|
|
2017-11-22 23:09:08 +01:00
|
|
|
extractor.tumblr.posts
|
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``"all"``
|
|
|
|
Example
|
|
|
|
``"video,audio,link"`` or ``["video", "audio", "link"]``
|
|
|
|
Description
|
|
|
|
A (comma-separated) list of post types to extract images, etc. from.
|
2017-11-22 23:09:08 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible types are ``text``, ``quote``, ``link``, ``answer``,
|
|
|
|
``video``, ``audio``, ``photo``, ``chat``.
|
2017-11-22 23:09:08 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
You can use ``"all"`` instead of listing all types separately.
|
2018-06-29 19:38:53 +02:00
|
|
|
|
|
|
|
|
2020-10-22 21:33:53 +02:00
|
|
|
extractor.twitter.cards
|
|
|
|
-----------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Fetch media from `Cards <https://developer.twitter.com/en/docs/twitter-for-websites/cards/overview/abouts-cards>`__.
|
|
|
|
|
|
|
|
|
2021-02-26 13:50:46 +01:00
|
|
|
extractor.twitter.conversations
|
|
|
|
-------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Fetch media from all Tweets and replies in a `conversation
|
|
|
|
<https://help.twitter.com/en/using-twitter/twitter-conversations>`__.
|
|
|
|
|
|
|
|
|
2021-10-05 18:58:10 +02:00
|
|
|
extractor.twitter.size
|
|
|
|
----------------------
|
|
|
|
Type
|
|
|
|
``list`` of ``strings``
|
|
|
|
Default
|
2021-12-15 23:17:07 +01:00
|
|
|
``["orig", "4096x4096", "large", "medium", "small"]``
|
2021-10-05 18:58:10 +02:00
|
|
|
Description
|
|
|
|
The image version to download.
|
|
|
|
Any entries after the first one will be used for potential
|
|
|
|
`fallback <extractor.*.fallback_>`_ URLs.
|
|
|
|
|
|
|
|
Known available sizes are
|
|
|
|
``4096x4096``, ``orig``, ``large``, ``medium``, and ``small``.
|
|
|
|
|
|
|
|
|
2021-08-16 01:31:39 +02:00
|
|
|
extractor.twitter.logout
|
|
|
|
------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Logout and retry as guest when access to another user's Tweets is blocked.
|
|
|
|
|
|
|
|
|
2021-10-29 22:10:58 +02:00
|
|
|
extractor.twitter.pinned
|
|
|
|
------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Fetch media from pinned Tweets.
|
|
|
|
|
|
|
|
|
2020-06-24 21:13:16 +02:00
|
|
|
extractor.twitter.quoted
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
2021-06-11 21:19:04 +02:00
|
|
|
``false``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
Fetch media from quoted Tweets.
|
2020-06-24 21:13:16 +02:00
|
|
|
|
|
|
|
|
2020-04-29 23:11:24 +02:00
|
|
|
extractor.twitter.replies
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Fetch media from replies to other Tweets.
|
2020-04-29 23:11:24 +02:00
|
|
|
|
2021-08-10 22:02:19 +02:00
|
|
|
If this value is ``"self"``, only consider replies where
|
|
|
|
reply and original Tweet are from the same user.
|
|
|
|
|
2020-04-29 23:11:24 +02:00
|
|
|
|
2018-08-17 20:04:11 +02:00
|
|
|
extractor.twitter.retweets
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
2021-06-11 21:19:04 +02:00
|
|
|
``false``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
Fetch media from Retweets.
|
2018-08-17 20:04:11 +02:00
|
|
|
|
2020-09-28 23:03:35 +02:00
|
|
|
If this value is ``"original"``, metadata for these files
|
|
|
|
will be taken from the original Tweets, not the Retweets.
|
|
|
|
|
2018-08-17 20:04:11 +02:00
|
|
|
|
2021-05-22 21:07:21 +02:00
|
|
|
extractor.twitter.text-tweets
|
|
|
|
-----------------------------
|
2021-05-22 17:01:49 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
2021-05-22 21:07:21 +02:00
|
|
|
Also emit metadata for text-only Tweets without media content.
|
2021-05-22 17:01:49 +02:00
|
|
|
|
|
|
|
This only has an effect with a ``metadata`` (or ``exec``) post processor
|
|
|
|
with `"event": "post" <metadata.event_>`_
|
|
|
|
and appropriate `filename <metadata.filename_>`_.
|
|
|
|
|
|
|
|
|
2020-01-18 21:26:46 +01:00
|
|
|
extractor.twitter.twitpic
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract `TwitPic <https://twitpic.com/>`__ embeds.
|
2020-01-18 21:26:46 +01:00
|
|
|
|
|
|
|
|
2021-03-15 22:55:24 +01:00
|
|
|
extractor.twitter.users
|
|
|
|
-----------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"timeline"``
|
2021-03-20 01:31:12 +01:00
|
|
|
Example
|
|
|
|
``"https://twitter.com/search?q=from:{legacy[screen_name]}"``
|
2021-03-15 22:55:24 +01:00
|
|
|
Description
|
2021-03-20 01:31:12 +01:00
|
|
|
| Format string for user URLs generated from
|
|
|
|
``following`` and ``list-members`` queries,
|
|
|
|
| whose replacement field values come from Twitter ``user`` objects
|
|
|
|
(`Example <https://gist.githubusercontent.com/mikf/99d2719b3845023326c7a4b6fb88dd04/raw/275b4f0541a2c7dc0a86d3998f7d253e8f10a588/github.json>`_)
|
|
|
|
|
|
|
|
Special values:
|
|
|
|
|
|
|
|
* ``"timeline"``: ``https://twitter.com/i/user/{rest_id}``
|
|
|
|
* ``"media"``: ``https://twitter.com/id:{rest_id}/media``
|
|
|
|
|
|
|
|
Note: To allow gallery-dl to follow custom URL formats, set the blacklist__
|
|
|
|
for ``twitter`` to a non-default value, e.g. an empty string ``""``.
|
2021-03-15 22:55:24 +01:00
|
|
|
|
2021-03-20 01:31:12 +01:00
|
|
|
.. __: `extractor.*.blacklist & .whitelist`_
|
2021-03-15 22:55:24 +01:00
|
|
|
|
|
|
|
|
2018-09-30 18:41:39 +02:00
|
|
|
extractor.twitter.videos
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Control video download behavior.
|
2019-11-01 22:06:07 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Download videos
|
|
|
|
* ``"ytdl"``: Download videos using `youtube-dl`_
|
|
|
|
* ``false``: Skip video Tweets
|
2018-09-30 18:41:39 +02:00
|
|
|
|
|
|
|
|
2021-01-21 22:41:49 +01:00
|
|
|
extractor.unsplash.format
|
|
|
|
-------------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"raw"``
|
|
|
|
Description
|
|
|
|
Name of the image format to download.
|
|
|
|
|
|
|
|
Available formats are
|
2021-03-03 03:05:25 +01:00
|
|
|
``"raw"``, ``"full"``, ``"regular"``, ``"small"``, and ``"thumb"``.
|
2021-01-21 22:41:49 +01:00
|
|
|
|
|
|
|
|
2019-12-19 17:20:51 +01:00
|
|
|
extractor.vsco.videos
|
|
|
|
---------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download video files.
|
2019-12-19 17:20:51 +01:00
|
|
|
|
|
|
|
|
2019-05-30 23:11:36 +02:00
|
|
|
extractor.wallhaven.api-key
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
2020-10-19 21:57:26 +02:00
|
|
|
Your `Wallhaven API Key <https://wallhaven.cc/settings/account>`__,
|
|
|
|
to use your account's browsing settings and default filters when searching.
|
2019-05-30 23:11:36 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
See https://wallhaven.cc/help/api for more information.
|
2019-05-30 23:11:36 +02:00
|
|
|
|
|
|
|
|
2020-10-15 15:12:09 +02:00
|
|
|
extractor.weasyl.api-key
|
2020-10-15 15:17:19 +02:00
|
|
|
------------------------
|
2020-10-15 15:12:09 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
2020-10-19 21:57:26 +02:00
|
|
|
Your `Weasyl API Key <https://www.weasyl.com/control/apikeys>`__,
|
|
|
|
to use your account's browsing settings and filters.
|
2020-10-15 15:12:09 +02:00
|
|
|
|
|
|
|
|
2020-04-29 23:27:29 +02:00
|
|
|
extractor.weibo.retweets
|
|
|
|
------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
2021-05-27 23:09:42 +02:00
|
|
|
Fetch media from retweeted posts.
|
|
|
|
|
|
|
|
If this value is ``"original"``, metadata for these files
|
|
|
|
will be taken from the original posts, not the retweeted posts.
|
2020-04-29 23:27:29 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.weibo.videos
|
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Download video files.
|
2020-04-29 23:27:29 +02:00
|
|
|
|
|
|
|
|
2021-07-10 20:47:33 +02:00
|
|
|
extractor.ytdl.enabled
|
|
|
|
----------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Match **all** URLs, even ones without a ``ytdl:`` prefix.
|
|
|
|
|
|
|
|
|
|
|
|
extractor.ytdl.format
|
|
|
|
---------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
youtube-dl's default, currently ``"bestvideo+bestaudio/best"``
|
|
|
|
Description
|
|
|
|
Video `format selection
|
|
|
|
<https://github.com/ytdl-org/youtube-dl#format-selection>`__
|
|
|
|
directly passed to youtube-dl.
|
|
|
|
|
|
|
|
|
2021-07-11 23:01:57 +02:00
|
|
|
extractor.ytdl.generic
|
|
|
|
----------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls the use of youtube-dl's generic extractor.
|
|
|
|
|
|
|
|
Set this option to ``"force"`` for the same effect as youtube-dl's
|
|
|
|
``--force-generic-extractor``.
|
|
|
|
|
|
|
|
|
2021-07-10 20:47:33 +02:00
|
|
|
extractor.ytdl.logging
|
|
|
|
----------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Route youtube-dl's output through gallery-dl's logging system.
|
|
|
|
Otherwise youtube-dl will write its output directly to stdout/stderr.
|
|
|
|
|
|
|
|
Note: Set ``quiet`` and ``no_warnings`` in
|
|
|
|
`extractor.ytdl.raw-options`_ to ``true`` to suppress all output.
|
|
|
|
|
|
|
|
|
|
|
|
extractor.ytdl.module
|
|
|
|
---------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
2021-11-29 04:36:43 +01:00
|
|
|
``null``
|
2021-07-10 20:47:33 +02:00
|
|
|
Description
|
|
|
|
Name of the youtube-dl Python module to import.
|
|
|
|
|
2021-11-29 04:36:43 +01:00
|
|
|
Setting this to ``null`` will try to import ``"yt_dlp"``
|
|
|
|
followed by ``"youtube_dl"`` as fallback.
|
|
|
|
|
2021-07-10 20:47:33 +02:00
|
|
|
|
|
|
|
extractor.ytdl.raw-options
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Example
|
|
|
|
.. code:: json
|
|
|
|
|
|
|
|
{
|
|
|
|
"quiet": true,
|
|
|
|
"writesubtitles": true,
|
|
|
|
"merge_output_format": "mkv"
|
|
|
|
}
|
|
|
|
|
|
|
|
Description
|
|
|
|
Additional options passed directly to the ``YoutubeDL`` constructor.
|
|
|
|
|
|
|
|
All available options can be found in `youtube-dl's docstrings
|
|
|
|
<https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L138-L318>`__.
|
|
|
|
|
|
|
|
|
2021-11-12 18:29:40 +01:00
|
|
|
extractor.ytdl.cmdline-args
|
|
|
|
---------------------------
|
|
|
|
Type
|
|
|
|
* ``string``
|
|
|
|
* ``list`` of ``strings``
|
|
|
|
Example
|
|
|
|
* ``"--quiet --write-sub --merge-output-format mkv"``
|
|
|
|
* ``["--quiet", "--write-sub", "--merge-output-format", "mkv"]``
|
|
|
|
Description
|
|
|
|
Additional options specified as youtube-dl command-line arguments.
|
|
|
|
|
|
|
|
|
|
|
|
extractor.ytdl.config-file
|
|
|
|
--------------------------
|
|
|
|
Type
|
|
|
|
|Path|_
|
|
|
|
Example
|
|
|
|
``"~/.config/youtube-dl/config"``
|
|
|
|
Description
|
|
|
|
Location of a youtube-dl configuration file to load options from.
|
|
|
|
|
|
|
|
|
2018-07-13 16:20:14 +02:00
|
|
|
extractor.[booru].tags
|
2018-06-29 19:38:53 +02:00
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Categorize tags by their respective types
|
|
|
|
and provide them as ``tags_<type>`` metadata fields.
|
2018-06-29 19:38:53 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: This requires 1 additional HTTP request for each post.
|
2017-11-22 23:09:08 +01:00
|
|
|
|
2021-07-10 20:47:33 +02:00
|
|
|
|
2021-04-13 23:40:24 +02:00
|
|
|
extractor.[booru].notes
|
2021-05-19 02:57:36 +02:00
|
|
|
-----------------------
|
2021-04-13 23:40:24 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Extract overlay notes (position and text).
|
|
|
|
|
|
|
|
Note: This requires 1 additional HTTP request for each post.
|
2017-11-22 23:09:08 +01:00
|
|
|
|
2021-07-10 20:47:33 +02:00
|
|
|
|
2019-01-07 18:22:33 +01:00
|
|
|
extractor.[manga-extractor].chapter-reverse
|
|
|
|
-------------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Reverse the order of chapter URLs extracted from manga pages.
|
2019-01-07 18:22:33 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Start with the latest chapter
|
|
|
|
* ``false``: Start with the first chapter
|
2019-01-07 18:22:33 +01:00
|
|
|
|
|
|
|
|
2021-09-18 02:15:42 +02:00
|
|
|
extractor.[manga-extractor].page-reverse
|
|
|
|
----------------------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Download manga chapter pages in reverse order.
|
|
|
|
|
2017-10-14 23:01:33 +02:00
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
Downloader Options
|
|
|
|
==================
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
|
|
|
|
downloader.*.enabled
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Enable/Disable this downloader module.
|
2018-11-16 18:02:24 +01:00
|
|
|
|
|
|
|
|
2020-09-01 22:05:17 +02:00
|
|
|
downloader.*.filesize-min & .filesize-max
|
|
|
|
-----------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Example
|
|
|
|
``"32000"``, ``"500k"``, ``"2.5M"``
|
|
|
|
Description
|
|
|
|
Minimum/Maximum allowed file size in bytes.
|
|
|
|
Any file smaller/larger than this limit will not be downloaded.
|
2020-09-01 22:05:17 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are valid integer or floating-point numbers
|
|
|
|
optionally followed by one of ``k``, ``m``. ``g``, ``t`` or ``p``.
|
|
|
|
These suffixes are case-insensitive.
|
2020-09-01 22:05:17 +02:00
|
|
|
|
|
|
|
|
2019-06-20 17:19:44 +02:00
|
|
|
downloader.*.mtime
|
|
|
|
------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Use |Last-Modified|_ HTTP response headers
|
|
|
|
to set file modification times.
|
2019-06-20 17:19:44 +02:00
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
downloader.*.part
|
|
|
|
-----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls the use of ``.part`` files during file downloads.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``true``: Write downloaded data into ``.part`` files and rename
|
|
|
|
them upon download completion. This mode additionally supports
|
|
|
|
resuming incomplete downloads.
|
|
|
|
* ``false``: Do not use ``.part`` files and write data directly
|
|
|
|
into the actual output files.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
downloader.*.part-directory
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
Alternate location for ``.part`` files.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Missing directories will be created as needed.
|
|
|
|
If this value is ``null``, ``.part`` files are going to be stored
|
|
|
|
alongside the actual output files.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2021-09-28 22:37:11 +02:00
|
|
|
downloader.*.progress
|
|
|
|
---------------------
|
|
|
|
Type
|
|
|
|
``float``
|
|
|
|
Default
|
|
|
|
``3.0``
|
|
|
|
Description
|
|
|
|
Number of seconds until a download progress indicator
|
|
|
|
for the current download is displayed.
|
|
|
|
|
|
|
|
Set this option to ``null`` to disable this indicator.
|
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
downloader.*.rate
|
|
|
|
-----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Example
|
|
|
|
``"32000"``, ``"500k"``, ``"2.5M"``
|
|
|
|
Description
|
|
|
|
Maximum download rate in bytes per second.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are valid integer or floating-point numbers
|
|
|
|
optionally followed by one of ``k``, ``m``. ``g``, ``t`` or ``p``.
|
|
|
|
These suffixes are case-insensitive.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
downloader.*.retries
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``integer``
|
|
|
|
Default
|
|
|
|
`extractor.*.retries`_
|
|
|
|
Description
|
|
|
|
Maximum number of retries during file downloads,
|
|
|
|
or ``-1`` for infinite retries.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
downloader.*.timeout
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``float`` or ``null``
|
|
|
|
Default
|
|
|
|
`extractor.*.timeout`_
|
|
|
|
Description
|
|
|
|
Connection timeout during file downloads.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
downloader.*.verify
|
|
|
|
-------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
`extractor.*.verify`_
|
|
|
|
Description
|
|
|
|
Certificate validation during file downloads.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2019-08-07 22:52:29 +02:00
|
|
|
downloader.http.adjust-extensions
|
|
|
|
---------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Check the file headers of ``jpg``, ``png``, and ``gif`` files
|
|
|
|
and adjust their filename extensions if they do not match.
|
2019-08-07 22:52:29 +02:00
|
|
|
|
|
|
|
|
2021-02-21 19:13:39 +01:00
|
|
|
downloader.http.headers
|
|
|
|
-----------------------
|
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Example
|
|
|
|
``{"Accept": "image/webp,*/*", "Referer": "https://example.org/"}``
|
|
|
|
Description
|
|
|
|
Additional HTTP headers to send when downloading files,
|
|
|
|
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
downloader.ytdl.format
|
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
youtube-dl's default, currently ``"bestvideo+bestaudio/best"``
|
|
|
|
Description
|
|
|
|
Video `format selection
|
|
|
|
<https://github.com/ytdl-org/youtube-dl#format-selection>`__
|
|
|
|
directly passed to youtube-dl.
|
2018-11-13 18:06:36 +01:00
|
|
|
|
|
|
|
|
2019-07-24 21:19:11 +02:00
|
|
|
downloader.ytdl.forward-cookies
|
|
|
|
-------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Forward cookies to youtube-dl.
|
2019-07-24 21:19:11 +02:00
|
|
|
|
|
|
|
|
2018-10-19 22:10:59 +02:00
|
|
|
downloader.ytdl.logging
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Route youtube-dl's output through gallery-dl's logging system.
|
|
|
|
Otherwise youtube-dl will write its output directly to stdout/stderr.
|
2018-10-19 22:10:59 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: Set ``quiet`` and ``no_warnings`` in
|
|
|
|
`downloader.ytdl.raw-options`_ to ``true`` to suppress all output.
|
2018-10-19 22:10:59 +02:00
|
|
|
|
|
|
|
|
2021-03-01 03:10:42 +01:00
|
|
|
downloader.ytdl.module
|
|
|
|
----------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
2021-11-29 04:36:43 +01:00
|
|
|
``null``
|
2021-03-01 03:10:42 +01:00
|
|
|
Description
|
|
|
|
Name of the youtube-dl Python module to import.
|
|
|
|
|
2021-11-29 04:36:43 +01:00
|
|
|
Setting this to ``null`` will first try to import ``"yt_dlp"``
|
|
|
|
and use ``"youtube_dl"`` as fallback.
|
|
|
|
|
2021-03-01 03:10:42 +01:00
|
|
|
|
2019-08-24 22:39:37 +02:00
|
|
|
downloader.ytdl.outtmpl
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
The `Output Template <https://github.com/ytdl-org/youtube-dl#output-template>`__
|
|
|
|
used to generate filenames for files downloaded with youtube-dl.
|
2019-08-24 22:39:37 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Special values:
|
2019-08-24 22:39:37 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``null``: generate filenames with `extractor.*.filename`_
|
|
|
|
* ``"default"``: use youtube-dl's default, currently ``"%(title)s-%(id)s.%(ext)s"``
|
2019-08-24 22:39:37 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: An output template other than ``null`` might
|
|
|
|
cause unexpected results in combination with other options
|
|
|
|
(e.g. ``"skip": "enumerate"``)
|
2019-08-24 22:39:37 +02:00
|
|
|
|
|
|
|
|
2018-10-19 22:10:59 +02:00
|
|
|
downloader.ytdl.raw-options
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Example
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2018-10-19 22:10:59 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
{
|
|
|
|
"quiet": true,
|
|
|
|
"writesubtitles": true,
|
|
|
|
"merge_output_format": "mkv"
|
|
|
|
}
|
2018-10-19 22:10:59 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
Additional options passed directly to the ``YoutubeDL`` constructor.
|
|
|
|
|
|
|
|
All available options can be found in `youtube-dl's docstrings
|
|
|
|
<https://github.com/ytdl-org/youtube-dl/blob/master/youtube_dl/YoutubeDL.py#L138-L318>`__.
|
2018-10-19 22:10:59 +02:00
|
|
|
|
|
|
|
|
2021-11-12 18:29:40 +01:00
|
|
|
downloader.ytdl.cmdline-args
|
|
|
|
----------------------------
|
|
|
|
Type
|
|
|
|
* ``string``
|
|
|
|
* ``list`` of ``strings``
|
|
|
|
Example
|
|
|
|
* ``"--quiet --write-sub --merge-output-format mkv"``
|
|
|
|
* ``["--quiet", "--write-sub", "--merge-output-format", "mkv"]``
|
|
|
|
Description
|
|
|
|
Additional options specified as youtube-dl command-line arguments.
|
|
|
|
|
|
|
|
|
|
|
|
downloader.ytdl.config-file
|
|
|
|
---------------------------
|
|
|
|
Type
|
|
|
|
|Path|_
|
|
|
|
Example
|
|
|
|
``"~/.config/youtube-dl/config"``
|
|
|
|
Description
|
|
|
|
Location of a youtube-dl configuration file to load options from.
|
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
Output Options
|
|
|
|
==============
|
|
|
|
|
2018-11-16 18:02:24 +01:00
|
|
|
|
2021-04-12 01:55:55 +02:00
|
|
|
output.fallback
|
|
|
|
---------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Include fallback URLs in the output of ``-g/--get-urls``.
|
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
output.mode
|
|
|
|
-----------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"auto"``
|
|
|
|
Description
|
|
|
|
Controls the output string format and status indicators.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``"null"``: No output
|
|
|
|
* ``"pipe"``: Suitable for piping to other processes or files
|
|
|
|
* ``"terminal"``: Suitable for the standard Windows console
|
|
|
|
* ``"color"``: Suitable for terminals that understand ANSI escape codes and colors
|
|
|
|
* ``"auto"``: Automatically choose the best suitable output mode
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
|
|
|
output.shorten
|
|
|
|
--------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls whether the output strings should be shortened to fit
|
|
|
|
on one console line.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2021-09-13 21:29:38 +02:00
|
|
|
Set this option to ``"eaw"`` to also work with east-asian characters
|
|
|
|
with a display width greater than 1.
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2021-05-04 18:07:08 +02:00
|
|
|
output.skip
|
|
|
|
-----------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Show skipped file downloads.
|
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
output.progress
|
|
|
|
---------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool`` or ``string``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Controls the progress indicator when *gallery-dl* is run with
|
|
|
|
multiple URLs as arguments.
|
|
|
|
|
|
|
|
* ``true``: Show the default progress indicator
|
|
|
|
(``"[{current}/{total}] {url}"``)
|
|
|
|
* ``false``: Do not show any progress indicator
|
|
|
|
* Any ``string``: Show the progress indicator using this
|
|
|
|
as a custom `format string`_. Possible replacement keys are
|
|
|
|
``current``, ``total`` and ``url``.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2018-05-27 16:48:54 +02:00
|
|
|
output.log
|
|
|
|
----------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or |Logging Configuration|_
|
|
|
|
Default
|
|
|
|
``"[{name}][{levelname}] {message}"``
|
|
|
|
Description
|
|
|
|
Configuration for standard logging output to stderr.
|
2018-05-27 16:48:54 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
If this is a simple ``string``, it specifies
|
|
|
|
the format string for logging messages.
|
2018-05-27 16:48:54 +02:00
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
output.logfile
|
|
|
|
--------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_ or |Logging Configuration|_
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
File to write logging output to.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
|
|
|
output.unsupportedfile
|
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_ or |Logging Configuration|_
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Description
|
|
|
|
File to write external URLs unsupported by *gallery-dl* to.
|
2018-05-27 17:08:22 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
The default format string here is ``"{message}"``.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2018-10-08 23:08:11 +02:00
|
|
|
output.num-to-str
|
|
|
|
-----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Convert numeric values (``integer`` or ``float``) to ``string``
|
|
|
|
before outputting them as JSON.
|
2018-10-08 23:08:11 +02:00
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
Postprocessor Options
|
|
|
|
=====================
|
|
|
|
|
2020-10-19 21:57:26 +02:00
|
|
|
This section lists all options available inside
|
|
|
|
`Postprocessor Configuration`_ objects.
|
|
|
|
|
2021-05-22 17:01:49 +02:00
|
|
|
Each option is titled as ``<name>.<option>``, meaning a post processor
|
2020-10-19 21:57:26 +02:00
|
|
|
of type ``<name>`` will look for an ``<option>`` field inside its "body".
|
|
|
|
For example an ``exec`` post processor will recognize
|
|
|
|
an `async <exec.async_>`__, `command <exec.command_>`__,
|
2020-11-25 12:12:41 +01:00
|
|
|
and `event <exec.event_>`__ field:
|
2020-10-19 21:57:26 +02:00
|
|
|
|
|
|
|
.. code:: json
|
|
|
|
|
|
|
|
{
|
|
|
|
"name" : "exec",
|
|
|
|
"async" : false,
|
|
|
|
"command": "...",
|
2020-11-25 12:12:41 +01:00
|
|
|
"event" : "after"
|
2020-10-19 21:57:26 +02:00
|
|
|
}
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
classify.mapping
|
|
|
|
----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Default
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
{
|
2020-10-19 21:57:26 +02:00
|
|
|
"Pictures": ["jpg", "jpeg", "png", "gif", "bmp", "svg", "webp"],
|
|
|
|
"Video" : ["flv", "ogv", "avi", "mp4", "mpg", "mpeg", "3gp", "mkv", "webm", "vob", "wmv"],
|
|
|
|
"Music" : ["mp3", "aac", "flac", "ogg", "wma", "m4a", "wav"],
|
|
|
|
"Archives": ["zip", "rar", "7z", "tar", "gz", "bz2"]
|
2020-09-26 13:33:46 +02:00
|
|
|
}
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
A mapping from directory names to filename extensions that should
|
|
|
|
be stored in them.
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Files with an extension not listed will be ignored and stored
|
|
|
|
in their default location.
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
|
2020-01-18 21:08:25 +01:00
|
|
|
compare.action
|
|
|
|
--------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"replace"``
|
|
|
|
Description
|
2021-10-05 03:03:37 +02:00
|
|
|
The action to take when files do **not** compare as equal.
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
* ``"replace"``: Replace/Overwrite the old version with the new one
|
2021-10-01 19:30:51 +02:00
|
|
|
|
2021-10-05 03:03:37 +02:00
|
|
|
* ``"enumerate"``: Add an enumeration index to the filename of the new
|
|
|
|
version like `skip = "enumerate" <extractor.*.skip_>`__
|
|
|
|
|
|
|
|
|
|
|
|
compare.equal
|
|
|
|
-------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"null"``
|
|
|
|
Description
|
|
|
|
The action to take when files do compare as equal.
|
2021-10-01 19:30:51 +02:00
|
|
|
|
2021-10-05 03:03:37 +02:00
|
|
|
* ``"abort:N"``: Stop the current extractor run
|
2021-10-01 19:30:51 +02:00
|
|
|
after ``N`` consecutive files compared as equal.
|
|
|
|
|
2021-10-05 03:03:37 +02:00
|
|
|
* ``"terminate:N"``: Stop the current extractor run,
|
|
|
|
including parent extractors,
|
2021-10-01 19:30:51 +02:00
|
|
|
after ``N`` consecutive files compared as equal.
|
|
|
|
|
2021-10-05 03:03:37 +02:00
|
|
|
* ``"exit:N"``: Exit the program
|
|
|
|
after ``N`` consecutive files compared as equal.
|
2020-01-18 21:08:25 +01:00
|
|
|
|
|
|
|
|
|
|
|
compare.shallow
|
|
|
|
---------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Only compare file sizes. Do not read and compare their content.
|
2020-01-18 21:08:25 +01:00
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
exec.async
|
|
|
|
----------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Controls whether to wait for a subprocess to finish
|
|
|
|
or to let it run asynchronously.
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
exec.command
|
|
|
|
------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string`` or ``list`` of ``strings``
|
|
|
|
Example
|
|
|
|
* ``"convert {} {}.png && rm {}"``
|
|
|
|
* ``["echo", "{user[account]}", "{id}"]``
|
|
|
|
Description
|
|
|
|
The command to run.
|
|
|
|
|
|
|
|
* If this is a ``string``, it will be executed using the system's
|
|
|
|
shell, e.g. ``/bin/sh``. Any ``{}`` will be replaced
|
|
|
|
with the full path of a file or target directory, depending on
|
2020-11-25 12:12:41 +01:00
|
|
|
`exec.event`_
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
* If this is a ``list``, the first element specifies the program
|
|
|
|
name and any further elements its arguments.
|
|
|
|
Each element of this list is treated as a `format string`_ using
|
|
|
|
the files' metadata as well as ``{_path}``, ``{_directory}``,
|
|
|
|
and ``{_filename}``.
|
|
|
|
|
2019-11-03 21:45:45 +01:00
|
|
|
|
2020-11-25 12:12:41 +01:00
|
|
|
exec.event
|
2019-11-03 21:45:45 +01:00
|
|
|
----------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2020-11-25 12:12:41 +01:00
|
|
|
``string``
|
2020-09-26 13:33:46 +02:00
|
|
|
Default
|
2020-11-25 12:12:41 +01:00
|
|
|
``"after"``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
2020-11-25 12:12:41 +01:00
|
|
|
The event for which `exec.command`_ is run.
|
|
|
|
|
|
|
|
See `metadata.event`_ for a list of available events.
|
2019-01-17 21:18:12 +01:00
|
|
|
|
|
|
|
|
|
|
|
metadata.mode
|
|
|
|
-------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"json"``
|
|
|
|
Description
|
|
|
|
Select how to write metadata.
|
|
|
|
|
|
|
|
* ``"json"``: all metadata using `json.dump()
|
|
|
|
<https://docs.python.org/3/library/json.html#json.dump>`_
|
|
|
|
* ``"tags"``: ``tags`` separated by newlines
|
|
|
|
* ``"custom"``: result of applying `metadata.content-format`_
|
|
|
|
to a file's metadata dictionary
|
|
|
|
|
2019-01-17 21:18:12 +01:00
|
|
|
|
2020-11-25 12:12:41 +01:00
|
|
|
metadata.filename
|
|
|
|
-----------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Example
|
|
|
|
``"{id}.data.json"``
|
|
|
|
Description
|
|
|
|
A `format string`_ to build the filenames for metadata files with.
|
|
|
|
(see `extractor.filename <extractor.*.filename_>`__)
|
|
|
|
|
|
|
|
If this option is set, `metadata.extension`_ and
|
|
|
|
`metadata.extension-format`_ will be ignored.
|
|
|
|
|
|
|
|
|
2020-01-02 20:58:10 +01:00
|
|
|
metadata.directory
|
|
|
|
------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"."``
|
|
|
|
Example
|
|
|
|
``"metadata"``
|
|
|
|
Description
|
|
|
|
Directory where metadata files are stored in relative to the
|
|
|
|
current target location for file downloads.
|
|
|
|
|
2020-01-02 20:58:10 +01:00
|
|
|
|
2019-01-17 21:18:12 +01:00
|
|
|
metadata.extension
|
|
|
|
------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"json"`` or ``"txt"``
|
|
|
|
Description
|
|
|
|
Filename extension for metadata files that will be appended to the
|
|
|
|
original file names.
|
|
|
|
|
2019-11-29 23:12:22 +01:00
|
|
|
|
|
|
|
metadata.extension-format
|
|
|
|
-------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Example
|
|
|
|
* ``"{extension}.json"``
|
|
|
|
* ``"json"``
|
|
|
|
Description
|
|
|
|
Custom format string to build filename extensions for metadata
|
|
|
|
files with, which will replace the original filename extensions.
|
|
|
|
|
|
|
|
Note: `metadata.extension`_ is ignored if this option is set.
|
2019-11-29 23:12:22 +01:00
|
|
|
|
2019-01-17 21:18:12 +01:00
|
|
|
|
2020-11-25 12:12:41 +01:00
|
|
|
metadata.event
|
|
|
|
--------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"file"``
|
|
|
|
Description
|
|
|
|
The event for which metadata gets written to a file.
|
|
|
|
|
|
|
|
The available events are:
|
|
|
|
|
|
|
|
``init``
|
2021-05-22 17:01:49 +02:00
|
|
|
After post processor initialization
|
2020-11-25 12:12:41 +01:00
|
|
|
and before the first file download
|
|
|
|
``finalize``
|
|
|
|
On extractor shutdown, e.g. after all files were downloaded
|
|
|
|
``prepare``
|
|
|
|
Before a file download
|
|
|
|
``file``
|
|
|
|
When completing a file download,
|
|
|
|
but before it gets moved to its target location
|
|
|
|
``after``
|
|
|
|
After a file got moved to its target location
|
|
|
|
``skip``
|
|
|
|
When skipping a file download
|
|
|
|
``post``
|
|
|
|
When starting to download all files of a `post`,
|
|
|
|
e.g. a Tweet on Twitter or a post on Patreon.
|
|
|
|
|
|
|
|
|
2019-11-30 17:27:49 +01:00
|
|
|
metadata.content-format
|
|
|
|
-----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2020-10-27 20:09:58 +01:00
|
|
|
``string`` or ``list`` of ``strings``
|
2020-09-26 13:33:46 +02:00
|
|
|
Example
|
2020-10-27 20:09:58 +01:00
|
|
|
* ``"tags:\n\n{tags:J\n}\n"``
|
|
|
|
* ``["tags:", "", "{tags:J\n}"]``
|
2020-09-26 13:33:46 +02:00
|
|
|
Description
|
|
|
|
Custom format string to build the content of metadata files with.
|
2019-01-17 21:18:12 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Note: Only applies for ``"mode": "custom"``.
|
2019-07-14 22:37:28 +02:00
|
|
|
|
|
|
|
|
|
|
|
mtime.key
|
|
|
|
---------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"date"``
|
|
|
|
Description
|
|
|
|
Name of the metadata field whose value should be used.
|
2019-07-14 22:37:28 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
This value must either be a UNIX timestamp or a
|
|
|
|
|datetime|_ object.
|
2019-07-14 22:37:28 +02:00
|
|
|
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
ugoira.extension
|
|
|
|
----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"webm"``
|
|
|
|
Description
|
|
|
|
Filename extension for the resulting video files.
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
ugoira.ffmpeg-args
|
|
|
|
------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
``null``
|
|
|
|
Example
|
|
|
|
``["-c:v", "libvpx-vp9", "-an", "-b:v", "2M"]``
|
|
|
|
Description
|
|
|
|
Additional FFmpeg command-line arguments.
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2021-05-26 02:04:21 +02:00
|
|
|
ugoira.ffmpeg-demuxer
|
|
|
|
---------------------
|
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``image2``
|
|
|
|
Description
|
|
|
|
FFmpeg demuxer to read input files with. Possible values are
|
|
|
|
"`image2 <https://ffmpeg.org/ffmpeg-formats.html#image2-1>`_" and
|
|
|
|
"`concat <https://ffmpeg.org/ffmpeg-formats.html#concat-1>`_".
|
|
|
|
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
ugoira.ffmpeg-location
|
|
|
|
----------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_
|
|
|
|
Default
|
|
|
|
``"ffmpeg"``
|
|
|
|
Description
|
|
|
|
Location of the ``ffmpeg`` (or ``avconv``) executable to use.
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2018-08-29 15:58:01 +02:00
|
|
|
ugoira.ffmpeg-output
|
|
|
|
--------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Show FFmpeg output.
|
|
|
|
|
2018-08-29 15:58:01 +02:00
|
|
|
|
2018-06-20 18:48:10 +02:00
|
|
|
ugoira.ffmpeg-twopass
|
|
|
|
---------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Enable Two-Pass encoding.
|
|
|
|
|
2018-06-20 18:48:10 +02:00
|
|
|
|
2018-07-20 22:06:48 +02:00
|
|
|
ugoira.framerate
|
|
|
|
----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"auto"``
|
|
|
|
Description
|
|
|
|
Controls the frame rate argument (``-r``) for FFmpeg
|
|
|
|
|
|
|
|
* ``"auto"``: Automatically assign a fitting frame rate
|
|
|
|
based on delays between frames.
|
|
|
|
* any other ``string``: Use this value as argument for ``-r``.
|
|
|
|
* ``null`` or an empty ``string``: Don't set an explicit frame rate.
|
2018-07-20 22:06:48 +02:00
|
|
|
|
|
|
|
|
2018-06-18 17:25:52 +02:00
|
|
|
ugoira.keep-files
|
|
|
|
-----------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Keep ZIP archives after conversion.
|
|
|
|
|
2018-06-18 17:25:52 +02:00
|
|
|
|
2018-09-21 19:52:45 +02:00
|
|
|
ugoira.libx264-prevent-odd
|
|
|
|
--------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Prevent ``"width/height not divisible by 2"`` errors
|
|
|
|
when using ``libx264`` or ``libx265`` encoders
|
|
|
|
by applying a simple cropping filter. See this `Stack Overflow
|
|
|
|
thread <https://stackoverflow.com/questions/20847674>`__
|
|
|
|
for more information.
|
|
|
|
|
|
|
|
This option, when ``libx264/5`` is used, automatically
|
|
|
|
adds ``["-vf", "crop=iw-mod(iw\\,2):ih-mod(ih\\,2)"]``
|
|
|
|
to the list of FFmpeg command-line arguments
|
|
|
|
to reduce an odd width/height by 1 pixel and make them even.
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
|
2021-05-26 02:26:26 +02:00
|
|
|
ugoira.repeat-last-frame
|
|
|
|
------------------------
|
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``true``
|
|
|
|
Description
|
|
|
|
Allow repeating the last frame when necessary
|
|
|
|
to prevent it from only being displayed for a very short amount of time.
|
|
|
|
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
zip.compression
|
|
|
|
---------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"store"``
|
|
|
|
Description
|
|
|
|
Compression method to use when writing the archive.
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Possible values are ``"store"``, ``"zip"``, ``"bzip2"``, ``"lzma"``.
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
zip.extension
|
|
|
|
-------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"zip"``
|
|
|
|
Description
|
|
|
|
Filename extension for the created ZIP archive.
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
zip.keep-files
|
|
|
|
--------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Keep the actual files after writing them to a ZIP archive.
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
2019-07-28 18:13:18 +02:00
|
|
|
zip.mode
|
|
|
|
--------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
Default
|
|
|
|
``"default"``
|
|
|
|
Description
|
|
|
|
* ``"default"``: Write the central directory file header
|
|
|
|
once after everything is done or an exception is raised.
|
2019-07-28 18:13:18 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* ``"safe"``: Update the central directory file header
|
|
|
|
each time a file is stored in a ZIP archive.
|
2019-07-28 18:13:18 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
This greatly reduces the chance a ZIP archive gets corrupted in
|
|
|
|
case the Python interpreter gets shut down unexpectedly
|
|
|
|
(power outage, SIGKILL) but is also a lot slower.
|
2019-07-28 18:13:18 +02:00
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
Miscellaneous Options
|
|
|
|
=====================
|
|
|
|
|
|
|
|
|
2020-10-25 03:05:10 +01:00
|
|
|
extractor.modules
|
|
|
|
-----------------
|
|
|
|
Type
|
|
|
|
``list`` of ``strings``
|
|
|
|
Default
|
|
|
|
The ``modules`` list in
|
|
|
|
`extractor/__init__.py <../gallery_dl/extractor/__init__.py#L12>`__
|
|
|
|
Example
|
|
|
|
``["reddit", "danbooru", "mangadex"]``
|
|
|
|
Description
|
|
|
|
The list of modules to load when searching for a suitable
|
|
|
|
extractor class. Useful to reduce startup time and memory usage.
|
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
cache.file
|
|
|
|
----------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
|Path|_
|
|
|
|
Default
|
|
|
|
* (``%APPDATA%`` or ``"~"``) + ``"/gallery-dl/cache.sqlite3"`` on Windows
|
|
|
|
* (``$XDG_CACHE_HOME`` or ``"~/.cache"``) + ``"/gallery-dl/cache.sqlite3"`` on all other platforms
|
|
|
|
Description
|
|
|
|
Path of the SQLite3 database used to cache login sessions,
|
|
|
|
cookies and API tokens across `gallery-dl` invocations.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
Set this option to ``null`` or an invalid path to disable
|
|
|
|
this cache.
|
2018-03-16 11:49:49 +01:00
|
|
|
|
|
|
|
|
2019-12-08 21:21:00 +01:00
|
|
|
pyopenssl
|
|
|
|
---------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``bool``
|
|
|
|
Default
|
|
|
|
``false``
|
|
|
|
Description
|
|
|
|
Use `pyOpenSSL <https://www.pyopenssl.org/en/stable/>`__-backed
|
|
|
|
SSL-support.
|
2019-12-08 21:21:00 +01:00
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2017-09-09 17:31:42 +02:00
|
|
|
API Tokens & IDs
|
|
|
|
================
|
|
|
|
|
2017-10-12 23:37:28 +02:00
|
|
|
All configuration keys listed in this section have fully functional default
|
|
|
|
values embedded into *gallery-dl* itself, but if things unexpectedly break
|
|
|
|
or you want to use your own personal client credentials, you can follow these
|
|
|
|
instructions to get an alternative set of API tokens and IDs.
|
|
|
|
|
2018-05-26 11:26:50 +02:00
|
|
|
|
2017-09-09 17:31:42 +02:00
|
|
|
extractor.deviantart.client-id & .client-secret
|
|
|
|
-----------------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
How To
|
|
|
|
* login and visit DeviantArt's
|
|
|
|
`Applications & Keys <https://www.deviantart.com/developers/apps>`__
|
|
|
|
section
|
|
|
|
* click "Register Application"
|
|
|
|
* scroll to "OAuth2 Redirect URI Whitelist (Required)"
|
|
|
|
and enter "https://mikf.github.io/gallery-dl/oauth-redirect.html"
|
|
|
|
* scroll to the bottom and agree to the API License Agreement.
|
|
|
|
Submission Policy, and Terms of Service.
|
|
|
|
* click "Save"
|
|
|
|
* copy ``client_id`` and ``client_secret`` of your new
|
|
|
|
application and put them in your configuration file
|
|
|
|
as ``"client-id"`` and ``"client-secret"``
|
2020-10-19 21:57:26 +02:00
|
|
|
* clear your `cache <cache.file_>`__ to delete any remaining
|
2021-09-21 21:58:17 +02:00
|
|
|
``access-token`` entries. (``gallery-dl --clear-cache deviantart``)
|
2020-10-19 21:57:26 +02:00
|
|
|
* get a new `refresh-token <extractor.deviantart.refresh-token_>`__ for the
|
|
|
|
new ``client-id`` (``gallery-dl oauth:deviantart``)
|
2017-09-09 17:31:42 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.flickr.api-key & .api-secret
|
|
|
|
--------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
How To
|
|
|
|
* login and `Create an App <https://www.flickr.com/services/apps/create/apply/>`__
|
|
|
|
in Flickr's `App Garden <https://www.flickr.com/services/>`__
|
|
|
|
* click "APPLY FOR A NON-COMMERCIAL KEY"
|
|
|
|
* fill out the form with a random name and description
|
|
|
|
and click "SUBMIT"
|
|
|
|
* copy ``Key`` and ``Secret`` and put them in your configuration
|
|
|
|
file
|
2017-09-09 17:31:42 +02:00
|
|
|
|
|
|
|
|
2017-10-10 17:29:46 +02:00
|
|
|
extractor.reddit.client-id & .user-agent
|
|
|
|
----------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
How To
|
|
|
|
* login and visit the `apps <https://www.reddit.com/prefs/apps/>`__
|
|
|
|
section of your account's preferences
|
|
|
|
* click the "are you a developer? create an app..." button
|
|
|
|
* fill out the form, choose "installed app", preferably set
|
|
|
|
"http://localhost:6414/" as "redirect uri" and finally click
|
|
|
|
"create app"
|
|
|
|
* copy the client id (third line, under your application's name and
|
|
|
|
"installed app") and put it in your configuration file
|
|
|
|
* use "``Python:<application name>:v1.0 (by /u/<username>)``" as
|
|
|
|
user-agent and replace ``<application name>`` and ``<username>``
|
|
|
|
accordingly (see Reddit's
|
|
|
|
`API access rules <https://github.com/reddit/reddit/wiki/API>`__)
|
2017-09-09 17:31:42 +02:00
|
|
|
|
|
|
|
|
2018-05-26 11:26:50 +02:00
|
|
|
extractor.smugmug.api-key & .api-secret
|
|
|
|
---------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
How To
|
|
|
|
* login and `Apply for an API Key <https://api.smugmug.com/api/developer/apply>`__
|
|
|
|
* use a random name and description,
|
|
|
|
set "Type" to "Application", "Platform" to "All",
|
|
|
|
and "Use" to "Non-Commercial"
|
|
|
|
* fill out the two checkboxes at the bottom and click "Apply"
|
|
|
|
* copy ``API Key`` and ``API Secret``
|
|
|
|
and put them in your configuration file
|
2018-05-26 11:26:50 +02:00
|
|
|
|
|
|
|
|
|
|
|
extractor.tumblr.api-key & .api-secret
|
|
|
|
--------------------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``string``
|
|
|
|
How To
|
|
|
|
* login and visit Tumblr's
|
|
|
|
`Applications <https://www.tumblr.com/oauth/apps>`__ section
|
|
|
|
* click "Register application"
|
|
|
|
* fill out the form: use a random name and description, set
|
|
|
|
https://example.org/ as "Application Website" and "Default
|
|
|
|
callback URL"
|
|
|
|
* solve Google's "I'm not a robot" challenge and click "Register"
|
|
|
|
* click "Show secret key" (below "OAuth Consumer Key")
|
|
|
|
* copy your ``OAuth Consumer Key`` and ``Secret Key``
|
|
|
|
and put them in your configuration file
|
2017-11-22 23:09:08 +01:00
|
|
|
|
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
|
2018-05-27 16:48:54 +02:00
|
|
|
Custom Types
|
|
|
|
============
|
|
|
|
|
|
|
|
|
2019-07-16 23:08:27 +02:00
|
|
|
Date
|
|
|
|
----
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-09-14 17:40:05 +02:00
|
|
|
* ``string``
|
|
|
|
* ``integer``
|
2020-09-26 13:33:46 +02:00
|
|
|
Example
|
|
|
|
* ``"2019-01-01T00:00:00"``
|
|
|
|
* ``"2019"`` with ``"%Y"`` as `date-format`_
|
|
|
|
* ``1546297200``
|
|
|
|
Description
|
|
|
|
A |Date|_ value represents a specific point in time.
|
2019-07-16 23:08:27 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
* If given as ``string``, it is parsed according to `date-format`_.
|
|
|
|
* If given as ``integer``, it is interpreted as UTC timestamp.
|
2019-07-16 23:08:27 +02:00
|
|
|
|
|
|
|
|
2021-09-14 17:40:05 +02:00
|
|
|
Duration
|
|
|
|
--------
|
|
|
|
Type
|
|
|
|
* ``float``
|
|
|
|
* ``list`` with 2 ``floats``
|
2021-12-18 23:04:33 +01:00
|
|
|
* ``string``
|
2021-09-14 17:40:05 +02:00
|
|
|
Example
|
|
|
|
* ``2.85``
|
|
|
|
* ``[1.5, 3.0]``
|
2021-12-18 23:04:33 +01:00
|
|
|
* ``"2.85"``, ``"1.5-3.0"``
|
2021-09-14 17:40:05 +02:00
|
|
|
Description
|
|
|
|
A |Duration|_ represents a span of time in seconds.
|
|
|
|
|
|
|
|
* If given as a single ``float``, it will be used as that exact value.
|
|
|
|
* If given as a ``list`` with 2 floating-point numbers ``a`` & ``b`` ,
|
|
|
|
it will be randomly chosen with uniform distribution such that ``a <= N <=b``.
|
|
|
|
(see `random.uniform() <https://docs.python.org/3/library/random.html#random.uniform>`_)
|
2021-12-18 23:04:33 +01:00
|
|
|
* If given as a ``string``, it can either represent a single ``float``
|
|
|
|
value (``"2.85"``) or a range (``"1.5-3.0"``).
|
2021-09-14 17:40:05 +02:00
|
|
|
|
|
|
|
|
2018-05-27 16:48:54 +02:00
|
|
|
Path
|
|
|
|
----
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
2021-09-14 17:40:05 +02:00
|
|
|
* ``string``
|
|
|
|
* ``list`` of ``strings``
|
2020-09-26 13:33:46 +02:00
|
|
|
Example
|
|
|
|
* ``"file.ext"``
|
|
|
|
* ``"~/path/to/file.ext"``
|
|
|
|
* ``"$HOME/path/to/file.ext"``
|
|
|
|
* ``["$HOME", "path", "to", "file.ext"]``
|
|
|
|
Description
|
|
|
|
A |Path|_ is a ``string`` representing the location of a file
|
|
|
|
or directory.
|
|
|
|
|
|
|
|
Simple `tilde expansion <https://docs.python.org/3/library/os.path.html#os.path.expanduser>`__
|
|
|
|
and `environment variable expansion <https://docs.python.org/3/library/os.path.html#os.path.expandvars>`__
|
|
|
|
is supported.
|
|
|
|
|
|
|
|
In Windows environments, backslashes (``"\"``) can, in addition to
|
|
|
|
forward slashes (``"/"``), be used as path separators.
|
|
|
|
Because backslashes are JSON's escape character,
|
|
|
|
they themselves have to be escaped.
|
|
|
|
The path ``C:\path\to\file.ext`` has therefore to be written as
|
|
|
|
``"C:\\path\\to\\file.ext"`` if you want to use backslashes.
|
2018-05-27 16:48:54 +02:00
|
|
|
|
|
|
|
|
|
|
|
Logging Configuration
|
|
|
|
---------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Example
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
{
|
2020-10-19 21:57:26 +02:00
|
|
|
"format" : "{asctime} {name}: {message}",
|
2020-09-26 13:33:46 +02:00
|
|
|
"format-date": "%H:%M:%S",
|
2020-10-19 21:57:26 +02:00
|
|
|
"path" : "~/log.txt",
|
|
|
|
"encoding" : "ascii"
|
2020-09-26 13:33:46 +02:00
|
|
|
}
|
|
|
|
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
{
|
2020-10-19 21:57:26 +02:00
|
|
|
"level" : "debug",
|
2020-09-26 13:33:46 +02:00
|
|
|
"format": {
|
|
|
|
"debug" : "debug: {message}",
|
|
|
|
"info" : "[{name}] {message}",
|
|
|
|
"warning": "Warning: {message}",
|
|
|
|
"error" : "ERROR: {message}"
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
Description
|
|
|
|
Extended logging output configuration.
|
|
|
|
|
|
|
|
* format
|
|
|
|
* General format string for logging messages
|
|
|
|
or a dictionary with format strings for each loglevel.
|
|
|
|
|
|
|
|
In addition to the default
|
|
|
|
`LogRecord attributes <https://docs.python.org/3/library/logging.html#logrecord-attributes>`__,
|
|
|
|
it is also possible to access the current
|
|
|
|
`extractor <https://github.com/mikf/gallery-dl/blob/33fe67b594dbf8fb742464c95e0f6952bb60eb8a/gallery_dl/extractor/common.py#L23>`__,
|
|
|
|
`job <https://github.com/mikf/gallery-dl/blob/33fe67b594dbf8fb742464c95e0f6952bb60eb8a/gallery_dl/job.py#L18>`__,
|
|
|
|
`path <https://github.com/mikf/gallery-dl/blob/33fe67b594dbf8fb742464c95e0f6952bb60eb8a/gallery_dl/util.py#L702>`__,
|
|
|
|
and `keywords` objects and their attributes, for example
|
|
|
|
``"{extractor.url}"``, ``"{path.filename}"``, ``"{keywords.title}"``
|
|
|
|
* Default: ``"[{name}][{levelname}] {message}"``
|
|
|
|
* format-date
|
|
|
|
* Format string for ``{asctime}`` fields in logging messages
|
|
|
|
(see `strftime() directives <https://docs.python.org/3/library/time.html#time.strftime>`__)
|
|
|
|
* Default: ``"%Y-%m-%d %H:%M:%S"``
|
|
|
|
* level
|
|
|
|
* Minimum logging message level
|
|
|
|
(one of ``"debug"``, ``"info"``, ``"warning"``, ``"error"``, ``"exception"``)
|
|
|
|
* Default: ``"info"``
|
|
|
|
* path
|
|
|
|
* |Path|_ to the output file
|
|
|
|
* mode
|
|
|
|
* Mode in which the file is opened;
|
|
|
|
use ``"w"`` to truncate or ``"a"`` to append
|
|
|
|
(see `open() <https://docs.python.org/3/library/functions.html#open>`__)
|
|
|
|
* Default: ``"w"``
|
|
|
|
* encoding
|
|
|
|
* File encoding
|
|
|
|
* Default: ``"utf-8"``
|
|
|
|
|
2020-10-19 21:57:26 +02:00
|
|
|
Note: path, mode, and encoding are only applied when configuring
|
2020-09-26 13:33:46 +02:00
|
|
|
logging output to a file.
|
2018-05-27 16:48:54 +02:00
|
|
|
|
|
|
|
|
2018-06-16 15:43:24 +02:00
|
|
|
Postprocessor Configuration
|
|
|
|
---------------------------
|
2020-09-26 13:33:46 +02:00
|
|
|
Type
|
|
|
|
``object``
|
|
|
|
Example
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
{ "name": "mtime" }
|
|
|
|
|
2020-10-19 21:57:26 +02:00
|
|
|
.. code:: json
|
2020-09-26 13:33:46 +02:00
|
|
|
|
|
|
|
{
|
2020-10-19 21:57:26 +02:00
|
|
|
"name" : "zip",
|
2020-09-26 13:33:46 +02:00
|
|
|
"compression": "store",
|
2020-10-19 21:57:26 +02:00
|
|
|
"extension" : "cbz",
|
2021-06-04 18:08:08 +02:00
|
|
|
"filter" : "extension not in ('zip', 'rar')",
|
2020-10-19 21:57:26 +02:00
|
|
|
"whitelist" : ["mangadex", "exhentai", "nhentai"]
|
2020-09-26 13:33:46 +02:00
|
|
|
}
|
|
|
|
Description
|
|
|
|
An ``object`` containing a ``"name"`` attribute specifying the
|
|
|
|
post-processor type, as well as any of its `options <Postprocessor Options_>`__.
|
2021-06-04 18:08:08 +02:00
|
|
|
|
|
|
|
It is possible to set a ``"filter"`` expression similar to
|
|
|
|
`image-filter <extractor.*.image-filter_>`_ to only run a post-processor
|
2021-06-08 02:06:19 +02:00
|
|
|
conditionally.
|
2021-06-04 18:08:08 +02:00
|
|
|
|
2020-09-26 13:33:46 +02:00
|
|
|
It is also possible set a ``"whitelist"`` or ``"blacklist"`` to
|
|
|
|
only enable or disable a post-processor for the specified
|
|
|
|
extractor categories.
|
|
|
|
|
|
|
|
The available post-processor types are
|
|
|
|
|
|
|
|
``classify``
|
|
|
|
Categorize files by filename extension
|
|
|
|
``compare``
|
|
|
|
| Compare versions of the same file and replace/enumerate them on mismatch
|
|
|
|
| (requires `downloader.*.part`_ = ``true`` and `extractor.*.skip`_ = ``false``)
|
|
|
|
``exec``
|
|
|
|
Execute external commands
|
|
|
|
``metadata``
|
|
|
|
Write metadata to separate files
|
|
|
|
``mtime``
|
|
|
|
Set file modification time according to its metadata
|
|
|
|
``ugoira``
|
|
|
|
Convert Pixiv Ugoira to WebM using `FFmpeg <https://www.ffmpeg.org/>`__
|
|
|
|
``zip``
|
|
|
|
Store files in a ZIP archive
|
2018-06-16 15:43:24 +02:00
|
|
|
|
|
|
|
|
2018-05-27 16:48:54 +02:00
|
|
|
|
2017-06-24 12:17:26 +02:00
|
|
|
.. |.netrc| replace:: ``.netrc``
|
2017-08-31 15:21:08 +02:00
|
|
|
.. |requests.request()| replace:: ``requests.request()``
|
|
|
|
.. |timeout| replace:: ``timeout``
|
|
|
|
.. |verify| replace:: ``verify``
|
2017-06-23 16:14:51 +02:00
|
|
|
.. |mature_content| replace:: ``mature_content``
|
|
|
|
.. |webbrowser.open()| replace:: ``webbrowser.open()``
|
2019-07-14 22:37:28 +02:00
|
|
|
.. |datetime| replace:: ``datetime``
|
2017-07-04 19:34:34 +02:00
|
|
|
.. |datetime.max| replace:: ``datetime.max``
|
2019-07-16 23:08:27 +02:00
|
|
|
.. |Date| replace:: ``Date``
|
2021-09-14 17:40:05 +02:00
|
|
|
.. |Duration| replace:: ``Duration``
|
2018-05-27 16:48:54 +02:00
|
|
|
.. |Path| replace:: ``Path``
|
2019-06-20 17:19:44 +02:00
|
|
|
.. |Last-Modified| replace:: ``Last-Modified``
|
2018-05-27 16:48:54 +02:00
|
|
|
.. |Logging Configuration| replace:: ``Logging Configuration``
|
2018-06-16 15:43:24 +02:00
|
|
|
.. |Postprocessor Configuration| replace:: ``Postprocessor Configuration``
|
2017-07-04 19:34:34 +02:00
|
|
|
.. |strptime| replace:: strftime() and strptime() Behavior
|
2020-10-19 21:57:26 +02:00
|
|
|
.. |postprocessors| replace:: ``postprocessors``
|
2017-07-04 19:34:34 +02:00
|
|
|
|
2018-03-16 11:49:49 +01:00
|
|
|
.. _base-directory: `extractor.*.base-directory`_
|
2019-07-16 23:08:27 +02:00
|
|
|
.. _date-format: `extractor.*.date-format`_
|
2021-03-19 16:24:23 +01:00
|
|
|
.. _deviantart.metadata: `extractor.deviantart.metadata`_
|
2020-10-19 21:57:26 +02:00
|
|
|
.. _postprocessors: `extractor.*.postprocessors`_
|
|
|
|
.. _download archive: `extractor.*.archive`_
|
2017-06-23 16:14:51 +02:00
|
|
|
|
2019-10-11 18:19:39 +02:00
|
|
|
.. _.netrc: https://stackoverflow.com/tags/.netrc/info
|
|
|
|
.. _Last-Modified: https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.29
|
|
|
|
.. _datetime: https://docs.python.org/3/library/datetime.html#datetime-objects
|
|
|
|
.. _datetime.max: https://docs.python.org/3/library/datetime.html#datetime.datetime.max
|
|
|
|
.. _strptime: https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior
|
|
|
|
.. _webbrowser.open(): https://docs.python.org/3/library/webbrowser.html
|
|
|
|
.. _mature_content: https://www.deviantart.com/developers/http/v1/20160316/object/deviation
|
|
|
|
.. _Authentication: https://github.com/mikf/gallery-dl#authentication
|
|
|
|
.. _OAuth: https://github.com/mikf/gallery-dl#oauth
|
2021-06-29 19:24:18 +02:00
|
|
|
.. _format string: https://github.com/mikf/gallery-dl/blob/master/docs/formatting.md
|
|
|
|
.. _format strings: https://github.com/mikf/gallery-dl/blob/master/docs/formatting.md
|
2019-10-11 18:19:39 +02:00
|
|
|
.. _youtube-dl: https://github.com/ytdl-org/youtube-dl
|
|
|
|
.. _requests.request(): https://requests.readthedocs.io/en/master/api/#requests.request
|
|
|
|
.. _timeout: https://requests.readthedocs.io/en/master/user/advanced/#timeouts
|
|
|
|
.. _verify: https://requests.readthedocs.io/en/master/user/advanced/#ssl-cert-verification
|
|
|
|
.. _`Requests' proxy documentation`: https://requests.readthedocs.io/en/master/user/advanced/#proxies
|