Mike Fährmann
413b77757b
implement 'contains()' ( #2446 )
...
and add it to globals() in compiled expressions for --filter etc
2022-03-30 16:18:33 +02:00
Mike Fährmann
ecee315bbf
[mangasee] unescape manga names ( fixes #2454 )
2022-03-30 16:18:18 +02:00
loragja
7e545a3ae9
[gofile] add gofile.io extractor ( #2364 )
...
* Add gofile extractor
* add gofile extractor to module list
* add support for tiny monitors and ancient python versions
* seriously, f-strings are not *that* new...
* i love flake8 :)
* add 'api-token' and 'recursive' options
* add tests
2022-03-29 17:31:57 +02:00
Layerex
625f4d4cc4
[telegraph] Add telegra.ph extractor ( #2312 )
2022-03-28 19:18:13 +02:00
Mike Fährmann
48cc4853be
[skeb] refactor 'sent-requests' and add tests
2022-03-28 11:26:24 +02:00
Mike Fährmann
1aafa1d12d
add '--ugoira-conv-copy' command-line option ( #1550 )
2022-03-27 15:31:24 +02:00
Mike Fährmann
a164f4b91e
[postprocessor:ugoira] put '=' before mkvmerge input filename
2022-03-27 14:46:03 +02:00
Mike Fährmann
5c2d953766
[postprocessor:ugoira] auto-select demuxer by default
2022-03-26 21:10:59 +01:00
Mike Fährmann
e45792a5c4
[postprocessor:ugoira] insert extra frame
...
into files generated with the 'image2' demuxer to compensate for the
last frame not being shown for as long as it should.
This only happens for ugoira with non-uniform delays between frames and
only when 'repeat-last-frame' is enabled.
2022-03-26 18:04:12 +01:00
Mike Fährmann
37d584a9b2
[hitomi] update metadata extraction ( fixes #2444 )
...
remove 'hitomi.metadata' option, as it is no longer necessary
to make additional HTTP requests to fetch all metadata.
2022-03-26 12:46:18 +01:00
Mike Fährmann
e718dd7b32
[postprocessor:ugoira] support setting timecodes with mkvmerge
...
by selecting the "mkvmerge" demuxer
(#1550 )
2022-03-26 12:41:55 +01:00
Mike Fährmann
71bba774da
respect 'output.private' in '-K/--list-keywords' output
2022-03-25 22:19:37 +01:00
Mike Fährmann
b03ca7f10c
[aryion] provide correct 'date' independent of dst
2022-03-24 22:57:18 +01:00
Mike Fährmann
ba69fb669d
[kemonoparty] add 'duplicates' option ( closes #2440 )
2022-03-24 11:58:38 +01:00
Mike Fährmann
e7b30866d0
[postprocessor:mtime] fix timestamps from datetime objects ( #2307 )
...
'datetime.timestamp()', which got used to convert datetime objects to
POSIX timestamps, assumes naive datetimes represent LOCAL time, while
datetimes in 'date' metadata fields represent UTC time.
Ref: https://docs.python.org/3/library/datetime.html#datetime.datetime.timestamp
> Naive datetime instances are assumed to represent local time
> you can obtain the POSIX timestamp by … calculating the timestamp directly
2022-03-23 23:05:14 +01:00
Mike Fährmann
29db716a63
implement 'datetime_to_timestamp()'
...
and rename 'to_timestamp()'
to the more descriptive 'datetime_to_timestamp_string()'
2022-03-23 22:36:01 +01:00
Mike Fährmann
c0c1277c5f
[downloader:http] support sending POST data ( #2433 )
...
by setting the '_http_data' metadata field for a file
needed in addition to be3492776b
2022-03-23 21:48:38 +01:00
Mike Fährmann
9313d4dc10
[pinterest] do not force 'm3u8_native' for video downloads ( #2436 )
2022-03-21 10:11:51 +01:00
Mike Fährmann
42f2fd2ed7
[twibooru] fix posts without 'name' ( fixes #2434 )
2022-03-21 10:08:37 +01:00
Mike Fährmann
40ce505420
[postprocessor:ugoira] add 'mtime' option ( #2307 )
2022-03-21 10:02:29 +01:00
Mike Fährmann
9bd27b1b8d
[postprocessor:metadata] implement archive options ( #2421 )
...
'archive', 'archive-format', and 'archive-prefix'
2022-03-20 21:16:46 +01:00
Mike Fährmann
be3492776b
[downloader:http] support using a different method than GET ( #2433 )
...
by setting the '_http_method' metadata field for a file
2022-03-20 10:09:05 +01:00
chinggg
6f1d5e8ab9
[unsplash] replace dash with space in search API queries ( #2429 )
2022-03-19 16:00:05 +01:00
Mike Fährmann
f8230dde43
[instagram] add 'previews' option ( #2135 )
2022-03-19 15:26:40 +01:00
Mike Fährmann
8295bc6d97
fix loading/storing cookies without domain
2022-03-19 15:14:55 +01:00
Mike Fährmann
500a479026
fix a third(!) bug in _check_cookies() ( #2372 )
...
turns out tests are worthless if you get em wrong ...
2022-03-18 19:52:37 +01:00
Mike Fährmann
c4cc387f7d
[furaffinity] fix search result pagination ( fixes #2402 )
2022-03-18 13:44:36 +01:00
Mike Fährmann
cf44aba333
[formatter] allow evaluating f-string literals
...
by starting a format string with '\fF'.
This was technically already possible with '\fE',
but this makes it a bit more convenient.
2022-03-18 13:31:01 +01:00
Mike Fährmann
d04e07d93b
build executables on windows-2019
...
windows-2016 gets removed on 2022.03.15
2022-03-14 18:06:36 +01:00
Mike Fährmann
3acfe683ea
release version 1.21.0
2022-03-14 15:42:43 +01:00
Mike Fährmann
281a5b3b28
[newgrounds] fix video descriptions ( #2328 )
2022-03-14 08:38:20 +01:00
Mike Fährmann
b1b15d6cef
[imagebam] add support for /view/ paths ( closes #2378 )
2022-03-14 08:38:20 +01:00
Mike Fährmann
e64c2b85d0
[fantia] apply patch ( #2381 )
...
from @thatfuckingbird with small adjustments
https://github.com/mikf/gallery-dl/issues/2381#issuecomment-1063208696
2022-03-11 18:02:31 +01:00
Mike Fährmann
f31ab0d2ec
[fanbox] fetch data for each individual post ( fixes #2388 )
...
Posts from 'https://api.fanbox.cc/post.listCreator '
do not contain a 'body' with all images anymore.
https://github.com/mikf/gallery-dl/pull/1459#discussion_r614322881
2022-03-11 17:36:05 +01:00
Mike Fährmann
fc277fa45f
[seiga] require authentication with 'user_session' cookie ( #2372 )
...
Login with username & password would now require entering a 2FA token.
see also 7b009cc893
2022-03-11 02:10:15 +01:00
Mike Fährmann
47cf05c4ab
refactor proxy handling code ( #2357 )
...
- allow gallery-dl proxy settings to overwrite environment proxies
- allow specifying different proxies for data extraction and download
- add 'downloader.proxy' option
- '-o extractor.proxy=–PROXY_URL -o downloader.proxy=null'
now has the same effect as youtube-dl's '--geo-verification-proxy'
2022-03-10 23:55:35 +01:00
Mike Fährmann
d50a1ec2cc
[subscribestar] unescape attachment URLs ( fixes #2370 )
2022-03-09 19:06:04 +01:00
Mike Fährmann
3ddc620ef6
[skeb] fix post extractor ( #2330 )
2022-03-09 18:45:07 +01:00
Orkun Koçyiğit
eb2bb7d998
[fantia] add 'num' enumeration index ( #2377 )
...
* Adding numerical ordering to fantia
* Fixed line to fit PEP8 line size limit
2022-03-08 22:06:41 +01:00
Mike Fährmann
fac8047899
[kemonoparty] limit default filename length ( #2373 )
2022-03-08 21:14:47 +01:00
Mike Fährmann
bfa5e61900
[patreon] add explicit 'image_large' file type ( #2257 )
...
to allow more control over when and if to download 'large_url' images
4fee3a0e52
2022-03-06 17:07:13 +01:00
Mike Fährmann
6ea3ff5173
[tumblr] notify users about registering an oauth application
...
if they hit the daily rate limit and are using default API credentials
2022-03-06 16:28:53 +01:00
Mike Fährmann
b5236656d5
[deviantart] notify users about registering an oauth application
...
if they get repeated 429 errors and are using default API credentials
2022-03-06 16:24:39 +01:00
Mike Fährmann
2aa47e8382
[twitter] handle Tweets with "softIntervention" entries
...
or other such things where the actual Tweet data is one level deeper
than usual
2022-03-03 02:06:54 +01:00
Mike Fährmann
64bbc7969d
[twitter] warn about age-restricted Tweets ( #2354 )
2022-03-03 02:03:27 +01:00
Mike Fährmann
e778be52bc
[twitter] update query hashes
2022-03-02 23:05:31 +01:00
Mike Fährmann
bddcec49f1
implement 'text.root_from_url()'
...
use domain from input URL for kemono
2022-03-01 03:09:57 +01:00
Mike Fährmann
92c492dc09
[kemonoparty] match beta.kemono.party URLs ( #2348 )
2022-03-01 03:02:30 +01:00
Mike Fährmann
4ea9157d51
[mangadex] fix chapters without 'translatedLanguage' ( #2352 )
2022-03-01 02:04:25 +01:00
Alice
f1cab23724
[skeb] add 'sent-requests' option ( #2322 ) ( #2330 )
...
* Update skeb.py
* Update configuration.rst
* flake8
2022-02-28 22:42:15 +01:00
dragobit
781fdfa212
[hentaicosplays] add Referer to headers ( #2317 )
2022-02-28 22:19:32 +01:00
Mike Fährmann
4385a34e05
[twitter] fix handling of 429 responses ( fixes #2339 )
...
Twitter doesn't return a valid JSON response for 429 errors anymore.
2022-02-28 16:42:55 +01:00
Mike Fährmann
5a50569360
[toyhouse] support 'art' listings ( #1546 , #2331 )
2022-02-27 16:22:50 +01:00
Mike Fährmann
1c79044433
[imagebam] set 'nsfw_inter' cookie ( fixes #2334 )
2022-02-27 16:12:28 +01:00
Mike Fährmann
d71c173150
[newgrounds] strip incomplete HTML tag from '_comment' ( #2328 )
2022-02-23 21:42:28 +01:00
Mike Fährmann
00fbc942e5
add 'timedelta' class to the global namespace in filter expressions
...
(https://github.com/mikf/gallery-dl/issues/2308#issuecomment-1046135924 )
2022-02-23 00:02:10 +01:00
Mike Fährmann
cf58048bd4
[newgrounds] add 'post_url' metadata field ( #2328 )
2022-02-23 00:00:23 +01:00
Mike Fährmann
59749555b1
[postprocessor:mtime] add 'event' option ( #2307 )
2022-02-22 23:27:40 +01:00
Mike Fährmann
7958995398
[postprocessor:metadata] implement 'mtime' option ( #2307 )
2022-02-22 23:02:13 +01:00
Mike Fährmann
7aa2e2cd84
[slideshare] fix extraction
2022-02-21 02:52:45 +01:00
Mike Fährmann
fdfdc1b614
[kissgoddess] add 'gallery' and 'model' extractors
...
(closes #1052 , #2304 )
2022-02-20 04:45:37 +01:00
Mike Fährmann
79a461a2c1
[mememuseum] add 'tag' and 'post' extractors ( closes #2264 )
2022-02-20 02:15:38 +01:00
Mike Fährmann
e5f6af6e32
[oauth:pixiv] add note about 'code' expiring in 30 seconds ( #2306 )
2022-02-19 23:47:30 +01:00
Mike Fährmann
bbc4190017
[bunkr] fix .mp4 downloads ( #2239 )
...
again ...
2022-02-19 03:55:14 +01:00
Mike Fährmann
254a5b26e0
[twibooru] add extractors for searches, galleries, and posts
...
(#2219 )
2022-02-18 23:43:57 +01:00
Mike Fährmann
9ebc20e290
[booru] call nameext_from_url() before update() and _prepare()
...
to be able to overwrite filename and extension in _prepare()
2022-02-18 00:37:59 +01:00
Mike Fährmann
4fee3a0e52
[patreon] download 'large_url' images if available ( #2257 )
2022-02-17 18:23:59 +01:00
Mike Fährmann
f5b2b9333f
fix another bug in _check:cookies ( #2160 )
...
regression introduced in ed317bfc
2022-02-16 22:58:57 +01:00
Mike Fährmann
c8414c9d95
build Linux executable in its own virtualenv
2022-02-16 03:56:48 +01:00
Mike Fährmann
6fdcfa941c
release version 1.20.5
2022-02-14 19:33:15 +01:00
Ailothaen
203a04a4a3
[reddit] Support of standalone submissions on personal pages of users ( #2301 )
...
* [reddit] Support of submissions on personal pages of users
* [reddit] Design improvement for user submissions
* [reddit] Removed functions declared twice
2022-02-13 23:03:46 +01:00
Mike Fährmann
f5b2a9fcc6
add 'signals-ignore' option ( #2296 )
2022-02-13 23:00:50 +01:00
Mike Fährmann
806bc62379
[redgifs] support 'i.redgifs.com' URLs ( closes #2300 )
2022-02-13 23:00:50 +01:00
Mike Fährmann
655b2de5d9
[vk] fix infinite pagination loops ( fixes #2297 )
2022-02-13 23:00:50 +01:00
Mike Fährmann
cc5b1ce91a
[inkbunny] rename search parameters to their API equivalents
...
(fixes #2292 )
2022-02-13 23:00:49 +01:00
Mike Fährmann
ed317bfcf1
warn about cookies expiring in less than 24 hours
...
requires an expiration timestamp,
so this only works with cookies from a cookies.txt file
2022-02-13 23:00:49 +01:00
David Hoppenbrouwers
b17e2dcf93
[wallpapercave] add extractor for images ( #2205 )
2022-02-11 23:44:51 +01:00
v-delta
c661737f36
[Imgbox] Fix ImgboxExtractor ( #2281 )
2022-02-11 22:17:02 +01:00
Thomas Jost
a7de819aca
[lightroom] add Lightroom gallery extractor ( #2263 )
2022-02-11 21:30:59 +01:00
Mike Fährmann
563bd0ecf4
[danbooru] inherit from BaseExtractor
...
- merge danbooru and e621 code
- support booru.allthefallen.moe (closes #2283 )
- remove support for old e621 tag search URLs
2022-02-11 21:01:51 +01:00
Mike Fährmann
bc0e853d30
combine KeyError & IndexError to common base class LookupError
2022-02-11 00:42:49 +01:00
Mike Fährmann
f1c853c6ef
[furaffinity] add 'layout' option ( #2277 )
...
to be able to force gallery-dl to parse according to a specific layout
in case its auto-detect fails
2022-02-11 00:28:47 +01:00
Mike Fährmann
b4f8e15a1f
allow BaseExtractors to use the domain pf the matched URL
2022-02-10 01:38:50 +01:00
Mike Fährmann
c0fddcefc5
[downloader:ytdl] make ImportErrors non-fatal ( #2273 )
2022-02-08 19:30:29 +01:00
Mike Fährmann
a57a44f510
[kemonoparty] handle files without 'name' ( fixes #2276 )
2022-02-08 18:27:05 +01:00
Mike Fährmann
4efe56f419
[furaffinity] improve new/old layout detection ( fixes #2277 )
2022-02-08 18:10:52 +01:00
Mike Fährmann
0f1e7ff319
[twitter] fix extraction ( #2275 )
2022-02-07 23:18:35 +01:00
Mike Fährmann
f351746483
release version 1.20.4
2022-02-06 22:02:36 +01:00
Mike Fährmann
dee0d22561
update extractor test results
2022-02-06 21:39:24 +01:00
Mike Fährmann
d7b8e04b50
[kemonoparty] use 'Accept-Encoding: identity' for all downloads
...
(#2267 )
fixes issues when data send with 'Content-Encoding: gzip' or other
encodings is larger than the actual file
2022-02-05 18:06:58 +01:00
enormous-muscles
55326377d8
Add Kohlchan extractor ( #2251 )
2022-02-04 23:22:17 +01:00
Mike Fährmann
cc7dce5755
[sexcom] add 'pins' extractor ( closes #2265 )
2022-02-04 20:55:00 +01:00
Mike Fährmann
02e18f56be
[e621] add 'favorite' extractor ( closes #2250 )
2022-02-04 20:54:48 +01:00
Mike Fährmann
70e6e1549e
[twitter] provide fallback URLs for card images
...
f2e8aedd74 (commitcomment-64057751)
2022-02-03 23:43:18 +01:00
Mike Fährmann
86fa412b47
[hitomi] add 'format' option ( #2260 )
...
default is 'webp' since downloading original files is no longer allowed
2022-02-03 23:32:19 +01:00
Mike Fährmann
492436f936
[twitter] add 'warnings' option ( #2258 )
...
disable reporting any non-fatal errors by default
2022-02-02 18:37:19 +01:00
Mike Fährmann
a5163e4c70
[twitter] restore 'logout' functionality ( #1719 )
2022-02-02 18:21:15 +01:00
Mike Fährmann
f58364f6a8
update Firefox cipher list
2022-02-01 02:33:01 +01:00
Mike Fährmann
7e6981dda6
rename 'disabletls12' to 'tls12'
...
and let config options override any default settings
2022-02-01 01:37:03 +01:00
Mike Fährmann
bb3e182562
overhaul session initialization
...
- share adapter & connection pool across sessions with the same
ssl options, ssl ciphers, and source address
- simplify browser emulation to just a list of headers and ciphers
2022-01-31 23:12:08 +01:00
