Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-25 12:12:34 +01:00
Code Issues Releases Wiki Activity
6,082 Commits 6 Branches 164 Tags 15 MiB
7d609b883e
Commit Graph

247 Commits

Author SHA1 Message Date
Mike Fährmann
6fdcfa941c
release version 1.20.5 2022-02-14 19:33:15 +01:00
Mike Fährmann
0f1e7ff319
[twitter] fix extraction (#2275) 2022-02-07 23:18:35 +01:00
Mike Fährmann
f351746483
release version 1.20.4 2022-02-06 22:02:36 +01:00
Mike Fährmann
e2be199124
[gelbooru] improve and fix pagination (#2230, #2232) 
Use 'id:<POSTID' as a tag instead of going through pages with 'pid'.

Something similar was already implemented in 93cef784,
but that got broken again in 3085aac4.
 2022-01-27 17:44:47 +01:00
Mike Fährmann
806badbeec
release version 1.20.3 2022-01-26 01:10:44 +01:00
Mike Fährmann
2bf554a896
[twitter] fix several errors (#2212, #2216, #2225) 
- fix Tweets with deleted quotes
- fix suspended Tweets without 'legacy' entry
- fix unified_cards without 'type'
 2022-01-25 16:13:22 +01:00
Mike Fährmann
fbd17547f5
release version 1.20.2 2022-01-24 18:24:50 +01:00
Mike Fährmann
4acc31bd9f
[newgrounds] set suitabilities filter before starting a search 2022-01-11 23:50:29 +01:00
Mike Fährmann
58a7921b5c
release version 1.20.1 2022-01-08 23:25:59 +01:00
Mike Fährmann
3b7c7daa76
improve UNC path handling (#2126) 
always call 'abspath()' on the directory path to handle cases when the
current working directory is UNC and 'base-directory' is relative.
 2021-12-30 22:22:19 +01:00
Mike Fährmann
47eae4c393
release version 1.20.0 2021-12-29 22:59:14 +01:00
Mike Fährmann
da14b3fe9f
[output] write download progress indicator to stderr 2021-11-29 04:41:17 +01:00
Mike Fährmann
604d5b8bb2
release version 1.19.3 2021-11-27 23:13:35 +01:00
Mike Fährmann
efa178cc91
[ytdl] implement parsing ytdl command-line options (#1680) 
- adds 'config-file' and 'cmdline-args' options
  for both ytdl downloader and extractor
- create 'ytdl' helper module, which combines YoutubeDL creation
  and option parsing.
- most likely a buggy mess due to incompatibilities between the
  original youtube-dl and yt-dlp.
 2021-11-07 02:44:11 +01:00
Mike Fährmann
a881305357
release version 1.19.2 2021-11-05 20:47:45 +01:00
Mike Fährmann
e436a2607b
[gfycat] consistent 'userName' values for 'user' downloads (#1962) 
by using the name from the input URL and not relying on possibly faulty
or incomplete API results.

'userData[username]', if available, will still have the original name.
 2021-10-26 19:15:30 +02:00
Mike Fährmann
ba9579c504
release version 1.19.1 2021-10-24 21:48:25 +02:00
Mike Fährmann
df8050b81d
[postprocessor:compare] add 'equal' option (#1592) 
Move functionality from cdd72e14 to its own option,
where it can be used with any 'action'
 2021-10-05 03:03:37 +02:00
Mike Fährmann
f8410203ef
release version 1.19.0 2021-10-01 22:02:07 +02:00
Mike Fährmann
d79bcb6236
allow extractors to register a 'finalize()' method 2021-09-07 21:15:30 +02:00
Mike Fährmann
9158853ff7
release version 1.18.4 2021-09-04 21:09:56 +02:00
Mike Fährmann
9cb5ea5eda
update default User-Agent headers 2021-08-14 04:01:41 +02:00
Mike Fährmann
1e0f59d902
release version 1.18.3 2021-08-13 21:20:36 +02:00
Mike Fährmann
2792ed6e4b
implement 'util.format_value()' 2021-07-26 02:11:22 +02:00
Mike Fährmann
552032d4fb
release version 1.18.2 2021-07-23 17:39:19 +02:00
Mike Fährmann
317ecc8180
use HTML tables in docs/formatting.md 2021-07-05 23:29:03 +02:00
Mike Fährmann
5f1b13d1a5
release version 1.18.1 2021-07-04 22:37:19 +02:00
Mike Fährmann
fd00d47116
implement conditional directories (#1394) 
They work the same way as conditional filenames (84d2e640), e.g.

"directory": {
    "score >= 20": ["high score"],
    "score >= 5" : ["mid score"],
    ""           : ["{category}", "default"]
}
 2021-06-20 20:09:35 +02:00
Mike Fährmann
ee1064a2b2
release version 1.18.0 2021-06-19 21:26:42 +02:00
Mike Fährmann
78f89d2e61
[idolcomplex] fix pagination (closes #1594) 2021-06-02 15:32:46 +02:00
Mike Fährmann
f14a36cfc9
release version 1.17.5 2021-05-30 19:24:00 +02:00
Mike Fährmann
c693db5b1a
add '"skip": "terminate"' option 
Stops not only the current extractor/job,
but all parent extractors/jobs as well.
 2021-05-12 02:22:28 +02:00
Mike Fährmann
4835888acc
release version 1.17.4 2021-05-07 21:37:48 +02:00
Mike Fährmann
e5123f56c9
fix crash when using --no-download with --ugoira-conv (#1507) 2021-04-26 23:35:44 +02:00
Mike Fährmann
07b6661a87
release version 1.17.3 2021-04-25 21:23:26 +02:00
Mike Fährmann
37940193a6
build executables with SOCKS proxy support (closes #1424) 2021-04-07 21:50:03 +02:00
Mike Fährmann
583bee7725
release version 1.17.2 2021-04-02 21:16:44 +02:00
Mike Fährmann
dc23cfd684
[deviantart] use fallback for /intermediary/ URLs 
instead of checking availability with HEAD requests
 2021-03-20 00:10:53 +01:00
Mike Fährmann
15daa62842
release version 1.17.1 2021-03-19 19:14:04 +01:00
Mike Fährmann
94faf8c85a
add type check before applying 'browser' option (fixes #1358) 2021-03-06 18:15:32 +01:00
Mike Fährmann
5cf593a00a
release version 1.17.0 2021-03-05 18:55:36 +01:00
Mike Fährmann
6e40585fb1
release version 1.16.5 2021-02-14 16:46:59 +01:00
Mike Fährmann
477ed010c1
release version 1.16.4 2021-01-23 17:34:59 +01:00
Mike Fährmann
780b6adb91
rename 'generate_csrf_token()' to just 'generate_token()' 
and add a 'size' argument
 2021-01-11 22:12:40 +01:00
Mike Fährmann
f277e48c77
release version 1.16.3 2021-01-10 00:31:52 +01:00
Mike Fährmann
0fdaea00a3
[postprocessor:metadata] sanitize filenames 2021-01-10 00:13:20 +01:00
Mike Fährmann
32fcc61b84
release version 1.16.2 2021-01-09 18:24:51 +01:00
Mike Fährmann
b656b829db
[twitter] fix login with username & password 
It is no longer possible to get an 'authenticity_token' from Twitter's
Javascript-free login form, which got disabled few days ago.

Generating a random 16 byte hex string client-side and sending that as
a cookie alongside the regular login form works just as well.
 2020-12-28 16:10:19 +01:00
Mike Fährmann
d1903589a5
release version 1.16.1 2020-12-27 18:28:33 +01:00
Mike Fährmann
8d2e4e5f13
[booru] improve error handling 
e.g. for posts without a valid 'file_url' (#1176)
 2020-12-17 01:16:45 +01:00
Mike Fährmann
1f9121fecb
release version 1.16.0 2020-12-12 23:08:25 +01:00
Mike Fährmann
06af57e84a
update CHANGELOG and README for 1.15.4 2020-11-28 00:09:34 +01:00
Mike Fährmann
9b1bd09454
change 'extension-map' default 
Replace all JPEG filename extensions with 'jpg'.
 2020-11-14 22:40:31 +01:00
Mike Fährmann
e5438b8a29
release version 1.15.3 2020-11-13 15:50:05 +01:00
Mike Fährmann
5bcf28de93
add a 'extractor.modules' option 2020-10-25 03:05:10 +01:00
Mike Fährmann
18213dc5ba
release version 1.15.2 2020-10-24 18:57:29 +02:00
Mike Fährmann
5b844a72b7
[newgrounds] handle embeds without scheme (#1033) 2020-10-15 15:13:54 +02:00
Mike Fährmann
35056a07d1
release version 1.15.1 2020-10-11 18:44:46 +02:00
Mike Fährmann
aeb0d32333
[twitter] improve twitpic extraction (fixes #1019) 
- ignore twitpic.com/photos/… URLs
- ignore empty image URLs
 2020-09-22 22:22:35 +02:00
Mike Fährmann
2184ec5d78
release version 1.15.0 2020-09-20 22:06:46 +02:00
Mike Fährmann
fd0685d9b5
[postprocessor:zip] defer zip file creation (fixes #968) 
don't try to create zip files on postprocessor construction,
wait until directory creation during file download,
 2020-08-31 21:53:18 +02:00
Mike Fährmann
33fe67b594
release version 1.14.5 2020-08-30 21:20:26 +02:00
Mike Fährmann
b2009ea39e
[aryion] update folder mime type list (fixes #945) 2020-08-16 22:30:15 +02:00
Mike Fährmann
688bd046fc
release version 1.14.4 2020-08-15 21:29:02 +02:00
Mike Fährmann
87202b8d74
[inkbunny] add 'user' and 'post' extractors (#283) 2020-07-22 22:21:30 +02:00
Mike Fährmann
b62ea72533
release version 1.14.3 2020-07-18 18:34:36 +02:00
Mike Fährmann
821524e4ee
[subscribestar] add 'user' and 'post' extractors (#852) 2020-07-03 21:08:47 +02:00
Mike Fährmann
4f16fd37fe
release version 1.14.2 2020-06-27 20:46:01 +02:00
Mike Fährmann
bb882b8cdb
improve output of '-K' for parent extractors (#825) 2020-06-14 21:39:21 +02:00
Mike Fährmann
6db7ed90cb
release version 1.14.1 2020-06-12 20:12:09 +02:00
Mike Fährmann
f1ef9082bb
release version 1.14.0 2020-05-31 20:39:36 +02:00
Mike Fährmann
0bf0146bfe
[reddit] don't send OAuth headers for file downloads (fixes #729) 2020-05-08 21:42:52 +02:00
Mike Fährmann
ba42ec321c
release version 1.13.6 2020-05-02 22:18:56 +02:00
Mike Fährmann
c56a751dae
[newgrounds] fix URLs produced by 'followng' extractors (#684) 2020-04-28 21:33:37 +02:00
Mike Fährmann
baa877d853
release version 1.13.5 2020-04-27 21:22:14 +02:00
Mike Fährmann
08674a91f3
[patreon] fix hash extraction from download URLs (closes #693) 
The old method was assuming every URL path ends with '/1'. For URLs
where this is not the case, the segment containing the post ID was
used as file hash.
 2020-04-15 23:28:57 +02:00
Mike Fährmann
a31c1aae72
release version 1.13.4 2020-04-12 21:24:52 +02:00
Mike Fährmann
fe96f99e4b
[hentainexus] reduce line length (flake8) & update test 2020-03-31 22:08:43 +02:00
Mike Fährmann
3ed72f82dc
release version 1.13.3 2020-03-28 22:03:33 +01:00
Mike Fährmann
90d15e3682
[instagram] use 'itertools.chain()' 2020-03-17 15:52:44 +01:00
Mike Fährmann
e0b0e8d62a
release version 1.13.2 2020-03-14 20:19:36 +01:00
Mike Fährmann
115fd2c6f2
"fix" incomplete MIME types (#632) 
e-/exhentai's original image downloads currently send
incomplete/invalid Content-Type headers, "jpg" instead
of "image/jpg" etc, since the last update.
(https://forums.e-hentai.org/index.php?showtopic=236113)

This change prepends any Content-Type value missing a
media type specification with "image/", transforming it
into a valid MIME type.

(A global solution to a local problem, but it shouldn't
 cause any issues anywhere else)
 2020-03-03 21:21:57 +01:00
Mike Fährmann
72122eb9b3
release version 1.13.1 2020-03-01 21:17:29 +01:00
Mike Fährmann
5cdf1b1319
fix --verbose/--quiet 
caused by 383795b
 2020-02-17 22:29:51 +01:00
Mike Fährmann
78e8d33c97
release version 1.13.0 2020-02-16 22:40:36 +01:00
Mike Fährmann
c1a6862863
implement functions to load/save cookies.txt files (closes #586) 
The methods of the standard libraries' MozillaCookieJar have
several shortcomings (#HttpOnly_ cookies, 0 expiration timestamps, etc.)
and require construction of an ultimately pointless CookieJar object.
 2020-01-21 21:59:36 +01:00
Mike Fährmann
5d73b7f29c
release version 1.12.3 2020-01-19 23:31:28 +01:00
Mike Fährmann
b9cbf932b4
[pixiv] update URL patterns (fixes #568) 
Pixiv now uses new URLs for
- user profiles and illustration listings:
  - https://www.pixiv.net/en/users/<ID>
  - https://www.pixiv.net/en/users/<ID>/artworks
- bookmarks:
  - https://www.pixiv.net/en/users/<ID>/bookmarks/artworks
 2020-01-10 13:38:09 +01:00
Mike Fährmann
9d369de592
release version 1.12.2 2020-01-05 21:47:13 +01:00
Mike Fährmann
bf3df3d0b0
[directlink] send Referer headers (closes #536) 2019-12-25 17:17:07 +01:00
Mike Fährmann
83909ab5d4
release version 1.12.1 2019-12-22 22:26:33 +01:00
Mike Fährmann
bbbafc1c24
[downloader:http] catch both possible SSLException instances 
With pyOpenSSL installed, but disabled, the SSLError exception
would be set to the one from pyOpenSSL, which could never get raised.

This commit solves this problem by catching both, the native SSLError
exception as well as the one from pyOpenSSL (if available.1)
 2019-12-09 20:34:10 +01:00
Mike Fährmann
f451be48c3
release version 1.12.0 2019-12-08 22:47:13 +01:00
Mike Fährmann
da6789b2b0
disable unique archive id checks for some tests 
- same image twice in a livedoor blog post
- unreliable results for related pinterest items
 2019-11-10 17:04:51 +01:00
Mike Fährmann
67e54ed8ea
release version 1.11.1 2019-11-09 00:52:06 +01:00
Mike Fährmann
ce98a86c0e
fix data file inclusion in source distributions 2019-11-09 00:47:13 +01:00
Mike Fährmann
6c86fbfe2a
release version 1.11.0 2019-11-08 22:34:56 +01:00
Mike Fährmann
6abe5f5bbb
[patreon] fix pagination (#444) 
The Patreon-provided URLs for the next set of posts aren't
always complete, i.e. they can be missing their scheme and
the subsequent double slash: "www.patreon.com/…"
 2019-10-12 22:30:51 +02:00
Mike Fährmann
ff1e4a86aa
release version 1.10.6 2019-10-11 20:15:56 +02:00
Mike Fährmann
8af59a4bba
fix & update docs 
- update Requests links
- add example for --exec
- set '-dev' version
 2019-10-11 18:36:25 +02:00