Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-25 20:22:36 +01:00
Code Issues Releases Wiki Activity
5,310 Commits 6 Branches 164 Tags 15 MiB
8a63801311
Commit Graph

4879 Commits

Author SHA1 Message Date
Mike Fährmann
d0effcae20
[kemonoparty] add 'revision_index' metadata field (#4727) 2023-10-26 22:26:38 +02:00
Mike Fährmann
3bbaa875f1
[kemonoparty] fix parsing of non-standard 'dates' (#4676) 2023-10-26 21:50:18 +02:00
Mike Fährmann
75dec71253
[idolcomplex] disable Referer headers by default (#4726) 2023-10-26 18:02:31 +02:00
Mike Fährmann
a09df34bcf
merge #4714: [4archive] add 'thread' and 'board' extractors 
(#1262, #2418, #4400, #4710)
 2023-10-25 20:12:07 +02:00
enduser420
acb713b95a [4archive] update 2023-10-25 23:08:45 +05:30
Mike Fährmann
6766877524
merge #4693: [reddit] support Reddit Mobile share links 2023-10-25 17:54:32 +02:00
Mike Fährmann
1042278bec
[misskey] support 'misskey.design' (#4713) 2023-10-25 17:47:03 +02:00
Mike Fährmann
12a800ce21
[patreon] improve 'campaign_id' handling (#4699, #4715) 
- add ways to directly specify a 'campaign_id'
  - 'campaign-id' config option
  - 'c' or 'campaign_id' URL query parameter
- more descriptive error messages
- show 'campaign_id' value in debug log
 2023-10-25 17:18:06 +02:00
Mike Fährmann
31dbbffc0b
[twitter] cache 'user_by_…' results (#4719) 2023-10-25 16:45:27 +02:00
enduser420
c0714d5585 [4archive] add 'thread' and 'board' extractors 2023-10-24 23:05:28 +05:30
inty
b68aad3dab [reddit] implement Reddit Mobile share links 2023-10-22 10:38:05 +00:00
Mike Fährmann
95a74be2a5
release version 1.26.1 2023-10-21 16:32:52 +02:00
Mike Fährmann
de224ef3e4
[cookies] include exception in fallback warning 2023-10-21 13:23:45 +02:00
Mike Fährmann
7958ab1946
[newgrounds] support 'imageData' files (#4642) 2023-10-21 13:22:55 +02:00
Mike Fährmann
b52fd91ac6
[sankaku] support '/posts/' URLs (#4688) 2023-10-21 13:20:35 +02:00
Mike Fährmann
b8674776e9
[4chanarchives] disable Referer headers by default (#4686) 2023-10-20 17:39:50 +02:00
Mike Fährmann
78493f0870
[bunkr] fix '/d/' file URLs (#4685) 2023-10-20 17:30:23 +02:00
Mike Fährmann
b2c3db3e24
[bunkr] add extractor for media URLs (#4684) 2023-10-20 15:22:44 +02:00
Mike Fährmann
0d52b775cb
[kemonoparty] add 'revisions' option (#4498, #4597) 2023-10-20 15:20:49 +02:00
Mike Fährmann
6e830ffc9e
[kemonoparty] support post searches (#3385, #4057) 2023-10-19 23:06:06 +02:00
Mike Fährmann
aaf539009b
[kemonoparty] initial support for post revisions (#4498, #4597) 
- single revision
https://kemono.party/SERVICE/user/12345/post/12345/revision/12345

- all revisions
https://kemono.party/SERVICE/user/12345/post/12345/revisions
 2023-10-19 22:32:51 +02:00
Mike Fährmann
174191cb79
[kemonoparty] restore discord pagination (#4676) 2023-10-19 21:57:27 +02:00
Mike Fährmann
c9a976d8a6
[kemonoparty] various updates and fixes (#4676, #4681) 
- fix pagination
- fix 'date' metadata
- fix discord channel API endpoint
 2023-10-19 17:36:16 +02:00
Klion Xu
dc1c2139b1
fix line too long 2023-10-19 10:54:08 +08:00
Klion Xu
6b22af9720
[kemonoparty] update API endpoint (#4676) 2023-10-19 10:32:59 +08:00
Mike Fährmann
bfdc07632a
[deviantart] expand nested comment replies (#4653) 2023-10-17 19:40:53 +02:00
Mike Fährmann
390d14dbcc
[chevereto] support 'img.kiwi' and 'deltaporno.com' (#4664, #1381) 2023-10-16 18:14:30 +02:00
Mike Fährmann
727c8eec6c
merge #4667: [redgifs] fix 'niches' extraction (#4666) 2023-10-16 14:20:01 +02:00
Mike Fährmann
2911ed1240
[chevereto] add generic extractors (#4664) 
- support jpgfish
- support pixl.li / pixl.is (#3179, #4357)
 2023-10-16 14:15:39 +02:00
enduser420
db3363ac0b [redgifs] fix 'niches' extraction 2023-10-16 16:51:30 +05:30
Mike Fährmann
ade8347ead
[kemonoparty] fix DM dates 2023-10-15 19:54:28 +02:00
Mike Fährmann
6dfe200ae4
[kemonoparty] support discord URLs with channel IDs (#4662) 2023-10-15 19:45:22 +02:00
Mike Fährmann
c6a3892210
[imgbb] update username extraction (#4626) 2023-10-14 20:55:39 +02:00
Mike Fährmann
830a48bca4
[fantia] bad workaround for 833dce14 (#4627) 
at least this makes "filter": "content_num == content_count+1"
with "event": "post-after" work
 2023-10-13 23:04:47 +02:00
Mike Fährmann
13ce3a9acb
[warosu] fix extraction (#4634) 2023-10-13 23:03:39 +02:00
Mike Fährmann
c4c4e4d2f4
[newgrounds] improve 'art-image' extraction (#4642) 
- download files in original resolution
- replace .webp with extension of first file
 2023-10-13 20:10:55 +02:00
Mike Fährmann
833dce141f
[fantia] add 'content_count' and 'content_num' metadata fields (#4627) 2023-10-13 20:10:55 +02:00
Mike Fährmann
2d41702762
[deviantart] implement '"group": "skip"' (#4630) 2023-10-12 22:14:20 +02:00
Mike Fährmann
992e86ec94
[deviantart] disable 'jwt' (#4652) 2023-10-12 21:37:10 +02:00
Mike Fährmann
2974b8e3c8
[moebooru] add 'metadata' option (#4646) 
for extended 'pool' metadata
 2023-10-12 21:34:25 +02:00
Mike Fährmann
d194ea68a9
[cookies] open cookie databases in read-only mode 
bypasses the need to copy the entire database file
might solve #4195
 2023-10-12 21:16:42 +02:00
Mike Fährmann
8bb7243c10
[reddit] fix wrong previews (#4649) 
caused by a failed comment URL
using the main submission's preview as fallback

14af15bd
4963bb9b
 2023-10-10 18:33:01 +02:00
Mike Fährmann
08bdde5aac
merge #4619: [twitter] add 'sensitive' metadata field 2023-10-09 15:40:58 +02:00
Mike Fährmann
f3d6aaff13
[twitter] rename to 'sensitive'; use 'tget()' 2023-10-09 15:39:09 +02:00
Mike Fährmann
95c280c59b
[imgbb] update pagination end condition (#4626) 2023-10-09 15:33:25 +02:00
Mike Fährmann
2e350dd82a
merge #4626: [imgbb] fix 'user' extraction, add 'displayname' 2023-10-09 15:31:23 +02:00
Mike Fährmann
a2daa9befe
[imgbb] fix flake8 and username order 2023-10-09 15:29:17 +02:00
Mike Fährmann
67ba4ee842
[pp:exec] support more replacement fields for '--exec' (#4633) 
- {_directory}
- {_filename}
- {_path} (alias for {})
 2023-10-09 12:50:10 +02:00
Mike Fährmann
9a008523ac
[hentaifoundry] fix '.swf' file downloads (#4641) 2023-10-09 11:45:55 +02:00
Mike Fährmann
15f940819b
[newgrounds] support 'art-image' files (#4642) 2023-10-09 11:20:10 +02:00
Mike Fährmann
63db54b905
[patreon] update 'campaign_id' path (#4639) 2023-10-09 09:41:38 +02:00
HRXN
b846f56c3a
[imgbb] Fix user extraction, add displayname 2023-10-05 12:18:58 +02:00
Mike Fährmann
efaab4fbfa
[twitter] fix crash due to missing 'source' (#4620) 
regression caused by 06aaedde
 2023-10-04 23:01:04 +02:00
Nahida
3438a3098d
[twitter] add possible_sensitive field 2023-10-04 10:34:02 +08:00
Mike Fährmann
85357c1ef8
release version 1.26.0 2023-10-03 21:22:01 +02:00
Mike Fährmann
64dbc58a5a
[deviantart] update Eclipse API endpoints 2 (#4615) 2023-10-03 14:58:52 +02:00
Mike Fährmann
84fbbd96aa
[shimmie2] remove 'meme.museum' 2023-10-02 20:41:25 +02:00
Mike Fährmann
aa77fda78c
[instagram] better error message for invalid users (#4606) 2023-10-02 20:29:10 +02:00
Mike Fährmann
482f002e1f
[nsfwalbum] detect '/error.jpg' images (#4598) 2023-10-02 15:38:22 +02:00
Mike Fährmann
eb230e4b77
[nsfwalbum] disable Referer headers by default (#4598) 2023-10-01 13:55:17 +02:00
Mike Fährmann
b92645cd37
[bunkr] fix extraction (#4514, #4532, #4529, #4540) 2023-09-30 18:05:12 +02:00
Mike Fährmann
4477808d1c
fix symlink resolution in __main__.py 
adapt ytdl order
 2023-09-30 17:02:57 +02:00
Mike Fährmann
be17103e21
[regifs] support 'order' parameter for user URLs (#4583) 2023-09-28 21:37:58 +02:00
Mike Fährmann
7150c4c76c
fix imports when using the gallery_dl directory as argument (#4581) 2023-09-26 22:07:50 +02:00
HRXN
ec91eeb7ef
Update gallery_dl/extractor/reddit.py 
Co-authored-by: Mike Fährmann <mike_faehrmann@web.de>
 2023-09-26 14:00:29 +02:00
HRXN
66613c3a32
[reddit] ignore '/message/compose' URLs without www subdomain 2023-09-25 02:24:44 +02:00
Mike Fährmann
bb39779e1a
[deviantart] use private tokens for 'is_mature' posts (#4563) 2023-09-24 19:15:35 +02:00
Mike Fährmann
0c5d8b1505
[deviantart] re-add 'quality' option and 'intermediary' transform 2023-09-24 17:36:05 +02:00
Mike Fährmann
20d1683c47
[deviantart] fix JWT replacement (#293, #4548, #4563) 
And again, a huge thank you to @Ironchest337
for discovering this.
 2023-09-24 15:06:16 +02:00
Mike Fährmann
d7aac9fc06
[reddit] ignore '/message/compose' URLs (#4482) 2023-09-23 00:13:05 +02:00
Mike Fährmann
1e31fce37b
[pillowfort] support '/tagged/' URLs (#4570) 2023-09-23 00:11:01 +02:00
Mike Fährmann
1d2fd0b831
[pillowfort] extract 'b2_lg_url' media (#4570) 2023-09-23 00:05:26 +02:00
Mike Fährmann
50e2ebaff0
[danbooru] support 'donmai.moe' URLs 2023-09-22 20:58:38 +02:00
Mike Fährmann
82296b1f05
[reddit] add 'previews' option (#4322) 
another way to disable this new behavior
 2023-09-22 18:10:44 +02:00
Mike Fährmann
918ba4f847
[redgifs] match gfycat image URLs (#4558) 2023-09-22 18:02:55 +02:00
Mike Fährmann
2ad75bab05
[deviantart] add 'is_original' metadata field (#4559) 
true for 'downloadable' content, journals, flash animations,
and images without '/v1/' in their URL; false otherwise
 2023-09-22 17:37:11 +02:00
Mike Fährmann
9d8317d963
[deviantart] disable JWT updates (#4548, #4563) 
back to lowres images ...
 2023-09-22 17:13:05 +02:00
Mike Fährmann
8064663bda
[deviantart] update Eclipse API endpoints (#4553) 2023-09-22 16:30:16 +02:00
Mike Fährmann
2cd801232b
fix --range causing crashes (#4557) 
regression caused by a383eca7
 2023-09-22 16:28:20 +02:00
Mike Fährmann
3528974459
[instagram] handle exceptions due to missing media (#4555) 2023-09-21 20:58:31 +02:00
Mike Fährmann
4963bb9b30
[reddit] improve comment metadata v2 (#4482) 
provide main submission metadata at the top level
and comment metadata inside the 'comment' field,
i.e. the other way round than in 1710f1e9
 2023-09-20 22:12:40 +02:00
Mike Fährmann
7592c5e566
[patreon] fix extraction (#4547) 2023-09-19 13:18:47 +02:00
Mike Fährmann
0655ce1bae
[mangakakalot] update domain 
the old one still works, but it incurs a redirect
 2023-09-19 00:18:21 +02:00
Mike Fährmann
3ecb512722
send Referer headers by default 2023-09-19 00:02:04 +02:00
Mike Fährmann
cb4798f07a
[architizer] fix extraction (#4537) 2023-09-17 15:19:19 +02:00
Mike Fährmann
6178177227
[twitter] fix '_extractor' of following results (#4536) 
regression from 20ed647f
 2023-09-15 23:04:30 +02:00
Mike Fährmann
d13c82eff1
[kemonoparty] update favorites API endpoint (#4522) 2023-09-14 14:57:01 +02:00
Mike Fährmann
27ec653991
fix bug in test_init and update example URLs 2023-09-14 13:27:03 +02:00
Mike Fährmann
24a1d46391
[mastodon] support '/@USER/following' URLs 
Previously, only '/users/USER/following' got matched.
 2023-09-13 23:42:51 +02:00
Mike Fährmann
9f75713e00
[recursive] simplify 2023-09-13 21:47:20 +02:00
Mike Fährmann
899df8f237
remove another '*' for keyword-only arguments 
076380e0
 2023-09-13 21:38:31 +02:00
Mike Fährmann
6ae92da57e
Merge branch 'tests' 2023-09-13 21:34:28 +02:00
Mike Fährmann
32da3c70d3
[behance] handle videos without 'renditions' (#4523) 2023-09-12 22:00:04 +02:00
Mike Fährmann
ae5e049c4f
[redgifs] provide 'collection' metadata in a separate field (#4508) 
instead of overwriting the actual metadata
 2023-09-12 22:00:04 +02:00
Mike Fährmann
1710f1e983
[reddit] improve comment metadata (#4482) 
- provide 'date'
- make metadata of the main submission available as 'submission[…]'
 2023-09-12 22:00:04 +02:00
Mike Fährmann
4cdab8074e
update/fix --list-extractors 2023-09-11 17:32:59 +02:00
Mike Fährmann
a453335a9f
remove test results in extractor modules 
and add generic example URLs
 2023-09-11 16:30:55 +02:00
Mike Fährmann
1d2b5d0c60
update test comment positions 
always put them above the test they're referring to
 2023-09-06 18:16:09 +02:00
Mike Fährmann
93a7a89cf6
[formatter] use value of last alternative (#4492) 
fixes {fieldname|''} evaluating to the value of 'keywords-default'
instead of an empty string
 2023-09-05 17:53:27 +02:00
Mike Fährmann
f856987297
[subscribestar] fix preview detection (#4468) 
and show a warning message when posts contain previews
 2023-09-04 22:21:14 +02:00
Mike Fährmann
4c0b3d5dc5
[twitter] fix crash when 'sortIndex' is None (#4499) 2023-09-04 18:28:43 +02:00
Mike Fährmann
f2de70f254
[gfycat] remove module 2023-09-04 18:27:11 +02:00
Mike Fährmann
6eca1fab9b
[gelbooru_v02] support 'xbooru.com' (#4493) 2023-09-03 15:39:02 +02:00
Mike Fährmann
23bac772f2
[jpgfish] update domain to 'jpg1.su' (#4494) 2023-09-03 15:03:47 +02:00
Mike Fährmann
ceb59e176f
fix default Firefox user agent string 
note to self: do not trust some random third-party website
 2023-09-02 22:22:23 +02:00
Mike Fährmann
8259a5abe4
flake8 2023-09-02 13:03:11 +02:00
Mike Fährmann
0b6e5b8161
[hiperdex] send Referer headers during file downloads (#4490) 2023-09-02 12:59:30 +02:00
Mike Fährmann
a05821f8b4
[hiperdex] fix 'manga' metadata 
remove trailing ' Manga'
 2023-09-02 12:59:30 +02:00
Mike Fährmann
03d471a0d4
merge #4481: [pixiv] handle errors for private novels 2023-09-02 12:50:23 +02:00
Cisney-Gassai
8c477f7146
[bunkr] Fixes media-files-pizza.bunkr.ru failed to resolve. 2023-09-02 01:45:59 -06:00
johnsmith1202gmail
c7e31b2724
Update pixiv.py 2023-08-30 12:42:31 -07:00
johnsmith1202gmail
d3046561d4 continue downloading when the item is made private on pixiv 2023-08-30 18:21:06 +00:00
Mike Fährmann
28798594e8
[gfycat] update pagination logic (#4479) 
Some searches do not use cursor based pagination
but an offset based one.
 2023-08-30 18:54:15 +02:00
Mike Fährmann
a783c4f0fe
[pornhub] add 'gif' support (#4463) 2023-08-29 19:34:27 +02:00
Mike Fährmann
ba842981af
[imagevenue] fix extraction (#4473) 2023-08-29 12:06:30 +02:00
Mike Fährmann
7defb24e1e
[reddit] provide video previews if available (#4322) 2023-08-28 22:22:10 +02:00
Mike Fährmann
fd65f27ede
[reddit] fix 'preview.redd.it' URLs (#4470) 2023-08-28 17:17:03 +02:00
Mike Fährmann
06aaedded5
[twitter] extract 'source' metadata (#4459) 2023-08-28 16:31:57 +02:00
Mike Fährmann
14af15bd18
[reddit] download preview for 404ed imgur links (#4322) 
This is a pretty ugly hack as the internal infrastructure doesn't
really support switching from external URL to regular download in
case the former fails, but it kind of works ...

Can be disabled by setting 'reddit.fallback' to 'false'.
 2023-08-24 15:41:05 +02:00
Mike Fährmann
d12a5e440a
update docs/supportedsites 2023-08-24 15:01:26 +02:00
Mike Fährmann
3a27150479
[instagram] add 'following' extractor (#1848) 2023-08-23 23:58:12 +02:00
Mike Fährmann
e0829ff0fd
[twitter] add 'date_original' metadata for retweets (#4337, #4443) 2023-08-23 23:58:11 +02:00
Mike Fährmann
5ed245317d
[exhentai] add 'fav' option (#4409) 
The name 'favorite' is already taken as extractor subcategory
 2023-08-23 23:58:11 +02:00
Mike Fährmann
fd6b413f3c
[exhentai] fix 'domain' option (#4458) 
regression from a383eca7
 2023-08-23 23:58:04 +02:00
Mike Fährmann
fdfb22c91f
[instagram] fix video preview archive IDs (#2135, #4455) 2023-08-23 12:29:32 +02:00
Mike Fährmann
92f98e6f5e
'sys.exit' -> 'SystemExit' 2023-08-21 23:46:39 +02:00
Mike Fährmann
410f783a33
implement 'subconfigs' option (#4440) 2023-08-21 21:18:40 +02:00
Mike Fährmann
2b88ad19e9
[twitter] accept 'x.com' URLs (#4452) 2023-08-21 19:47:07 +02:00
Mike Fährmann
c1c73c0b0e
[pp:ugoira] add '"framerate": "uniform"' (#4421) 2023-08-21 19:43:47 +02:00
Mike Fährmann
2a3acd318a
[pp:ugoira] fix high frame rates (#4421) 
only return an output frame rate for non-uniform ugoira
when the frame delay gcd is >= 10, i.e. 100 fps
 2023-08-21 19:17:14 +02:00
Mike Fährmann
70bdf32a88
[pp:ugoira] extend 'ffmpeg-output' (#4421) 
- when setting this option to a string value,
  pass -hide-banner and -loglevel to FFmpeg
- change default to "error"
 2023-08-21 18:19:53 +02:00
Mike Fährmann
8dceea3384
[shimme2] move 'giantessbooru' back into shimmie module (#4373) 
Do the same thing as for 'realbooru' and override 'posts()'
insteadd of using a separate module.
 2023-08-18 15:25:28 +02:00
Mike Fährmann
6482f9453b
[behance] fix cookie usage (#4417) 2023-08-18 14:48:20 +02:00
Mike Fährmann
d34195b41d
[behance] fix and update 'user' extractor (#4417) 2023-08-17 16:06:35 +02:00
Mike Fährmann
4d3cf709da
[behance] add 'date' metadata field (#4417) 2023-08-17 15:33:47 +02:00
Mike Fährmann
c689cd9720
[behance] show error for mature content (#4417) 2023-08-17 15:31:37 +02:00
Mike Fährmann
33d912490f
merge #4419: [bunkr] Fix extracting wmv files 2023-08-17 15:28:29 +02:00
Mike Fährmann
01610a6e9e
merge #4412: [bunkr] fix media domain for cdn9 2023-08-17 15:18:49 +02:00
Mike Fährmann
b19d62263b
merge #4420: [issuu] fix extraction 2023-08-17 15:17:09 +02:00
ClosedPort22
6dc8be5e48
[issuu] fix extraction 2023-08-13 21:13:50 +08:00
Luc Ritchie
85a070b9e6
[bunkr] Fix extracting wmv files 2023-08-12 16:53:14 -04:00
Mike Fährmann
3f8ff692a7 [bunkr] fix media domain for cdn9 
Fixes #4386
 2023-08-11 18:14:47 -04:00
Mike Fährmann
d8b21a97bf
[formatter] use 'rpartition' for \fM format strings 
fixes using absolute module paths like C:\path\module.py on Windows
 2023-08-11 22:29:33 +02:00
Mike Fährmann
f9fb276e81
[postprocessor] add 'prepare-after' event (#4083) 2023-08-10 21:28:48 +02:00
Mike Fährmann
0ef1fcab20
[postprocessor] update 'finalize' events 
Add 'finalize-error' and 'finalize-success' events that trigger
depending on whether error(s) did or did not happen.

'finalize' itself now always triggers regardless of error status.
(was supposed to have the same behavior as the new 'finalize-success')
 2023-08-10 19:46:37 +02:00
Mike Fährmann
af4bdb62a7
merge #4403: [downloader:http] close connection when file already exists 2023-08-10 18:35:13 +02:00
Mike Fährmann
15275b3524
[postprocessor:ugoira] restore 'libx264-prevent-odd' (#4407) 
was accidentally removed in commit be9547a5
 2023-08-10 18:32:52 +02:00
Mike Fährmann
391a7d74c8
[giantessbooru] fix and move to separate module (#4373) 
too many differences to the other shimmie2 sites
 2023-08-09 18:36:56 +02:00
ClosedPort22
5448268d5c
[downloader:http] close connection when file already exists (#3748) 2023-08-08 23:35:43 +08:00
Mike Fährmann
3963dbe5e4
extend 'parent>child' categories 
continuation of ed21908f

allow for children to have an arbitrary distance from their parent,
e.g. reddit -> danbooru -> imgur:gallery -> imgur:album
would still be covered by 'reddit>imgur' or even 'danbooru>imgur'
 2023-08-07 23:22:12 +02:00
Mike Fährmann
089d1a4f67
[twitter] fix 'TweetWithVisibilityResults' (#4369) 2023-08-06 22:08:50 +02:00
Mike Fährmann
a4f7f7da17
add '_dump()' convenience method to Extractor 2023-08-06 17:03:09 +02:00
Mike Fährmann
df5c7ee03e
[deviantart] fix search (#4384) 
send correct usernames instead of 'u'
 2023-08-04 17:16:04 +02:00
Mike Fährmann
a60db454af
[sankaku] update/fix API headers 
'Referer' and 'Origin' were both empty
 2023-08-04 17:14:43 +02:00
Mike Fährmann
fb3f0453db
[twitter] improve error messages for single Tweets (#4369) 
also fixes '"quoted": false' not having any effect
 2023-08-03 22:02:07 +02:00
Mike Fährmann
541bff5a37
[pururin] fix extraction (#4375) 
- rename 'title_jp' to 'title_ja'
- change type of 'collection', 'convention',  and 'scanlator' to list
 2023-08-03 14:40:44 +02:00
Mike Fährmann
6a87c314af
[instagram] fix private posts with long shortcodes (#4362) 2023-08-03 13:51:03 +02:00
Mike Fährmann
f899fac4c5
[giantessbooru] fix extraction (#4373) 
This does not fix anything Cloudflare related,
just other things caused by a site update.
 2023-08-03 13:40:11 +02:00
Mike Fährmann
136283d402
[shimmie2] update base URL pattern 
to match new giantessbooru URLs
 2023-08-03 13:34:48 +02:00
Mike Fährmann
9d67655397
add "ascii+" as a special 'path-restrict' value (#4371) 2023-08-01 19:33:53 +02:00
Mike Fährmann
c79359eb3a
[fantia] improve metadata extraction (#4126) 
extract all metadata and URLs before starting to download
 2023-07-31 22:31:50 +02:00
Mike Fährmann
48ef062867
fix issues with 'Extractor.finalize()' 
- prevent crash in InstagramUserExtractor (#4359)
- call it at the end of every DownloadJob
- add it to tests
 2023-07-29 13:43:27 +02:00
Mike Fährmann
ed21908fda
initial support for child extractor options 
Using "parent-category>child-category" as extractor category in a config
file allows to set options for a child extractor when it was spawned by
that parent.

For example "reddit>gfycat" to set gfycat options for when it was found
in a reddit post.

{
    "extractor": {
        "gfycat": {
            "filename": "regular filename"
        },
        "reddit>gfycat": {
            "filename": "reddit-specific filename"
        }
    }
}

Note: This does currently not work for most imgur links due to how its
extractor hierarchy is structured.
 2023-07-28 17:07:25 +02:00
Mike Fährmann
255d08b79e
add test for 'Extractor.initialize()' (#4359) 2023-07-28 16:58:16 +02:00
Mike Fährmann
2bcf0a4c49
[instagram] fix initialization order (#4359) 
regression caused by the changes in a383eca7
 2023-07-28 14:25:37 +02:00
Mike Fährmann
7eab101144
[acidimg] fix extraction 
swap ' and " again (2e309a13)
and add a fallback in case this happens yet another time
 2023-07-28 14:23:11 +02:00
Mike Fährmann
62fce6a75f
[imagehosts] adjust variable names (#4358) 
prefix them with underscores to prevent a clash
with the new 'self.cookies' from d97b8c2f
 2023-07-28 14:18:47 +02:00
Mike Fährmann
e8299b459a
[moebooru] match search URLs with empty 'tags' (#4354) 2023-07-26 18:02:26 +02:00
Mike Fährmann
7fbc304ae9
[twitter] fix crash on private user (#4349) 2023-07-26 17:53:51 +02:00
Mike Fährmann
1ece3b92ff
[mangadex] allow multiple values for 'lang' (#4093) 
This was already possible by setting 'lang' to a list of strings,
but now it can also be done as a more command-line friendly string.

-o lang=fr,it
 2023-07-26 17:39:27 +02:00
Mike Fährmann
52053b58f0
[lensdump] fix extraction (#4352) 2023-07-26 14:24:19 +02:00
Mike Fährmann
11f71a9cba
remove 'mememuseum' module 
This was forgotten when adding generic Shimmie2 support in 7865067d
 2023-07-25 22:22:27 +02:00
Mike Fährmann
a383eca7f6
decouple extractor initialization 
Introduce an 'initialize()' function that does the actual init
(session, cookies, config options) and can called separately from
the constructor __init__().

This allows, for example, to adjust config access inside a Job
before most of it already happened when calling 'extractor.find()'.
 2023-07-25 22:16:16 +02:00
Mike Fährmann
6c9432165e
add return value to 'PostProcessor._init_archive()' 2023-07-24 14:32:25 +02:00
Mike Fährmann
54d974deb0
add 'python' post processor 
similar to 'exec' but calls a Python function
 2023-07-24 14:06:14 +02:00
Mike Fährmann
1baf83a9e5
[hiperdex] fix for unicode titles (#4325) 2023-07-22 16:20:57 +02:00
Mike Fährmann
7da954f810
[flickr] update default API credentials (#4332) 
and add a delay between API requests
 2023-07-22 15:38:33 +02:00
Mike Fährmann
a45a17ddb7
[pixiv] ignore 'limit_sanity_level' images (#4328) 2023-07-22 14:57:38 +02:00
Mike Fährmann
088e8d5fcf
[pornhub] fix extraction (#4301) 2023-07-22 14:05:40 +02:00
Mike Fährmann
d97b8c2fba
consistent cookie-related names 
- rename every cookie variable or method to 'cookies_*'
- simplify '.session.cookies' to just '.cookies'
- more consistent 'login()' structure
 2023-07-22 01:20:50 +02:00
Mike Fährmann
ceebacc9e1
remove 'pyopenssl' option 2023-07-19 20:44:07 +02:00
Mike Fährmann
3c2c7e21dd
merge #4319: [zerochan] fix 'tags' extraction 2023-07-18 18:37:50 +02:00
Mike Fährmann
0ba8d1f168
merge #4312: [redgifs] add 'niches' extractor 2023-07-18 18:36:15 +02:00
Mike Fährmann
c5565f79f7
merge #4096: [danbooru] add support for booru.borvar.art instance 2023-07-18 18:33:08 +02:00
Mike Fährmann
63326e3168
[danbooru] add tests for booruvar 2023-07-18 18:29:57 +02:00
Mike Fährmann
5171d8975c
[E621] support 'e6ai.net' (#4320) 2023-07-18 18:16:30 +02:00
Mike Fährmann
a996d936d2
[imagefap] fix pagination (#3013) 2023-07-18 17:56:33 +02:00
Mike Fährmann
22099422ca
[deviantart] fix shortened URLs (#4316) 2023-07-18 17:55:13 +02:00
Mike Fährmann
90231f2d5a
[twitter] add 'tweet-endpoint' option (#4307) 
use the newer TweetResultByRestId only for guests by default
 2023-07-18 17:19:32 +02:00
Mike Fährmann
20ed647f6f
[twitter] add 'user' extractor and 'include' option (#4275) 2023-07-18 16:42:55 +02:00
Mike Fährmann
86be197d11
[twitter] remove '/search/adaptive.json' 2023-07-18 15:45:37 +02:00
enduser420
d52ed2bc5a [zerochan] fix 'tags' extraction 2023-07-18 16:38:04 +05:30
enduser420
12cd85658b [redgifs] add 'niches' extractor 2023-07-16 21:22:06 +05:30
Mike Fährmann
248e8bc699
release version 1.25.8 2023-07-15 19:24:43 +02:00
Mike Fährmann
bc9123cfee
[naverwebtoon] fix 'comic' metadata extraction 2023-07-14 22:41:36 +02:00
Mike Fährmann
ab5dde7221
[mangaread] fix 'tags' extraction 2023-07-14 22:36:06 +02:00
Mike Fährmann
c9a82c9313
[erome] ignore duplicate album IDs 2023-07-14 22:21:02 +02:00
Mike Fährmann
c84397023a
[slideshare] fix extraction 2023-07-14 21:52:53 +02:00
Mike Fährmann
ffbbbd3baf
[gelbooru_v01] 'vidyart' -> 'vidyart2' 2023-07-14 15:09:39 +02:00
Mike Fährmann
e40b90e137
merge #4303: [gelbooru_v01] fix 'source' (#4302) 2023-07-14 15:00:58 +02:00
Mike Fährmann
c6b31a2169
[reddit] set default 0.6s delay between requests (#4292) 
to limit API requests to 100 per minute
https://www.reddit.com/r/redditdev/comments/14nbw6g/
 2023-07-14 14:41:16 +02:00
Mike Fährmann
20da41018d
[pornhub] set 'accessAgeDisclaimerPH' cookie (#4301) 2023-07-14 14:30:27 +02:00
ncaat
75757c4ace
[gelbooru_v01] fix 'source' (#4302) 2023-07-14 12:53:24 +02:00
Mike Fährmann
2dd6942d1c
[jpgfish] update domain to 'jpeg.pet' 2023-07-13 23:21:01 +02:00
Mike Fährmann
1137b89ed4
[lineblog] remove module 
"LINE BLOGは2023年6月29日をもちましてサービスを終了いたしました"
 2023-07-13 20:46:04 +02:00
Mike Fährmann
86560fe0cd
[bcy] remove module 
"The website was shut down on July 12, 2023"
https://danbooru.donmai.us/wiki_pages/bcy
 2023-07-13 20:46:04 +02:00
Mike Fährmann
fceabee433
[philomena] use API interface class 
handle 429 errors and retry after 10min (#4288)
 2023-07-13 20:46:04 +02:00
Mike Fährmann
f079d9a703
[reddit] notify users about registering an oauth application 
(#4292, #4253, #3943)
 2023-07-12 21:43:00 +02:00
Mike Fährmann
fb3d1462b1
merge #4291: [wikifeet] fix 'tag' extraction 2023-07-10 14:42:56 +02:00
Mike Fährmann
0b08e2e8a8
merge #4287: [twitter] Fix following extractor not getting all users 2023-07-10 14:41:00 +02:00
Mike Fährmann
f6553ffd2f
[twitter] simplify '_pagination_users' 
- remove 'stop' variable
- call 'cursor.startswith()' only once
 2023-07-10 14:39:09 +02:00
Mike Fährmann
1590124aae
[twibooru] fix '--range' 2023-07-10 14:12:56 +02:00
enduser420
a2111dd025 [wikifeet] fix 'tag' extraction 2023-07-09 12:48:47 +05:30
Mike Fährmann
a1ffa1ff09
[philomena] fix '--range' (#4288) 2023-07-08 23:17:27 +02:00
Mike Fährmann
a27dbe8c82
[twitter] use 'TweetResultByRestId' endpoint (#4250) 
allows accessing single Tweets without login
 2023-07-08 23:17:10 +02:00
Mike Fährmann
d3d639a159
[twitter] don't treat missing 'TimelineAddEntries' as fatal (#4278) 2023-07-08 22:49:34 +02:00
ActuallyKit
c321c773f2 make the code less ugly 2023-07-09 02:52:04 +07:00
ActuallyKit
a437a34bcf fix lint i guess? 2023-07-09 02:41:46 +07:00
ActuallyKit
6cbc434b54 Fix users pagination 2023-07-09 02:28:35 +07:00
Mike Fährmann
d5b6802774
[seiga] set 'skip_fetish_warning' cookie (#4242) 2023-07-07 20:51:49 +02:00
Mike Fährmann
88d1e29401
[bunkr] use '.la' TLD for 'media-files12' servers (#4147, #4276) 2023-07-07 20:10:28 +02:00
Mike Fährmann
f0cb951566
[paheal] unescape 'source' 2023-07-07 20:03:00 +02:00
Mike Fährmann
b480b7076a
[paheal] fix a78f8ce5 for enabled 'metadata' (#4262) 2023-07-07 20:00:49 +02:00
Mike Fährmann
384337d3dd
[fantia] send 'X-Requested-With' header only for API requests (#4273) 2023-07-07 15:16:18 +02:00
Mike Fährmann
c2ac665ff7
[fantia] send 'X-Requested-With' header (#4273) 2023-07-06 19:03:53 +02:00
Mike Fährmann
7444fc125b
[gfycat] implement login support (#3770, #4271) 
For the record: '/webtoken' and '/weblogin' are not the same ...
 2023-07-06 18:56:34 +02:00
Mike Fährmann
e9b9f751bf
[gfycat] support '@me' user (#3770, #4271) 2023-07-06 16:03:35 +02:00
Mike Fährmann
5b59a0d143
update default User-Agent header to Firefox 115 ESR 2023-07-05 15:12:50 +02:00
Mike Fährmann
0556e1ad45
merge #4268: [newgrounds] extract & pass auth token for login 2023-07-04 21:51:03 +02:00
Mike Fährmann
a16d7c59cb
[newgrounds] access 'response.text' only once 2023-07-04 21:49:57 +02:00
Mike Fährmann
1bf9f52c99
[twitter] add 'ratelimit' option (#4251) 2023-07-04 18:17:32 +02:00
Mike Fährmann
f86fdf64a6
[twitter] use GraphQL search by default (#4264) 2023-07-04 17:55:22 +02:00
Mike Fährmann
1d4db83d49
[weibo] fix end of cursor based pagination 2023-07-04 17:41:22 +02:00
Mike Fährmann
a78f8ce5b0
[paheal] fix extraction (#4262) 
swap ' and "
 2023-07-04 17:36:41 +02:00
FrostTheFox
9576652fa5
extract & pass auth token for newgrounds 2023-07-04 02:35:48 -04:00
Mike Fährmann
5457007dd3
release version 1.25.7 2023-07-02 22:25:45 +02:00
Mike Fährmann
3d8de383bf
[mangapark] extract 'source_id' for manga 
forgot to add this to 6ae3101f
 2023-07-02 15:17:10 +02:00
Mike Fährmann
6ae3101fd0
[mangapark] add 'source' option (#3969) 2023-07-02 15:07:22 +02:00
Mike Fährmann
c45a913bfd
[flickr] add 'exif' option 2023-07-01 19:19:39 +02:00
Mike Fährmann
3845c0256d
[sankaku] improve warnings for unavailable posts 2023-07-01 19:11:41 +02:00
Mike Fährmann
46cae04aa3
[piczel] update API server (#4244) 2023-06-30 17:19:53 +02:00
Mike Fährmann
3479646f65
[mangapark] update and fix 'manga' extractor (#3969) 
TODO:
- non-English chapters
- 'source' option
 2023-06-30 17:17:54 +02:00
Mike Fährmann
10786c657e
[mangapark] update and fix 'chapter' extractor (#3969) 2023-06-29 23:44:44 +02:00
Mike Fährmann
9c31c2daef
[poipiku] improve error detection (#4206) 2023-06-27 23:12:26 +02:00
Mike Fährmann
260ff55e19
[senmanga] ensure download URLs have a scheme (#4235) 2023-06-27 13:49:33 +02:00
Mike Fährmann
ccbc1a1d55
[flickr] add 'metadata' option (#4227) 2023-06-26 16:49:48 +02:00
Mike Fährmann
c1cce4a80b
[twitter] extend 'conversations' option (#4211) 2023-06-24 21:34:34 +02:00
Mike Fährmann
b6c959744d
[furaffinity] improve 'description' HTML (#4224) 
- ignore header
- include footer and closing <div> if present
 2023-06-24 20:30:40 +02:00
Mike Fährmann
8357acf359
[gelbooru_v01] replace 'extract_all()' with 'extract_from()' 
It's even slightly faster, especially on Python before 3.11
 2023-06-23 15:17:42 +02:00
Mike Fährmann
068aa26c3e
[gelbooru_v01] fix '--range' (#4167) 2023-06-22 11:54:33 +02:00
Mike Fährmann
2052e7ce59
[hentaifox] fix titles containing '@' (#4201) 2023-06-18 20:01:33 +02:00
Mike Fährmann
92d98697b2
[wallhaven] update API error message 2023-06-18 20:00:34 +02:00
Mike Fährmann
a673998b1e
release version 1.25.6 2023-06-17 15:27:09 +02:00
Mike Fährmann
339fcdb8ad
[wallhaven] handle '429 Too Many Requests' errors (#4192) 
- set 1.4s delay between API requests
  (WH allows 45 requests per minute)
- wait and retry on 429 errors
 2023-06-17 13:47:00 +02:00
Mike Fährmann
ef9891ec9d
[fantia] extract 'plan' metadata (#2477, #4128) 2023-06-16 15:43:07 +02:00
Mike Fährmann
f8452984fa
[fantia] emit warning for non-visible contents (#4128) 2023-06-16 15:40:59 +02:00
Mike Fährmann
dc7af00014
[fantia] refactor 
- embed response data as hidden '_data' field
  (instead of returning/passing 'resp')
- split _get_urls_from_post()
 2023-06-15 20:11:54 +02:00
Mike Fährmann
6c8bf9a762
[pornhub] improve redirect handling (#4188) 2023-06-15 16:32:53 +02:00
Mike Fährmann
654267a335
[weibo] fix 'json' extension for some videos 2023-06-15 13:49:17 +02:00
Mike Fährmann
ce93c460a6
[formatter] implement 'H' conversion (#4164) 
to remove HTML tags and unescape HTML entities
 2023-06-15 13:07:51 +02:00
Mike Fährmann
deff3b434d
[vipergirls] implement login support (#4166) 2023-06-13 21:05:09 +02:00
Mike Fährmann
db20a645c5
[vipergirls] use API endpoints (#4166) 2023-06-13 20:54:02 +02:00
Mike Fährmann
0b34a444e0
[pixiv:novel] only detect Pixiv embeds (#4175) 2023-06-13 18:58:35 +02:00
Mike Fährmann
9f1aee3884
[vipergirls] limit number of requests per second (#4166) 2023-06-10 17:57:04 +02:00
Mike Fährmann
21c75d03a3
merge #4133: [furaffinity] extract 'favorite_id' metadata 2023-06-09 16:37:52 +02:00
Mike Fährmann
5e3a1749c8
[furaffinity] simplify 'favorite_id' assignment 2023-06-09 16:35:59 +02:00
Mike Fährmann
ad882291d3
[instagram] fix retrieving '/tagged' posts (#4122) 
reduce number of retrieved posts per API request from 50 to 20
 2023-06-09 16:04:39 +02:00
Mike Fährmann
0a9aaa7a8d
[weibo] prevent fatal exception due to missing video (#4150) 2023-06-08 22:22:43 +02:00
Mike Fährmann
ac651c604c
[senmanga] fix and update (#4160) 2023-06-08 22:18:43 +02:00
Mike Fährmann
df106fb58b
[bunkr] fix video downloads 2023-06-08 17:21:37 +02:00
Mike Fährmann
aad5e6490c
merge #4159: [bunkr] update domain to bunkrr.su 2023-06-08 17:02:54 +02:00
Mike Fährmann
e0522ffb3d
[bunkr] update 2023-06-08 17:01:04 +02:00
Mike Fährmann
e04796e04b
merge #3447: [jschan] add generic extractors for jschan imageboards 2023-06-08 16:52:26 +02:00
Mike Fährmann
b9692341fe
[jschan] update 2023-06-08 16:50:09 +02:00
Stephan
a7c066cbac
Update bunkr.py 2023-06-08 14:10:25 +02:00
Stephan
72e697b8b5
Update bunkr.py 
Support bunkrr.su
 2023-06-08 14:01:28 +02:00
Mike Fährmann
4ae925c88f
[kemonoparty] support '.su' TLD (#4139) 2023-06-06 20:55:03 +02:00
Mike Fährmann
2d9e3093ca
merge #4134: [postimage] add gallery support, update image extractor 2023-06-03 15:56:35 +02:00
Mike Fährmann
e64b521287
merge #4136: [acidimg] fix extractor 2023-06-03 15:54:09 +02:00
Mike Fährmann
a90974178d
[jpgfish] update domain to 'jpg.pet' (#4138) 2023-06-03 15:49:56 +02:00
Mike Fährmann
ee959052ac
merge #4138: add jpg.pet as alias for jpgfish 2023-06-03 15:44:20 +02:00
Mike Fährmann
0281cc7d08
[fanbox] skip 404ed fanbox embeds (#4088) 
continuation of 4fc9675d
 2023-06-03 15:40:21 +02:00
Prinz23
97c0d13cbb add jpg.pet as alias for jpgfish 2023-06-03 15:02:36 +02:00
chio0hai
2e309a13a7 [acidimg] fix extractor 2023-06-03 00:58:33 -04:00
chio0hai
92178b369c [postimage] add gallery support, update image extractor to download 
original image instead of main image
 2023-06-03 00:23:34 -04:00
Bad Manners
952c03bc9e Add fav_id data to FuraffinityFavoriteExtractor 
An extra field is collected when paginating favorites, and saved to
a temporary cache variable. This field is identical for both the old
and the new page layouts for FurAffinity, but can only be collected
during pagination, hence the cache variable. Other FurAffinity
extractors should be unaffected by this change.
 2023-06-02 19:53:47 -03:00
Mike Fährmann
54cf1fa3e7
[twitter] use GraphQL search endpoint (#3942) 
for guest users; selectable with 'search-endpoint' option.

adapted from 9c7b888ffa
 2023-06-01 21:37:31 +02:00
Mike Fährmann
864a654b25
[twitter] update query hashes 2023-06-01 21:37:31 +02:00
Mike Fährmann
45cc7cee1a
[twitter] better error message for guest searches (#3942) 2023-06-01 21:37:11 +02:00
Mike Fährmann
271f23d971
[twitter] extract 'conversation_id' metadata (#3839) 2023-06-01 15:31:52 +02:00
Mike Fährmann
94b6a67666
[reddit] fix crash with empty 'crosspost_parent_lists' (#4120) 2023-06-01 14:51:58 +02:00
Mike Fährmann
0cf7282fa0
[pixiv] add 'full-series' option for novels (#4111) 2023-06-01 13:07:20 +02:00
Mike Fährmann
bab13402df
[redgifs] update 'search' URL pattern (#4115) 2023-05-30 15:42:31 +02:00
Mike Fährmann
5a6fd8027d
[redgifs] support galleries (#4021) 2023-05-30 15:02:18 +02:00
Mike Fährmann
0ad59c92b1
[blogger] download files from 'lh*.googleusercontent.com' (4070) 2023-05-28 19:58:20 +02:00
Mike Fährmann
ffed7efb6f
[pixiv] use BASE_PATTERN 2023-05-28 18:06:47 +02:00
Mike Fährmann
b286efefcc
[pixiv] add 'novel-bookmark' extractor (#4111) 2023-05-28 16:30:17 +02:00
Mike Fährmann
5283db1aae
release version 1.25.5 2023-05-27 17:08:25 +02:00
Mike Fährmann
28f6487c64
[instagram] add 'metadata' option (#3107) 2023-05-27 15:51:13 +02:00
Mike Fährmann
8cf13f8696
merge #4104: [lensdump] add lensdump.com extractors 2023-05-26 23:42:02 +02:00
Mike Fährmann
58f7480d46
[lensdump] update 
- update docs/supportedsites.md
- add GPL2 header
- use BASE_PATTERN
- improve LensdumpImageExtractor
 2023-05-26 23:39:17 +02:00
Mike Fährmann
3516fdae74
[kemonoparty] fix kemono and coomer logins using the same cache 
(#4098)
 2023-05-26 13:35:02 +02:00
chio0hai
d5300cf381 [lensdump] subcategory 2023-05-26 03:51:42 -04:00
chio0hai
82ba6bfdc0 [lensdump] f-string fix 2023-05-26 03:46:12 -04:00
chio0hai
9b2326e4e1 [lensdump] add lensdump.com extractor 2023-05-26 03:22:23 -04:00
Mike Fährmann
a5d0b03bde
[ytdl] fix crash due to removed 'no_color' attribute 
8417f26b8a
 2023-05-25 23:11:49 +02:00
Mike Fährmann
148bdc04a4
merge #2719: [jpgfish] add 'jpgfish' extractors 2023-05-25 23:04:46 +02:00
Mike Fährmann
609c4f3e07
[jpgfish] simplify and improve 2023-05-25 22:58:42 +02:00
Mike Fährmann
2b1f875ef4
[jpgchurch] update to 'jpgfish' 2023-05-25 18:58:51 +02:00
Mike Fährmann
3d29c42142
[mangaread] fix 'tags' extraction 2023-05-25 17:04:20 +02:00
Mike Fährmann
5f86527cbe
merge #2781: [mangaread] Add Mangaread extractor 2023-05-25 17:03:28 +02:00
Mike Fährmann
cdc6549fd2
merge #3329: [8muses] Add 'parts' to album data 
and fix 'album[url]'
 2023-05-25 16:31:29 +02:00
Mike Fährmann
ad760429b1
[8muses] update 2023-05-25 16:25:53 +02:00
Mike Fährmann
d0184fddcf
[twitter] optimize '_extract_twitpic()' 
- use findall instead of finditer
- store URLs in a dict to discard duplicates
 2023-05-25 15:18:49 +02:00
Mike Fährmann
3dc862c7fc
merge #3796: [twitter] extract TwitPic URLs in text (#3792) 2023-05-25 14:59:07 +02:00
Mike Fährmann
243de697b9
merge #3976: [reddit] support cross-posted media (#887, #3586) 2023-05-25 13:16:25 +02:00
Mike Fährmann
f8c4c5eef9
[reddit] simplify and add tests 2023-05-25 13:15:11 +02:00
thatfuckingbird
822a77d846 [danbooru] add support for booru.borvar.art instance 2023-05-24 19:06:40 +02:00
Mike Fährmann
f3cca50b9e
[mangadex] update links to API docs 2023-05-24 12:32:13 +02:00
Mike Fährmann
65a9f4b124
merge #3950: [misskey] add 'favorite' extractor 2023-05-23 22:18:32 +02:00
Mike Fährmann
c76f0f3a1b
[misskey] update 
- rename to 'MisskeyFavoriteExtractor'
- add 'access-token' option to docs
- add test URLs for other instances
- simplify 'pattern'
 2023-05-23 22:17:17 +02:00
Mike Fährmann
3fca455b82
[pixiv] add 'embeds' option (#1241) 2023-05-23 12:14:06 +02:00
Mike Fährmann
d1f2ef3b7b
[imagechest] update 
- don't load HTML page when using API
- restructure some code
- add more methods to ImagechestAPI
 2023-05-22 22:42:16 +02:00
Mike Fährmann
856f6c10cd
allow for GalleryExtractors to skip loading gallery_url 2023-05-22 22:29:30 +02:00
Mike Fährmann
4fc9675d48
[fanbox] skip 404ed or otherwise invalid posts (#4088) 2023-05-22 18:49:06 +02:00
Mike Fährmann
69865dcc05
[formatter] implement slicing strings as bytes (#4087) 
prefixing a slice '[10:30]' with a lowercase b '[b10:30]' encodes
the string to bytes in filesystem encoding before applying the slice
 2023-05-22 18:30:45 +02:00
Mike Fährmann
56b8b8cd36
[pixiv] support short novel URLs 
https://www.pixiv.net/n/<ID>
 2023-05-21 14:26:30 +02:00
Mike Fährmann
e6f55d1555
[imagechest] add API support and 'access-token' option (#4065) 2023-05-20 17:00:27 +02:00
Mike Fährmann
77abcf5ab3
[gofile] automatically fetch 'website-token' by default 
the hardcoded token changed yet again
 2023-05-20 17:00:27 +02:00
Mike Fährmann
e3fed9bd17
[tcbscans] update domain to 'tcbscans.com' (#4080) 2023-05-20 13:59:35 +02:00
Mike Fährmann
a83983c651
[instagram] add 'order-posts' option (#4017, #3993) 2023-05-18 22:50:12 +02:00
Mike Fährmann
d680623db3
[instagram] add 'order-files' option (#4017, #3993) 2023-05-18 22:34:33 +02:00
Naatie
f9b7a033e0 [misskey] refactor misskey extractor 2023-05-18 20:15:59 +07:00
Naatie
04dbfd994e [misskey] add my favorites extractor 2023-05-18 20:15:59 +07:00
Mike Fährmann
82a12d6126
[nsfwalbum] detect placeholder images 
patch by an anonymous contributor
 2023-05-17 15:18:10 +02:00
Mike Fährmann
011e4607c3
[poipiku] extract full 'descriptions' (#4066) 
don't cut it off after the first line
 2023-05-15 23:16:31 +02:00
Mike Fährmann
5037013e2b
[gofile] update 'website-token' (#4056) 2023-05-15 14:59:44 +02:00
Mike Fährmann
6b6bb4be73
[weibo] require numeric IDs to have length >= 10 (#4059) 2023-05-14 18:45:37 +02:00
Mike Fährmann
494acabd38
[danbooru] refactor pagination logic (#4002) 
- only use 'b<ID>' when no other order is specified
- support 'a<ID>' when  using 'order:id' as tag
 2023-05-14 18:39:59 +02:00
Mike Fährmann
fd0e1ffd6e
[danbooru] improve 75666cf9 (#4002) 
Search for direct post IDs instead of trying to
replicate the same results as the initial request.
 2023-05-14 16:35:31 +02:00
Mike Fährmann
e41e45ff6b
[gofile] add basic password support (#4056) 2023-05-13 15:46:29 +02:00
Mike Fährmann
790dd365e1
[postprocessor:exec] support tilde expansion for 'command' 
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1544733532
 2023-05-12 16:34:17 +02:00
Mike Fährmann
2e6cea95db
[cookies] update logging behavior (#4050) 
- only show the same warning/error once
- simplify and capitalize logging messages
 2023-05-12 16:34:17 +02:00
Mike Fährmann
20dc13f832
[pixiv] initial 'novel' support (#1241, #4044) 
supported URLs are
- https://www.pixiv.net/novel/show.php?id=<ID>
- https://www.pixiv.net/novel/series/<ID>
- https://www.pixiv.net/en/users/<ID>/novels
 2023-05-12 16:34:08 +02:00
Mike Fährmann
c698c3de44
[newgrounds] add default delay between requests (#4046) 2023-05-11 16:04:37 +02:00
Mike Fährmann
708f478d15
[danbooru][e621] add 'date' metadata field (#4047) 2023-05-11 15:59:42 +02:00
Mike Fährmann
306e13a4d4
release version 1.25.4 2023-05-07 20:57:36 +02:00
Mike Fährmann
35c23a2fd8
merge #4031: [mangadex] add 'status' and 'tags' metadata 2023-05-07 20:16:23 +02:00
Mike Fährmann
2266fc8cc5
[mangadex] update and extend test results 2023-05-07 20:14:49 +02:00
Janne Alaranta
1ce5dc9e18 fix whitespaces 2023-05-07 18:47:04 +03:00