Mike Fährmann
ed859f05ed
[deviantart] fix & improve journal/literature extraction ( #6254 , #6207 )
...
fetch text from HTML __INITIAL_STATE__,
since the API doesn't reliably work and is unusable for sta.sh journals
2024-10-01 14:36:49 +02:00
Mike Fährmann
8f09e4eb02
[cohost] remove 'plainTextBody' from default filenames ( #6262 )
2024-10-01 14:36:49 +02:00
Mike Fährmann
2e87a91e97
[oauth] prevent empty browser names
2024-10-01 08:22:50 +02:00
Mike Fährmann
3d4191fc0d
[patreon] handle exception for suspended accounts
2024-10-01 08:22:50 +02:00
Mike Fährmann
319c3a4c9b
[patreon] use 'self.groups' to access matched URL values
2024-10-01 08:22:50 +02:00
Mike Fährmann
321161c769
[patreon] use mobile UA ( #6241 , #6239 , #6140 )
2024-10-01 08:22:16 +02:00
Mike Fährmann
7d0e5877ce
[newgrounds] add more fallback URLs for 'art-images' files
2024-10-01 08:22:16 +02:00
Mike Fährmann
f3f27496d6
[newgrounds] support more comment embed formats ( #6253 )
2024-09-30 08:20:50 +02:00
Mike Fährmann
987f6c36c0
[civitai] ensure image files have an 'id' ( #6251 )
2024-09-29 19:25:55 +02:00
Mike Fährmann
a937b72034
[ao3] add 'subscriptions' extractor ( #6247 )
2024-09-29 13:01:51 +02:00
Mike Fährmann
9a0acbe7c4
[flickr] remove debug remains ( #6252 )
...
fixes regression introduced in a051e1c9
2024-09-29 13:01:51 +02:00
Mike Fährmann
0389575e0f
[8chan] dynamically generate 'TOS' cookie name
2024-09-28 11:04:27 +02:00
Mike Fährmann
4e9dd036e7
[civitai] add 'nsfw' option ( #3706 )
2024-09-28 08:44:35 +02:00
Mike Fährmann
1ab90dd9e2
[civitai] improve model 'gallery' extraction ( #3706 )
2024-09-27 22:14:14 +02:00
Mike Fährmann
bc11dc0de2
[deviantart] add 'previews' option ( #3782 , #6124 )
2024-09-27 10:41:26 +02:00
Mike Fährmann
928e170721
[deviantart] work around OAuth API returning empty journal texts
...
(#6196 , #6207 , #5916 )
2024-09-27 10:41:26 +02:00
Mike Fährmann
f8f67dab22
[cookies] add 'cookies-select' option
2024-09-27 10:41:26 +02:00
Mike Fährmann
a783d6473c
[jpgfish] update domain to 'jpg5.su' ( #6231 )
2024-09-26 15:23:22 +02:00
Mike Fährmann
f0267f969a
[skeb] prevent 429 errors and need for 'request_key' cookie
...
https://github.com/danbooru/danbooru/pull/5795
2024-09-26 15:15:37 +02:00
Mike Fährmann
e8c5b57a33
merge #6115 : [inkbunny:favorite] update default directory
...
- extract 'favs_username' metadata
- store files in 'favs_username' directory
2024-09-25 20:28:14 +02:00
Mike Fährmann
ac3ff25236
[inkbunny:favorite] update test
2024-09-25 20:25:33 +02:00
Mike Fährmann
050d87d024
merge #6113 : [weasyl:favorite] fix pagination, support readable URLs
2024-09-25 20:08:36 +02:00
Mike Fährmann
9d7f8f892d
[weasyl:favorite] update
...
- use 'self.groups' to access userid and username
- safe one request by not doing an explicit username -> userid lookup
- safe one request by following the 'Next' link instead of detecting an
empty page
2024-09-25 20:02:01 +02:00
Mike Fährmann
3eb3564b5d
[civitai] support using internal tRPC API endpoints ( #3706 )
2024-09-25 18:46:18 +02:00
Mike Fährmann
daa220370c
[civitai] improve image URL handling ( #3706 )
...
- support building URLs from URL-hash values
- rename '_orig' to '_url'
2024-09-25 17:29:35 +02:00
Mike Fährmann
a2db0d5c0d
[civitai] add 'quality' option ( #3706 )
...
download 'original=true' files by default
2024-09-25 17:23:08 +02:00
Mike Fährmann
92bbb9bf31
[civitai] add 'image' extractors ( #3706 , #6220 )
2024-09-23 10:13:30 +02:00
Deer-Spangle
cbecaecc43
Fix infinite loop in Weasyl Favs downloader
2024-09-22 11:12:34 +01:00
Mike Fährmann
8bf081d295
[ao3] fix bookmark extraction, support 'series' bookmarks ( #6013 )
2024-09-21 13:54:42 +02:00
Mike Fährmann
5c062f95c1
[ao3] fix 'title' of restricted works ( #6013 )
2024-09-21 13:21:45 +02:00
Mike Fährmann
91eb04b0d0
[ao3] detect member-only and adult works ( #6013 )
2024-09-21 13:20:42 +02:00
Mike Fährmann
3348b05df0
[ao3] implement login with username & password ( #6013 )
2024-09-21 13:15:50 +02:00
Mike Fährmann
93eca64a73
[civitai] add initial support ( #3706 , #3787 , #4129 , #5995 )
2024-09-20 17:21:17 +02:00
Mike Fährmann
fe190be589
[8chan] update 'TOS' cookie name
2024-09-19 20:50:58 +02:00
Mike Fährmann
2c7a0c3ca8
add alternatives for deprecated utc datetime functions
2024-09-19 20:47:05 +02:00
Mike Fährmann
a051e1c955
directly pass exception instances as 'exc_info' logger argument
2024-09-19 14:50:08 +02:00
Mike Fährmann
2dfdc92b21
[ao3] extract detailed 'chapters' metadata ( #6013 )
2024-09-18 09:48:57 +02:00
Mike Fährmann
1d7df9cef2
[ao3] extract 'date_completed' and 'date_updated' metadata ( #6013 )
2024-09-17 17:26:54 +02:00
Mike Fährmann
064bc890fc
[ao3] parse 'series' metadata ( #6013 )
2024-09-17 17:14:44 +02:00
Mike Fährmann
6b8628b23f
merge #6191 : [cohost] support downloading likes
2024-09-17 10:50:28 +02:00
Mike Fährmann
8f777668d0
[ao3] match URLs with 'com' and 'net' TLDs and 'ao3.org' ( #6013 )
2024-09-17 09:59:55 +02:00
Kae
31af0a12c7
[cohost] Support for downloading likes
...
turns out it's almost the same as tags, so I was able to get it working pretty easily
2024-09-17 12:50:59 +10:00
Mike Fährmann
a9dc7a6f98
[deviantart] warn about empty journal texts ( #5916 )
...
https://github.com/mikf/gallery-dl/issues/5916#issuecomment-2351069092
2024-09-16 10:12:25 +02:00
Mike Fährmann
4c391805de
[bluesky] simplify file extraction code ( #6183 )
2024-09-16 09:37:18 +02:00
Mike Fährmann
638a676495
[ao3] add initial support ( #6013 )
2024-09-15 22:38:21 +02:00
Mike Fährmann
7d6520e15d
[bluesky] support video downloads ( #6183 )
2024-09-15 22:38:03 +02:00
Mike Fährmann
af8cba089e
[zzup] support 'up.zzup.com' galleries ( #6181 )
2024-09-15 11:01:27 +02:00
Mike Fährmann
7abf6e445c
[cohost] add 'tag' extractor ( #4483 )
2024-09-13 11:11:53 +02:00
Mike Fährmann
0d67d54162
[cohost] support 'attachment-row' blocks ( #4483 )
2024-09-13 10:33:05 +02:00
Mike Fährmann
29279f07d4
[cohost] warn about inaccessible posts ( #4483 )
2024-09-13 10:31:28 +02:00
Mike Fährmann
df0d7d4a12
[cohost] add 'user' and 'post' extractors ( #4483 )
2024-09-11 18:03:33 +02:00
Mike Fährmann
0db3c11ab0
[common] use 'cf-mitigated' header to detect challenges
2024-09-07 20:16:06 +02:00
Mike Fährmann
4b04ccb3a1
[wikimedia] probe possible API endpoints when none is defined
...
instead of hardcoding it to '/api.php'
2024-09-07 20:11:44 +02:00
Mike Fährmann
7260e236d8
[furaffinity] fix 'favorite' pagination ( #6151 )
2024-09-06 08:52:05 +02:00
Mike Fährmann
7fe0f35998
[sexcom] add 'likes' extractor ( #6149 )
2024-09-06 07:44:06 +02:00
Mike Fährmann
513fd26616
[bcbnsfw] use '*' as query when retrieving all posts ( #6135 )
2024-09-05 20:43:23 +02:00
Mike Fährmann
07bd967f59
[pp:ugoira] update ( #6056 )
...
- introduce '_ugoira_frame_index' metadata field
- store Ugoira file exts separately
- add 'skip' option
2024-09-05 20:25:20 +02:00
Mike Fährmann
32d2e686c2
[batoto] return 'chapter_url' as string ( #5562 )
...
don't try to parse it as integer
2024-09-05 17:47:38 +02:00
Mike Fährmann
6f611de889
[ytdl] simplify '_process_entries()'
2024-09-03 21:17:31 +02:00
Mike Fährmann
4a3a255ddc
[ytdl] fix processing playlists of playlists ( #6127 )
2024-09-03 08:39:04 +02:00
Mike Fährmann
655145853a
[toyhouse] fix extraction for image URLs
2024-09-02 17:15:02 +02:00
Mike Fährmann
864484e4c6
[instagram] add 'info' as a possible 'include' value
2024-09-02 15:43:55 +02:00
Mike Fährmann
0b53f3d0e9
[deviantart] fix '"original": "images"' ( #6124 )
2024-09-02 15:15:37 +02:00
Mike Fährmann
9d1e5f3c9b
[pixiv] allow downloading "original" ugoira frames ( #6056 )
2024-09-01 13:37:32 +02:00
Deer-Spangle
8c40094905
Store Inkbunny favourites in the Favorites directory of the user's favs being downloaded, rather than in different directories for each submission
2024-08-31 16:37:20 +01:00
Mike Fährmann
bd932b6860
[twitter] add 'info' as a possible 'include' value ( #6114 )
2024-08-31 17:04:22 +02:00
Deer-Spangle
10543c64c6
Store favourites in the Favorites directory of the user's favs being downloaded, rather than in different directories for each submission
2024-08-31 15:54:04 +01:00
Deer-Spangle
01ace529b6
Fixing syntax on python 3.5, the sad days before f-strings
2024-08-31 15:22:39 +01:00
Deer-Spangle
311272f55a
Linting
2024-08-31 15:10:02 +01:00
Deer-Spangle
b1b01f56af
Add support for the readable URL format of Weasyl favourites page
2024-08-31 12:56:50 +01:00
Mike Fährmann
ef8b1bc56e
[twitter] extract 'type' metadata ( #6111 )
2024-08-31 13:16:51 +02:00
Mike Fährmann
8eaae29bf9
[exhentai] fix 'limits' ( #6090 )
2024-08-31 10:23:07 +02:00
Mike Fährmann
e464c955b3
merge #6104 : [generic] better directory names
2024-08-30 11:47:09 +02:00
Mike Fährmann
58113b73d1
[flickr] make album metadata extraction non-fatal ( #3441 )
...
https://github.com/mikf/gallery-dl/issues/3441#issuecomment-2313679156
2024-08-30 10:24:03 +02:00
Allen
a552fb5bcd
[generic] better directory formatting/md
2024-08-29 22:30:48 +02:00
Mike Fährmann
cf8e04d999
[koharu] improve format selection ( #6088 )
...
- allow specifying more than one possible format
- ignore not available formats
2024-08-29 09:33:24 +02:00
Mike Fährmann
c51938b82b
[twitter] fix pinned Tweet extraction ( #6102 )
2024-08-29 08:53:48 +02:00
Mike Fährmann
f669b3f046
[instagram] fix "KeyError - 'post_date'" ( #6081 , #6091 )
...
fixes regression introduced in a3642638
2024-08-27 12:39:20 +02:00
Mike Fährmann
4b286e80fd
merge #6050 : [wikimedia] add 'wiki' extractor
2024-08-25 09:38:24 +02:00
Mike Fährmann
65cae11b54
merge #6077 : [flickr] fix bug introduced in e92a9ae3
2024-08-25 09:30:07 +02:00
Mike Fährmann
46c3971c88
[bunkr] add 'tlds' option to allow URLs with all TLDs ( #5875 , #6017 )
2024-08-24 20:45:44 +02:00
Mike Fährmann
a364263810
[instagram] add 'post_date' metadata field ( #6081 )
...
and restore 'date' metadata for GraphQL file results,
which got accidentally removed in a001c9c0
2024-08-24 15:53:18 +02:00
Mike Fährmann
3bffe7a8bd
[deviantart] fix 'manual' cursor-based pagination ( #6079 )
2024-08-23 16:03:38 +02:00
Pedro Cunha
dcd44cf423
[flickr] reference the correct function
2024-08-22 17:00:35 +01:00
Mike Fährmann
80c423a444
[cyberdrop] fix extraction
2024-08-19 13:37:14 +02:00
Mike Fährmann
f32e10f6a5
[instagram] add 'max-posts' option ( #6054 )
2024-08-19 10:19:44 +02:00
Mike Fährmann
6af26a424a
[hitomi] extract 'extension_original' metadata ( #6049 )
2024-08-18 12:56:50 +02:00
ClosedPort22
968c04a27c
[wikimedia] raise default value for 'limit' from 10 to 50
2024-08-18 16:33:13 +08:00
ClosedPort22
5704024662
[wikimedia] improve handling of warnings and errors
2024-08-18 16:31:49 +08:00
ClosedPort22
33fe1b68b7
[wikimedia] add 'limit' option
2024-08-18 13:39:48 +08:00
ClosedPort22
33d2ddd9fb
[wikimedia] add 'wiki' extractor
2024-08-18 13:20:59 +08:00
ClosedPort22
a3b2c88fbe
[wikimedia] refactor
2024-08-18 12:59:58 +08:00
Mike Fährmann
8ea75202ed
[bunkr] fix file downloads ( #6037 )
2024-08-17 19:25:19 +02:00
Mike Fährmann
ef4c1b4fc5
[sankaku] restore old 'tags' format ( #6043 )
...
lowercase + words separated by underscores
2024-08-17 19:25:19 +02:00
Mike Fährmann
e29f4ef226
[instagram] allow disabling 'cursor' output
2024-08-17 19:25:19 +02:00
Mike Fährmann
c0668f5106
[twitter] allow disabling 'cursor' output ( #5990 )
2024-08-17 19:24:38 +02:00
Mike Fährmann
0bbdeaaa50
[bunkr] support 'bunkr:' URL prefix ( #6017 )
...
and support 'bunkr.ax' by default
2024-08-14 19:00:59 +02:00
Mike Fährmann
e92a9ae343
[flickr] make exif and context metadata extraction non-fatal ( #6002 )
2024-08-14 09:44:04 +02:00
Mike Fährmann
f0de1685ae
[batoto] extract 'chapter_url' ( #5562 )
...
i.e. the chapter number found in the full chapter URL
2024-08-13 20:57:33 +02:00
Mike Fährmann
3b0753f6ea
[batoto] extract 'title' independent of chapter info ( #5988 )
2024-08-13 20:20:32 +02:00
Mike Fährmann
802d74bdf6
[batoto] improve chapter info regex 2
...
- make regex case-insensitive
- match 'Season 12' and 'S12' as volume numbers
- match 'Episode 12' as chapter number
2024-08-13 20:19:17 +02:00
Mike Fährmann
c495bfe460
[newgrounds] fix warning for age-restricted posts ( #6005 )
2024-08-13 08:16:15 +02:00
Mike Fährmann
54e1023046
merge #6001 : [e621] cache pool metadata API calls
2024-08-12 18:04:52 +02:00
Mike Fährmann
eb5955bacc
[e621] reorder
2024-08-12 18:01:29 +02:00
Mike Fährmann
6193c8eabc
merge #5997 : [batoto] improve chapter info regex ( #5988 )
...
… and add 'chapter_string' metadata
2024-08-12 17:54:17 +02:00
Mike Fährmann
224821bd21
[batoto] add test, improve 'info' handling
2024-08-12 17:52:28 +02:00
ClosedPort22
0f0ded53ae
[e621] cache pool metadata results
2024-08-12 21:03:35 +08:00
enduser420
aa9e24ce0d
[batoto] update
2024-08-12 15:52:18 +05:30
Mike Fährmann
8cd625fe2b
merge #5983 : [pixiv] use mobile API for series
2024-08-12 11:53:37 +02:00
Mike Fährmann
b33a53968d
[pixiv] incorporate into '_pagination'
2024-08-12 11:50:41 +02:00
enduser420
bc43c35847
[bato] fix chapter info regex
2024-08-12 13:24:44 +05:30
Mike Fährmann
785e6f2911
[tumblr] fix 401 Unauthorized for likes when using api-key ( #5994 )
...
fixes regression introduced in 540eaa5a
2024-08-12 09:09:59 +02:00
Mike Fährmann
7f0f407d88
[gofile] fix "KeyError: 'childrenIds'" ( #5993 )
2024-08-11 15:26:58 +02:00
mix5003
b6a40217d6
[pixiv] use api for illust series
2024-08-10 20:29:13 +07:00
Mike Fährmann
d1fe81dab0
merge #5970 : [bunkr] support 'bunkr.ci' and 'bunkrrr.org'
2024-08-10 12:26:39 +02:00
Mike Fährmann
ad6ed72307
[bunkr] add tests for new domains
2024-08-10 12:25:43 +02:00
Mike Fährmann
8d1805e336
[hentaicosplays] support 'hentai-cosplay-xxx.com' ( #5959 )
2024-08-10 12:14:18 +02:00
Mike Fährmann
87a14a50e7
[zerochan] improve redirect handling, add 'redirects' option ( #5891 )
2024-08-10 11:32:30 +02:00
Mike Fährmann
6e7da6310c
[behance] fix video extraction ( #5965 )
...
a lot slower than before since each video now requires an extra HTTP
request and 'sleep-request' is set to 2s-4s by default.
it now also requires ytdl.
2024-08-10 11:06:54 +02:00
Larsluph
8e2a2bd221
[bunkr] Add support for more domains
...
Supported domains added:
- https://bunkr.ci
- https://bunkrrr.org
2024-08-08 12:38:32 -05:00
Mike Fährmann
9783d95585
[behance] fix "KeyError: 'fields'" ( #5965 )
2024-08-08 16:29:56 +02:00
Mike Fährmann
269b2808f1
[deviantart] fix "KeyError - 'category'" ( #5960 , #5961 )
...
'deviation' objects returned by the OAuth API no longer include a
'category' or 'category_path' value.
2024-08-08 09:34:51 +02:00
Mike Fährmann
c07a074418
[furaffinity] remove end of system messages
2024-08-07 18:39:34 +02:00
Mike Fährmann
846512f6cd
[furaffinity] add 'submissions' extractor ( #5954 )
2024-08-07 16:34:32 +02:00
Mike Fährmann
b4733b7ea8
[fanbox] fix pagination ( #5949 , #5951 , #5956 )
2024-08-07 10:16:14 +02:00
Mike Fährmann
34433f5a5d
[bunkr] fail downloads for 'maintenance' files ( #5952 )
2024-08-07 08:33:10 +02:00
Mike Fährmann
575e5df539
[hotleak] fix AttributeError ( #5950 )
...
fixes regression introduced in 0432e057
2024-08-07 08:26:23 +02:00
Mike Fährmann
ff4e53adac
[instagram] restore GraphQL API functionality ( #5920 )
...
by reducing the number of requested posts from 50 to 24
https://github.com/mikf/gallery-dl/issues/5920#issuecomment-2269712540
2024-08-07 08:17:59 +02:00
Mike Fährmann
21734ab69e
[twitter] update 'x-csrf-token' header during login ( #5945 )
2024-08-07 08:11:44 +02:00
Mike Fährmann
008f644ba3
[cien] fix extraction when authenticated ( #5934 )
...
'articleBody' doesn't always contain the entire article, it seems.
2024-08-03 21:57:18 +02:00
Mike Fährmann
bb07313227
[deviantart] try to work around journal/status API changes ( #5916 )
...
The new /user/profile/posts endpoint is documented to
"return all journals & status updates for a given user in a single feed"
but it does not do this.
There are usually several or even all status updates missing.
2024-08-03 13:17:27 +02:00
Mike Fährmann
f87783ff26
[cien] add 'files' option ( #2885 )
2024-08-03 11:37:14 +02:00
Mike Fährmann
f68b0efc1b
[wallpapercave] support 'album' listings ( #5925 )
2024-08-02 16:13:09 +02:00
Mike Fährmann
a5be680596
[directlink] extend recognized file extensions ( #5924 )
...
bmp, svg, avif, heic, psd, pdf, m4v, mov, wav, mp3, zip, rar, 7z, swf
2024-08-02 16:13:09 +02:00
Mike Fährmann
d9ff862fbd
[koharu] reduce login cache expiration to 24h ( #5893 )
2024-08-02 16:13:09 +02:00
Mike Fährmann
9be7896f78
[koharu] fix 'count' for 'cbz' downloads ( #5893 )
2024-08-02 16:13:03 +02:00
Mike Fährmann
d65fc2435c
[cien] extract 'gallery' images ( #2885 )
2024-08-01 14:01:21 +02:00
Mike Fährmann
0432e05783
[hotleak] fix faulty image URLs ( #5915 )
2024-08-01 12:35:01 +02:00
Mike Fährmann
7d15ea6274
[instagram] fix 'AttributeError' in GraphQL pagination ( #5920 )
2024-08-01 12:35:01 +02:00
Mike Fährmann
c372242a06
[koharu] add 'favorite' extractor ( #5893 )
2024-08-01 12:33:18 +02:00
Mike Fährmann
2bf76461ce
[deviantart:following] use OAuth API endpoint ( #2511 )
2024-07-31 17:21:39 +02:00
Mike Fährmann
095f278d6f
[vsco] add 'include' option ( #5911 )
2024-07-31 12:32:04 +02:00
Mike Fährmann
ff58683b76
[koharu] send necessary headers for image downloads ( #5893 )
2024-07-30 19:39:50 +02:00
Mike Fährmann
399ba85841
[fallenangels] remove module
2024-07-30 17:33:16 +02:00
Mike Fährmann
84eefeebd6
[sankaku] match URLs with 'www' subdomain ( #5907 )
2024-07-30 17:05:22 +02:00
Mike Fährmann
279854cd9e
[agnph] implement 'tags' option ( #5284 )
2024-07-30 14:16:19 +02:00
Mike Fährmann
d2dda2bc00
[paheal] implement fast '--range' support ( #5905 )
2024-07-30 12:55:57 +02:00
Mike Fährmann
ca3e7a5e5a
[koharu] add 'date', simplify 'tags' ( #5893 )
2024-07-30 12:38:19 +02:00
Mike Fährmann
9c91935fe4
merge #5904 : [dynastyscans] extract chapter 'tags'
2024-07-29 21:17:13 +02:00
enduser420
5a36da2968
[dynastyscans] extract chapter 'tags'
2024-07-29 16:50:47 +05:30
