Mike Fährmann
6af26a424a
[hitomi] extract 'extension_original' metadata ( #6049 )
2024-08-18 12:56:50 +02:00
ClosedPort22
968c04a27c
[wikimedia] raise default value for 'limit' from 10 to 50
2024-08-18 16:33:13 +08:00
ClosedPort22
5704024662
[wikimedia] improve handling of warnings and errors
2024-08-18 16:31:49 +08:00
ClosedPort22
33fe1b68b7
[wikimedia] add 'limit' option
2024-08-18 13:39:48 +08:00
ClosedPort22
33d2ddd9fb
[wikimedia] add 'wiki' extractor
2024-08-18 13:20:59 +08:00
ClosedPort22
a3b2c88fbe
[wikimedia] refactor
2024-08-18 12:59:58 +08:00
Mike Fährmann
8ea75202ed
[bunkr] fix file downloads ( #6037 )
2024-08-17 19:25:19 +02:00
Mike Fährmann
ef4c1b4fc5
[sankaku] restore old 'tags' format ( #6043 )
...
lowercase + words separated by underscores
2024-08-17 19:25:19 +02:00
Mike Fährmann
e29f4ef226
[instagram] allow disabling 'cursor' output
2024-08-17 19:25:19 +02:00
Mike Fährmann
c0668f5106
[twitter] allow disabling 'cursor' output ( #5990 )
2024-08-17 19:24:38 +02:00
Mike Fährmann
78ae0ba9f7
[formatter] implement 'A' format specifier ( #6036 )
2024-08-16 20:34:23 +02:00
Mike Fährmann
01e8433889
fix tests & syntax for older Python versions
2024-08-15 18:16:55 +02:00
Mike Fährmann
0bbdeaaa50
[bunkr] support 'bunkr:' URL prefix ( #6017 )
...
and support 'bunkr.ax' by default
2024-08-14 19:00:59 +02:00
Mike Fährmann
2d55521582
[cookies:firefox] extract only cookies without container by default ( #5957 )
...
add support for 'all' to extract all cookies regardless of container
2024-08-14 18:59:41 +02:00
Mike Fährmann
e92a9ae343
[flickr] make exif and context metadata extraction non-fatal ( #6002 )
2024-08-14 09:44:04 +02:00
Mike Fährmann
f0de1685ae
[batoto] extract 'chapter_url' ( #5562 )
...
i.e. the chapter number found in the full chapter URL
2024-08-13 20:57:33 +02:00
Mike Fährmann
3b0753f6ea
[batoto] extract 'title' independent of chapter info ( #5988 )
2024-08-13 20:20:32 +02:00
Mike Fährmann
802d74bdf6
[batoto] improve chapter info regex 2
...
- make regex case-insensitive
- match 'Season 12' and 'S12' as volume numbers
- match 'Episode 12' as chapter number
2024-08-13 20:19:17 +02:00
Mike Fährmann
5208c0d28a
[util] extend CustomNone with an __index__ method ( #6009 )
...
- Make it compatible with functions expecting integer arguments
- Simplify and reuse some method definitions
2024-08-13 16:05:54 +02:00
Mike Fährmann
ad73789256
[util] extend CustomNone with arithmetic operators ( #6007 )
2024-08-13 09:41:12 +02:00
Mike Fährmann
c495bfe460
[newgrounds] fix warning for age-restricted posts ( #6005 )
2024-08-13 08:16:15 +02:00
Mike Fährmann
54e1023046
merge #6001 : [e621] cache pool metadata API calls
2024-08-12 18:04:52 +02:00
Mike Fährmann
eb5955bacc
[e621] reorder
2024-08-12 18:01:29 +02:00
Mike Fährmann
6193c8eabc
merge #5997 : [batoto] improve chapter info regex ( #5988 )
...
… and add 'chapter_string' metadata
2024-08-12 17:54:17 +02:00
Mike Fährmann
224821bd21
[batoto] add test, improve 'info' handling
2024-08-12 17:52:28 +02:00
ClosedPort22
0f0ded53ae
[e621] cache pool metadata results
2024-08-12 21:03:35 +08:00
enduser420
aa9e24ce0d
[batoto] update
2024-08-12 15:52:18 +05:30
Mike Fährmann
8cd625fe2b
merge #5983 : [pixiv] use mobile API for series
2024-08-12 11:53:37 +02:00
Mike Fährmann
b33a53968d
[pixiv] incorporate into '_pagination'
2024-08-12 11:50:41 +02:00
enduser420
bc43c35847
[bato] fix chapter info regex
2024-08-12 13:24:44 +05:30
Mike Fährmann
785e6f2911
[tumblr] fix 401 Unauthorized for likes when using api-key ( #5994 )
...
fixes regression introduced in 540eaa5a
2024-08-12 09:09:59 +02:00
Mike Fährmann
9db361e4cd
remove debug remains
2024-08-12 08:00:59 +02:00
Mike Fährmann
7f0f407d88
[gofile] fix "KeyError: 'childrenIds'" ( #5993 )
2024-08-11 15:26:58 +02:00
mix5003
b6a40217d6
[pixiv] use api for illust series
2024-08-10 20:29:13 +07:00
Mike Fährmann
c51475273f
release version 1.27.3
2024-08-10 13:05:57 +02:00
Mike Fährmann
d1fe81dab0
merge #5970 : [bunkr] support 'bunkr.ci' and 'bunkrrr.org'
2024-08-10 12:26:39 +02:00
Mike Fährmann
ad6ed72307
[bunkr] add tests for new domains
2024-08-10 12:25:43 +02:00
Mike Fährmann
8d1805e336
[hentaicosplays] support 'hentai-cosplay-xxx.com' ( #5959 )
2024-08-10 12:14:18 +02:00
Mike Fährmann
87a14a50e7
[zerochan] improve redirect handling, add 'redirects' option ( #5891 )
2024-08-10 11:32:30 +02:00
Mike Fährmann
6e7da6310c
[behance] fix video extraction ( #5965 )
...
a lot slower than before since each video now requires an extra HTTP
request and 'sleep-request' is set to 2s-4s by default.
it now also requires ytdl.
2024-08-10 11:06:54 +02:00
Larsluph
8e2a2bd221
[bunkr] Add support for more domains
...
Supported domains added:
- https://bunkr.ci
- https://bunkrrr.org
2024-08-08 12:38:32 -05:00
Mike Fährmann
9783d95585
[behance] fix "KeyError: 'fields'" ( #5965 )
2024-08-08 16:29:56 +02:00
Mike Fährmann
269b2808f1
[deviantart] fix "KeyError - 'category'" ( #5960 , #5961 )
...
'deviation' objects returned by the OAuth API no longer include a
'category' or 'category_path' value.
2024-08-08 09:34:51 +02:00
Mike Fährmann
c07a074418
[furaffinity] remove end of system messages
2024-08-07 18:39:34 +02:00
Mike Fährmann
846512f6cd
[furaffinity] add 'submissions' extractor ( #5954 )
2024-08-07 16:34:32 +02:00
Mike Fährmann
b4733b7ea8
[fanbox] fix pagination ( #5949 , #5951 , #5956 )
2024-08-07 10:16:14 +02:00
Mike Fährmann
605f6fba2c
[pp:metadata] implement 'base-directory' option ( #5262 , #5728 )
2024-08-07 10:15:06 +02:00
Mike Fährmann
34433f5a5d
[bunkr] fail downloads for 'maintenance' files ( #5952 )
2024-08-07 08:33:10 +02:00
Mike Fährmann
575e5df539
[hotleak] fix AttributeError ( #5950 )
...
fixes regression introduced in 0432e057
2024-08-07 08:26:23 +02:00
Mike Fährmann
ff4e53adac
[instagram] restore GraphQL API functionality ( #5920 )
...
by reducing the number of requested posts from 50 to 24
https://github.com/mikf/gallery-dl/issues/5920#issuecomment-2269712540
2024-08-07 08:17:59 +02:00
Mike Fährmann
21734ab69e
[twitter] update 'x-csrf-token' header during login ( #5945 )
2024-08-07 08:11:44 +02:00
Mike Fährmann
008f644ba3
[cien] fix extraction when authenticated ( #5934 )
...
'articleBody' doesn't always contain the entire article, it seems.
2024-08-03 21:57:18 +02:00
Mike Fährmann
5f6a8915af
release version 1.27.2
2024-08-03 19:22:13 +02:00
Mike Fährmann
bb07313227
[deviantart] try to work around journal/status API changes ( #5916 )
...
The new /user/profile/posts endpoint is documented to
"return all journals & status updates for a given user in a single feed"
but it does not do this.
There are usually several or even all status updates missing.
2024-08-03 13:17:27 +02:00
Mike Fährmann
f87783ff26
[cien] add 'files' option ( #2885 )
2024-08-03 11:37:14 +02:00
Mike Fährmann
f68b0efc1b
[wallpapercave] support 'album' listings ( #5925 )
2024-08-02 16:13:09 +02:00
Mike Fährmann
a5be680596
[directlink] extend recognized file extensions ( #5924 )
...
bmp, svg, avif, heic, psd, pdf, m4v, mov, wav, mp3, zip, rar, 7z, swf
2024-08-02 16:13:09 +02:00
Mike Fährmann
d9ff862fbd
[koharu] reduce login cache expiration to 24h ( #5893 )
2024-08-02 16:13:09 +02:00
Mike Fährmann
9be7896f78
[koharu] fix 'count' for 'cbz' downloads ( #5893 )
2024-08-02 16:13:03 +02:00
Mike Fährmann
d65fc2435c
[cien] extract 'gallery' images ( #2885 )
2024-08-01 14:01:21 +02:00
Mike Fährmann
1bdb0f7a49
[path] leave '..' segments as is ( #5728 )
2024-08-01 13:35:42 +02:00
Mike Fährmann
0432e05783
[hotleak] fix faulty image URLs ( #5915 )
2024-08-01 12:35:01 +02:00
Mike Fährmann
7d15ea6274
[instagram] fix 'AttributeError' in GraphQL pagination ( #5920 )
2024-08-01 12:35:01 +02:00
Mike Fährmann
c372242a06
[koharu] add 'favorite' extractor ( #5893 )
2024-08-01 12:33:18 +02:00
Mike Fährmann
2bf76461ce
[deviantart:following] use OAuth API endpoint ( #2511 )
2024-07-31 17:21:39 +02:00
Mike Fährmann
095f278d6f
[vsco] add 'include' option ( #5911 )
2024-07-31 12:32:04 +02:00
Mike Fährmann
ff58683b76
[koharu] send necessary headers for image downloads ( #5893 )
2024-07-30 19:39:50 +02:00
Mike Fährmann
399ba85841
[fallenangels] remove module
2024-07-30 17:33:16 +02:00
Mike Fährmann
84eefeebd6
[sankaku] match URLs with 'www' subdomain ( #5907 )
2024-07-30 17:05:22 +02:00
Mike Fährmann
279854cd9e
[agnph] implement 'tags' option ( #5284 )
2024-07-30 14:16:19 +02:00
Mike Fährmann
d2dda2bc00
[paheal] implement fast '--range' support ( #5905 )
2024-07-30 12:55:57 +02:00
Mike Fährmann
ca3e7a5e5a
[koharu] add 'date', simplify 'tags' ( #5893 )
2024-07-30 12:38:19 +02:00
Mike Fährmann
9c91935fe4
merge #5904 : [dynastyscans] extract chapter 'tags'
2024-07-29 21:17:13 +02:00
enduser420
5a36da2968
[dynastyscans] extract chapter 'tags'
2024-07-29 16:50:47 +05:30
Mike Fährmann
8a6e208605
[zerochan] fix 'Invalid control character' errors ( #5892 )
2024-07-29 11:24:17 +02:00
Mike Fährmann
aa6d00613f
[cien] initial support ( #2885 , #4103 , #5240 )
2024-07-28 19:27:12 +02:00
Mike Fährmann
65d7cccaf9
merge #5899 : [redgifs] support URLs with numeric IDs ( #5898 )
2024-07-28 12:35:57 +02:00
Mike Fährmann
c9aeedeafd
[koharu] add 'gallery' and 'search' extractors ( #5893 , #4707 )
2024-07-28 12:22:18 +02:00
reasonabledoubt
f8a7a61f67
Update redgifs.py to include 0-9 regex
2024-07-28 02:34:17 -04:00
Mike Fährmann
226ead728e
[agnph] add 'tag' and 'post' extractors ( #5284 , #5890 )
2024-07-27 12:17:47 +02:00
Mike Fährmann
304bb4bbb8
merge #5887 : [imagefap] add '{num}' to default filenames ( #1746 )
2024-07-27 09:26:48 +02:00
ShruggZoltan
20f670c3b6
Reduce default numbering to 4 digits of zero-padding.
2024-07-27 01:11:50 +00:00
Mike Fährmann
0fcd603498
[warosu] fix extraction
2024-07-26 21:09:07 +02:00
Mike Fährmann
d7a2c73274
[util] let a CustomNone instance be equal to itself
2024-07-26 20:56:01 +02:00
Mike Fährmann
8ecd408f53
add '-J/--resolve-json' command-line option ( #5864 )
2024-07-26 20:41:35 +02:00
Mike Fährmann
70f18b7a78
[zerochan] fix tag redirections ( #5891 )
2024-07-26 20:41:34 +02:00
Mike Fährmann
b5e141ed6e
[sankakucomplex] update domain to 'news.sankakucomplex.com'
2024-07-26 20:39:55 +02:00
ShruggZoltan
db2a61ba08
Numbers images obtained from imagefap by default.
2024-07-26 06:04:10 +00:00
Mike Fährmann
f321272b7c
[ytdl] fix --cookies-from-browser option parsing
2024-07-25 17:45:25 +02:00
Mike Fährmann
540eaa5add
[tumblr] implement 'pagination' option ( #5880 )
...
restore pagination behavior from before
de670bd7de
2024-07-23 20:31:04 +02:00
Mike Fährmann
7b445ec255
[readcomiconline] update ( #5866 )
2024-07-23 18:56:49 +02:00
Mike Fährmann
5207a0c2e0
[zerochan] implement 'tags' option ( #5874 )
...
allow splitting tags into separate lists by category
2024-07-23 10:21:33 +02:00
Mike Fährmann
1aadc29c5b
[zerochan] fix 'source' extraction
2024-07-23 09:34:44 +02:00
Mike Fährmann
ae40c61c21
[zerochan] fix tag category extraction ( #5874 )
2024-07-23 09:16:32 +02:00
Mike Fährmann
3b26748264
[inkbunny:following] fix infinite loop
...
happens when number of watched users is a multiple of 20
2024-07-22 17:59:58 +02:00
Mike Fährmann
db9833c28a
merge #5870 : [aryion] add 'favorite' extractor ( #4511 )
2024-07-21 12:37:47 +02:00
Mike Fährmann
156a70bec0
[aryion] update favorite extractor
...
- add test case
- add docs/supportedsites entry
- add custom directory_fmt and archive_fmt
- remove constructor
- appease flake8
2024-07-21 12:34:06 +02:00
Mike Fährmann
36a64a3aa7
[behance] fix image extraction ( #5873 )
2024-07-21 10:54:12 +02:00
Mike Fährmann
727e53f513
[bunkr] support 'bunkr.fi' URLs ( #5872 )
2024-07-21 06:41:50 +02:00
walkenjoyer
19e98ef8e9
[aryion] Add favorite extractor ( #4511 )
2024-07-20 18:49:59 +02:00
Mike Fährmann
a595949840
[8chan] update 'TOS' cookie name ( #5868 )
2024-07-20 02:15:29 +02:00
Mike Fährmann
fef80a2f55
[zerochan] fetch metadata for each post separately ( #5869 )
...
instead of processing all posts at once before returning any of them
2024-07-20 02:11:27 +02:00
Mike Fährmann
84a634fc14
[job] add 'resolve' argument to DataJob ( #5864 )
2024-07-19 14:32:42 +02:00
Mike Fährmann
287a7d13cf
[sankaku] implement 'notes' extraction ( #5865 )
2024-07-18 20:44:49 +02:00
Mike Fährmann
026e0b97db
merge #5824 : [furaffinity] add 'folders' and 'thumbnail' ( #1284 )
2024-07-18 01:41:15 +02:00
Mike Fährmann
9a8ec95398
[furaffinity] improve
2024-07-18 01:39:58 +02:00
Mike Fährmann
6940ad0e72
[booru] allow multiple 'url' keys ( #5859 )
2024-07-17 20:49:52 +02:00
Mike Fährmann
bf44add132
[booru] prevent crash when file URL is empty ( #5859 )
...
- catch all exceptions, including IndexError, and not just a select few
- change logging message level to warning
2024-07-17 20:20:17 +02:00
Mike Fährmann
51fd14f87d
[gelbooru_v02] use total number of posts as end marker ( #5830 )
...
… and potentially retry on empty responses
2024-07-12 22:51:46 +02:00
Mike Fährmann
6110e3f940
[common] fix Logger names of BaseCategory extractors
...
update of d11ec0090c178846
2024-07-12 22:51:46 +02:00
Mike Fährmann
668219b36e
[exhentai] improve error message when temporarily banned ( #5845 )
2024-07-12 22:51:46 +02:00
Mike Fährmann
f7383a56f8
wrap filters/conditionals in a try-except block
...
allows accessing undefined fields without exception or locals().get(…)
but hides mistakes/typos/etc by evaluating to False without feedback
performance loss compared to the previous version without try-except
is negligible (~20ns for me)
2024-07-12 22:51:11 +02:00
Mike Fährmann
c83c812a1e
[instagram][twitter] rename 'profile' to 'info' ( #5262 , #3623 )
2024-07-11 00:22:39 +02:00
Mike Fährmann
16e276fca4
[nijie] fix image URLs for single image posts ( #5842 )
...
fixes regression introduced in 2e11b6e7
2024-07-10 15:06:25 +02:00
Mike Fährmann
eb3ef13d28
include 'zstd' in Accept-Encoding header when supported
...
… and slightly update optional dependency list
2024-07-10 00:33:35 +02:00
Mike Fährmann
8aca0e6970
update default User-Agent header to Firefox 128 ESR
2024-07-09 20:42:06 +02:00
Mike Fährmann
8e747b6dee
[twitter] send initial 'cursor' only when given via option
2024-07-09 20:42:06 +02:00
Mike Fährmann
ff39d28ef4
[twitter] fix 'user' when providing 'cursor' ( #5833 )
2024-07-08 23:16:47 +02:00
Mike Fährmann
65385e09cb
[twitter] remove break ( #5831 )
2024-07-07 01:08:30 +02:00
Mike Fährmann
1656c9287d
[pp:metadata] fix 'directory' when file directory has conditions ( #5728 )
2024-07-06 22:07:17 +02:00
Mike Fährmann
da9916c01f
[pp:metadata] implement format strings for 'directory' ( #5728 )
2024-07-06 03:08:59 +02:00
Mike Fährmann
8f3f061daf
[hentainexus] fix error for spread pages ( #5827 )
2024-07-05 21:36:29 +02:00
Mike Fährmann
b3aded1939
add '--print-traffic'
2024-07-05 21:34:33 +02:00
Mike Fährmann
b376fa814e
[zerochan] handle "KeyError - 'items'" ( #5826 )
...
Zerochan sometimes sends an empty response when there are no more
accessible posts to be had.
2024-07-05 21:34:33 +02:00
Mike Fährmann
97a50a23d2
[twitter] implement 'cursor' support ( #5753 )
2024-07-05 00:03:02 +02:00
Nicholas Bishop
4f3be2f571
Remove leftover split() statement
...
Forgot to remove this from a previous implementation attempt
2024-07-04 16:50:49 -04:00
Mike Fährmann
162756b684
[toyhouse] fix Content Warning bypass ( #5820 )
2024-07-04 22:42:36 +02:00
Nicholas Bishop
f43bccb5be
[furaffinity] Add 'thumbnail' ( #1284 ) and 'folders' properties
...
Retrieve 'thumbnail' and 'folders' properties for each post.
'thumbnail' (#1284 ):
- Preview image used for search results, writing posts, music, etc.
- Filename format: <post_id>@600-<directory_containing_full_image>.jpg
'Folders' (related to #1817 ):
- A list of all gallery folders containing this post
- Folder name format: [<folder_category> - ]<folder_name>
- Only works on new layout; old layout does not show folders, so list will be empty
A test is included for each property.
2024-07-04 15:41:14 -04:00
Mike Fährmann
e03b99ba0e
[actions] emit logging messages before waiting/exiting/etc
...
distinguish between actions that should run before or after
a logging message gets emitted
2024-07-01 01:19:25 +02:00
Mike Fährmann
f7a6401031
[actions] move LoggerAdapter from 'output' to 'actions'
2024-06-30 20:41:51 +02:00
Mike Fährmann
f41a5065b2
[actions] support multiple actions per pattern
2024-06-30 02:36:33 +02:00
Mike Fährmann
8219a78b8d
[actions] extend 'wait'
...
allow specifying a duration like for --sleep
2024-06-29 23:58:38 +02:00
Mike Fährmann
84b3da54aa
[actions] add 'exec' action ( #5619 )
2024-06-29 23:02:48 +02:00
Mike Fährmann
c9860002ba
[actions] add 'abort' and 'terminate' actions ( #5778 )
2024-06-29 19:20:17 +02:00
Mike Fährmann
d80f4fbc10
[actions] optimize empty patterns
2024-06-29 18:57:48 +02:00
Mike Fährmann
d10bfa9065
[vipergirls] improve 'thread' URL pattern
...
allow for query parameters and fragments at tne end of URLs
2024-06-29 17:38:57 +02:00
Mike Fährmann
ba51c1a8e4
[path] fix 1ca2cf51
2024-06-29 02:41:56 +02:00
Mike Fährmann
1ca2cf51a2
merge #5807 : [path] fix moving files across drives on Windows
2024-06-29 02:37:42 +02:00
AyluinReymaer
de61b3b546
Update path.py
2024-06-29 00:58:03 +01:00
AyluinReymaer
91a989a172
path.py - fix system cannot move file to different drive
...
This fixes the below error message:
OSError(18, 'The system cannot move the file to a different disk drive')
2024-06-29 00:08:53 +01:00
Mike Fährmann
29eb535e18
merge #5802 : [twitter] extract 'bookmark_count' and 'view_count'
2024-06-28 23:21:57 +02:00
rameerez
ee93dfeb8d
Add bookmark_count and view_count to the X / Twitter extractor
...
- Parse `view_count` as integer in Twitter extractor
- Style fix
- optimize 'view_count' extraction
2024-06-28 23:17:11 +02:00
Mike Fährmann
c2bf4ff99c
[twitter] add 'profile' extractor ( #3623 )
2024-06-28 22:55:48 +02:00
Mike Fährmann
44896b0296
[instagram] add 'profile' extractor ( #5262 )
...
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-2188915210
2024-06-28 22:54:07 +02:00
Mike Fährmann
ea81fa985f
[archive] implement 'archive-event' option ( #5784 )
...
With this, IDs of skipped files will no longer be written to an archive
by default. Use "archive-event": "file,skip" to restore the previous
behavior.
2024-06-27 22:00:59 +02:00
Mike Fährmann
51fdfbe6fc
[erome] extract 'date' metadata ( #5796 )
2024-06-27 21:07:23 +02:00
Mike Fährmann
9329c30b41
merge #5783 : [subscribestar] fix 'date' parsing in updated posts
2024-06-27 18:12:05 +02:00
Mike Fährmann
ea7ccb55a1
merge #5781 : [cookies] add 'thorium' support
2024-06-27 17:58:39 +02:00
joskezelensky
b2a40a5b93
add thorium cookies support
...
- Update cookies.py
- preserve lexicographical order
2024-06-27 17:56:53 +02:00
Arased
03486599af
Fix subscribestar date parsing in udated posts
2024-06-24 16:40:59 +02:00
Mike Fährmann
0bb8831853
[twitter] fix 'username-alt' option name ( #5715 )
2024-06-22 17:49:58 +02:00
Mike Fährmann
8bb793e21b
release version 1.27.1
2024-06-22 17:11:44 +02:00
Mike Fährmann
606202fb04
[fanbox] remove debug code ( #5759 )
2024-06-21 21:05:01 +02:00
Mike Fährmann
8f50c04af2
[formatter] implement 'X' format specifier ( #5770 )
2024-06-21 20:56:19 +02:00
Mike Fährmann
b707b31264
merge #5774 : [tcbscans] support other domains
2024-06-21 19:06:11 +02:00
Mike Fährmann
ad25be3420
[fanbox] fix 8452d04a ( #5759 )
...
The previous code would raise an exception when there are no or
more than 1 plans with a fee higher than 'feeRequired'.
2024-06-21 18:56:43 +02:00
enduser420
5adbfe526d
[tcbscans] support other domains
2024-06-21 21:29:11 +05:30
Mike Fährmann
f58b0e6fc7
[twitter] ignore 'Unavailable' media ( #5736 )
...
… including geo-restricted content.
add 'unavailable' option to allow re-enabling them again
2024-06-21 00:15:10 +02:00
Mike Fährmann
8452d04a33
[fanbox] handle KeyError for no longer existing plans ( #5759 )
...
return the plan of the next higher tier instead
2024-06-21 00:13:25 +02:00
Mike Fährmann
11421cf940
[skeb] fix '429 Too Many Requests' errors ( #5766 )
...
Introduce '_handle_429' method to make it easier for Extractors to react
to 429 errors regardless of 'sleep-429' settings.
2024-06-21 00:12:05 +02:00
Mike Fährmann
60b4541199
improve a1bb3279, fix oauth:pixiv ( #5757 )
...
Check 'input' option only when required.
This also fixes an exception in oauth:pixiv caused by using the same
'_input' name as a method defined there.
2024-06-18 16:50:04 +02:00
Mike Fährmann
9671bd6d40
[actions] fix exception when 'msg' is not a string ( #5683 )
2024-06-18 16:49:45 +02:00
Mike Fährmann
4727ad6f57
[pp:ugoira] include traceback in exception debug log ( #5683 )
2024-06-16 18:42:32 +02:00
Mike Fährmann
a1bb32792b
do not try to read from stdin when it is non-interactive ( #5733 )
...
add '--no-input' command-line option and 'input' config file option
to allow users to manually configure this
2024-06-16 18:31:39 +02:00
Mike Fährmann
76385f5f18
re-categorize some command-line options in --help
2024-06-15 23:29:09 +02:00
Mike Fährmann
ae3706286a
[speakerdeck] inherit from GalleryExtractor
2024-06-15 21:56:51 +02:00
Mike Fährmann
9c65db2a92
consistent 'with open(…) as fp:' syntax
2024-06-14 01:22:00 +02:00
Mike Fährmann
3fc2e61818
add '--config-open' and '--config-status' ( #5713 )
...
adapted from 119a25a59c
2024-06-13 23:22:56 +02:00
Mike Fährmann
a92c0bef96
merge #5730 : [speakerdeck] fix extraction
2024-06-13 15:55:57 +02:00
Yuta Hayashibe
f272b5d2ac
fix: extractor for speakerdeck.com
2024-06-13 15:54:54 +02:00
Mike Fährmann
c699ce8ebb
[twitter] add 'username-alt' option ( #5715 )
2024-06-13 00:54:43 +02:00
Mike Fährmann
c6fc0281e8
[newgrounds] extend 'format' option ( #5709 )
...
- check more extensions for original formats (mp4, webm, m4v, mov, mkv)
- allow specifying which extensions and recoded formats to check
2024-06-12 20:46:45 +02:00
Mike Fährmann
86f0c3baaf
[szurubooru] support empty tag searches ( #5711 )
2024-06-11 20:25:06 +02:00
Mike Fährmann
40c685dc43
merge #5712 : [hentainexus] update key
2024-06-10 19:58:03 +02:00
KageNiHisomu
87859ff200
fix: static method defines hostname, update url link
2024-06-10 14:35:03 +10:00
KageNiHisomu
68540aef2a
update nexus key
2024-06-10 14:25:51 +10:00
Mike Fährmann
2e11b6e756
[nijie] support downloading videos ( #5707 , #5617 )
2024-06-08 22:55:28 +02:00
Mike Fährmann
e17b1d5073
[deviantart] fix 2c8aa9b2 ( #5696 )
2024-06-08 02:00:36 +02:00
Mike Fährmann
f160859c5c
[hitomi] extract 'title_jpn' metadata ( #5706 )
2024-06-08 00:05:19 +02:00
Mike Fährmann
722b4b513d
merge #5609 : [instagram] extract 'liked' metadata
2024-06-07 01:43:13 +02:00
Mike Fährmann
7b0c07ee69
[instagram] rename to 'liked', enable for GraphQL results
2024-06-07 01:41:56 +02:00
Mike Fährmann
aa9be75d44
[twitter] fix duplicate ArkoseLogin check
...
forgot to replace this in 9e5d65fbf3
2024-06-06 19:44:42 +02:00
Mike Fährmann
2c8aa9b2de
[deviantart] fix watching module ID extraction ( #5696 )
2024-06-06 15:21:04 +02:00
Mike Fährmann
9abeab5ecf
[shimmie2] support 'vidya.pics' ( #5632 )
2024-06-06 15:08:56 +02:00
Mike Fährmann
162d4269ec
[twitter] extend 'ratelimit' option ( #5532 )
...
allow waiting for a set amount of seconds
2024-06-06 01:18:08 +02:00
Mike Fährmann
2cab87c5b6
[kemonoparty:favorite] fix exception when sorting None ( #5692 )
2024-06-05 16:49:30 +02:00
Mike Fährmann
7614bc458e
[util] extend CustomNone with comparison operators
2024-06-05 16:49:30 +02:00
Mike Fährmann
1ce5de0290
[formatter] implement 'C' format specifier ( #5647 )
...
to apply a conversion after ':' or
to apply multiple conversions
for example {tags:CSl} or {tags:J - /Cl}
to convert list to string and lowercase it
2024-06-05 16:49:29 +02:00
Mike Fährmann
9b99d2c886
[philomena] support downloading SVG files ( #5643 )
2024-06-05 16:48:51 +02:00
Mike Fährmann
8fce9ea6d5
[hentainexus] restore module ( #5275 )
...
revert 97641cd151
2024-06-05 16:48:25 +02:00
Mike Fährmann
3ec7ee6baa
[downloader:ytdl] fix exception due to missing 'ext' ( #5675 )
2024-06-02 18:16:53 +02:00
Mike Fährmann
5d3d03a1f1
fix 6cfbc107
...
the former condition would return True for 2.31.*
6cfbc1071f (commitcomment-142642913)
2024-06-02 18:16:53 +02:00
Mike Fährmann
4d11cd9ffb
[vichan] remove 'wikieat.club'
...
redirects to some non-vichan site
2024-06-02 18:16:24 +02:00
Mike Fährmann
31133b97fb
[nitter] remove instances
2024-06-02 18:15:53 +02:00
Mike Fährmann
ce228ee163
[photobucket] remove module
...
had been broken for years and the new site is payed access only
2024-06-02 01:40:31 +02:00
Mike Fährmann
612435480a
[workflows:tests] restore Python 3.5 usage
...
https://github.com/actions/setup-python/issues/866
2024-06-02 01:32:54 +02:00
Mike Fährmann
31bdb288ef
release version 1.27.0
2024-06-01 21:06:12 +02:00
Mike Fährmann
6cfbc1071f
workaround for requests 2.32.3 ( #5665 )
...
manually call 'load_default_certs()' for SSLContexts
in custom HTTPAdapter instances
2024-06-01 16:02:18 +02:00
Mike Fährmann
807e2f7094
[realbooru] fix videos and provide fallback URLs ( #2530 )
...
revert acc94ac187
2024-05-31 23:58:40 +02:00
Mike Fährmann
deacbcbfdf
[tbscans] remove excess whitespace in 'manga'
2024-05-31 22:32:59 +02:00
Mike Fährmann
00440b8fb1
[slideshare] fix extraction
2024-05-31 21:05:50 +02:00
Mike Fährmann
281ab33f87
[4archive] fix extraction
2024-05-31 17:42:53 +02:00
Mike Fährmann
a53db09ca0
[artstation] disable TLS 1.2 ciphers by default ( #5564 , #5658 )
2024-05-30 23:54:19 +02:00
Mike Fährmann
7ba2b209f1
fix exit status of --clear-cache/--list-modules/--list-extractors
...
All three would always cause gallery-dl to exit with status 1,
regardless of error or success.
2024-05-29 02:56:25 +02:00
Mike Fährmann
3d033a48ea
[workflows:executables] change dev/linux binary name to …_linux
...
This reverts commit 05f8435e09
2024-05-29 02:46:29 +02:00
Mike Fährmann
53aadb0c44
[update] implement '--update-to'
...
to switch between update channels (stable or dev (alias nightly & master))
or to install a specific version
Examples:
--update-to dev
--update-to dev@2024.05.25
--update-to v1.25.2
2024-05-28 20:40:03 +02:00
Mike Fährmann
5018bd42f8
[update] implement '--update-check'
...
to check if a new update is available without downloading it
2024-05-28 03:42:58 +02:00
Mike Fährmann
2e323ef0a2
[update] implement '-U/--update'
...
to auto-update executable files in-place
2024-05-28 02:26:13 +02:00
Mike Fährmann
601f5becc8
[version] add __variant__
...
Specifies origin and OS of executable files.
For example 'stable/windows'.
2024-05-27 21:37:01 +02:00
Mike Fährmann
020050ea8b
merge #5641 : [pixeldrain] add support for single file album download
2024-05-25 23:43:43 +02:00
Mike Fährmann
154a890399
[pixeldrain] integrate into 'album' extractor
2024-05-25 23:42:23 +02:00
HornyQT
24e70b956b
[pixeldrain] add support for single file album download
2024-05-25 16:06:50 +02:00
Mike Fährmann
0761b22a7f
[hiperdex] update domain to 'hiperdex.top' ( #5635 )
2024-05-24 17:13:10 +02:00
Mike Fährmann
ea434963ae
[subscribestar] fix file URLs ( #5631 )
2024-05-23 19:12:01 +02:00
Mike Fährmann
6198dfa424
[seiga] re-implement login with username & password
2024-05-23 18:52:53 +02:00
Mike Fährmann
329da4db5e
[newgrounds] update and fix login procedure ( #5109 )
2024-05-23 17:06:22 +02:00
Mike Fährmann
895e633c44
implement 'keywords-eval' option ( #5621 )
...
to allow evaluating 'keywords' values as format strings
2024-05-22 22:53:34 +02:00
Mike Fährmann
bc25636e63
[patreon] fix bootstrap data extraction ( #5624 )
...
again
2024-05-22 00:17:49 +02:00
Mike Fährmann
141efc2ad3
[twitter] transfer 'twitter.com' cookies to 'x.com' ( #5597 )
2024-05-21 23:46:42 +02:00
Jan Wikholm
ece3837408
Enable user_has_liked as IG download filter
...
Background and context:
1. Instagram by default requires you to be logged in.
2. Your account can LIKE posts.
3. Getting all of your likes from IG can be a pain.
As an active IG user, I want to be able to download only posts I've liked.
The information of whether or not your logged-in user has liked the target post is available in the API results, but was not available to the end-user for filtering.
This small change allows `user_has_liked` to be used to e.g. download only liked posts of an entire IG account.
Example use:
```
$ gallery-dl --cookies-from-browser FIREFOX--filter 'user_has_liked' https://www.instagram.com/imgur/
```
This will only download the posts that have `has_liked: True` set in the returned REST API data.
Note: it will still need to loop through the entire account's posts since this is done per-post basis instead of looping just the liked posts, but at least you are able to use this IG feature as a download marker.
2024-05-18 12:05:08 +03:00
Mike Fährmann
6276f5cd94
[twitter] update domain to x.com ( #5597 )
2024-05-17 22:59:52 +02:00
Mike Fährmann
bfcb66af05
[poipiku] send LANG cookie to ensure English UI ( #5590 )
2024-05-17 22:59:31 +02:00
Mike Fährmann
f651b3b6ab
merge #5601 : [twitter] match '/video/' Tweet URLs
2024-05-17 22:49:12 +02:00
Mike Fährmann
2dec4bef60
merge #5591 : [poipoku] avoid language-specific extraction
2024-05-17 22:47:51 +02:00
Mike Fährmann
7f1ed909d5
[imgur] match gallery/album/image URLs with title slugs ( #5593 )
2024-05-17 22:44:37 +02:00
Delphox
8ba73e2ec9
[twitter] match /video/ tweet urls
2024-05-17 16:50:51 -03:00
fireattack
831f922c1c
[poipoku] avoid language-specific extr
2024-05-14 02:42:02 +08:00
Mike Fährmann
33b07c4603
[twitter] wait for rate limit reset before 429 error ( #5532 )
2024-05-12 01:13:30 +02:00
Mike Fährmann
3b6f306a8b
[exhentai] fix blank page detection
2024-05-11 22:01:34 +02:00
Mike Fährmann
90b3741643
[cookies] set proper 'expires' value for Chrome session cookies
...
https://github.com/yt-dlp/yt-dlp/pull/9747
2024-05-11 19:46:35 +02:00
Mike Fährmann
f178839fc2
[exhentai] fix multi-page viewer detection ( #4969 )
2024-05-11 18:32:52 +02:00
Mike Fährmann
5b6b5dd81b
[8chan] fix downloaded files by sending 'TOS' cookie ( #5578 )
2024-05-11 15:34:21 +02:00
Mike Fährmann
d2f50ecf09
add 'skip-filter' option ( #5255 )
2024-05-10 22:59:52 +02:00
Mike Fährmann
fd734b9222
[archive] add 'archive-mode' option ( #5255 )
2024-05-10 22:59:51 +02:00
Mike Fährmann
28039229fe
[common] use 'create_urllib3_context' for creating SSLContexts
...
enables dumping TLS session keys by setting SSLKEYLOGFILE (#5215 )
as well as other potentially useful settings.
2024-05-10 22:59:29 +02:00
Mike Fährmann
33006fe126
[common] disable 'check_hostname' for non-urllib3 SSLContexts
...
e.g. when 'browser' is set to a non-empty value and gallery-dl creates
its own SSLContext instance instead of using requests' and urllib3's
defaults.
urllib3 disables this option for its default contexts,
since it does this check on its own.
Fixes "ValueError: Cannot set verify_mode to CERT_NONE when
check_hostname is enabled" when using --no-check-certificate.
(#3614 , #4891 , #5576 )
2024-05-10 18:20:08 +02:00
Mike Fährmann
215abbc3e4
[archive] implement DownloadArchiveMemory class ( #5255 )
...
keeps archive IDs in memory
and only writes them to disk in a 'finalize' step
2024-05-10 01:15:44 +02:00
Mike Fährmann
88f94190f4
[archive] move DownloadArchive into its own module
2024-05-10 01:05:28 +02:00
Mike Fährmann
2ee9ffeed6
merge #5568 : [furaffinity] match 'xfuraffinity' URLs
2024-05-09 19:20:12 +02:00
Mike Fährmann
b27ffd790a
[poipiku] fix downloading R-18 posts ( #5567 )
...
… by automatically sending a `POIPIKU_CONTENTS_VIEW_MODE=1` cookie
to enable "adult" mode.
2024-05-09 15:14:08 +02:00
Delphox
11109d5bad
[furaffinity] match xfuraffinity.com
2024-05-08 12:15:47 -03:00
Mike Fährmann
dc9d83e64b
[output] support 'NO_COLOR' environment variable
2024-05-03 12:25:41 +02:00
Mike Fährmann
625740c3d2
merge #5538 : [cookies] optimize _find_most_recently_used_file for exact profiles
2024-05-02 18:19:19 +02:00
Jan Wikholm
06d102f19a
optimize _find_most_recently_used_file for exact profile
...
When reading cookies from the browser, the user is able to give either just the browser name, or also provide profile/container information.
If an exact profile is provided, there is no need to find the latest profile with `os.walk` which is very expensive.
This change optimizes that case and the performance increase is significant (~8 sec to 0.6 sec).
```
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
./imgur_OO4UNqJ.jpg
real 0m8.429s
user 0m0.216s
sys 0m0.431s
$ time gallery-dl --config-ignore -d . -D . --cookies-from-browser FIREFOX:bgamf5r6.default-release https://imgur.com/OO4UNqJ
[cookies][info] Extracted 16 cookies from Firefox
./imgur_OO4UNqJ.jpg
real 0m0.456s
user 0m0.183s
sys 0m0.011s
$ gallery-dl --version
1.26.9
```
2024-05-02 18:16:13 +02:00
Mike Fährmann
bd8e4797e5
[vsco] add 'avatar' extractor ( #5341 )
2024-05-02 18:12:19 +02:00
Mike Fährmann
d0cead105b
[formatter] allow dots etc in '…' literals ( #5539 )
...
don't parse fields starting with '
this disables the ability to directly apply […] to '…' literals,
but that's not really useful anyway and can still be done with _lit
2024-05-02 17:43:24 +02:00
Mike Fährmann
619bf5c644
[inkbunny] retry API calls with a loop instead of recursion
2024-05-01 22:46:24 +02:00
Mike Fährmann
d11ec00908
[common] fix _cfgpath for BaseExtractor objects
...
After the changes in 0c17884673
2024-05-01 16:00:07 +02:00
Mike Fährmann
6db1837a6d
[bluesky] filter reposts only for user timelines ( #5528 )
2024-05-01 16:00:07 +02:00
Mike Fährmann
3cf5366143
[mastodon] add support for card images
2024-05-01 16:00:07 +02:00
