blankie
df718887c2
[webtoons] fix extracting comic and episode name with commas
2024-01-21 09:50:27 +11:00
Mike Fährmann
0d367ce1b9
[tests] update extractor results
2024-01-20 18:02:36 +01:00
Mike Fährmann
9ca6117c67
[hbrowse] remove module
...
website gone
2024-01-20 02:53:44 +01:00
Mike Fährmann
375eefb886
[chevereto] remove 'pixl.li'
...
"Pixl is closing down"
"All images will be deleted January 1st."
2024-01-20 02:21:40 +01:00
Mike Fährmann
b0a441f1e3
[nitter] remove 'nitter.lacontrevoie.fr'
...
"Fermeture de Nitter / Closing down Nitter"
2024-01-19 19:34:16 +01:00
Mike Fährmann
a1c1e80f67
[giantessbooru] update domain
2024-01-19 14:21:56 +01:00
Mike Fährmann
2007cb2f59
[tests] check extractor category values
2024-01-19 14:21:09 +01:00
Mike Fährmann
93b4120e77
[gelbooru] support 'all' and empty tag ( #5076 )
2024-01-18 21:49:33 +01:00
Mike Fährmann
a416d4c3d5
[sankaku] support post URLs with alphanumeric IDs ( #5073 )
2024-01-18 16:23:14 +01:00
Mike Fährmann
ea553a1d55
[wikimedia] generalize ( #1443 )
...
- support mediawiki.org
- support mariowiki.com (#3660 )
- combine code into a single extractor
(use prefix as subcategory)
- handle non-wiki instances
- unescape titles
2024-01-18 15:36:16 +01:00
Mike Fährmann
c3c1635ef3
[wikimedia] update
...
- rewrite using BaseExtractor
- support most Wiki* domains
- update docs/supportedsites
- add tests
2024-01-17 00:08:06 +01:00
Mike Fährmann
3d68eda4ab
[kemonoparty] add 'revision_hash' metadata ( #4706 , #4727 , #5013 )
...
A SHA1 hexdigest of other relevant metadata fields like
title, content, file and attachment URLs.
This value does NOT reflect which revisions are listed on the website.
Neither does 'edited' or any other metadata field (combinations).
2024-01-16 00:38:10 +01:00
Mike Fährmann
799a8206ad
merge #5061 : [webtoons] extract more metadata
...
- author_name
- comic_name
- episode_name
- username
2024-01-15 18:27:12 +01:00
Mike Fährmann
8ffa0cd3c8
[webtoons] small optimization
...
don't extract the entire 'author_area' and
avoid creating a second 'text.extract_from()' object
2024-01-15 18:24:47 +01:00
Mike Fährmann
68196589c4
[2ch] update
...
- simplify extractor code
- more metadata
- add tests
2024-01-15 04:09:05 +01:00
Mike Fährmann
69726fc82c
[tests] skip tests requiring auth when non is provided
2024-01-14 22:47:16 +01:00
blankie
bb446b1598
[webtoons] extract more metadata
2024-01-14 19:26:49 +11:00
Mike Fährmann
355b909f46
merge #5041 : [steamgriddb] add support ( #5033 )
2024-01-13 00:59:15 +01:00
Mike Fährmann
71e2c3e5a2
merge #5037 : [hatenablog] add support ( #5036 )
2024-01-13 00:57:21 +01:00
Mike Fährmann
b97af09e03
[tests] include URL in failure report
2024-01-12 03:23:21 +01:00
Mike Fährmann
58e0665fbc
[tests] load config from external file
2024-01-12 03:21:44 +01:00
Mike Fährmann
2dcfb012ea
[patreon] download 'm3u8' manifests with ytdl
2024-01-12 02:33:27 +01:00
Mike Fährmann
2191e29e14
[nijie] fix image URL for single image posts ( #5049 )
2024-01-11 05:07:38 +01:00
Mike Fährmann
39904c9e4e
[deviantart:avatar] add 'formats' option ( #4995 )
2024-01-10 17:13:34 +01:00
Mike Fährmann
887ade30a5
[batoto] support more mirror domains ( #5042 )
2024-01-09 18:02:49 +01:00
blankie
2ccb7d3bd3
[steamgriddb] add support
2024-01-09 17:12:56 +11:00
blankie
2cfe788f93
[hatenablog] fix extractor naming errors
2024-01-09 01:42:57 +11:00
blankie
61f3b2f820
[hatenablog] add support
2024-01-09 01:29:47 +11:00
Mike Fährmann
657ed93a22
[batoto] improve v2 manga URL pattern
...
and add tests
2024-01-07 22:23:30 +01:00
Mike Fährmann
33f228756a
[mangadex] add 'list' extractor ( #5025 )
...
supports listing manga and chapters from list feed
2024-01-07 02:59:35 +01:00
Mike Fährmann
c25bdbae91
[komikcast] fix 'manga' extractor ( #5027 )
2024-01-06 14:19:44 +01:00
Mike Fährmann
8e1a2b5446
[komikcast] update domain to 'komikcast.lol' ( #5027 )
2024-01-06 02:16:43 +01:00
Mike Fährmann
a441249ea2
merge #4979 : [batoto] add 'chapter' and 'manga' extractors ( #1434 , #2111 )
2024-01-06 01:53:26 +01:00
Mike Fährmann
b11c352d66
[bato] rename to 'batoto'
...
to use the same category name as the previous bato.to site
2024-01-06 01:49:34 +01:00
Mike Fährmann
3aa24c3744
[bato] simplify and update
2024-01-06 01:10:04 +01:00
Mike Fährmann
11150a7d72
[nudecollect] remove module
2024-01-05 21:32:04 +01:00
Mike Fährmann
c158927c38
merge #5016 : [zzup] add 'gallery' extractor ( #4517 , #4604 , #4659 , #4863 )
2024-01-05 21:25:46 +01:00
Mike Fährmann
217fa7f8a1
include 'test/results' in flake8 checks
2024-01-05 18:16:33 +01:00
Mike Fährmann
e61f016465
[szurubooru] support 'snootbooru.com' ( #5023 )
2024-01-05 17:56:39 +01:00
Mike Fährmann
b4bcf40278
[weibo] fix AttributeError in 'user' extractor ( #5022 )
...
yet another bug caused by a383eca7
2024-01-05 17:18:33 +01:00
Mike Fährmann
0ab0a10d2d
[jpgfish] update domain
2024-01-05 02:27:20 +01:00
enduser420
0f30136109
[zzup] add 'gallery' extractor
2024-01-04 21:38:59 +05:30
Mike Fährmann
7eaf648f2e
[fanbox] add 'metadata' option ( #4921 )
...
extracts 'plan' and extended 'user' metadata
2024-01-04 15:01:33 +01:00
Mike Fährmann
4f3671458e
[deviantart] add 'avatar' and 'background' extractors ( #4995 )
2024-01-03 00:07:55 +01:00
Mike Fährmann
63f649cd92
[idolcomplex] fix extraction & update URL patterns ( #5002 )
2024-01-01 17:38:32 +01:00
Mike Fährmann
7aa1c9671b
[tests] fix 'invalid escape sequence' warnings
2024-01-01 16:12:43 +01:00
Mike Fährmann
b6903a4c90
[nijie] add 'count' metadata field
...
https://github.com/mikf/gallery-dl/issues/146#issuecomment-1812849102
2023-12-30 22:25:59 +01:00
Mike Fährmann
b93b351db9
merge #4962 : [poringa] add support ( #4675 )
2023-12-30 20:39:35 +01:00
Mike Fährmann
9f21c839ad
[poringa] improvements and fixes
...
- add 'num' and 'count' metadata fields
- prevent crash for "private" posts
- prevent crash when there's no 'main-info'
- update tests
2023-12-30 20:37:09 +01:00
Mike Fährmann
caceb14fc2
[tests] fail when a results file contains syntax errors
...
or is otherwise not importable
2023-12-30 17:26:57 +01:00
Mike Fährmann
085411f3f1
[rule34] recognize URLs with 'www' subdomain ( #4984 )
2023-12-30 16:07:56 +01:00
Antonio
e348da7a06
[poringa] add support
2023-12-27 00:07:23 -06:00
bug-assassin
74c225f94e
[bato] add support
2023-12-26 22:33:33 -05:00
Mike Fährmann
f9544194c0
[paheal] restore 'extension' metadata ( #4976 )
2023-12-26 16:09:26 +01:00
Mike Fährmann
77d46e6f0c
[lynxchan] update 'bbw-chan' domain ( #4970 )
2023-12-25 15:29:05 +01:00
Mike Fährmann
108c978073
merge #4919 : [postmill] add support ( #4917 )
2023-12-23 15:23:56 +01:00
Mike Fährmann
2a60645095
[deviantart] set 'is_original' for intermediary URLs to 'false'
2023-12-22 14:49:10 +01:00
Mike Fährmann
01bb75f6cb
merge #4945 : {shimmie2[ support 'rule34hentai.net' ( #861 , #4789 )
2023-12-22 00:10:26 +01:00
Mike Fährmann
79e4606893
[rule34hentai] cleanup
...
- fix using 'self._posts_rule34hentai'
- fix 'file_url' for posts
- update docs/supportedsites
- add tests
2023-12-22 00:01:36 +01:00
Mike Fährmann
627ed794a2
[danbooru] provide 'tags' as list ( #4942 )
...
keep the old 'tag_string' values around, similar to sankaku
a lot of repeat code ...
would be a lot less bad if "".split(" ") returned an empty list
2023-12-21 14:39:38 +01:00
Mike Fährmann
99aa923322
[inkbunny] improve '/submissionsviewall.php' patterns ( #4934 )
...
allow 'mode=…' to be in any position
don't require it to be somewhere in the middle
2023-12-16 19:21:20 +01:00
Mike Fährmann
3f9c113d78
[mastodon] Support non-numeric status IDs ( #4936 )
2023-12-16 01:52:31 +01:00
Mike Fährmann
2852404e49
[inkbunny] add 'unread' extractor ( #4934 )
2023-12-15 21:20:12 +01:00
Mike Fährmann
a37b7759bc
[myhentaigallery] recognize '/g/' URLs ( #4920 )
2023-12-12 20:02:28 +01:00
blankie
fbe14a2745
[postmill] add support
2023-12-12 21:36:52 +11:00
Mike Fährmann
bf74eb5c46
merge #4886 : [urlgalleries] add 'gallery' extractor ( #919 , #1184 , #2905 )
2023-12-08 22:55:58 +01:00
Mike Fährmann
ade93c5397
[urlgalleries] add tests
2023-12-08 22:55:16 +01:00
Mike Fährmann
4eb3590103
[nijie] fix image URLs of multi-image posts ( #4876 )
2023-12-05 17:48:50 +01:00
Mike Fährmann
c83fbe6c2d
merge #4855 : [nitter] fix video extraction ( #4853 )
2023-11-27 18:39:05 +01:00
Mike Fährmann
1137d72d48
[tests] skip test_init for BaseExtractor classes without instances
2023-11-27 18:36:15 +01:00
Mike Fährmann
625e94fa7d
update extractor test results
...
still not everything, but good enough for now
2023-11-27 18:30:53 +01:00
enduser420
1e9bacd169
[nitter] fix video extraction
2023-11-27 21:58:06 +05:30
Mike Fährmann
95c1dfb089
[tests] swap assertEqual argument order
...
before this, it would show test failures as
+ test value
- extracted value
when it should be the other way round
2023-11-27 01:06:13 +01:00
Mike Fährmann
bdb3ce7217
[foolslide] remove 'powermanga.org'
2023-11-26 23:19:05 +01:00
Mike Fährmann
f9dac43be9
[warosu] fix file URLs
2023-11-24 02:44:55 +01:00
Mike Fährmann
645b4627ef
[sankaku] update URL patterns
2023-11-24 02:41:52 +01:00
Mike Fährmann
119755a5a3
[tests] implement skipping/failing tests when pressing ctrl+c
2023-11-24 00:48:37 +01:00
Mike Fährmann
1ae43d8123
merge #4841 : [fapello] support '.su' TLD ( #4840 )
2023-11-22 20:18:32 +01:00
Mike Fährmann
e1404827a6
[pixeldrain] add 'file' and 'album' extractors ( #4839 )
2023-11-22 19:01:19 +01:00
enduser420
2402162e8a
[fapello] support '.su' TLD
2023-11-22 19:35:43 +05:30
Mike Fährmann
725c8dd55a
[tmohentai] 'categories' -> 'genres'
...
quite likely that the site meant 'genres' by "Genders"
2023-11-21 22:11:43 +01:00
Mike Fährmann
ce7c4cb544
merge #4832 : [tmohentai] add 'gallery' extractor ( #4808 )
2023-11-21 20:25:49 +01:00
Mike Fährmann
c4a201ed42
[tmohentai] simplify + tests
2023-11-21 20:24:07 +01:00
Mike Fährmann
e17a48fe56
[blogger] inherit from BaseExtractor
...
- support www.micmicidol.club (#4759 )
2023-11-21 16:52:25 +01:00
Mike Fährmann
0fa85360a0
merge #4812 : [erome] add 'count' metadata field
2023-11-20 22:42:02 +01:00
Mike Fährmann
a43cf78bb7
[erome] tests
2023-11-20 22:41:12 +01:00
Mike Fährmann
07cb584231
[behance] add 'modules' option ( #4799 )
2023-11-17 22:54:38 +01:00
Mike Fährmann
ea78f67860
[downloader:http] skip files not passing filesize-min/-max ( #4821 )
...
instead of failing the download
2023-11-17 22:54:20 +01:00
Mike Fährmann
3f591d5a4e
[mastodon] update test results
2023-11-11 21:24:07 +01:00
Mike Fährmann
6402f2950f
[pp:metadata] ignore non-string tag values ( #4764 )
2023-11-04 17:33:14 +01:00
Mike Fährmann
007c433677
[patreon] support 'id:<campaign_id>' in place of a user name
...
https://patreon.com/id:12345
… and remove 'campaign-id' config option
2023-11-04 00:17:41 +01:00
Mike Fährmann
43a3d93467
merge #4755 : [twitter] recognize fixupx.com URLs
2023-11-02 15:33:29 +01:00
Mike Fährmann
cdf77e326f
[twitter] add test for fixupx.com
2023-11-02 15:32:48 +01:00
Mike Fährmann
fc8f86bf24
[hitomi] recognize 'imageset' gallery URLs ( #4756 )
2023-11-02 15:29:44 +01:00
Mike Fährmann
72b18d701f
represent util.NONE as 'null' in JSON output
...
was '"None"' before
2023-11-02 15:23:28 +01:00
Mike Fährmann
68e72a836c
[exhentai] fix extraction ( #4730 )
...
- update to new API response layout
- use proper API server URL
- fix 'filesize' metadata
2023-10-30 13:38:49 +01:00
Mike Fährmann
fd8f58ad76
[behance] unescape embed URLs ( #4742 )
2023-10-30 13:38:49 +01:00
Mike Fährmann
c9a2be36d4
[sankaku] support '/posts/' tag search URLs ( #4740 )
2023-10-29 13:48:42 +01:00
Mike Fährmann
218295a4c6
[twitter] fix avatars without 'date' information ( #4696 )
2023-10-27 17:58:02 +02:00
Mike Fährmann
d0effcae20
[kemonoparty] add 'revision_index' metadata field ( #4727 )
2023-10-26 22:26:38 +02:00
