Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-23 19:22:32 +01:00
Code Issues Releases Wiki Activity
3,240 Commits 6 Branches 164 Tags 16 MiB
52984f7e22
Commit Graph

44 Commits

Author SHA1 Message Date
Mike Fährmann
dfe1f490e9
[mangadex] use custom User-Agent header (#1535) 2021-07-15 16:39:32 +02:00
Mike Fährmann
53dab5c289
[mangadex] revert chapter handling (#1535) 
Spawn a new ChapterExtractor for each individual chapter
instead of handling them directly with a MangaExtractor.

Doing it that way broke too many features like
--chapter-filter, --chapter-range, --zip, etc.
 2021-06-09 21:54:40 +02:00
Mike Fährmann
1197ee2c20
[mangadex] add extractor for a user's followed feed (#1535) 2021-06-08 02:45:36 +02:00
Mike Fährmann
07c8adbd8b
[mangadex] implement login with username & password (#1535) 2021-06-08 02:12:57 +02:00
Mike Fährmann
3e332eaf53
[mangadex] update to API v5 (#1535) 2021-06-07 01:29:42 +02:00
Mike Fährmann
69ea781d32
[mangadex] improve caching of manga results 
'manga_id' being a string or integer are treated as two different keys
 2021-02-28 01:34:58 +01:00
Mike Fährmann
e58039358d
[mangadex] use 'api.mangadex.org' as default API server 
The caching issues seem to be gone.
(#1290, #1310)
 2021-02-28 01:31:50 +01:00
Mike Fährmann
79c0fc249b
[mangadex] add 'api-server' option (#1309) 
and change the API server back to 'https://mangadex.org/api' for now
 2021-02-11 19:52:16 +01:00
Mike Fährmann
193dca2ce1
update extractor test results 2021-01-21 21:35:42 +01:00
Mike Fährmann
b0cf968115
[mangadex] update API URLs 
https://mangadex.org/thread/351011/9/#post_4238014
 2021-01-15 21:11:58 +01:00
Mike Fährmann
2c4d4a75db
[mangadex] respect 'chapter-reverse' settings (closes #1194) 
The extractor in question doesn't inherit from MangaExtractor
and therefore didn't do this automatically.
 2020-12-22 15:08:10 +01:00
Mike Fährmann
e5d81bdc7b
[mangadex] handle 'external' chapters (closes #1154) 2020-12-04 20:56:30 +01:00
Mike Fährmann
42ccae53c4
[mangadex] switch to API v2 
https://mangadex.org/api/v2/
https://mangadex.org/thread/351011
 2020-11-16 11:05:17 +01:00
Mike Fährmann
43dab3a228
[mangadex] unescape more metadata fields (fixes #1066) 
like 'manga', 'author', 'artist', etc.
 2020-10-16 00:41:15 +02:00
Mike Fährmann
7daef6ee70
update extractor test results 
- certain posts on Instagram now return
  https://static.cdninstagram.com/rsrc.php/null.jpg
  for public users
- MangaDex is deploying its new MangaDex@Home network similar to
  exhentai's Hentai@Home
- realbooru has a new site layout, but the underlying booru API still
  works like before
 2020-06-12 00:36:06 +02:00
Mike Fährmann
3bad1579ee
update extractor test results 2020-05-31 17:42:07 +02:00
Mike Fährmann
45baa13615
update extractor test results 
- don't run Instagram tests on Travis anymore
- replace Twitter test because timeline was made private
- update Hiperdex domain to '.com' (again ...)
 2020-05-28 02:18:06 +02:00
Mike Fährmann
2530db3f4d
[mangadex] transform 'date' timestamps to datetime objects 2020-03-18 02:19:19 +01:00
Mike Fährmann
af42c75152
[mangadex] revert domain to 'mangadex.org' 2020-01-14 09:49:57 +01:00
Mike Fährmann
988cc2ec23
[mangadex] change domain to mangadex.cc (closes #559) 2020-01-05 21:21:40 +01:00
Mike Fährmann
a2af2d2965
adjust cache maxage values 2019-03-14 22:21:49 +01:00
Mike Fährmann
5530871b5a
change results of text.nameext_from_url() 
Instead of getting a complete 'filename' from an URL and splitting that
into 'name' and 'extension', the new approach gets rid of the complete
version and renames 'name' to 'filename'. (Using anything other than
{extension} for a filename extension doesn't really work anyway)

Example: "https://example.org/path/filename.ext"

before:
- filename : filename.ext
- name     : filename
- extension: ext

now:
- filename : filename
- extension: ext
 2019-02-14 16:07:17 +01:00
Mike Fährmann
61741d7333
provide type information for Queue messages 
Child extractors are now directly constructed with Extractor.from_url()
if the extractor class is known beforehand, instead of using
extractor.find() and searching through all possible extractor classes.
 2019-02-12 21:32:32 +01:00
Mike Fährmann
580baef72c
change Chapter and MangaExtractor classes 
- unify and simplify constructors
- rename get_metadata and get_images to just metadata() and images()
- rename self.url to chapter_url and manga_url
 2019-02-11 18:38:47 +01:00
Mike Fährmann
4b1880fa5e
propagate 'match' to base extractor constructor 2019-02-11 13:31:10 +01:00
Mike Fährmann
6284731107
simplify extractor constants 
- single strings for URL patterns
- tuples instead of lists for 'directory_fmt' and 'test'
- single-tuple tests where applicable
 2019-02-08 13:45:40 +01:00
Mike Fährmann
a303efb597
[mangadex] handle manga pages without chapters 2019-01-03 16:22:12 +01:00
Mike Fährmann
b47af4637a
[mangadex] update URL pattern 
Manga URLs now begin with /title/ instead of /manga/
 2018-08-31 20:16:50 +02:00
Mike Fährmann
2af2bb7911
[mangadex] fix relative page URLs 2018-08-25 11:07:26 +02:00
Mike Fährmann
b55e39d1ee
[mangadex] improve extraction 
- cache manga API results
- add artist, author and date fields to chapter metadata
- remove Manga-/ChapterExtractor inheritance
- minor code simplifications and improvements
 2018-08-10 16:50:07 +02:00
Mike Fährmann
b1c4c1e13c
[mangadex] fix extraction 2018-08-08 18:08:26 +02:00
Mike Fährmann
2d1a104739
[mangadex] unescape manga names and chapter titles 
pretty sure I previously tested if unescaping strings from the
embedded JSON object was necessary ... maybe they changed it
 2018-06-11 17:53:21 +02:00
Mike Fährmann
a47c6136cd
[simplyhentai] avoid redirects for all-pages.json (#89) 2018-06-01 22:06:34 +02:00
Mike Fährmann
15cce22d82
[mangadex] fix parsing of unusual chapter strings 2018-05-23 18:40:39 +02:00
Mike Fährmann
7f899bd5d8
Merge branch 'master' into 1.4-dev 2018-05-14 14:50:02 +02:00
Mike Fährmann
e2157f594e
[mangadex] fix manga extraction (closes #84) 
Chapter listings for manga now use
https://mangadex.org/manga/<id>/_/chapters/2/
as URL instead of
https://mangadex.org/manga/<id>/_//2/
 2018-05-06 17:43:50 +02:00
Mike Fährmann
95392554ee
use text.urljoin() 2018-04-26 17:00:26 +02:00
Mike Fährmann
2721417dd8
Merge branch 'master' into 1.4-dev 2018-04-24 11:33:02 +02:00
Mike Fährmann
e54b43be08
[mangadex] add title info for chapter extractors 2018-04-22 16:20:04 +02:00
Mike Fährmann
cc36f88586
rename safe_int to parse_int; move parse_* to text module 2018-04-20 14:53:21 +02:00
Mike Fährmann
d1c91a1f2b
[mangadex] fix manga-page extraction 2018-03-25 17:22:12 +02:00
Mike Fährmann
85ed023c2e
[mangadex] remove the trailing ' - MangaDex' in a better way 
str.rstrip() works differently than assumed.
 2018-03-10 15:54:50 +01:00
Mike Fährmann
1400868f53
[mangadex] general improvements 
- support >100 chapter entries per manga
- custom archive ID format
- detect non-existing chapters
 2018-03-06 14:15:15 +01:00
Mike Fährmann
749fbbfa6c
[mangadex] add chapter- and manga-extractor 2018-03-05 18:37:21 +01:00