Mike Fährmann
c51616f8d8
[foolslide] fix minor chapter number
2017-09-26 12:49:50 +02:00
H R X N
77bf923c56
Update imgur.py to include 'title' of single image ( #40 )
...
Add {title} keyword..
Images on Imgur don't necessarily have a title, but I think most of them do, and since this should not break anything else..
2017-09-26 12:48:48 +02:00
Mike Fährmann
a85f06d2d1
[foolslide] restructure; convert suitable values to int
2017-09-24 16:57:47 +02:00
Mike Fährmann
deb2e803ba
simplify MangaExtractor class
2017-09-24 16:05:43 +02:00
Mike Fährmann
9fc1d0c901
implement and use 'util.safe_int()'
...
same as Python's 'int()', except it doesn't raise any exceptions and
accepts a default value
2017-09-24 15:59:25 +02:00
Mike Fährmann
8a97bd0433
rename '--images' and '--chapters'
...
... to '--range' and '--chapter-range' to be consistent with
'--filter' and '--chapter-filter'
2017-09-23 17:31:40 +02:00
Mike Fährmann
8963da8fd8
[spectrumnexus] extract manga metadata
2017-09-23 16:49:33 +02:00
Mike Fährmann
a3e40734d1
[mangareader] extract manga metadata
2017-09-23 15:42:50 +02:00
Mike Fährmann
9196005a4d
[mangazuki] extract manga metadata
2017-09-22 20:53:43 +02:00
Mike Fährmann
543ba245eb
[deviantart] update test results
...
thumbnail URLs changed from //tXX.… to //t00.…
2017-09-22 17:53:59 +02:00
Mike Fährmann
b7a54a51d0
[mangapark] extract manga metadata + code improvements
2017-09-22 17:53:32 +02:00
Mike Fährmann
d39b8779af
[mangahere] extract manga metadata
2017-09-22 14:55:37 +02:00
Mike Fährmann
c265cc074a
[hbrowse] fix syntax for Python3.3 and 3.4
2017-09-20 16:41:39 +02:00
Mike Fährmann
a9e7145651
[hbrowse] extract hmanga metadata & general maintenance
2017-09-20 16:25:25 +02:00
Mike Fährmann
92c8a6cb01
[hentai2read] extract hmanga metadata
2017-09-20 13:28:57 +02:00
Mike Fährmann
de174b40d6
[hentaihere] extract hmanga metadata
2017-09-20 13:13:14 +02:00
Mike Fährmann
04cc1ffe34
[kissmanga] extract manga metadata
2017-09-19 16:25:04 +02:00
Mike Fährmann
885bd4cbe2
[readcomiconline] extract comic metadata
2017-09-18 19:18:24 +02:00
Mike Fährmann
cebf800a7f
[foolfuuka] add support for more sites ( #18 )
...
- https://arch.b4k.co
- https://archive.whatisthisimnotgoodwithcomputers.com
- https://archive.yeet.net
Notes:
- The name "whatisthisimnotgoodwithcomputers" is way too long ...
- archive.yeet.net is out of date and also blocked by 4chan servers
- newest threads are 2 weeks old
- using "https://archive.yeet.net " as Referer header results in
"403 Forbidden" when accessing 4chan
2017-09-16 21:36:16 +02:00
Mike Fährmann
84d4450410
[fallenangels] extract manga metadata
2017-09-15 20:51:40 +02:00
Mike Fährmann
f32b1a0292
[imgyt] fix extraction
2017-09-14 15:04:32 +02:00
Mike Fährmann
4ad903b797
[warosu] fix extraction
2017-09-14 14:57:40 +02:00
Mike Fährmann
b84f48dfa5
[batoto] extract manga metadata
2017-09-14 14:55:57 +02:00
Mike Fährmann
4ceb176c6b
[foolslide] extract manga metadata
...
enables chapter filtering for
- https://kobato.hologfx.com/
- https://jaiminisbox.com/
- https://reader.kireicake.com/
- https://powermanga.org/
- https://reader.seaotterscans.com/
- http://sensescans.com/
- http://www.slide.world-three.org/
2017-09-12 16:44:38 +02:00
Mike Fährmann
24e5f154a4
[deviantart] update test results
...
API responses now contain proper https:// URLs and their image download
server is now "orig00.deviantart.net" for all images.
2017-09-12 16:38:57 +02:00
Mike Fährmann
0dedbe759c
enable '--chapter-filter'
...
The same filter infrastructure that can be applied to image URLS now
also works for manga chapters and other delegated URLs.
TODO: actually provide any metadata (currently supported is only
deviantart and imagefap).
2017-09-12 16:19:00 +02:00
Mike Fährmann
31cd5b1c1d
[luscious] detect high-load responses
2017-09-12 15:46:21 +02:00
Mike Fährmann
470bbe9d8c
fix smaller stuff
...
- change filename option in example config file
- adapt default filename format for mangafox
- remove unnecessary newline
[skip ci]
2017-09-11 17:07:29 +02:00
Mike Fährmann
6f30cf4c64
change keyword names to valid Python identifiers
...
This commit mostly replaces all minus-signs ('-') in keyword names with
underscores ('_') to allow them to be used in filter-expressions. For
example 'gallery-id' got renamed to 'gallery_id'.
(It is theoretically possible to access any variable, regardless of its
name, with 'locals()["NAME"]', but that seems a bit too convoluted if
just 'NAME' could be enough)
2017-09-10 22:20:47 +02:00
Mike Fährmann
81877bb5f6
add '-K' as shortcut for '--list-keywords'
2017-09-09 18:48:28 +02:00
Mike Fährmann
54c0715135
allow users to set their own API access_tokens/client_ids
2017-09-09 17:50:19 +02:00
Mike Fährmann
49c7e70c10
[acidimg] add image extractor
2017-09-09 15:19:18 +02:00
Mike Fährmann
5704c709fa
apply filter before range
2017-09-09 14:51:31 +02:00
Mike Fährmann
9b21d3f13c
add '--filter' command-line option
...
This allows for image filtering via Python expressions by the same
metadata that is also used to build filenames (--list-keywords).
The usually shunned eval() function is used to evaluate
filter-expressions, but it seemed quite appropriate in this case and
shouldn't introduce any new security issues, as any attacker that could do
> gallery-dl --filter "delete-everything()" ...
could as well do
> python -c "delete-everything()"
2017-09-08 17:52:00 +02:00
Mike Fährmann
268cfa3cfe
filter duplicate URLs ( #36 )
...
Duplicate URLs might occur if, for example, an artist adds another
image to his gallery while an extractor is running and images are being
downloaded on sites like pixiv/nijie/hentaifoundry.
The next image on the next page will have already been downloaded and
will cause a premature end if '--abort-on-skip' is being used.
2017-09-06 17:08:50 +02:00
Mike Fährmann
00420ff202
[booru] consistent order for "popular" results
2017-09-06 12:33:19 +02:00
Mike Fährmann
83cf1e1d6d
[sankaku] unescape image URLs
2017-09-04 17:41:11 +02:00
Mike Fährmann
f98e3e8002
[luscious] fix tag extraction
2017-09-01 16:29:52 +02:00
Mike Fährmann
b8862ff15e
add 'downloader.http.verify' option
...
(also: change the default 'timeout' from None to 30)
2017-08-31 15:21:08 +02:00
Mike Fährmann
65997d835b
replace popular/ranking tests with older ones
...
Metadata of several year old lists shouldn't change as much as it
would for newer ones, which makes metadata-comparisons of the output
of build_testresult_db.oy easier.
2017-08-31 15:09:18 +02:00
Mike Fährmann
be30fb2f98
add common config category for boorus and foolslide
2017-08-29 22:42:48 +02:00
Mike Fährmann
c0755a4d5e
[exhentai] revert login-method to its old version ( #37 )
...
Additional cookies don't seem to help and have to be manually set
anyway. The older method is more likely to succeed, so I'd rather
use this one.
2017-08-29 22:10:38 +02:00
Mike Fährmann
3ee39ffd93
[exhentai] update login procedure ( #37 )
...
This new version behaves pretty much exactly like a browser would and
caches all cookies sent to it and not just "ipb_member_id" and
"ipb_pass_hash".
2017-08-28 21:03:32 +02:00
Mike Fährmann
88a386977e
[booru] add "popular" extractors for more sites
...
- konachan.com
- behoimi.org
- e621.net
2017-08-26 23:08:52 +02:00
Mike Fährmann
07214f4007
[booru] place subcategories into base classes
2017-08-26 22:27:55 +02:00
Mike Fährmann
60a888a1e4
[foolfuuka] add common config category
...
All FoolFuuka based 4chan-archive extractors can now be configured using
their own config keys (extractor.<category>) as well as a common shared
one (extractor.foolfuuka).
2017-08-25 22:53:37 +02:00
Mike Fährmann
47bcf53ec1
implement support for additional unit test result types
...
- "pattern" matches all resulting URLs against the given regex
- "count" allows to specify the amount of returned URLs
2017-08-25 22:01:14 +02:00
Mike Fährmann
2d0dfe9d56
[exhenai] init headers before login and detect sadpanda
...
- also debug-logs html after failed login
- #37
2017-08-25 16:44:59 +02:00
Mike Fährmann
c7ec103e15
[batoto] fix extraction of chapter URLs
2017-08-25 16:34:42 +02:00
Mike Fährmann
18e6ed1c7e
[booru] add extractors for "Popular" images
2017-08-24 21:29:22 +02:00
