1
0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-22 18:53:21 +01:00
Commit Graph

24 Commits

Author SHA1 Message Date
Mike Fährmann
e2710702d4
fix Cloudflare bypss 2019-12-01 01:07:24 +01:00
Mike Fährmann
76df628b13
rewrite invalid cloudflare redirect locations
After solving a challenge on komikcast.com, cloudflare would redirect to
https:/komikcast.com (with only one '/') when testing on TravisCI.
2019-04-27 16:22:42 +02:00
Mike Fährmann
49a6522c38
ensure consistent headers and params ordering
Necessary to avoid being labeled a bot and getting a CAPTCHA response
after solving a Cloudflare challenge.
2019-04-09 10:52:27 +02:00
Mike Fährmann
9ebd29fcc1
update cloudflare bypass (wip)
This commit adds support for the two new JS expressions embedded in the
overall challenge code.

It does compute the correct 'js_answer' value, but the HTTP request to
/cdn-cgi/l/chk_jschl to get the 'cf_clearance' cookie always results in
a 403 response with a CAPTCHA inside (hence 'wip')

All steps to make this HTTP request indistinguishable from a regular web
browser (which passes the test) show no effect. This includes:
- using the exact same HTTP headers as a web browser
- follow query argument order
- different wait times
2019-04-01 15:14:59 +02:00
Mike Fährmann
5476404a5c
update and fix Cloudflare bypass 2019-03-25 22:53:36 +01:00
Mike Fährmann
f612284d24
cache cfclearance cookies 2019-03-14 16:14:29 +01:00
Mike Fährmann
6dae6bee37
automatically detect and bypass cloudflare challenge pages
TODO: cache and re-apply cfclearance cookies
2019-03-10 15:31:33 +01:00
Mike Fährmann
0328a04a65
[cloudflare] don't output the whole challenge page
thanks to the embedded animated gifs this is just a bit too much
2018-12-30 13:09:23 +01:00
Mike Fährmann
95392554ee
use text.urljoin() 2018-04-26 17:00:26 +02:00
Mike Fährmann
ff643793bd
improve and document cloudflare bypass code 2018-04-19 21:32:10 +02:00
Mike Fährmann
d11fcf4804
smaller changes and fixes
- fix the cloudflare challenge result if the last decimal places
  are zero (JS`s toFixed() removes trailing zeroes)
- fix downloading of kissmanga chapter-pages hosted on blogspot
  (accessing blogspot with "kissmanga.com" as referrer yields a 401)
- disable certificate validation for 'mangahere' tests
- update flickr test result
2018-04-06 15:30:09 +02:00
Mike Fährmann
f6c95dccf9
[cloudflare] fix bypass procedure
Cloudflare challenges, at least for kissmanga and readcomiconline,
now use slightly different Javascript expressions.

Instead of a single value per expression, they now have a numerator
and a denominator of a fractional value, which in the end gets
truncated to 10 decimal places.
2018-04-05 20:28:04 +02:00
Mike Fährmann
67d4462d26
[batoto] rudimentary Cloudflare bypass 2018-01-11 18:49:19 +01:00
Mike Fährmann
cf79a47b59
update unit tests 2017-04-08 11:02:32 +02:00
Mike Fährmann
cdecd0b37b
add notification when solving cloudflare challenge 2017-03-25 12:23:30 +01:00
Mike Fährmann
4f123b8513
code adjustments according to pep8 2017-01-30 19:40:15 +01:00
Mike Fährmann
40dbea7ed2
rewrite parts of the cloudflare bypass system 2016-12-16 13:28:36 +01:00
Mike Fährmann
9e3788175e
implement decorator for cloudflare bypass
this method for enabling and caching a cloudflare bypass for a
requests.session object allows for different cache-timeouts for
different domains
2016-11-20 18:05:49 +01:00
Mike Fährmann
b634ace39e
[readcomiconline] add comic-issue and comic extractor 2016-11-14 18:29:45 +01:00
Mike Fährmann
3c94d85409
fix cloudflare bypass 2016-07-12 12:03:25 +02:00
Mike Fährmann
8accf4b790
use cache-decorator for cloudflare bypass 2016-03-07 16:34:42 +01:00
Mike Fährmann
e78937a580
cache cookies from cloudflare-bypass 2015-11-27 21:32:07 +01:00
Mike Fährmann
079330785a
document and fix smaller bugs 2015-11-07 13:06:23 +01:00
Mike Fährmann
822527a2ce
module to bypass cloudflare protection 2015-11-07 02:30:08 +01:00