69 Commits

Author	SHA1	Message	Date
Mike Fährmann	ea6fbfdd39	simplify build_supportedsites.py	2019-02-21 14:29:19 +01:00
Mike Fährmann	ee2ad76620	update and improve create_test_data.py script	2019-02-20 15:48:31 +01:00
Mike Fährmann	abbd45d0f4	update handling of extractor URL patterns ... When loading extractor classes during 'extractor.find(…)', their 'pattern' attribute will be replaced with a compiled version of itself.	2019-02-08 20:08:16 +01:00
Mike Fährmann	53c2fd4664	add mastodon/foolslide/foolfuuka examples to example config	2019-02-05 16:17:25 +01:00
Mike Fährmann	373cb07b28	update .travis.yml and run_tests.sh ... - add python3.8 and pypy3 builds - remove deprecated 'sudo: true' and 'sudo: false' - enable builds for 'test-...' branches	2019-01-31 15:58:52 +01:00
Mike Fährmann	bfbbac4495	[tsumino] add login capabilities (#161)	2019-01-30 17:58:48 +01:00
Mike Fährmann	0156189468	[hentaifox] add chapter extractor (#160)	2019-01-28 18:00:32 +01:00
Mike Fährmann	e4171d6baf	[luscious] add login capabilities (closes #159)	2019-01-28 17:14:15 +01:00
Mike Fährmann	e53cdfd6a8	update build_supportedsites.py	2019-01-09 14:58:35 +01:00
Mike Fährmann	6c71e9cf5d	[deviantart] add separate 'sta.sh' extractor (#113) ... - supports multiple stashed deviations per page - explicitly mentions sta.sh support on supportedsites.rst	2018-12-26 18:56:57 +01:00
Mike Fährmann	fb53b5dd55	fix control+c during -j and range tests	2018-11-25 18:54:05 +01:00
Mike Fährmann	3c25fa2dad	update build_testresult_db.py script	2018-11-15 22:58:14 +01:00
Mike Fährmann	2cf3f53839	[yuki] add thread extractor (closes #111)	2018-09-28 12:47:32 +02:00
Mike Fährmann	17611bfec0	update build_supportedsites.py script	2018-09-28 12:43:19 +02:00
Mike Fährmann	c402cc4047	[hentaifoundry] add 'popular' and 'recent' extractors ... for "Popular Pictures" and "Recent Pictures" listings	2018-09-24 13:11:18 +02:00
Mike Fährmann	6ea9a78588	[wallhaven] add login capabilities ... Being logged in is required to access NSFW wallpapers.	2018-09-19 21:04:01 +02:00
Mike Fährmann	69fd61ea86	[bobx] add gallery and model extractors	2018-09-13 20:13:12 +02:00
Mike Fährmann	3ecea4cf36	[hentaicafe] add chapter and manga extractors (#101)	2018-09-05 21:08:40 +02:00
Mike Fährmann	75862715ac	[behance] add user extractor	2018-08-31 17:42:09 +02:00
Mike Fährmann	e9dd2eff1d	[twitter] add extractor for media-tweet timelines (#96) ... For example "https://twitter.com/PicturesEarth/media". They are different from normal timelines in that they do not contain any (re)tweets from other users and feature all media the user ever posted, including responses to other tweets.	2018-08-19 20:46:12 +02:00
Mike Fährmann	63fa0b2006	[pinterest] add extractors for related pins ... Related pins can not be accessed by adding a "#related" fragment to the end of a Pinterest URL, for example: - https://www.pinterest.com/pin/858146903966145189/#related - https://www.pinterest.com/g1952849/test-/#related There are no explicit real URLs for related pins, using an option to enable them results in "clunky" code, and a custom "related:<URL>" scheme doesn't feel right either.	2018-08-15 21:49:45 +02:00
Mike Fährmann	84854fcad7	[myportfolio] add user and gallery extractors (#95)	2018-07-19 18:56:45 +02:00
Mike Fährmann	a6fe2bb594	[whatisthisimnotgoodwithcomputers] remove extractor	2018-07-14 09:53:16 +02:00
Mike Fährmann	55b0913412	[simplyhentai] add gallery extractor (#89)	2018-05-27 15:25:04 +02:00
Mike Fährmann	f3483a2b7c	[smugmug] add OAuth support	2018-05-10 18:58:05 +02:00
Mike Fährmann	82c50fa609	release version 1.3.5	2018-05-04 10:03:20 +02:00
Mike Fährmann	16e014baaa	[smugmug] added image and album extractor ... just some initial code that still requires a lot of work ... TODO: - folders - old-style albums (which are nearly all of them ...) - images from users - OAuth It could also happen that the API credentials used will become invalid whenever my 14 day trial period ends (7 days remaining), but that would just require users to supply their own.	2018-04-29 21:27:25 +02:00
Mike Fährmann	d10579edb5	[pinterest] improve PinterestAPI code; remove OAuth mentions ... on another note: access_tokens have been set to only allow for 10 requests per hour (from 200 yesterday)	2018-04-17 17:12:42 +02:00
Mike Fährmann	4bd182c107	[pinterest] implement oauth:pinterest (#83) ... Pinterest access tokens are rate limited at 200 requests per hour (or maybe per 2 or 3 hours?) so having just one access token for all users isn't going to work in the long run.	2018-04-16 20:03:28 +02:00
Mike Fährmann	7b562907c3	[nijie] add favorites extractor ... adds support for 'https://nijie.info/user_like_illust_view.php?id=...'	2018-03-31 18:54:25 +02:00
Mike Fährmann	299ae24996	[test] add a few downloader tests	2018-03-25 15:10:25 +02:00
Mike Fährmann	dd314279fb	[test] add unit tests for extractor module functions	2018-03-25 11:49:42 +02:00
Mike Fährmann	d75dc71394	fix travis tests	2018-03-19 18:07:19 +01:00
Mike Fährmann	8ef790de12	update .travis.yml ... - restrict builds to master branch and release tags - implement 'core' and 'results' test categories	2018-03-19 17:57:32 +01:00
Mike Fährmann	7121eeae8b	check supportedsites.rst in release script	2018-03-17 15:35:38 +01:00
Mike Fährmann	1400868f53	[mangadex] general improvements ... - support >100 chapter entries per manga - custom archive ID format - detect non-existing chapters	2018-03-06 14:15:15 +01:00
Mike Fährmann	40d35c87bc	[paheal] add tag- and post-extractors (closes #69)	2018-01-15 16:39:05 +01:00
Mike Fährmann	35e09869d1	[mangapark] fix image URLs and use HTTPS	2018-01-12 14:59:49 +01:00
Mike Fährmann	29d75fc3fa	[tumblr] add support for OAuth authentication (#65)	2018-01-11 14:11:37 +01:00
Mike Fährmann	b33efc99a4	[idolcomplex] add support for idol.sankakucomplex.com	2018-01-09 17:54:37 +01:00
Mike Fährmann	5b094328b5	[puremashiro] add chapter- and manga-extractor (closes #66) ... Also adds support for region subtags in language codes (e.g. en-us)	2018-01-07 21:50:43 +01:00
Mike Fährmann	fc7d165c97	[deviantart] add support for OAuth2 authentication ... Some user galleries [*] require you to be either logged in or authenticated via OAuth2 to access their deviations. [*] e.g. https://polinaegorussia.deviantart.com/gallery/ -------------- known issue: A deviantart 'refresh_token' can only be used once and gets updated whenever it is used to request a new 'access_token', so storing its initial value in a config file and reusing it again and again is not possible.	2017-12-18 01:16:46 +01:00
Mike Fährmann	0a9a07a6e1	[slideshare] improve metadata; flake8 ... - added 'views' and 'published' keywords - fixed longer titles and descriptions	2017-12-13 21:16:49 +01:00
Mike Fährmann	2c1adda784	update release.sh script ... - update CHANGELOG on new releases - change issue references to actual links - replace "Unreleased" with new version and date - fix filenames of old Windows executables [no ci]	2017-11-08 17:47:52 +01:00
Mike Fährmann	2d0cfb33e1	[xvideos] add user profile extractor (#45)	2017-11-02 17:28:35 +01:00
Mike Fährmann	035ef655f1	[imagefap] update unit tests ... old gallery/image has been deleted	2017-10-27 12:22:16 +02:00
Mike Fährmann	6af921a952	[sankaku] rewrite/improve (fixes #44) ... - add wait-time between HTTP requests similar to exhentai - add 'wait-min' and 'wait-max' options - increase retry-count for HTTP requests to 10 - implement user authentication (non-authenticated users can only view images up to page 25) - implement 'skip()' functionality (only works up to page 50) - implement image-retrieval for pages >= 51 - fix issue with multiple tags	2017-10-14 23:01:33 +02:00
Mike Fährmann	0386503c80	fix (sub)category-transfer for DownloadJob instances (#41) ... ... and extend "parent" parameters to TestJob- and DataJob-classes as well.	2017-10-06 15:38:35 +02:00
Mike Fährmann	cebf800a7f	[foolfuuka] add support for more sites (#18) ... - https://arch.b4k.co - https://archive.whatisthisimnotgoodwithcomputers.com - https://archive.yeet.net Notes: - The name "whatisthisimnotgoodwithcomputers" is way too long ... - archive.yeet.net is out of date and also blocked by 4chan servers - newest threads are 2 weeks old - using "https://archive.yeet.net" as Referer header results in "403 Forbidden" when accessing 4chan	2017-09-16 21:36:16 +02:00
Mike Fährmann	18e6ed1c7e	[booru] add extractors for "Popular" images	2017-08-24 21:29:22 +02:00