Mike Fährmann
fca87974fe
[sexcom] fix video downloads by sending specific Referer headers
2019-11-19 23:52:34 +01:00
Mike Fährmann
bbbeff4c41
[downloader.http] implement file-specific HTTP headers
2019-11-19 23:50:54 +01:00
Mike Fährmann
977026c5ad
update .travis.yml
...
- remove any 'dist: xenial'
- add Python 3.7 and 3.8 to the default build list
- run "results" tests with Python 3.8
2019-11-19 17:47:59 +01:00
Mike Fährmann
edc080468d
[instagram] make 'video_url' fields optional ( fixes #479 )
...
[ci skip]
2019-11-19 11:18:43 +01:00
Mike Fährmann
9fdc5e74cb
[deviantart] ensure consistent username capitalization ( #455 )
...
The 'username' field was capitalized in a very inconsistent manner:
Either all lowercase, or as given by the input URL, or with the
"original" capitalization, depending on the extractor used among
other things.
Now usernames use their original capitalization for all extractors.
('UserName' instead of 'username' or 'uSeRnAmE')
2019-11-18 22:09:58 +01:00
Mike Fährmann
b1f0609de5
[newgrounds] rewrite ( #394 )
...
- restructure extractor hierarchy
- extract more metadata
- extract videos without youtube-dl
- be more resilient to errors
TODO:
- favorites
- games, but that might be near impossible for non-flash titles
2019-11-18 21:13:33 +01:00
Mike Fährmann
3ece3976ae
[newgrounds] implement login support ( #394 )
2019-11-16 23:45:32 +01:00
Mike Fährmann
3a07c06865
[newgrounds] update
...
- create directory per post
- rename variables and methods
2019-11-14 23:17:14 +01:00
Mike Fährmann
5513b66eb0
[vsco] fix user profile extraction
2019-11-12 23:36:48 +01:00
Mike Fährmann
abfcb356fc
[flickr] support 3k, 4k, 5k, and 6k photo sizes ( closes #472 )
2019-11-10 17:52:51 +01:00
Mike Fährmann
521fcd2eb9
[imgbb] fix error in galleries without user info ( closes #471 )
2019-11-10 17:10:51 +01:00
Mike Fährmann
8061263d4c
[imgbb] improve pagination logic
...
- avoid unnecessary API calls for small or empty galleries
- combine duplicate code
2019-11-10 17:07:27 +01:00
Mike Fährmann
da6789b2b0
disable unique archive id checks for some tests
...
- same image twice in a livedoor blog post
- unreliable results for related pinterest items
2019-11-10 17:04:51 +01:00
Mike Fährmann
67e54ed8ea
release version 1.11.1
2019-11-09 00:52:06 +01:00
Mike Fährmann
ce98a86c0e
fix data file inclusion in source distributions
2019-11-09 00:47:13 +01:00
Mike Fährmann
6c86fbfe2a
release version 1.11.0
2019-11-08 22:34:56 +01:00
Mike Fährmann
ba083b30b2
fix snap build
...
… hopefully
2019-11-08 21:44:12 +01:00
Mike Fährmann
94a94f3b86
miscellaneous stuff
2019-11-08 20:58:53 +01:00
Mike Fährmann
4867ece13b
update README.rst
...
- use 'python3 -m pip' instead of pure 'pip'
- advice Windows users to use 'py -3'
- use '.tar.gz' for archive links since they are quite a bit smaller
- reword and simplify some stuff
2019-11-08 16:41:27 +01:00
Mike Fährmann
4f39d2ae64
install bash completion into share/bash-completion/completions
2019-11-07 23:23:48 +01:00
Mike Fährmann
b0197098e6
[imgur] get title from webpage if missing in API response
...
(closes #467 )
2019-11-07 21:10:04 +01:00
Mike Fährmann
dd5d2b2eac
[deviantart] add user profile extractor ( #377 , #419 )
2019-11-07 18:29:49 +01:00
Mike Fährmann
a437e78620
[deviantart] minimize cookie usage during scraps extraction
...
(#445 )
2019-11-05 21:55:13 +01:00
Mike Fährmann
1a197d2195
store the original cookiejar as Extractor._cookiejar
2019-11-05 21:53:22 +01:00
Mike Fährmann
de83ae4576
make 'method' argument of Extractor.request keyword-only
2019-11-05 17:28:09 +01:00
Mike Fährmann
a5be08a830
[downloader:ytdl] forward proxy settings
2019-11-05 16:16:26 +01:00
Mike Fährmann
4325695d74
[luscious] expand GraphQL queries
2019-11-04 21:17:22 +01:00
Mike Fährmann
94dbdbf506
[nijie] change default filename format
...
… to be consistent with Pixiv filenames
2019-11-04 20:47:38 +01:00
HRXN
794aace0bc
Add basic package version notice to README.rst ( #465 )
2019-11-03 23:52:50 +01:00
Mike Fährmann
9e88e7a344
[postprocessor:exec] improve ( #421 , #413 )
...
- add 'final' option
- include job status in pp finalization
- improve and extend documentation
2019-11-03 21:45:45 +01:00
Mike Fährmann
c18fadc221
[instagram] extract videos without youtube-dl ( #391 )
2019-11-03 14:02:56 +01:00
Mike Fährmann
f15eedb634
[sexcom] set Referer header for file downloads ( closes #464 )
2019-11-03 13:27:58 +01:00
Mike Fährmann
2a3bd4e3c7
rename extractor classes starting with a digit
2019-11-02 20:42:09 +01:00
Mike Fährmann
b3b9da6d74
[photobucket] replace test URL
...
The other user deleted all of is images.
2019-11-02 20:17:08 +01:00
Mike Fährmann
64786363be
[4chan] simplify
...
- remove 'chan.py'
- slight adjustments to directory and filenames
2019-11-02 20:11:21 +01:00
Mike Fährmann
557e2c018b
[8chan] remove module
2019-11-02 20:06:47 +01:00
Mike Fährmann
e14782a948
[instagram] simplify graphql extraction for post pages
2019-11-01 22:08:25 +01:00
Mike Fährmann
c01ff78467
[twitter] extend 'videos' option to force extraction with ytdl
...
(closes #459 )
2019-11-01 22:06:07 +01:00
Mike Fährmann
f8ac67ce50
[hitomi] extend URL pattern + follow redirects
2019-11-01 21:40:10 +01:00
Mike Fährmann
e782b4b230
update setup.py
...
- add 'cloudflare' and 'video' extra install target (#460 )
(e.g. 'pip install gallery-dl[cloudflare]')
- remove py2exe and distutils code
- get version without calling exec()
- inline variables
2019-11-01 21:33:48 +01:00
Mike Fährmann
e877ca97c3
[naver] adjust directory names and metadata structure
2019-10-31 16:53:48 +01:00
Mike Fährmann
702f2fbd1f
[issuu] add publication and user extractors ( #413 )
2019-10-31 16:52:57 +01:00
Mike Fährmann
8361d874d7
[hitomi] fix extraction
2019-10-29 16:23:20 +01:00
Mike Fährmann
5fa6ff04dd
[instagram] extract '__additionalDataLoaded' ( #391 )
...
The '_sharedData' of Post pages is missing its 'graphql' part for
logged in users. This data is now included in the parameters of a
function call to '__additionalDataLoaded(...)'
And, of course, video extraction with youtube-dl broke because of
this change as well.
2019-10-29 16:00:31 +01:00
Mike Fährmann
5af291ba5c
include failed downloads and child extractors in exit status
2019-10-29 15:56:54 +01:00
Mike Fährmann
322c2e7ed4
renaming variables
...
mostly 'keyword(s)' to 'kwdict'
2019-10-29 15:46:35 +01:00
Mike Fährmann
87a87bff7e
[simplyhentai] fix image URLs
2019-10-28 21:11:06 +01:00
Mike Fährmann
4409d00141
embed error messages in StopExtraction exceptions
2019-10-28 16:39:49 +01:00
Mike Fährmann
d5e3910270
adjust 'util.raises()'
2019-10-28 15:06:17 +01:00
Mike Fährmann
d44f790e81
adjust output for HTTP status related errors
2019-10-27 23:55:02 +01:00
