Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-22 18:53:21 +01:00
Code Issues Releases Wiki Activity
1,599 Commits 6 Branches 164 Tags 15 MiB
eb3522601d
Commit Graph

1599 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
林博仁(Buo-ren Lin)
eb3522601d snap: Support official config paths via *-files confinement interfaces (#197) 
Refer-to: The personal-files interface - doc - snapcraft.io <https://forum.snapcraft.io/t/the-personal-files-interface/9357>
Refer-to: The system-files interface - doc - snapcraft.io <https://forum.snapcraft.io/t/the-system-files-interface/9358>
Signed-off-by: 林博仁(Buo-ren Lin) <Buo.Ren.Lin@gmail.com>
 2019-03-21 13:49:13 +01:00
Mike Fährmann
8d96a8ce4c
[500px] add user-, gallery-, and image-extractors (#185) 2019-03-20 17:32:36 +01:00
Mike Fährmann
d0f88c35be
[komikcast] fix extraction 2019-03-18 11:12:19 +01:00
Mike Fährmann
6277a739e4
[35photo] add user-, genre-, and image-extractors (#162) 2019-03-18 01:11:30 +01:00
Mike Fährmann
fb14f80d62
[tumblr] fix avatar URLs for non-OAuth1.0 calls (closes #193) 2019-03-17 11:07:22 +01:00
Mike Fährmann
8c20443839
release version 1.8.0 2019-03-15 15:27:11 +01:00
Mike Fährmann
973a720a7a
[weibo] fix unit test URL patterns 2019-03-15 15:19:39 +01:00
Mike Fährmann
a2af2d2965
adjust cache maxage values 2019-03-14 22:21:49 +01:00
Mike Fährmann
f612284d24
cache cfclearance cookies 2019-03-14 16:14:29 +01:00
Mike Fährmann
34ea0d6a10
rewrite cache module 
less complexity, better performance,
but some duplicate code here and there
 2019-03-14 15:55:48 +01:00
Mike Fährmann
12482553bd
update links to youtube-dl 2019-03-13 22:03:02 +01:00
Mike Fährmann
591a07f20c
small code changes and cleanups 2019-03-13 22:03:02 +01:00
Mike Fährmann
6f57d44ec2
[seaotterscans] remove extractor 
http://seaotterscans.com/ now redirects to their MangaDex profile
 2019-03-13 22:02:45 +01:00
林博仁(Buo-ren Lin)
8aaa4edbcd Implement selective pulling for building tagged release snaps (#187) 
This patch re-implements the pull step that will only build
development snapshots snaps if the latest tagged release has been
promoted to the stable channel.  This ensures that there's always
a revision of the stable release snap available in the edge channel
for the publisher to promote to stable as currently the build
infrastructure only supports build on code push (but not new tagged
releases) at this time.

The implementation is inspired from the snapcrafters adopted snaps workflow,
documented in the following forum topic:

    The automatic build and pubish process of snaps owned by the Snapcrafters - doc - snapcraft.io
    https://forum.snapcraft.io/t/autopublishing-of-snapcrafters-organizations-snaps-how/7954/2

Refer the following forum topic for a possible solution proposal of this problem:

    Proposal: Allow overriding the source-tag property for an one-time build in the build infrastructure - snapcraft - snapcraft.io
    https://forum.snapcraft.io/t/proposal-allow-overriding-the-source-tag-property-for-an-one-time-build-in-the-build-infrastructure/10303

Signed-off-by: 林博仁(Buo-ren Lin) <Buo.Ren.Lin@gmail.com>
 2019-03-13 21:58:46 +01:00
林博仁(Buo-ren Lin)
d398b1de40 snap: Use SNAPCRAFT_ARCH_TRIPLET for pulseaudio library workaround (#188) 
Signed-off-by: 林博仁(Buo-ren Lin) <Buo.Ren.Lin@gmail.com>
 2019-03-13 21:52:35 +01:00
Mike Fährmann
6dae6bee37
automatically detect and bypass cloudflare challenge pages 
TODO: cache and re-apply cfclearance cookies
 2019-03-10 15:31:33 +01:00
Mike Fährmann
25aaf55514
[smugmug] improve format selection (closes #183) 
- use original image if available
- support video formats
- remove user info for ImageExtractor (it is no longer possible to get
  image owner information for a single image)
 2019-03-10 15:20:35 +01:00
Mike Fährmann
7c1cb923a4
[myportfolio] replace unit test 
the old gallery got removed
 2019-03-10 15:06:16 +01:00
Mike Fährmann
fffbfd3dce
[imgspice] fix extraction 2019-03-09 20:29:23 +01:00
Mike Fährmann
4ca4631bad
simplify auto-disabling certificate verification 
if no certificate bundle is found
 2019-03-08 16:34:01 +01:00
Mike Fährmann
09d872a2b1
generalize extractor creation code 2019-03-07 22:55:26 +01:00
Mike Fährmann
8dc6be246b
[shopify] add custom retry logic for 430 status codes (#175) 2019-03-07 15:31:15 +01:00
Mike Fährmann
0887fb61f4
[komikcast] update test results 2019-03-07 14:55:52 +01:00
Mike Fährmann
a881537b91
more util.py tests 2019-03-06 21:09:37 +01:00
Mike Fährmann
976ccb267f
[myportfolio] combine gallery and user extractors 
An URL alone isn't good enough to distinguish between a gallery or a
gallery-listing, so the new extractor decides what to do based on the
page's content.
 2019-03-06 19:45:01 +01:00
Mike Fährmann
efd104e45e
[instagram] reject more non-user URLs (#180) 2019-03-06 10:26:01 +01:00
HRXN
56e0e92e0d [shopify] cosmetic changes in shopify.py (#181) 
Glanced over the commits, randomly spotted some minor things.
 2019-03-06 09:16:27 +01:00
Mike Fährmann
23baecb29e
fix 'CONVERSIONS' variable name 2019-03-05 22:50:56 +01:00
Mike Fährmann
9c0e2f294b
[shopify] add generic collection and product extractors (#175) 
with fashionnova.com  as a default domain
 2019-03-05 22:33:37 +01:00
Mike Fährmann
105097ddcf
add 'S' conversion options for format string fields 
Same as 's' (convert to string), but has a better, human-readable
conversion for lists.
 2019-03-04 21:13:34 +01:00
Mike Fährmann
1578013efc
remove unused default config path 2019-03-04 20:53:58 +01:00
Mike Fährmann
26c4365baa
adjust metadata types for GalleryExtractors 2019-03-02 14:53:04 +01:00
Mike Fährmann
13e0f2a78f
[deviantart] add 'scraps' extractor (closes #168) 2019-03-01 14:13:34 +01:00
Mike Fährmann
3ea11f5d5e
[nhentai] rewrite 
- use GalleryExtractor as base class
- extract a lot more metadata (artist, tags, etc.)
 2019-03-01 14:13:34 +01:00
Mike Fährmann
176b7253a1
update function signature for config.load() 2019-03-01 14:13:34 +01:00
Mike Fährmann
3595cd582f
use GalleryExtractor as common base class 2019-03-01 14:13:16 +01:00
林博仁(Buo-ren Lin)
77551bf01b Implement snap packaging, snaps are universal linux packages (#170) 
This patch implements the necessary details to package gallery-dl as a snap that can be run on a broad range of supporting GNU/Linux distributions[1].

[1] https://snapcraft.io/

Signed-off-by: 林博仁(Buo-ren Lin) <Buo.Ren.Lin@gmail.com>
 2019-03-01 14:12:41 +01:00
Mike Fährmann
a138d5873d
[hentaifoundry] improve/fix extraction 
- Sometimes an ad interfered when trying to get a download URL
- Resolving "www.hentai-foundry.com" yields an invalid(?) IPv6 address
  (2607:5300:60:ca9e:feed:dead:beef:1) and urllib3 only tries to connect
  to the IPv4 variant after a rather long wait time
 2019-02-25 16:16:09 +01:00
Mike Fährmann
280531c8ff
[pururin] add gallery extractor (closes #174) 2019-02-25 14:54:57 +01:00
Mike Fährmann
3159dd79d5
[seiga] use HTTPS 2019-02-21 22:51:11 +01:00
Mike Fährmann
ea6fbfdd39
simplify build_supportedsites.py 2019-02-21 14:29:19 +01:00
Mike Fährmann
ee2ad76620
update and improve create_test_data.py script 2019-02-20 15:48:31 +01:00
Mike Fährmann
f6734142ee
[komikcast] remove 'width' and 'height' info 2019-02-19 15:12:40 +01:00
Mike Fährmann
d0059cab79
[tumblr] check for null URLs (closes #165) 2019-02-19 13:49:55 +01:00
Mike Fährmann
e687a6095e
[luscious] raise exception if album is not available 2019-02-19 13:30:39 +01:00
Mike Fährmann
22d3a2fcc8
[artstation] add extractor for artwork listings (#80) 
like https://www.artstation.com/artwork?sorting=latest
or   https://www.artstation.com/artwork?sorting=picks
 2019-02-18 12:45:44 +01:00
Mike Fährmann
937a802b49
[dynastyscans] add extractors for images and image searches 
(closes #163)
 2019-02-18 12:25:52 +01:00
Mike Fährmann
b09a8184ca
move TestJob into test module; test _extractor values 2019-02-17 18:18:31 +01:00
Mike Fährmann
19860655a3
[weibo] add 'user' and 'status' extractors 2019-02-17 18:18:31 +01:00
Mike Fährmann
f8782c05f2
[paheal] rename "tags" to "search_tags" 
to better match field names of other booru extractors
 2019-02-17 18:18:09 +01:00