Mirrors/gallery-dl
1
0
Fork 0
mirror of https://github.com/mikf/gallery-dl.git synced 2024-11-25 20:22:36 +01:00
Code Issues Releases Wiki Activity
6,090 Commits 6 Branches 164 Tags 15 MiB
docs/gallery-dl.conf
Commit Graph

224 Commits

Author SHA1 Message Date
Mike Fährmann
bced143750
[tests] add workaround for compile_expression_defaultdict in pypy3 2024-11-16 19:35:28 +01:00
Mike Fährmann
5bc3657c59
[util] implement 'compile_filter()' (#5262) 
https://github.com/mikf/gallery-dl/issues/5262#issuecomment-2477029728

allow (theoretically*) all filter expression statements
to be a list of individual filters

(*) except for 'filename' and 'directory' conditionals,
as dict keys cannot be lists
 2024-11-14 22:47:36 +01:00
Mike Fährmann
0b99d9e6b9
[util] add "defaultdict" filters-environment 
allows accessing undefined values without raising an exception,
but preserves other errors like TypeError, AttributeError, etc
 2024-11-14 22:47:25 +01:00
Mike Fährmann
4667833195
[util] add 'std' object to global eval namespace (#6330) 
allows accessing standard library modules (and other external modules)
in a more straightforward manner than '__import__(...)'

* std.os.getcwd()
* std["os"].getcwd()
 2024-10-17 21:08:07 +02:00
Mike Fährmann
36883e458e
use 'v[0] == "c"' instead of 'v.startswith("c")' 2024-10-15 08:24:06 +02:00
Mike Fährmann
3946fe5ac4
[cookies] return loaded cookies as list 
don't set_cookie() them immediately into a CookieJar
also, give some more consistent names to chrome/chromium functions
 2024-10-14 14:24:27 +02:00
Mike Fährmann
a120295632
[util] use minimal separators for 'json_dumps()' 2024-10-01 17:03:13 +02:00
Mike Fährmann
2c7a0c3ca8
add alternatives for deprecated utc datetime functions 2024-09-19 20:47:05 +02:00
Mike Fährmann
4601aa9794
[util] add 'NullContext' 
an even more minimal 'contextlib.nullcontext'
that also works on older Python versions
 2024-09-09 20:28:18 +02:00
Mike Fährmann
4da3347d18
allow filtering '--list-extractors' results 
with blacklist/whitelist syntax, e.g.

--list-extractors pixiv
--list-extractors pixiv:user pixiv:work
--list-extractors :search
 2024-09-08 09:27:32 +02:00
Mike Fährmann
01e8433889
fix tests & syntax for older Python versions 2024-08-15 18:16:55 +02:00
Mike Fährmann
5208c0d28a
[util] extend CustomNone with an __index__ method (#6009) 
- Make it compatible with functions expecting integer arguments
- Simplify and reuse some method definitions
 2024-08-13 16:05:54 +02:00
Mike Fährmann
ad73789256
[util] extend CustomNone with arithmetic operators (#6007) 2024-08-13 09:41:12 +02:00
Mike Fährmann
d7a2c73274
[util] let a CustomNone instance be equal to itself 2024-07-26 20:56:01 +02:00
Mike Fährmann
f7383a56f8
wrap filters/conditionals in a try-except block 
allows accessing undefined fields without exception or locals().get(…)
but hides mistakes/typos/etc by evaluating to False without feedback

performance loss compared to the previous version without try-except
is negligible (~20ns for me)
 2024-07-12 22:51:11 +02:00
Mike Fährmann
7614bc458e
[util] extend CustomNone with comparison operators 2024-06-05 16:49:30 +02:00
Mike Fährmann
88f94190f4
[archive] move DownloadArchive into its own module 2024-05-10 01:05:28 +02:00
Mike Fährmann
9a8403917a
restore LD_LIBRARY_PATH for PyInstaller builds (#5421) 2024-04-06 16:58:33 +02:00
Mike Fährmann
8a11b72253
remove extractor/test.py (#4504) 2024-02-27 01:37:57 +01:00
Mike Fährmann
a24b82e67d
add 'util.repeat()' 2023-12-11 23:32:28 +01:00
Mike Fährmann
e256434c9e
use custom HTTPBasicAuth class 
to support LazyPrompt as password
and to generate the Authorization header only once
instead of for every request
 2023-12-07 01:40:31 +01:00
Mike Fährmann
bdebe4597a
fix util.dump_response to work with bytes as header values 2023-12-06 23:03:34 +01:00
Mike Fährmann
9dd5cb8c8a
interactively prompt for passwords on login when none is provided 2023-12-06 22:12:59 +01:00
Mike Fährmann
4700051562
rework and extend input file processing (#4732) 
- add 2 command-line options to modify input file contents
  - -I/--input-file-comment
  - -x/--input-file-delete
- implement InputManager class
- move code from util.py to __init__.py
  (mainly to avoid import cycles)
 2023-11-14 20:50:11 +01:00
Mike Fährmann
72b18d701f
represent util.NONE as 'null' in JSON output 
was '"None"' before
 2023-11-02 15:23:28 +01:00
Mike Fährmann
899df8f237
remove another '*' for keyword-only arguments 
076380e0
 2023-09-13 21:38:31 +02:00
Mike Fährmann
26d06e0bb2
move executable check into util.py 2023-02-28 23:10:23 +01:00
Mike Fährmann
d788e6c60c
implement 'globals' option 2023-02-28 18:18:55 +01:00
Mike Fährmann
3d8777fbc1
move user agent string to util.py 2023-02-22 11:09:17 +01:00
Mike Fährmann
56039d2456
add 'hash_md5' and 'hash_sha1' functions (#3679) 
... to global eval namespace
 2023-02-22 10:58:44 +01:00
Mike Fährmann
d4232f3a8b
implement restarting an extractor (#3338) 2023-02-11 21:06:14 +01:00
Mike Fährmann
1aae72773f
put argument init on separate lines 2023-02-10 17:05:20 +01:00
Mike Fährmann
5503ac4d5e
replace json.dumps with direct calls to JSONEncoder.encode 2023-02-09 15:51:40 +01:00
Mike Fährmann
dd884b02ee
replace json.loads with direct calls to JSONDecoder.decode 2023-02-09 15:22:00 +01:00
Mike Fährmann
762a68996b
implement 'archive-pragma' option 2023-02-05 17:00:31 +01:00
Mike Fährmann
d6793b2c7d
include request body in 'write-pages=all' output 2023-01-14 22:33:59 +01:00
Mike Fährmann
d651d45239
implement specifying ranges in slice notation (#918, #2865) 
e.g.
- '1:101'   or ':101' or ':101:'  for files 1 to 100
- '1::2'    or '::2'              for every second file
- '1:101:5' or ':101:5'           for files 1, 6, 11, ..., 91, 96

(the second argument specifies the first index NOT included)
 2022-12-27 18:21:12 +01:00
Mike Fährmann
3616adfc75
implement '--range' with Python ranges 2022-12-26 18:32:34 +01:00
Mike Fährmann
1800bd7d14
allow '*-filter' options to be a list of expressions 2022-12-23 22:20:21 +01:00
Mike Fährmann
43c211f1a7
extend and rename util.CustomNone 2022-12-06 22:08:51 +01:00
Mike Fährmann
39d9c362e4
include 'http-metadata' in '-K' output 2022-11-07 16:33:26 +01:00
Mike Fährmann
597b63d922
move git head functionality to function in util.py 2022-11-04 19:49:39 +01:00
ClosedPort22
455e34113e
Improve compatibility of DownloadArchive (#3078) 
Other programs can add additional columns to the table without affecting
gallery-dl
 2022-10-21 15:56:40 +02:00
Mike Fährmann
a6e2d96dde
fix bug when processing input file comments (#2808) 
and move 'parse_inputfile()' to util.py
 2022-10-08 15:41:39 +02:00
Mike Fährmann
dd3a6a9fd1
make 'enumerate_reversed()' work with generators (#2795) 2022-08-01 14:08:44 +02:00
Mike Fährmann
a96f9d29b5
create missing directories for archive files (#2597) 2022-05-17 12:57:01 +02:00
Mike Fährmann
c4b9f7bab8
update functions working with cookies.txt files 
- rename
  - load_cookiestxt -> cookiestxt_load
  - save_cookiestxt -< cookiestxt_store
- in cookiestxt_load, add cookies directly to a cookie jar
  instead of storing them in a list first
- other unnoticeable performance increases
 2022-05-06 13:21:29 +02:00
Mike Fährmann
4c8e28db82
avoid circular import in util.py 2022-05-04 17:42:37 +02:00
Mike Fährmann
61afa973b7
use extended format strings for archive keys 2022-04-28 20:26:12 +02:00
Mike Fährmann
ca3a364db7
fix build_duration_func() (#2533) 
for extractors with request_interval_min > 0
 2022-04-27 20:28:14 +02:00