1
0
mirror of https://github.com/instaloader/instaloader.git synced 2024-11-04 17:32:30 +01:00
Commit Graph

485 Commits

Author SHA1 Message Date
Alexander Graf
52eb6d8e27 Fix minor issues regarding last commit
- Close session from InstaloaderContext.anonymous_copy().

- Have graphql_count_per_slidingwindow default value in one place rather than
  three.
2018-05-14 10:27:34 +02:00
André Koch-Kramer
3211d63ec1 Replace deleted field has_highlight_reel
Since the property has_highlight_reel is no longer available throught
the previously used graphql query, this information needs to be obtained
in another way. Therefore the properties has_highlight_reels,
has_public_story and has_viewable_story were added to the Profile class.
Since has_public_story can be obtained throught graphql queries without
being rate limited when invoked anonymously, the ability to use an
anonymous copy of the context was added to to the InstaloaderContext class.
Fixes #116
2018-05-13 19:43:49 +02:00
André Koch-Kramer
e1228501d2 Set graphql_count_per_slidingwindow to 20
Instagram tightened their rate limits again.
Implemented hidden switch --graphql-rate-limit for easier adjustment of
graphql_count_per_slidingwindow.
Fixes #118
2018-05-13 19:09:40 +02:00
Alexander Graf
b8797cf337 Release of version 4.0.1 2018-05-08 19:16:55 +02:00
Alexander Graf
fa06e90e09 Fix anonymous retry logic 2018-05-08 19:12:42 +02:00
Alexander Graf
d464396d6e Adjust rate control behavior
Increment graphql_count_per_slidingwindow to 25; Let context._sleep() choose
sleep times from upper-bounded exponential distribution with similar expected
value as before, letting wait times between queries seem more natural.
2018-05-08 18:54:28 +02:00
Alexander Graf
9961312d34 Fix anonymous retry on profile that blocked viewer 2018-05-08 18:15:17 +02:00
Alexander Graf
211b2269ff download_profile: Request story only if has_highlight_reel
This is a simple modification to save GraphQL queries, hence to more likely
stay within rate limits without having to wait between profiles.

Fixes #112.
2018-05-08 18:10:28 +02:00
Alexander Graf
b3df5a5f07 Fix bugs in Instaloader.update_comments()
- Cast id and created_at from JSON to int to be compatible with comments JSON
  files from Instaloader 3.3,
- Do not fail if comment JSON files are empty,
- Close file descriptors of comment JSON files when reading,
- Do no create empty comment JSON files if generating content fails.

Closes #114.
2018-05-08 17:39:00 +02:00
Alexander Graf
5204990db9 Release of version 4.0 2018-05-02 17:38:07 +02:00
Alexander Graf
94df221da2 Merge branch 'v4-dev' 2018-05-02 17:29:12 +02:00
Alexander Graf
e7decba5ee main: Enumerate profiles separated with ' ' in log 2018-05-02 17:27:46 +02:00
Alexander Graf
a3a09e1952 Very minor tweak to a paragraph in basic-usage.rst 2018-05-02 17:26:53 +02:00
Alexander Graf
b1e55a41a4 Do not log Post's username in download_saved_posts
This improves speed, as get_saved_posts() returns Posts that do not have
owner_profile info. So, printing the owner_username often required an
additional request.
2018-05-01 20:22:52 +02:00
Alexander Graf
b5ad92e236 Trivial improvement to Profile.from_id() 2018-05-01 19:52:39 +02:00
Alexander Graf
084cfb102c Handle ValueError on bad profile id file
Fixes "ValueError: invalid literal for int() with base 10: b''", reported in #95.
2018-05-01 15:14:19 +02:00
Alexander Graf
2722da6ae4 Fix anon username from id (find renamed profile)
maybe related to #95, #104, 67ac8f3397.
2018-05-01 13:11:40 +02:00
Alexander Graf
1469064176 Minor documentation tweaks
- Instaloader as Cronjob sections

- Many new internal links to ease navigation
2018-04-30 09:56:43 +02:00
Alexander Graf
18ef484f3a First Release Candidate for Version 4.0 2018-04-29 22:53:20 +02:00
Alexander Graf
dba74ab8d4 Minor documentation improvements 2018-04-29 16:41:24 +02:00
Alexander Graf
dd80ef484c Improve formatting of basic-usage/what-to-download 2018-04-29 16:26:31 +02:00
Alexander Graf
6117e223a7 Describe Metadata Text Files Feature 2018-04-29 16:24:55 +02:00
Alexander Graf
7c76202f1c Update documentation regarding v4 changes
as-module.rst: List how Post and Profile is created

Describe new filename specification logic

Update filter-posts section
2018-04-29 13:33:25 +02:00
Alexander Graf
5036cc7b44 main: Ignore profile JSON files as target
It might annoy or be undesired when using Instaloader with a globbed
JSON file list, i.e. instaloader profile/*.json to just update all post
metadata.
2018-04-29 11:22:56 +02:00
Alexander Graf
8a8ea2913c Improve post/storyitem-metadata-txt behavior
If an expression evaluates to None, an empty string is yielded, rather
than 'None'.

Blanks (and newlines) are strip()ed from the formatted strings, which
also avoids creation of whitespace-only textfiles.

save_captions parameter to Instaloader constructor has been removed, as
it is equivalent to setting both {post,storyitem}_metadata_txt_pattern
to '' (empty string).

Instaloader.save_caption() now prints the '[{old}] updated [{new}]' if
the caption was updated, rather than printing the new caption twice,
which was confusing.
2018-04-29 11:17:48 +02:00
Alexander Graf
b2dd395a86 Link cli-options in Instaloader __init__ docstring 2018-04-29 10:13:31 +02:00
Alexander Graf
d21fb4154e fine-tune as-module.rst 2018-04-28 21:45:57 +02:00
Alexander Graf
7edc69454d doc: Fix links to Instaloader classes 2018-04-28 19:50:14 +02:00
Alexander Graf
147cd1a580 Post_get_comments() yield namedtuple PostComment 2018-04-28 18:17:00 +02:00
Alexander Graf
0e40da1c70 Let Post.get_likes() yield Profile instances 2018-04-28 17:32:13 +02:00
Alexander Graf
4dcb23cfe5 {Post,Profile,StoryItem}.get_node() -> ._as_dict()
This is consistend with namedtuple._as_dict() and marks it as a
protected member, which is good as it should be only accessed by
save_structure_to_file().
2018-04-28 17:15:42 +02:00
Alexander Graf
3dcfcff04c namedtuples PostLocation and PostSidecarNode
Post.get_sidecar_edges() has been renamed to Post.get_sidecar_nodes().

Post.get_location() has been made to a property Post.location and it now
internally stores location struct and makes it JSON store-/loadable.
2018-04-28 17:11:08 +02:00
Alexander Graf
2c50972e08 Update as-module.rst and docstrings 2018-04-28 15:47:04 +02:00
Alexander Graf
b9ac40899d Theme-related Documentation Enhancements
- Consistent background color in all types of codes
- Improve dd (list item) padding
- Link to Index in global TOC in navbar
- Briefly describe Instaloader in `index.html` title
- Remove page contents (navbartoc) in index, genindex and installation
- Link to new pypi.org rather than old pypi.python.org
2018-04-28 15:47:04 +02:00
André Koch-Kramer
e25eb2a948 Limit GraphQl queries to 20 per 11 minutes
cherry-picked from commit d90c05e0a4
- Set GRAPHQL_PAGE_LENGTH to 50 what appears to be the new working maximum.
- Limit GQL queries to 20 per 666 seconds.
- Remove logic for tracking queries per query identifier as Instagram
  only allows 20 overall GQL queries per sliding window.

Related to #101
2018-04-26 17:13:59 +02:00
André Koch-Kramer
440c59ecb9 Release of version 3.3.10 2018-04-20 18:51:55 +02:00
André Koch-Kramer
c9198d232f Add 'disable=protected-access' to satisfy pylint 2018-04-20 18:47:36 +02:00
André Koch-Kramer
e689dadf00 Output warning when fetching special data
When fetching comments, likes, followees or followers, output a warning
that the used queries are rate limited.
2018-04-20 15:34:43 +02:00
André Koch-Kramer
d90c05e0a4 Limit GraphQl queries to 20 per 11 minutes
- Set GRAPHQL_PAGE_LENGTH to 50 what appears to be the new working maximum.
- Limit GQL queries to 20 per 666 seconds.
- Remove logic for tracking queries per query identifier as Instagram
  only allows 20 overall GQL queries per sliding window.

Related to #101
2018-04-20 12:59:41 +02:00
Alexander Graf
57ce51d4ef Support datetime objects in filter strings
Now --only-if="date_utc<datetime(2018,1,1,hour=12)" is possible.
2018-04-19 15:42:26 +02:00
Alexander Graf
7d2830b8b2 Have --storyitem-filter (--only-if for StoryItems) 2018-04-19 15:12:05 +02:00
Alexander Graf
7688bdce45 Reverse story download order to enable fast-update
Port of 729aa31d86 to v4-dev branch.
2018-04-18 16:38:13 +02:00
Alexander Graf
eecaca7c7b Release of version 3.3.9 2018-04-18 16:25:53 +02:00
Alexander Graf
729aa31d86 Reverse story download order to enable fast-update
Fixes #100.
2018-04-18 16:24:47 +02:00
Alexander Graf
94df60292e Fix bug about 429 handling
It caused Instaloader to fail with a TypeError on 429 Too Many Requests.
2018-04-18 16:24:20 +02:00
Alexander Graf
2d9acd9989 Slightly improve error handling
Use raise from rather than raise where appropriate

Have a logic to reduce graphql page length if 400 Bad Request is
returned
2018-04-18 15:58:45 +02:00
Alexander Graf
09f2eb3601
Set GRAPHQL_PAGE_LENGTH to 12
This is the length the browser requests. Might alleviate #94.
2018-04-18 11:41:00 +02:00
Alexander Graf
066c3de113 get_followe{e,r}s yield Profiles rather than dicts 2018-04-17 15:11:50 +02:00
Alexander Graf
9d8175354b Add paging tests for profiles and stories 2018-04-17 14:23:12 +02:00
Alexander Graf
dcd16f29c2 Split tests TestInstaloader{Anonymously,LoggedIn} 2018-04-17 14:10:27 +02:00