Instead of retrying a download attempt answered with a 404, the download
is aborted after the first attempt. Thanks to the _error_catcher(), a
message is printed and Instaloader goes on with the next files to
download.
Further, this commit removes the unneeded NodeUnavailableException and
adjusts docstrings accordingly.
Additional sleeps are necessary because Instagram is rate limiting
GraphQL queries. The error does not occur if not more than 100 queries
are made in a sliding window of eleven minutes.
Fixes #29
If get_node_metadata() is able to provide all comments of a node, no
additional query is needed. Especially GraphQL queries are time
expensive because no more than 100 can be queried in ten minutes. Since
get_node_metadata() does not use GraphQL queries, this is a usefull
tradeoff.
+ Additional error handling
Now, setup.py does not assume to be called from the path where the
source tree resides. This fixes getting the long_description and the
version if setup.py is called from outside.
These options instruct instaloader to not save captions or geotags
respectively, even if the regarding information can be obtained without
any additional queries to Instagram.
This feature was proposed in #25, and thus this commit should close #25.
Get rid of NonfatalException (an exception is nonfatal iff it is
catched somewhere)
Foreport fixes for #26 and #30.
The current __sersion__ string is now kept in instaloader.py rather than
setup.py. This lets instaloader.__version__ always deliver the version
of the actually-loaded Instaloader module.
Minor changes to README.rst, error handling and which class methods are
public.
With these and the changes of the previous commit, we saved 31 lines of
code, indicating that it might be easier to understand and to maintain.
Instead of relying on that datetime.fromtimestamp() raises ValueError if
the timestamp of a story is in milliseconds rather than seconds, we now
compare the timestamp value with a timestamp of the year 2286 to decide
whether to divide it by 1000 or not.
This is motivated by #30, where an OSError is raised in
datetime.fromtimestamp() under Windows.
Remove many code duplications, merely by using more pythonic idioms.
Use GraphQL more often.
Better cope with errors: All requests can be retried; failed requests do
not cause program termination; all error strings are repeated to the
user at the end of execution.
download_post() (formerly download_node()) does not repeat node metadata
request (before this commit, this request was executed up to three
times).
In case a node can not be downloaded or its metadata is needed and can
not be retrieved, a NodeUnavailableException is raised and the according
node will be skipped.
Concerns #26
download_stories() did and does not check if a story is "unseen". The
response to the query of the '/feed/reels_tray/' URL provides all
available stories of the user's followees. Nevertheless, some of them do
not contain an 'items' field which has no causal relationship with their
status in terms of "seen" or "unseen". Therefore, to overcome this lack
of 'items' the 'feed/user/TARGET_USERID/reel_media/' URL needs to be
queried for each relevant followee whose 'items' were not provided in
the first place.
Unless --no-sleep is given, Instaloader waits between requests to the
Instagram servers.
This commit fixes and enhances this behavior. Now, --no-sleep is always
obeyed. Between requests to the instagram.com servers, there is now a
delay of 250 ms ~ 2000 ms. Requests to the file servers do not cause a
delay.
Instaloader downloads all posts in
<DIRNAME>/<FILENAME>+(suffix and extension)
which are now generated by the templates given with --dirname-pattern
and --filename-pattern. These templates may contain specifiers such as
'{target}', '{profile}', '{date}' and '{shortcode}'.
Default for --dirname-pattern is '{target}', default for
--filename-pattern is '{date:%Y-%m-%d_%H-%M-%S}'
The former options --no-profile-subdir and --hashtag-username were
removed, because their behavior can now be achieved like this:
--no-profile-subdir and --hashtag-username:
--dirname-pattern='.' --filename-pattern='{profile}__{date:%Y-%m-%d_%H-%M-%S}'
--no-profile-subdir, but not --hashtag-username:
--dirname-pattern='.' --filename-pattern='{target}__{date:%Y-%m-%d_%H-%M-%S}'
--hashtag-username but not --no-profile-subdir:
--dirname-pattern='{profile}'
This adds the option proposed in #23, to encode both the hashtag and the
profile name in the file's path when downloading by hashtag, e.g.:
--dirname-pattern='{target}' --filename-pattern='{profile}_{date:%Y-%m-%d_%H-%M-%S}'
(Closes #23)
