Adds the --latest-stamps command line option, pointing to a file where
the latest time each profile was scraped. On the next run, only posts
newer than that time are downloaded.
Fixes #1122.
With this change, Instaloader is capable of resuming a previously-aborted download loop. To do so, it creates a JSON file within the target directory when interrupted, that contains all the necessary information to later resume that operation.
Resuming an interrupted download is supported for most, but not all targets. It is supported for:
- Regular profile posts,
- IGTV posts
- Saved posts,
- Tagged posts,
- Explore posts.
This reverts commit 8423e3bd00. The channel did
not have as much activity as expected and an IRC channel does not work if there
are not at least a few people in it.
caption_hashtags is a list of all hashtags that are mentioned in the
Post's caption. It allows to easily filter Posts that have multiple
hashtags, and as such fixes #24.
Further, the documentation of --only-if has been completed by linking to
a description of the syntax in the Python documentation, and by linking
to a list of all defined properties with their meanings. So, this commit
also closes #42.
These are now adapted to how many requests have already been done. With
the current settings, Instaloader does not more than
12 request in the first ten seconds,
28 requests in the first minute,
40 requests in the first two minutes,
63 requests in the first five minutes,
90 requests in the first ten minutes,
and after that 50 requests per ten minutes.
This should make it less likely that Instaloader is rate-limited by
Instagram, while still being fast if downloading only a few posts.
Further, option --no-sleep is hidden in --help output and README.rst.
where FILTER is a boolean expression in python syntax where all names
are evaluated to instaloader.Post properties.
Examples:
instaloader --login=USER --only-if='viewer_has_liked' :feed
instaloader --only-if='likes>1000 and comments>5' profile
This simplifies accessing properties of a Post. Method download_post()
remains to class Instaloader rather than Post, as it fits there better.
Also, since it is now easily possible, all download_*() functions now
have a filter_func parameter. Its meaning has been reverted to be
consistent of how a filter is commonly understood: A post is downloaded
iff filter_func is None or evaluates to True.
Post.get_comments() foreports commit 86fb80d ("Avoid GraphQL queries if
all comments in metadata").
These options instruct instaloader to not save captions or geotags
respectively, even if the regarding information can be obtained without
any additional queries to Instagram.
This feature was proposed in #25, and thus this commit should close #25.
Get rid of NonfatalException (an exception is nonfatal iff it is
catched somewhere)
Foreport fixes for #26 and #30.
The current __sersion__ string is now kept in instaloader.py rather than
setup.py. This lets instaloader.__version__ always deliver the version
of the actually-loaded Instaloader module.
Minor changes to README.rst, error handling and which class methods are
public.
With these and the changes of the previous commit, we saved 31 lines of
code, indicating that it might be easier to understand and to maintain.
Remove many code duplications, merely by using more pythonic idioms.
Use GraphQL more often.
Better cope with errors: All requests can be retried; failed requests do
not cause program termination; all error strings are repeated to the
user at the end of execution.
download_post() (formerly download_node()) does not repeat node metadata
request (before this commit, this request was executed up to three
times).
Instaloader downloads all posts in
<DIRNAME>/<FILENAME>+(suffix and extension)
which are now generated by the templates given with --dirname-pattern
and --filename-pattern. These templates may contain specifiers such as
'{target}', '{profile}', '{date}' and '{shortcode}'.
Default for --dirname-pattern is '{target}', default for
--filename-pattern is '{date:%Y-%m-%d_%H-%M-%S}'
The former options --no-profile-subdir and --hashtag-username were
removed, because their behavior can now be achieved like this:
--no-profile-subdir and --hashtag-username:
--dirname-pattern='.' --filename-pattern='{profile}__{date:%Y-%m-%d_%H-%M-%S}'
--no-profile-subdir, but not --hashtag-username:
--dirname-pattern='.' --filename-pattern='{target}__{date:%Y-%m-%d_%H-%M-%S}'
--hashtag-username but not --no-profile-subdir:
--dirname-pattern='{profile}'
This adds the option proposed in #23, to encode both the hashtag and the
profile name in the file's path when downloading by hashtag, e.g.:
--dirname-pattern='{target}' --filename-pattern='{profile}_{date:%Y-%m-%d_%H-%M-%S}'
(Closes #23)
