New doc section: codesnippets / Advanced Examples

Presents code examples that use the instaloader module for more advanced tasks
than what is possible with the Instaloader command line interface.

Presents #46, #56, #110, #113, #120, #121.

docs/as-module.rst

@@ -97,6 +97,9 @@ metadata of a Profile. :class:`Profile` instances can be created with:
 A reference of the many methods provided by the :mod:`instaloader` module is
 provided in the remainder of this document.
 
+For a list of real code examples using the Instaloader module for advanced
+tasks, see :ref:`codesnippets`.
+
 ``Instaloader`` (Main Class)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 

docs/codesnippets.rst

@@ -0,0 +1,105 @@
+.. _codesnippets:
+
+Advanced Instaloader Examples
+=============================
+
+.. currentmodule:: instaloader
+
+.. highlight:: python
+
+.. contents::
+   :backlinks: none
+
+Here we present code examples that use the :ref:`python-module-instaloader` for
+more advanced tasks than what is possible with the Instaloader command line
+interface.
+
+.. For each code snippet:
+   - title
+   - brief description of what it does / motivation / how it works
+   - code, or link to code
+   - link to discussion issue
+   - link used methods
+
+Download Posts in a specific period
+-----------------------------------
+
+To collect pictures (and metadata) only from a specific period, you can play
+around with :func:`~itertools.dropwhile` and :func:`~itertools.takewhile` from
+:mod:`itertools` like in this snippet.
+
+.. literalinclude:: codesnippets/121_since_until.py
+
+See also :class:`Post`, :meth:`Instaloader.download_post`.
+
+Discussed in :issue:`121`.
+
+Likes of a Profile / Ghost Followers
+------------------------------------
+
+To store inactive followers, i.e. followers that did not like any of your
+pictures, into a file you can use this approach.
+
+.. literalinclude:: codesnippets/120_ghost_followers.py
+
+See also :meth:`Profile.get_posts`, :meth:`Post.get_likes`,
+:meth:`Profile.get_followers`, :meth:`Instaloader.load_session_from_file`,
+:meth:`Profile.from_username`.
+
+Discussed in :issue:`120`.
+
+Track Deleted Posts
+-------------------
+
+This script uses Instaloader to obtain a list of currently-online posts, and
+generates the matching filename of each post. It outputs a list of posts which
+are online but not offline (i.e. not yet downloaded) and a list of posts which
+are offline but not online (i.e. deleted in the profile).
+
+.. literalinclude:: codesnippets/56_track_deleted.py
+
+See also :func:`load_structure_from_file`, :meth:`Profile.from_username`,
+:meth:`Profile.get_posts`, :class:`Post`.
+
+Discussed in :issue:`56`.
+
+Only one Post per User
+----------------------
+
+To download only the one most recent post from each user, this snippet creates a
+:class:`set` that contains the users of which a post has already been
+downloaded. When iterating the posts, check whether the post's owner already is
+in the set. If so, skip the post. Otherwise, download it and add the user to
+that set.
+
+.. literalinclude:: codesnippets/113_only_one_per_user.py
+
+See also :class:`Post`, :meth:`Instaloader.download_post`,
+:attr:`Post.owner_profile`, :class:`Profile`.
+
+Discussed in :issue:`113`.
+
+Upgrade Images by local Copies
+------------------------------
+
+The following script finds local versions of images fetched by Instaloader, in
+order to ugprade the downloaded images by locally-found versions with better
+quality. It uses image hashing to identify similar images.
+
+`updgrade-instaloader-images.py <https://gist.github.com/pavelkryukov/15f93d19a99428a284a8bcec27e0187b>`__ (external link to GitHub Gist)
+
+Discussed in :issue:`46`.
+
+Render Captions to downloaded Images
+------------------------------------
+
+Instaloader does not modify the downloaded JPEG file. However, one could combine
+it with an imaging library such as Pillow or PIL to render the
+:attr:`Post.caption` on pictures.  The following shows an approach.
+
+.. literalinclude:: codesnippets/110_pil_captions.py
+
+See also :attr:`Post.caption`, :attr:`Post.url`, :meth:`Post.from_shortcode`,
+:func:`load_structure_from_file`.
+
+Discussed in :issue:`110`.

docs/codesnippets/110_pil_captions.py

@@ -0,0 +1,20 @@
+from io import BytesIO
+
+from requests import get
+from PIL import Image, ImageDraw
+from instaloader import *
+
+L = Instaloader()
+
+# Load Post instance
+post = load_structure_from_file(L.context, '2017-10-01_18-53-03_UTC.json.xz')
+# or post = Post.from_shortcode(L.context, SHORTCODE)
+
+# Render caption
+image = Image.open(BytesIO(get(post.url).content))
+draw = ImageDraw.Draw(image)
+color = 'rgb(0, 0, 0)'  # black color
+draw.text((300,100), post.caption.encode('latin1', errors='ignore'), fill=color)
+
+# Save image
+image.save('test.jpg')

docs/codesnippets/113_only_one_per_user.py

@@ -0,0 +1,14 @@
+import instaloader
+
+L = instaloader.Instaloader()
+
+posts = L.get_hashtag_posts('milfgarden')
+
+users = set()
+
+for post in posts:
+    if not post.owner_profile in users:
+        L.download_post(post, '#milfgarden')
+        users.add(post.owner_profile)
+    else:
+        print("{} from {} skipped.".format(post, post.owner_profile))

docs/codesnippets/120_ghost_followers.py

@@ -0,0 +1,27 @@
+import instaloader
+
+L = instaloader.Instaloader()
+
+USER = 'your_account'
+PROFILE = USER
+
+# Your preferred way of logging in:
+L.load_session_from_file(USER)
+
+profile = instaloader.Profile.from_username(L.context, PROFILE)
+
+likes = set()
+print('Fetching likes of all posts of profile {}.'.format(profile.username))
+for post in profile.get_posts():
+    print(post)
+    likes = likes | set(post.get_likes())
+
+print('Fetching followers of profile {}.'.format(profile.username))
+followers = set(profile.get_followers())
+
+ghosts = followers - likes
+
+print('Storing ghosts into file.')
+with open('/YOUR PATH/inactive-users.txt', 'w') as f:
+    for ghost in ghosts:
+        print(ghost.username, file=f)

docs/codesnippets/121_since_until.py

@@ -0,0 +1,17 @@
+from datetime import datetime
+from itertools import dropwhile, takewhile
+
+import instaloader
+
+L = instaloader.Instaloader()
+
+posts = L.get_hashtag_posts('milfgarden')
+# or
+# posts = instaloader.Profile.from_username(L.context, PROFILE).get_posts()
+
+SINCE = datetime(2015, 5, 1)
+UNTIL = datetime(2015, 3, 1)
+
+for post in takewhile(lambda p: p.date > UNTIL, dropwhile(lambda p: p.date > SINCE, posts)):
+    print(post.date)
+    L.download_post(post, '#milfgarden')

docs/codesnippets/56_track_deleted.py

@@ -0,0 +1,34 @@
+from glob import glob
+from sys import argv
+from os import chdir
+
+from instaloader import Instaloader, Post, Profile, load_structure_from_file
+
+# Instaloader instantiation - you may pass additional arguments to the constructor here
+L = Instaloader()
+
+# If desired, load session previously saved with `instaloader -l USERNAME`:
+#L.load_session_from_file(USERNAME)
+
+try:
+    TARGET = argv[1]
+except IndexError:
+    raise SystemExit("Pass profile name as argument!")
+
+# Obtain set of posts that are on hard disk
+chdir(TARGET)
+offline_posts = set(filter(lambda s: isinstance(s, Post),
+                           (load_structure_from_file(L.context, file)
+                            for file in (glob('*.json.xz') + glob('*.json')))))
+
+# Obtain set of posts that are currently online
+post_iterator = Profile.from_username(L.context, TARGET).get_posts()
+online_posts = set(post_iterator)
+
+if online_posts - offline_posts:
+    print("Not yet downloaded posts:")
+    print(" ".join(str(p) for p in (online_posts - offline_posts)))
+
+if offline_posts - online_posts:
+    print("Deleted posts:")
+    print(" ".join(str(p) for p in (offline_posts - online_posts)))

docs/conf.py

@@ -39,7 +39,8 @@ import docs.sphinx_autodoc_typehints as sphinx_autodoc_typehints
 extensions = [
     'sphinx.ext.autodoc',
     'sphinx.ext.githubpages',
-    'sphinx.ext.intersphinx'
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.extlinks'
 ]
 
 autodoc_default_flags = ['show-inheritance', 'members', 'undoc-members']
@@ -49,6 +50,8 @@ autodoc_member_order = 'bysource'
 intersphinx_mapping = {'python': ('https://docs.python.org/3', None),
                        'requests': ('http://docs.python-requests.org/en/master', None)}
 
+extlinks = {'issue': ('https://github.com/instaloader/instaloader/issues/%s', 'issue #')}
+
 # Add any paths that contain templates here, relative to this directory.
 templates_path = ['_templates']
 

docs/index.rst

@@ -59,6 +59,7 @@ Instaloader Documentation
    basic-usage
    cli-options
    as-module
+   codesnippets
    contributing
 
 Useful Links