archive about

Showing twitter references in a static blog

Weekend project: How to show under each static blog engine post the tweets that mentioned it.

bucket3 is my static-blog-generation engine, but it's also my little software project that helps me relax and learn new things.

After some time I hadn't added any (important) new features, this weekend I decided to add something I've been thinking about for years: I wanted my blog to show under each post tweets that mentioned it.

There are some easy ways to do it, probably by using Disqus or something like it. But I want my blog to be

  1. static: all important elements have to be served by a simple HTTP server (Amazon S3 currently)
  2. independent: I don't want to rely on any other sites for my blog to render.

I ended up using TwitterSearch, a pytrhon library "to easily iterate tweets found by the Twitter Search API" to add a new command to bucket3, invoked using bucket3 mentions.

When I run bucket3 mentions on my computer (where I write and render this blog, and then upload it to S3), bucket3 queries twitter for any reference to "blog.vrypan.net". The script checks each reference to see if there is a link to a specific blog post, and if so saves the tweet in a local file. It also saves the twitter user avatar, because I didn't want to reference the one hosted by twitter in my pages. Then it marks the corresponding post source (usually a markdown file) as modified.

So, next time I run bucket3 update all modified posts are re-rendered (this has nothing to do with the post pub date) to include the referring tweets.

Some thoughts:

  1. Twitter users can change their profile images any time. Displaying the avatar as it was the time the user mentioned my posts, even if it is later changed, is something I like.
  2. Bucket3 saves the last tweet ID it got, and will only query twitter for tweets after it next time it runs. This means it is easy to delete unwanted tweets (ex. spam) by editing the file (it's a YAML file) that holds the tweets referencing each post.
  3. Twitter's simple design (a single global timeline, with immutable entries that can only be deleted) makes it much easier to implement such a feature, compared to other social networks where posts can be edited, they have comments, etc.
  4. On the other hand, the Twitter API has many limitations, including that it will only return tweets no older than 9 days and that "Search is focused in relevance and not completeness. This means that some Tweets and users may be missing from search results".
  5. I'll probably put bucket3 mentions in a cron job that runs every 30 minutes or so.
  6. It may be a good idea to do some grouping of identical tweets, but I'm not 100% sure. Maybe next time I dig into bucket3's code, I'll try it.

In a way, my static blog has a commenting engine too, now.

In the future, I intend to add more types of "mentions" in addition to twitter. It would be great for example if I could use webmention.io to get and show webmentions.