Anatomy of a Better Twitter Bot

Standard

So you may recall that I put together a Twitter bot for the lyric of the day club a few months ago. You may also recall that I mentioned that my co-conspirator Daryn Nakhuda has the ability to write code that doesn’t, you know, “fail mysteriously and silently” and such.

Well while it’s taken us a while to get there, but Daryn has now rewritten the LOTD bot. While I was on vacation and virtually inaccessible, Twitter disabled the “replies” API call, which meant that the LOTD bot couldn’t access the lyrics that people were posting. And since I was without internet access and had intermittent cell phone coverage, the bot went down and stayed down until Daryn managed to contact me.

After an amusing little incident where I accidentally sent the admin password for the server to hundreds of people, Daryn got coding and wrote LOTD Bot, Mark II. In addition to general improvements, Daryn added one completely awesome feature: now, if the bot can’t get replies from the Twitter API it fails over to do a search on Summize to find new replies to post. Sweet, no? Here are the (slightly updated) requirements and info for LOTD Bot, Mark II:

Requirements
To run a variation of this @lotd script, you must be able to understand Perl well enough to make some basic modifications to the script, and must be able to set up a simple database table (phpMyAdmin is your friend). In addition, you need a server or account at a Web hosting service that:

  • Allows you to set up a (small) database.
  • Allows you to run perl.
  • Has the XML::Simple and LWP::UserAgent perl modules installed, or will allow you to install them.
  • Allows you to run scheduled jobs (i.e. cronjobs)

Overview
It’s a very simple setup: there’s a single perl script running on my server that gets the replies posted to @lotd via the Twitter API (or via a Summize search, if the API call fails), loads them into a database table, and then republishes the posts in the lotd account (again via the API). It’s scheduled to run once every 15 minutes, around the clock. The script uses the XML::Simple and DBI modules, but doesn’t have any other dependencies.

To run your own bot using this script, you’ll need to:

  1. Create the Twitter account that will be the repost bot.
  2. Create a database table as described in the next section.
  3. Update the script below:
    • The DB connect information for your database. (Line 14)
    • The Twitter username/password information for your Twitterbot account (there are “read from” and “write to” values, but they’ll point to the same Twitter account in most cases). (Lines 16 – 20)
    • The regex to remove @YourTwitterBot from replies before it reposts them. (Line 40)
  4. Upload the finished script to your server.
  5. Set up a cron job to periodically run your script (@lotd runs every 15 minutes).

…and that’s pretty much it.

Get the Database Table Structure (MySQL CREATE)
Click the link above, copy and paste, and change the table or field names as seems appropriate.

Get the Example Script
Click the link above, copy and paste, and make the updates noted in the post above. If you change the database table or field names, make sure that you also update the script appropriately.

Anatomy of a Twitter Bot

Standard

Update: If you’re here in search of sample code for a Twitter repost bot, I would strongly recommend going to the Anatomy of a Better Twitter Bot post, which has a much-improved iteration of the LOTD bot code.

Update: Since Fred Twittered this post, it has reached a somewhat larger audience than expected and I’ll add an addtional note: there will be some interesting new stuff happening with @lotd. I’ve been chatting back and forth with Daryn Nakhuda, a gentleman and a real programmer, about possibilities, and when you put together his ideas, my ideas, and his ability to write sane, reliable code, cool things should result. Stay tuned.

While the logic of “hide your shame” should dictate that I never, ever reveal the code that runs the Twitter Lyric of the Day Club, there’s been enough interest that I’m just going to suck it up and publish it. For any programmers who end up at this post: yes, the code is a little eccentric, and yes, there are a number of ways that it could fall down and hurt itself. Believe me, I know. That said, here’s the rundown:

Requirements
To run a variation of this @lotd script, you must be able to understand Perl well enough to make some basic modifications to the script, and must be able to set up a simple database table (phpMyAdmin is your friend). In addition, you need a server or account at a Web hosting service that:

  • Allows you to set up a (small) database.
  • Allows you to run perl.
  • Has the XML::Simple perl module installed, or will allow you to install it.
  • Allows you to run scheduled jobs (i.e. cronjobs)

Overview
It’s a very simple setup: there’s a single perl script running on my server that gets the replies posted to @lotd via the Twitter API, loads them into a database table, and then republishes the posts in the lotd account (again via the API). It’s scheduled to run once every 15 minutes, around the clock. The script uses the XML::Simple and DBI modules, but doesn’t have any other dependencies.

To run your own bot using this script, you’ll need to:

  1. Create the Twitter account that will be the repost bot.
  2. Create a database table as described in the next section.
  3. Update the script below:
    • The DB connect information for your database. (Lines 21, 43, 75)
    • The Twitter username/password information for your Twitterbot account. (Lines 6, 84)
    • The regex to remove @YourTwitterBot from replies before it reposts them. (Line 18)
  4. Upload the finished script to your server.
  5. Set up a cron job to periodically run your script (@lotd runs every 15 minutes).

…and that’s pretty much it. As I said in my first email to Fred about this, “the hardest part of making something like the @lotd bot is just having the idea — once you’ve got the idea, Twitter makes it easy to build what you’ve got in your head.”

Good luck, have fun, and let me know about anything interesting that you make!

Update: In response to a couple of questions, I guess I didn’t make it explicit—this stuff is released under the Woodie Guthrie license: “This song is Copyrighted in U.S., under Seal of Copyright # 154085, for a period of 28 years, and anybody caught singin it without our permission, will be mighty good friends of ourn, cause we don’t give a dern. Publish it. Write it. Sing it. Swing to it. Yodel it. We wrote it, that’s all we wanted to do.”

Get the Database Table Structure (MySQL CREATE)
Click the link above, copy and paste, and change the table or field names as seems appropriate.

Get the Example Script
Click the link above, copy and paste, and make the updates noted in the post above. If you change the database table or field names, make sure that you also update the script appropriately.