Anatomy of a Better Twitter Bot

Standard

So you may recall that I put together a Twitter bot for the lyric of the day club a few months ago. You may also recall that I mentioned that my co-conspirator Daryn Nakhuda has the ability to write code that doesn’t, you know, “fail mysteriously and silently” and such.

Well while it’s taken us a while to get there, but Daryn has now rewritten the LOTD bot. While I was on vacation and virtually inaccessible, Twitter disabled the “replies” API call, which meant that the LOTD bot couldn’t access the lyrics that people were posting. And since I was without internet access and had intermittent cell phone coverage, the bot went down and stayed down until Daryn managed to contact me.

After an amusing little incident where I accidentally sent the admin password for the server to hundreds of people, Daryn got coding and wrote LOTD Bot, Mark II. In addition to general improvements, Daryn added one completely awesome feature: now, if the bot can’t get replies from the Twitter API it fails over to do a search on Summize to find new replies to post. Sweet, no? Here are the (slightly updated) requirements and info for LOTD Bot, Mark II:

Requirements
To run a variation of this @lotd script, you must be able to understand Perl well enough to make some basic modifications to the script, and must be able to set up a simple database table (phpMyAdmin is your friend). In addition, you need a server or account at a Web hosting service that:

  • Allows you to set up a (small) database.
  • Allows you to run perl.
  • Has the XML::Simple and LWP::UserAgent perl modules installed, or will allow you to install them.
  • Allows you to run scheduled jobs (i.e. cronjobs)

Overview
It’s a very simple setup: there’s a single perl script running on my server that gets the replies posted to @lotd via the Twitter API (or via a Summize search, if the API call fails), loads them into a database table, and then republishes the posts in the lotd account (again via the API). It’s scheduled to run once every 15 minutes, around the clock. The script uses the XML::Simple and DBI modules, but doesn’t have any other dependencies.

To run your own bot using this script, you’ll need to:

  1. Create the Twitter account that will be the repost bot.
  2. Create a database table as described in the next section.
  3. Update the script below:
    • The DB connect information for your database. (Line 14)
    • The Twitter username/password information for your Twitterbot account (there are “read from” and “write to” values, but they’ll point to the same Twitter account in most cases). (Lines 16 – 20)
    • The regex to remove @YourTwitterBot from replies before it reposts them. (Line 40)
  4. Upload the finished script to your server.
  5. Set up a cron job to periodically run your script (@lotd runs every 15 minutes).

…and that’s pretty much it.

Get the Database Table Structure (MySQL CREATE)
Click the link above, copy and paste, and change the table or field names as seems appropriate.

Get the Example Script
Click the link above, copy and paste, and make the updates noted in the post above. If you change the database table or field names, make sure that you also update the script appropriately.

  • http://www.tachophobia.com RacerRick

    Thank you whitney.

    We're updating our twitter bots to work with both identica and twitter, btw.

  • http://www.tachophobia.com RacerRick

    Thank you whitney.

    We're updating our twitter bots to work with both identica and twitter, btw.

  • Pingback: seamonkeyrodeo » Anatomy of a Twitter Bot

  • http://www.coovents.com Gregory Schnese

    Hi,

    Thanks for updatting this script. I'm running your new version but I'm getting this error: Not an ARRAY reference … line 141.

    Line 141 starts with “foreach my $item (@{$data->{status}}).”

    Is anyone else getting this error?

    Thanks,

    Greg

  • http://www.coovents.com Gregory Schnese

    Hi,

    Thanks for updatting this script. I'm running your new version but I'm getting this error: Not an ARRAY reference … line 141.

    Line 141 starts with “foreach my $item (@{$data->{status}}).”

    Is anyone else getting this error?

    Thanks,

    Greg

  • http://smr.absono.us whitneymcn

    Hey, Greg -

    My first guess is that it's a known bug that I should document (or fix,
    for that matter), where the script doesn't gracefully handle cases where
    the call to the Twitter API to get replies is successful but returns no
    replies. (When I originally wrote the script @lotd already had a bunch
    of replies so I didn't encounter the bug while putting it together.)

    If that's it, the simplest workaround is to just seed the Twitter
    account by sending it an @reply — as long as there's at least one for
    the script to find you're good from there on out.

    Let me know if that doesn't work and I'll dig into it a little.

  • http://smr.absono.us whitneymcn

    Hey, Greg -

    My first guess is that it's a known bug that I should document (or fix,
    for that matter), where the script doesn't gracefully handle cases where
    the call to the Twitter API to get replies is successful but returns no
    replies. (When I originally wrote the script @lotd already had a bunch
    of replies so I didn't encounter the bug while putting it together.)

    If that's it, the simplest workaround is to just seed the Twitter
    account by sending it an @reply — as long as there's at least one for
    the script to find you're good from there on out.

    Let me know if that doesn't work and I'll dig into it a little.

  • http://www.coovents.com Gregory Schnese

    Thanks for the help, it works now!

  • http://www.coovents.com Gregory Schnese

    Thanks for the help, it works now!

  • http://tony.eggtea.com eggtea

    Sweet script I'm using it @ twitter.com/ourideas

  • da smart

    Im have that same issue:
    Not an ARRAY reference at ./crifbot.pl line 141.

    I have one some tweets and I have one @reply as well.

    the perl prints out:

    root@tools ~]# ./crifbot.pl
    TWITTER content: <?xml version=”1.0″ encoding=”UTF-8″?>
    <statuses type=”array”>
    <status>
    <created_at>Thu Jan 29 15:24:29 +0000 2009</created_at>
    <id>1158687922</id>
    <text>@crifdogs test?</text>
    <source>web</source>
    <truncated>false</truncated>
    <in_reply_to_status_id></in_reply_to_status_id>
    <in_reply_to_user_id>19604210</in_reply_to_user_id>
    <favorited>false</favorited>
    <in_reply_to_screen_name>mysuername</in_reply_to_screen_name>
    <user>
    <id>5371582</id>
    <name>Guy that replied</name>
    <screen_name>guys_screenname</screen_name>
    <location>nyc</location>
    <description>pithy description</description>
    <profile_image_url>http://s3.amazonaws.com/twitter_production/profile_images/normal.jpg</profile_image_url&gt;
    <url>http://google.com</url&gt;
    <protected>false</protected>
    <followers_count>42</followers_count>
    </user>
    </status>
    </statuses>
    Not an ARRAY reference at ./crifbot.pl line 141.

  • Pingback: Knuckle Sandwich » Blog Archive » links for 2009-02-05

  • wreckandsalvage

    Complete noob question, but how do I save the script? As a .cgi or .pl?

  • http://smr.absono.us whitneymcn

    No question too simple… :)

    Save as .pl

  • wreckandsalvage

    Thanks!

    I'm still running into problems though for some reason. I have the thing chmod 755 in http://www.bullemhead.com/cgi-bin/

    and it's directed to the right location of perl on my (Dreamhost) server.

    Still giving me errors. Any ideas?

  • wreckandsalvage

    Sorry, I figured it out. Thanks!

  • http://smr.absono.us whitneymcn

    No worries, glad you got it working.

  • http://smr.absono.us whitneymcn

    No worries, glad you got it working.

  • Pingback: @SanMo, a Twitter service for locals « RobotSkirts

  • Pingback: Retweeting in Python

  • http://icfun.blogspot.com wolf
  • http://www.webhostingreviewz.com/ web hosting review

    Have to say Thank You for the script. Scripts like this just are new life way.

  • http://www.tourtravelchina.com/ China Tour

    Perfect – Just what I was looking for. Thanks for showing this!

  • http://www.101waystomakemoney.com/ Ways to make quick money

    I can imagine what it was in head of this businessman. Funny video. I will forward this video to my friends.

  • http://gadgettechblog.com/ Gadget_Blog

    Very very interesting post..I like this one. gotta bookmark this one.rnrnCheers,rngadgettechblog.comrn

  • http://www.passiontips.info wwryte

    There are quite a few uses I could think of for an automated Twitter bot that posts new tweets for you throughout the day. While this sounds like it would be a hard task it’s actually quite easy and a great project for anyone who wants to learn how to use the Twitter API within PHP

  • http://www.premierhealthcareonline.com/ affordable health care

    Different point of view from that post. Interesting to say the least.
    affordable health care

  • http://ericbaumer.com/ Eric

    just a quick note. I noticed that, if there is more than one tweet when the script is run, the bot account sends them out in a psuedo random order (using an associative array doesn't guarantee anything about the order in which they'll be returned). I'm happy to give you comment of the modified perl script if you're interested.

  • http://absono.us whitneymcn

    Excellent, Eric — if you want to post the updated version just the me
    the URL and I'll drop a link to it in here, or if you prefer to send
    it along I'll update this post.

  • Conner Web

    Twitter is really becoming a big yet. I think one day it may even surpass Myspace. Make Money Online

  • dinheiro

    Congratulations, this is an excelent post!

  • makemoneyonlinegenuine

    great stuff thanks : )

  • bob m

    I learned a lot from this post!

  • http://www.freelegitmoney.com/ Make money online

    Thanks for the great tips. I was really impressed by this one.

  • http://www.bondmedia.co.uk/ Web Design London

    I must say that overall I am really impressed with this blog.It is easy to see that you are impassioned about your writing. I wish I had got your ability to write. I look forward to more updates and will be returning.

  • http://jermainelovepoemsandlovequotes.blogspot.com/ Love Poems & Love Quotes

    That is pretty cool! Twitter is affecting so many people these days.

  • http://www.premiumseo.co.uk SEO Cambridge

    Great! love to hear about the identica bot too

  • http://www.loftbedplan.org Joy

    Twitter is dominating now. Good job for them.

  • http://www.superawesomedating.com awesome

    You could also you yahoo pipes, its a bit complicated because their are so many steps, but technically, its not to difficult.