Sure, plenty of people found it uninteresting, stupid, or felt that something like favstar.fm’s dynamic “most favorited” list creator was a more useful tool (and the favstar.fm lists are definitely awesome, by the way), but that comes with the territory.

The one case where criticism struck me as a little odd was also — perhaps unfortunately — the case where it was coming from the highest profile source. Over the weekend Robert Scoble Twittered “Yo @nk can you block http://conversationlist.com/ from posting to “listed?” That service is VERY spammy and hides real value of lists.”

I know, ouch, right?

I made a little effort to reach out to Scoble for some more detail on his complaints about conversationlists, but haven’t (yet, at least) heard anything back, so I’ll address the possible complaints that I’ve dreamed up here and keep an eye on the comments to discuss anything else that comes up.

Problem One: being added to a bunch of lists called “conversationlist” doesn’t tell me anything about how people view me. It is, therefore, spammy.

I agree that being added to someone’s conversationlist is something different from being added to someone’s “influencers” list, or their “tech” list, but I don’t believe that it’s less valuable information. When you’re added to a conversationlist it’s because that person is actually paying attention to you on Twitter, either trying to engage with you or talking about you. It’s a dynamic state, rather than a static classification.

Granted, you can’t easily build a scorecard from this kind of list. Instead of “500 people have added me to lists named ‘tech’, and I am therefore one of the top tech resources on Twitter,” you have to say something like “I consistently appear on 50 people’s conversationlists, so I am having a measurable impact on the discussions happening on Twitter.”

Better still, you could say something like “why am I significant to this particular group of people? Do they agree or disagree with what I’m saying? Who and what else are they talking about?”

Conversationlists don’t tell you how people think about you directly, they tell you that people are actually interested in you on an ongoing basis.

Problem Two: they’re not [ahem] curated lists, it’s a machine making them. I want to know what people actually think.

Conversationlists are certainly not curated lists in the current buzzword sense, but nor are they created by machines; they are created by people just doing what they already do on Twitter. The machine is only there to keep track of what the people are doing.

Consider this: yes, my “tech” list (if I had one) would a be curated public statement. I would include the people I think are influential (even if I don’t really pay attention to them). I would probably make a point of excluding certain people that I think are overrated (even if I secretly pay attention to those people). I would try to include lesser-known “insider” people, so that other geeks would look at my tech list and say “hey, @whitneymcn didn’t just create that same tech list everybody else did, he knows about @ObscureCoolTechPerson, too.”

My tech list would (of course) be full of fascinating people, but it would also be a view into what I want you to think about me. That’s certainly useful, but it’s not “right” in and of itself. By offering insight on who I’m actually talking to and about, conversationlists give a different view, and one that I think is equally useful. I’m curating that list with my actions, every single day.

Hell, @scobleizer is on my conversationlist right now. I obviously don’t agree with what he’s saying, but I’m sure paying attention to him, and I think there’s value in exposing that fact.

Problem Three: I @reply lots and lots of people, and my friend doesn’t @reply anyone. Conversationlists are useless to us.

I don’t see much of a problem here: don’t use conversationlists if it doesn’t fit what you’re doing on Twitter. [Though do check out the additional tools that we're rolling out, okay?]

Some of the earliest feedback we got on conversationlists, from someone whose opinion I respect, included these two bullet points:

• I don’t do much public conversation in Twitter so my conversationlist is kind of weird.
• Killer idea.

So if I may paraphrase, this person said both “this thing is pretty much useless for me personally” and “I think that this thing is interesting” in the course of about four sentences. Is that crazy? No, I don’t think so. Conversationlists aren’t going to be useful for everyone. For what it’s worth, nor are Twitter lists going to be useful to everyone. The issue, however, is that “not useful to me” is rather different from “not useful.”

Conclusion

There may be other things that bug people about conversationlists, and Kevin and I really would love to hear them. Leave a comment, send an email, let us know. We won’t make every change that everyone would like to see, but we’ll absolutely listen to everything that people have to say.

Before we left Brooklyn I was thinking about Foursquare; I think what Jon Steinberg and crew have done with Social Great is really interesting, but it’s not the direction that really excites me about the potential of the real time Web.

In case you’ve been living under a rock, Social Great tracks the Foursquare checkins across a number of cities and provides near real-time trending data: where are people (in the aggregate sense) going, what places and sorts of places are most popular now, and what trending is happening in those various cities. Really cool.

The issue for me, though, is that while this is a great way to transform Foursquare’s data, I think it doesn’t go far enough in the transformation; while it’s far less “now” focused than Foursquare itself, Social Great is still pointing in that direction. As the poster child for the “real time Web,” Twitter has set the default mindset and behavior to taking real-time input and making it into real-time output–or as close to real-time as the service can handle, anyway. We tend to assume that the value of these little chunks of data decreases like a new car leaving the lot: if you don’t catch it immediately it’s lost a huge part of its value.

But that doesn’t have to be the case. Taking Foursquare as an example, what I was interested in seeing was a map overlay of the full historical data set for my friends: if I’m trying to kill an hour or two in an unfamiliar neighborhood, being able to see which places my friends have gone to  (and which of those they’ve gone to frequently) has more value than knowing where those friends are right now. In aggregate, the little snippets of data become hugely valuable to me.

But there’s another issue here: I’m increasingly irritated by the fact that Twitter is the dumping ground for a lot of this data. Everybody makes it easy to dump their data into Twitter, because Twitter is…yes, you guessed it, the poster child for the “real time Web.” Since Twitter is input == output, though, people’s timelines become increasingly littered with context-free “I’m here,” and “I just bought this,” and “I just heard this” tweets.

It’s pretty close to a 180 for me to say it, but what since Facebook is already somewhat further along in managing distinct classes of real time data that may or may not be visible (think Facebook apps)–and they dearly want in on the real time scene, as long as we’re on the topic–so what if we let them have all this data? As long as I can pull all the data out via an API [yes, that's a big caveat there] I really don’t care where it lives, so why not let Facebook have a shot?

Maybe it’s a terrible idea, but I’m starting to think that we really need an aggregator for all of this stuff, and one that acts as a storehouse rather than a repeater.

Like your favorite bar in the East Village, Twitter sucks now that it’s popular.

That’s the feeling I’ve had for sometime, and — as it turns out — founder @ev and investor @fredwilson agree… at least statistically.

A week or so later, Mark Schoneveld added an interesting note to the discussion that was swirling around Trent Reznor’s very public decision to give up on Twitter. [Um, again, I mean. Give up Twitter again.] As above, you should really read the source document, but here’s an elegant snippet from Mark’s post:

The direct-to-fan connection that Twitter offers can be fun at first – or was fun at first – but dealing with the mainstreamed onslaught must be incredibly difficult for artists. To keep up with and maintain boundaries of privacy and priority starts to be impossible as a point.

On the face of it this looks pretty bad: there are many similar rumbles (and not just appearing on Tumblr, you cynics), and these two examples come from pretty insightful guys who are also tied into communities that have been responsible for a lot of enthusiasm about Twitter.

I agree with Nate and Mark as far as they push it, but I also wonder whether the very real problems that they point to—problems that are unfortunately linked to Twitter’s increased popularity—signal the start of Twitter’s decline as a communication tool, or signal the start of its evolution into a tool that plays a different role in the “real time web.”

While I, too, kind of miss the Twitter that was, I believe that Twitter from 2006 through 2008-ish was one facet of something new and rather different being jury-rigged to fit into an older model of the Web. It was an exciting social communication tool in that context, but I think that where Twitter will be valuable as we go forward is as plumbing, rather than wallpaper.

We’re used to thinking about Twitter as a simple transport mechanism, a largely unfiltered input == output system: each and every tweet is picked up by a little universe of followers, and that’s the end of it. Occasionally we might check Twitter’s trending topics to see what the spammers have to say, but the value of any given tweet is seen as having a mayfly-scale lifetime, where you either catch it the moment when it’s relevant or you just let it flow past.

But if Twitter is just a transport mechanism, we’re pretty close to done.

Trent Reznor simply can’t handle tens or hundreds of thousands of people nitpicking at him in 140 character chunks any better than he can handle email or phone calls from that many people. Nate Westheimer (a really open, accessible guy in my experience, so no criticism of him) will find that his Twitter followers have changed from a community into…well, followers.

If, on the other hand, Twitter (and let’s make Twitter a portmanteau word that includes the rest of the mobile/social world) is nothing more or less than a huge data set about what’s happening right now, then we’ve got something interesting. Yes, this stuff can be rendered right now for the small number of people interested in seeing it, but it can be used in many other ways as well.

If things go as I expect (and hope), yes…any given individual will Twitter less often and there will be less of the “neighborhood bar” feel to the service, people will check in less frequently using Foursquare, and the ambient intimacy thing that has been so compelling will at the large scale become a “hey, do you remember when…” curiosity.

Let’s just accept that and figure out how to use real-time data to inform and enhance what’s happening online, rather than just enjoying the fact that we can know that something happened just a minute ago. If you immediately respond to every single thing that happens, you’re a jittery mess; if you take in everything that happens and give it a little processing time…well, let’s see what happens, shall we?

Well while it’s taken us a while to get there, but Daryn has now rewritten the LOTD bot. While I was on vacation and virtually inaccessible, Twitter disabled the “replies” API call, which meant that the LOTD bot couldn’t access the lyrics that people were posting. And since I was without internet access and had intermittent cell phone coverage, the bot went down and stayed down until Daryn managed to contact me.

After an amusing little incident where I accidentally sent the admin password for the server to hundreds of people, Daryn got coding and wrote LOTD Bot, Mark II. In addition to general improvements, Daryn added one completely awesome feature: now, if the bot can’t get replies from the Twitter API it fails over to do a search on Summize to find new replies to post. Sweet, no? Here are the (slightly updated) requirements and info for LOTD Bot, Mark II:

Requirements
To run a variation of this @lotd script, you must be able to understand Perl well enough to make some basic modifications to the script, and must be able to set up a simple database table (phpMyAdmin is your friend). In addition, you need a server or account at a Web hosting service that:

• Allows you to set up a (small) database.
• Allows you to run perl.
• Has the XML::Simple and LWP::UserAgent perl modules installed, or will allow you to install them.
• Allows you to run scheduled jobs (i.e. cronjobs)

Overview
It’s a very simple setup: there’s a single perl script running on my server that gets the replies posted to @lotd via the Twitter API (or via a Summize search, if the API call fails), loads them into a database table, and then republishes the posts in the lotd account (again via the API). It’s scheduled to run once every 15 minutes, around the clock. The script uses the XML::Simple and DBI modules, but doesn’t have any other dependencies.

To run your own bot using this script, you’ll need to:

1. Create the Twitter account that will be the repost bot.
2. Create a database table as described in the next section.
3. Update the script below:
   • The DB connect information for your database. (Line 14)
   • The Twitter username/password information for your Twitterbot account (there are “read from” and “write to” values, but they’ll point to the same Twitter account in most cases). (Lines 16 – 20)
   • The regex to remove @YourTwitterBot from replies before it reposts them. (Line 40)
4. Upload the finished script to your server.
5. Set up a cron job to periodically run your script (@lotd runs every 15 minutes).

…and that’s pretty much it.

Get the Database Table Structure (MySQL CREATE)
Click the link above, copy and paste, and change the table or field names as seems appropriate.

Get the Example Script
Click the link above, copy and paste, and make the updates noted in the post above. If you change the database table or field names, make sure that you also update the script appropriately.

On a related topic, one of the biggest issues for anyone presenting is getting a read on the audience: how is this going over? Do I need to pick up the pace a little bit, or slow down? Did they all stop listening five minutes ago? One excellent speaker I know builds in a number of jokes as his thermometer, evaluating both how much response he gets to a joke and how quickly that response comes.

There have been a number of experiments with integrating the audience backchannel into a presentation, but as much as I like discussion and exchange, if somebody has a well-constructed, well-rehearsed presentation I don’t want them looking at their laptop every other minute to check out the latest comments from the peanut gallery. I want the actual presentation, but I’d be interested in seeing what presenters could do with some form of audience feedback. Thus the visual backchannel is born.

You make a little device…or rather you make of whole bunch of them. It is perhaps shaped like a ball small enough to fit nicely in a single hand. When someone holds the device, their hand completes a circuit, and via the little wifi radio that you cleverly included in it, the device sends a notification of that change in state to a nearby server. For extra credit, have the device measure the force of the grip and send that data, as well.

You then hand out one of these devices to every audience member as they enter the presentation space, telling them to grip the ball when they’re interested and let go of it when they’re not interested.

All of this data is sliced, diced, and mashed together by the receiving server and presented as some sort of interesting (and very simple) visualization on a monitor that’s visible to the presenter. Something as simple as a graphical representation of how much of the audience is “engaged” and what the engaged/disengaged trend is could be both beautiful and extremely useful.

And from there you’ve got a universe of other interesting possibilities for data analysis and visualization available. Put a more granular view up on a big screen behind the presenter, too, if you want. And on the Web–why not? A site that compares the visualizations from a variety of presenters would be fascinating, right? Or an annotated engagement timeline of the next Jobsnote?

This would, of course, have the side effect of forcing livebloggers to decide whether it was more important to provide feedback to the room in which they are actually sitting or type their every thought for the whole Internet to enjoy, but I think that might actually be a good thing. [Please note that I deleted the joke about how being able to type with one hand while the other was busy may actually be a skill that most bloggers have mastered, as I felt it was in poor taste.]

Any questions? I’m looking in your direction, potential ITP 2009 show participants…

Update (4 hours post-publishing): To expand a little bit on the above, a snippet from an email that I sent to a correspondent who prefers not to be named. I’ll just say that I’m pleased and impressed when people respond very reasonably to criticism that might perhaps be called “snarky.”

What struck me about the email, though, is that in a message that’s about social communication tools being introduced, the email tells people how they shouldn’t communicate (all caps “PLEASE DO NOT REPLY TO THIS EMAIL”) rather than telling them what they should do if they want to communicate with you. Why not shift the focus to providing guidance to subscribers, rather than stopping one particular behavior?

Amazon Recommends Pretending to Target Emails
In other news, the Amazon recommendation email situation that I’ve blogged about before continues its slide into absurdity. For a quick recap:

• Back in the winter of 2005 Amazon would email me about stuff like an opportunity to get an 18% discount when pre-ordering an M83 album because I’d bought “Svefn-G-Englar” by Sigur Rós. Good.
• In the spring of 2007 Kim Cameron wrote about the same sort of positive experience with Amazon recommendation emails. Excellent.
• Come the winter of 2008, things get ugly. Because I have “shopped for electronics” (not purchased, mind you, but shopped for) Amazon emails me to recommend that I buy an Archos DVR. Lame. I have to assume there’s a new hire in the marketing department.

And now it’s the spring of 2008 and that new marketing hire appears to have settled in for the long haul. What’s the latest recommendation email from Amazon?

As someone who purchased video games or music from genres included in the game, you might be interested in our Grand Theft Auto IV music downloads store.

Seriously, Amazon? I’ve purchased “video games or music from genres included in the game?” Wouldn’t it be simpler to just skip a couple of steps and move directly into emailing me whatever your marketing department wants to send, whenever they want to send it? Once useful email program, fast becoming a sad joke.

If we accept that Twitter is in large part a social technology, then we’re presented with a universe of questions about Twitter’s social character. At the most basic level, Twitter’s relationships can be either one way (I follow you) or two way (we both follow each other). One could look at a user to see what percentage of their relationships are mutual follows, but that doesn’t really interest me much, since I suspect there’s a lot of “oh, I guess I should follow them back” noise in there.

Where it gets interesting to me is when groups are exposed: not just you and me following one another, but you, me, and Dr. X all following each other. A mutual follow might just be someone’s Twitter etiquette (Twittiquette?) exposed, but a set of mutual follow relationships seems a more reliable indicator of substantive relationships.

Definition of a Twitter Set
A Twitter Set is a group of three or more Twitter users such that every member of the set follows every other member, and no other members exist for the set. Note that set members need not only follow one another. Also note that I’m leaving the question of whether @Scobleizer and @JasonCalacanis should count for this metric as an exercise for the student.

Why Track This?
I’m not sure. For me, I’m interested in tracking it (or rather, having someone else track it) for pretty much the only reason I ever do anything: it seems like there could be something interesting there; I don’t know what, exactly, but something interesting.

As a starting point, one could build some fascinating visualizations off of this data: seeing where sets exist, how big they are, and how they overlap would be fascinating. How many sets do users tend to belong to? What’s the largest set that exists on Twitter, and who’s in it? How are sets distributed across the universe of Twitter users? Really nice visualization fodder.

More Details, Please
I think you could also reduce this to an interesting metric that shows something (again, I’m not sure exactly what) about how different people qw( use establish expose ) relationships on Twitter. If we say that a user who is part of a single, closed set (i.e. all set members follow only one another) should have a value of one as a baseline, we could use something like this:

A = Total number of sets within the user’s Twitter relationships
B = Number of people that the user is following
C = Size of largest set within the user’s Twitter relationships
User’s Twitter Set Value = (A / B) * C

Since the number of people that the user is following still seems relevant, I’m inclined to explicitly include the following count in the expressed metric, like so:

If you and your four closest friends only follow one another, the “twitter set value” for each of you would be…
(1 / 5) * 5 = 1
…and I’d express that as Twitter Set Value (TSV) = 1(5)

My own case (eyeball estimate, haven’t checked real stats) would be something like:
(8 / 84) * 4 = 0.38
TSV = 0.38(84)

Again, I’m not sure exactly what it would mean, but I’d be really interested in seeing this in action…any takers?

Update: Since Fred Twittered this post, it has reached a somewhat larger audience than expected and I’ll add an addtional note: there will be some interesting new stuff happening with @lotd. I’ve been chatting back and forth with Daryn Nakhuda, a gentleman and a real programmer, about possibilities, and when you put together his ideas, my ideas, and his ability to write sane, reliable code, cool things should result. Stay tuned.

While the logic of “hide your shame” should dictate that I never, ever reveal the code that runs the Twitter Lyric of the Day Club, there’s been enough interest that I’m just going to suck it up and publish it. For any programmers who end up at this post: yes, the code is a little eccentric, and yes, there are a number of ways that it could fall down and hurt itself. Believe me, I know. That said, here’s the rundown:

Requirements
To run a variation of this @lotd script, you must be able to understand Perl well enough to make some basic modifications to the script, and must be able to set up a simple database table (phpMyAdmin is your friend). In addition, you need a server or account at a Web hosting service that:

• Allows you to set up a (small) database.
• Allows you to run perl.
• Has the XML::Simple perl module installed, or will allow you to install it.
• Allows you to run scheduled jobs (i.e. cronjobs)

Overview
It’s a very simple setup: there’s a single perl script running on my server that gets the replies posted to @lotd via the Twitter API, loads them into a database table, and then republishes the posts in the lotd account (again via the API). It’s scheduled to run once every 15 minutes, around the clock. The script uses the XML::Simple and DBI modules, but doesn’t have any other dependencies.

To run your own bot using this script, you’ll need to:

1. Create the Twitter account that will be the repost bot.
2. Create a database table as described in the next section.
3. Update the script below:
   • The DB connect information for your database. (Lines 21, 43, 75)
   • The Twitter username/password information for your Twitterbot account. (Lines 6, 84)
   • The regex to remove @YourTwitterBot from replies before it reposts them. (Line 18)
4. Upload the finished script to your server.
5. Set up a cron job to periodically run your script (@lotd runs every 15 minutes).

…and that’s pretty much it. As I said in my first email to Fred about this, “the hardest part of making something like the @lotd bot is just having the idea — once you’ve got the idea, Twitter makes it easy to build what you’ve got in your head.”

Good luck, have fun, and let me know about anything interesting that you make!

Update: In response to a couple of questions, I guess I didn’t make it explicit—this stuff is released under the Woodie Guthrie license: “This song is Copyrighted in U.S., under Seal of Copyright # 154085, for a period of 28 years, and anybody caught singin it without our permission, will be mighty good friends of ourn, cause we don’t give a dern. Publish it. Write it. Sing it. Swing to it. Yodel it. We wrote it, that’s all we wanted to do.”

Get the Database Table Structure (MySQL CREATE)
Click the link above, copy and paste, and change the table or field names as seems appropriate.

Get the Example Script
Click the link above, copy and paste, and make the updates noted in the post above. If you change the database table or field names, make sure that you also update the script appropriately.

You initially followed a couple of friends and some internet-famous people. Then you added some of the people you work with and some people who always seem to come up with funny things to Twitter. Then some more internet-famous people, and a few genuinely interesting people that you’ve never heard of. And a few more friends. And then a couple of people that you found through @messages sent by the people you already follow.

How many people is that you’re following, now? Do you actually have enough spare bandwidth to make effective use of an input stream that’s made up of the output from 50, or 500, or 5,000 [yes, I'm looking at you, jowyang] people?

Sure, one good answer is to modify Dave Winer’s RSS insight and view Twitter as a river of tweets. You hop in and catch what you can, and don’t worry to much about the stuff that gets carried past you by the current. But I think there’s a limit to how well that works. The different people in my Twitterstream are important and interesting to me for many different reasons; some people I always want to hear from, and and I may not want to hear from (and about) all of them all the time.

So here’s what I propose: a Twitter client that (a) allows you to flag each person that you follow as a member of one or more groups, and (b) allows you to dynamically filter which tweets are displayed, based on group.

With that functionality I can follow all the people who interest me while also ensuring that I don’t miss the output from the small group of people that I’m most interested in. I like both the [shudder] “ambient intimacy” experience of updates from people I know and the “exhibitionist meets stalker” experience of updates from people that I don’t know, and I’d like to be able to keep both of them as a part of my Twitter experience.

Maybe it’s just me, but this feels like something that could be really useful. Any takers?

Like all right-thinking people, I’ve followed Techdirt for years, and over the past couple of years Mike Masnick and I have exchanged a number of emails on various topics. I suppose I vaguely assumed that sooner or later I’d run into Mike in person somewhere along the line, but we’re just email acquaintances who share some interests, you know? There was no specific reason that we would plan to meet, so I never really gave it much thought.

Then last week I wondered aloud — via twitter — how much Mamoun’s falafel my father and I have eaten together over the past 36 years. Four days and an assortment of @replies and d messages later, Mike and I are sitting in Mamoun’s, eating falafel and chatting. Without Twitter, most (if not all) of the data points below would never have been exposed:

• I am a fan of Mamoun’s falafel.
• I live in New York City.
• Mike Masnick is a fan of Mamoun’s falafel.
• Mike Masnick happens to be in New York City this week.

Kind of neat, and an excuse for me to go eat falafel while talking to a smart person, which is always a good thing.

With Twitter, as with Tumblr, there’s an interesting strain of online community floating around. If mySpace is the ugly side of objective-free explicitly social software, maybe these “tools for creating user-defined community” are the up side of the deal. In both cases you build your own community (and therefore your experience of the tool) entirely based on who you follow — there’s no single “there” there (though I mean that in the best possible sense).

While I’m not yet clear on how this plays out on any grander scale (what happens when, with greater numbers of users, the volume of data flowing through makes users more inclined to limit input to only source they already know?) it’s a lot more fun than I expected to be along for the ride.