Scripting News for 3/16/2007

NY Times on Twitter 

Okay, I got the bugs worked out, and it seems to work, fingers crossed, praise Murhpy, I am not a lawyer and I don’t work for the NY Times. 🙂

It’s set to post one new story per minute, as long as they are available. I may decrease that to once every three minutes or every five minutes. I’m interested in what people think.

“All the news that’s fit to twit.”

How important it is for publications and publishing platforms to be open. Because Twitter has a public API that allows anyone to add a feature, and because the NY Times offers its content as a set of feeds, I was able to whip up a connection between the two in a few hours. That’s the power of open APIs.

Rex Hammock: “Hey, Twitter is a newsreader.”

Don Park: “There he goes again.”

Link of the day 

A new feature! When Today’s Links would only have one item, we give an award to the one link that made a Links section necessary. The first such honor goes to Mary Hodder.

Mary Hodder: “Cingular == evil.” Concise.

Microsoft in it to win? 

I see Scoble is up pretty late too. 🙂

I had this thought when I read that Microsoft is “in it to win,” even before I read Scoble’s essay.

I’m a fan of Deal or No Deal. I know it’s a stupid show, and that’s why I like it.

I can’t believe the misplaced arrogance of some of the contestants. Last night I saw a guy who had one big number left on the board decide to go for it. The box he opened of course was the big number. So he went from having an offer on the table of $40K to $1K in a single move. That’s okay, I’m still going to win, he says. His family backs him up.

Okay, what else is he supposed to say, you might ask? Well, good question.

He should be saying “My it’s nice to have made $40K for 25 minutes worth of work.”

In other words, when you boast of how smart you are, when smarts has nothing to do with winning or losing, you look pretty fcuking dumb.

Microsoft isn’t in it to win because Microsoft can’t win, any more than the guy with just one big number on the board can. Sure there’s a infinitesmal sliver of hope, but not enough to bet your future on.

I don’t know what I’d say if I were Microsoft now. I would try to divest in the system that produced Vista as much as I possibly could. When a big tree falls, it creates room for new growth. It takes a long time for a tree the size of Microsoft to fall on its own. And it’s very hard for an exec at such a company to make big parts of it fall before they have to.

That’s why IBM was the last of the Big Iron companies to collapse after the advent of the fractional horsepower computer (aka the PC). They were the biggest, so they had the furthest to fall. Microsoft is in the unenviable position of being the IBM of our age.

Programming Twitter at 4AM 

I guess the next step in my exploration is to program a simple app that uses the Twitter API. And just like the old days, in the 70s, I’m up in the middle of the night because that’s when the system is fast enough to actually get some work done.

During the day, Twitter is very slow. As a user this is somewhat workable, but if you’re developing an app, it’s excruciating. And it’s possible that my app might not function at all when the system is overloaded. I’m feeling this stuff out. Any help from someone experienced at developing on Twitter would be much appreciated.

Okay, I don’t want to announce what I’m trying to create, because I might not be able to get it to work, and then there will be users who will be disappointed when I turn the thing off (that never really worked). But in general, it’s a newsbot. I’m sure a lot of other people are doing these, it’s kind of an obvious thing to try, right?

Anyway, I’m wondering if Twitter keeps you from posting too frequently and if so, what’s the maximum rate you can post? I’m finding that after doing some work, the connection closes as I’m waiting for a response. It could be that I’m being throttled, or it could be that the server is having problems even in the middle of the night, even when the web UI is fast.

If you have any insight, please post a comment.

WSJ head buried in sand? 

This WSJ article reminds me of the early articles in MSM about blogging. I wonder if any programmers at the WSJ are experimenting with newsbots for Twitter (see above) or if they’re just burying their heads in the sand.

At the very least this is a good sandbox for experimentation. As I said yesterday, what matters is that there are users. I wonder if they’ll ever figure this out, that when they dis something that people like, they’re losing the respect of those people.

They really make Tara Hunt sound like a b*tch. I know enough to make a distinction between sounding like one when passed through their filters and actually being one.

20 responses to this post.

  1. middle of the night for you is noon for someone on the other side of the planet. i see the UI fast and slow at all hours.


  2. Dave, fyi – be aware that when you send the same update twice over the API, it shouldn’t be posted twice. This is what makes RSS-to-Twitter scripts easy to code, because you don’t have to check with Twitter to see if it’s already been posted, you just post it again and let Twitter sort it out. My ‘bartsf’ and Tube scripts just pull the data in, look for any ‘problems’ and send them. If the same problem is being reported, it shouldn’t send out two updates unless there is a change in the actual message.

    Does it work? Yes. I’ve been subscribed to my own transport update feed on Twitter for the last few weeks and I haven’t had any updates that I shouldn’t have had. It can lead to unexpected stuff when testing scripts though.


  3. Tom, I’m seeing something else.

    Everything I post to Twitter is failing.

    Could you try posting something through the API right now and see if it works?

    I can’t tell if it’s me or the backend.


  4. I should add, about an hour ago, everything was getting through, so I believe the code on my end is doing the right thing.


  5. Just did an update via curl on my Mac. Seems to be working for me.

    The command to test the API from the command line:
    curl -u [username]:[password] -d “status=[status]”|json]

    Set username, password, xml/json, and status needs to be escaped as with URLs. I just posted a message to my account fine. Try running this curl command on your Mac and see if it works. I tested the JSON and the XML versions and they both seem to be fine.


  6. Twittermap isn’t updating it appears. So I don’t have a clue.


  7. Dave. I’m currently working on a tool called Twitter POP. It allows you to post ever n minutes. I’ve had it running every 15 minutes with no problems, but usually have it set for every 60 minutes. A few beta testers have been posting more often with no problems. Colin


  8. Posted by Mikael on March 16, 2007 at 5:51 am

    I do know the BBC is posting heavily to Twitter.


  9. Twitter IM is offline currently (and every time I tried it, it didn’t work). Possibly overloaded.


  10. Posted by stephen bove on March 16, 2007 at 7:41 am

    Danny Hillis and co at MetaWeb launch…

    It’s a closed beta for now…

    Here’s what Esther Dyson is saying about it:


  11. Twitter is indeed excrutiatingly slow to develop on at times. Last night is was so bad. Then suddenly it was ok.

    And OK enough for me to finish up my Interactive TwitterBot – it’s alot of fun

    If you want to test a post direct via a API post try

    It uses the same simple code as my Twitter Confessional

    Which I also published the source code to on my blog

    I also used the API to create an OPML file of the last posts of my friends or followers, I think this should work with the OPML Editor ‘Instant Outliner’ / Subscription system (which dodesn’t seem to be working any more)

    Trying to help.


  12. You can also get a feed of fresh podcasts at – but it’s quite high volume – which is in itself a good thing


  13. Posted by Kosso on March 16, 2007 at 9:46 am

    Deal or No Deal – while being a brain dead gameshow anyway – SUCKS in the USA.

    They have to use models to hold the boxes and open them. And also ruin the surprise with their plastic facial expressions as they open them up.

    In the UK, the people with the other boxes are the other contestants. Much better if you ask me. They get a real sense of community going there.

    The USA version is typical American Schmaltz-A-Vison


  14. Just saw the NYT River; exactly what I was looking to do for our blog feed using a Twitterbot..awesome! BTW: I read about the new river on Twitter as it hasn’t shown up in my RSS reader (Google) yet. THAT’s what I like about Twitter, the instant gratification factor. 🙂


  15. Kevin, glad you like it!

    I’m going to do a general web service that will poll a feed and post updates to a Twitterbox. I’d be honored to use your feed as a test case.


  16. Sounds good to me and I’ll be curious to poll our reader base to see what impacts, implications, benefits there are. That’s the other half of the equation IMO, the user experience. Great stuff as always Dave; thanks!


  17. Hope you will decrease time between NYT posts.

    every five or 10 minutes would work.

    thanks, it’s an interesting app.


  18. Dave,

    I like your NY Times river of news but I found a number of them already on twitter. Check out the following twitter names: CNN CNNBRK NYTIMES BBCNEWS (just type BBC and you will see a number of feeds) CNETNEWS etc. I see GOOGLE and YAHOO are taken but no postings yet (not sure if they are the real thing or not).

    You type in the first few chars in the “Find name” box on your twitter page and it will bring back matches starting with those characters.


  19. Dave, Of course you want to play with the new toys. And why not?

    But (best “Carthage must be destroyed” voice) your newsrivers were way sleeker and more usable than this is so far. I still say they were worth paying $ for, more than Times Select. Bring back the NYT River.


  20. Hey Dave, very cool but a bit too much flow for me (I use the SMS service and use twitter between all my friends). The first example of pumping RSS into twitter that I have seen was Gabe Rivera with Techmeme. What is interesting about it is he joined twitter himself, he announced himself on twitter, he immediately had a few requests to have techmeme ‘join’ twitter as well, in an interesting series of events it all played out on twitter within a few hours, see Gabe’s posts:

    Dave, as a suggestion, how about a generic mashup interface that lets me add any feed into my twitter? Not as a seperate source, but for the feed to be posted into twitter under my username The first thing I would do is to add flickr,, youtube, my blog, my diggs all into my stream so that my subscribers can *really* see everything I am doing at any moment. It would be my ‘life stream’ – more than just my twittering (though that would be a component). I think twitter would be the ultimate aggregator/social network to have around your ‘life stream’..

    imagine all your friends being able to see almost every action that you take on the web, except for those you wish to keep private. i think that is a lot more powerful that just posting short messages. if you don’t write it, i will do it myself 🙂 I don’t think anybody else has though of this (all the RSS action i have seen is to bring RSS in as another twitter source – a bit like a proxy)


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: