Scripting News for 10/17/07

NY Times topics in OPML, the mother lode? 

Amyloo was digging around the NY Times code weblog and found this OPML file, weighing in at a monstrous 3.3MB that contains some mysterious but rich data about the NY Times and a guide to using the Times to cover special topics that I don’t think anyone outside the Times knew existed, but there it is, in a public folder, so lets have a look.

1. There are 10522 top-level headlines. There’s no structure to the OPML, it’s absolutely flat.

Here’s an HTML rendering of the list: timestopics.html.

2. It’s a subscription list. Each item has four attributes, type, title, htmlUrl and xmlUrl.

3. The htmlUrl for each element points to a page of stories for the topic. For example, here’s a page of stories about table tennis. On that page is a link to an RSS 2.0 feed containing the same information.

4. The xmlUrl links for at least some of the elements are broken, the error appears to be very simple, if you replace the ampersand with a question mark, it works.

If you look around at the topics you’ll see it’s an incredibly rich set of data. Here are just some of the topics that begin with the letter T: Tableware, Taste, Tattoos, Tax Credits, Tax Evasion, Taxation, Taxicabs and Taxicab Drivers, Tea, Teachers and School Employees, TED Conference News, Teflon, Telephones and Telecommunications, Television, Television Sets, Table Tennis, Terra Cotta, Terrorism, Tests and Testing, Textbooks, Thanksgiving Day.

NY Times metadata 

If you do a View Source on a NY Times story, you’ll see that there’s lots of metadata in the HTML, including keywords for most of the of the stories.

Behind the keywords is a taxonomy that I haven’t seen, but would like to. I asked them to make this public, both at my meeting there last Thursday and in a phone talk this morning. I think there could be a lot of value in the Times taxonomy, it might even set a standard.

In the meantime, I wrote a script last night that tracks the keywords in NY Times stories as they flow through the nytimesriver application. Here’s a report that’s updated once per hour.

http://nytimesriver.com/keywords.html

Obviously it would be interesting to be able to click on the keywords to see what articles reference each of the keywords. And it would also be nice to have a cumulative list and a daily list. Right now all we have is the cumulative version.

But it’s still pretty interesting, bordering on fascinating to think of the possibilities if they provide the framework behind these keywords.

When the pros try to figure out how what they do will continue to make sense after the Internet achieves all its promise, this may be an example. The metadata is generated by librarians, and we don’t as yet have our own librarians in the blogosphere (though some might disagree). And it’s possible that after a release of the taxonomy that something like Wikipedia may happen, with the public taking over maintenence of the taxonomy. No one knows what will happen, but one thing seems clear, there can be value in a news organization beyond the reporting and editing it does.

Unsung flow-builders 

Over the last week, I’ve been writing about the disconnect between flow and rank. Paradoxically, sites that are ranked high don’t always deliver a lot of hits when they link to you.

On the flipside, there are some sites that are rarely on Top 100 lists, or talked about very much, that deliver substantial flow. Two of them stand out, one a veteran site, and the other a relative newcomer.

1. Daring Fireball is a thoughtful blog written by John Gruber that focuses on the Macintosh. Since I’ve returned to the Mac in 2005, and have been writing more about Mac issues, I’ve started getting links from this site, and when I do, they usually send between 1000 and 2000 readers my way. And they’re generally interesting people with useful information and ideas. I follow Gruber on Twitter and have learned that he is a Phillies fan and therefore disappointed this year. His posts are interesting there too, and irreverent, which I like of course. 🙂

2. A Digg-like memetracker, news.ycombinator.com is in the same league as TechMeme, about 1000 hits for a highly ranked piece. I don’t know much about the site, I’m not a regular reader, and I don’t know much about the people who visit from this site.

iPhone SDK coming in Feb 

Apple announced that there will be an SDK for the iPhone.

Hooray!

Funny sign at Web 2.0 

Thanks to Bijan Sabet!

Nokia N810 

Just read about this on Engadget.

I know there’s a Nokia breakfast in SF starting at 8AM, which I will not be able to make, but as an N800 user, if this product really is coming, I can see two thing right off the bat that address major problems with the previous model. 1. Nokia makes good keyboards, but the old model doesn’t have one. On-screen keyboards are a pain, even relatively good ones like the one in the iPhone, but the one in the N800 is not particularly good. 2. The other notable feature is the screen resolution, which looks pretty fantastic.

Anyway, I’ve asked my contacts at Nokia for info as soon as it’s available, but it seems like the Engadget guys are on top of it. If you have any more info, please post a comment here. Thanks.

Later…

Nokia did announce the N810 (data sheet pdf).

Here’s a high-res picture.

A video showing the N810 in action.

Flors: What the N810 means for maemo developers.

Apple’s iPhone SDK announcement 

Note: There was no permalink for the story on Apple’s news website, here’s the full text, with permalink. DW

Let me just say it: We want native third party applications on the iPhone, and we plan to have an SDK in developers’ hands in February. We are excited about creating a vibrant third party developer community around the iPhone and enabling hundreds of new applications for our users. With our revolutionary multi-touch interface, powerful hardware and advanced software architecture, we believe we have created the best mobile platform ever for developers.

It will take until February to release an SDK because we’re trying to do two diametrically opposed things at once—provide an advanced and open platform to developers while at the same time protect iPhone users from viruses, malware, privacy attacks, etc. This is no easy task. Some claim that viruses and malware are not a problem on mobile phones—this is simply not true. There have been serious viruses on other mobile phones already, including some that silently spread from phone to phone over the cell network. As our phones become more powerful, these malicious programs will become more dangerous. And since the iPhone is the most advanced phone ever, it will be a highly visible target.

Some companies are already taking action. Nokia, for example, is not allowing any applications to be loaded onto some of their newest phones unless they have a digital signature that can be traced back to a known developer. While this makes such a phone less than “totally open,” we believe it is a step in the right direction. We are working on an advanced system which will offer developers broad access to natively program the iPhone’s amazing software platform while at the same time protecting users from malicious programs.

We think a few months of patience now will be rewarded by many years of great third party applications running on safe and reliable iPhones.

Steve

P.S.: The SDK will also allow developers to create applications for iPod touch. [Oct 17, 2007]