Scripting News for 8/26/2006

Heads-up: NYTimesRiver now includes podcasts. The new feeds were added at about 9PM this evening, so most of the items in the river now are podcasts. This should even out over the next few hours. If you want to listen to the podcast, click on the sound icon. Note that some mobile devices can’t play MP3s; this unfortunately includes the Blackberry.  

Something to ponder. How close the various PDA-makers are to something that would kick the iPod in the behind. We just need a quick way to get an MP3 over HTTP. These cell phones, with built-in speakers even, are good at audio.  

NewsRiver feature: Eliminate duplicate stories? 

NewsRiver feature: Goodbye annoying “web bugs!” 

Andrew Baron: “Google Video is set up so that you can actually link to any second in the video.” 

I learned something today on my walk that I did not know. Writely, the Google-acquired web-based word processor, can generate RSS. This I have to check out. If it works well, that means that Writely is actually a blogging tool. Could it possibly be that cool? I’ll report back. BTW, it came from the NY Times Tech Talk podcast.  

Postscript: Hold on to your hats, Writely supports the Metaweblog API. That’s awesome! 

Library Camp East, Darien CT, Sept 25. 

Phil Windley: Which Mobile Device Should I Buy? 

Earlier this week, while trying to sip from a firehose of feedback, Ed Vielmetti pointed out that the BBC river derived from their UK Edition, and that there was more in the World Edition. So it went on my to-do list, and this morning I was able to find them and now they’re tributaries.  

Why carry the water, David? 

David Weinberger: “If FOO doesn’t make an effort to be diverse, the old boys will just naturally become better friends because they spent 2.25 days camping, eating and peeing together.”

As long as you keep carrying their water you’ll be peeing with them. The minute you say something honest and honorable, you’re out. This is why the tech industry is so rotten.

Tatoo this on your forehead: It only works when it’s open to all comers. No matter how much of an “effort” O’Reilly makes, a closed community is still closed.

BTW, keeping it closed is a perfect way to be sure you’ve got a bubble. If you want sustainable growth, you have to keep investing in strange new ideas. That’s what I told John Doerr just after the last bubble burst, but he didn’t listen. You guys aren’t listening now.

By keeping it nice and closed, and keeping everyone you invite too scared to say what’s obvious, you make sure that this little euphoria you think you’re having will pop at the first sign of trouble. There’s nothing sustainable about what you all are doing.

You need to invest as you’re spending, but there’s none of that going on. If you have any guts insist that Tim ask me to keynote his Web 2.0 conference and I’ll lay out a roadmap for investment. That’ll be a good first step toward insurance against this being a bubble.

Why the title works 

In a perfect world every item would have a unique guid.

In a better than perfect world, not only would they all have unique guids, but they’d all be permalinks.

In an even better world, if an item appeared in two or more feeds it would have the same guid (and it would also be a permalink).

But it’s not a perfect world, and even if you put it in the spec that it must work the best possible way (as the Atom 1.0 spec did) there’s no guarantee that it will actually work that way. But it sure would be nice if it did!

We have to live in a less than perfect world. I know many people think I’m omniscient and all-powerful and it’s my fault that the world isn’t perfect, so go ahead and blame me, if it makes you feel better, but that’s not the point of this piece (although my detractors will surely stop right there).

You can do a decent job of figuring out if you’ve seen an item before and not show it to the user if you look at the title of the story. Like the situation I describe above, it ain’t perfect, but then neither is anything else in the world.

Why the title works…

I noticed that headline writers tend to be creative, they don’t come up with different headlines when a story appears in more than one feed, so I tried using that as the guid, it worked! Poof there go the dupes.

Now I’m sure you’ll miss some articles as headlines get re-used, but I’ve found that it’s much more likely to go the other way. As a day goes by the editors play with the titles, making slight, subtle editorial changes. I have a table that tracks this, and it can get really humorous. An inside peek into the mind of an editor. šŸ™‚

Anyway I thought I should share this bit. Have a great Saturday!

12 responses to this post.

  1. Posted by Anton2000 on August 26, 2006 at 6:39 am

    Hello Dave, Opera web browser has a feature to test how a webpage looks in xhtml – View – Small screen.


  2. Posted by Anton2000 on August 26, 2006 at 6:45 am

    And for all the kvetchers and complainers – opera mini is something to kvetch about



  3. A lot of people seem to be asking for this. I’ll create more when I can, with job categories too.

    For people who’d like to create their own the one line of code it takes is available at:


  4. Dave, I’ll bet you’d be interested in Library Camp East, Darien CT, Sept 25 2006.

    You missed the first Library Camp, looks like this is going to be a world tour in the making.


  5. Ed, I would love to go, but I’m going to be in Europe at the end of Sept!

    Please keep me posted, I’ll make it to one of them for sure…


  6. Dave,

    Writely supports posting to your blog. Supports, Blogger API, MetaWeblog API, and Moveable Type API. Cool!


  7. Will keep you posted, Dave – lead time for planning one of these things seems to be three months, so that makes it possible (even reasonable) to think of a Library Camp West in January – just need to find a library willing and able to host it.


  8. I love the idea of NYTimesrRiver and BBCriver but I’m disappointed that they don’t work on most mainstream phones. The reason is simple and easy to fix – page size. The Rivers are 50 – 75 KBytes for the main page. Ordinary mobile phones like the RAZR for example, and just about anything without a keyboard or touchscreen have a page size limit of between 10 and 30 KB. If you limited the number of stories or broke the list of articles of articles up into multiple pages and kept page size down to under 10 KB you’d have a much larger mobile audience for these sites.



  9. Page size can easily be fixed, just let some users go to a page that is shoved through the Google WAP Proxy. It chunks the page up into amounts that smaller phones can handle. Here’s the link to Google’s version of your page, I got this right off my phone (since going to from a desktop browser doesn’t work so hot) which is XHTML compatible. I should probably go from a WAP phone and see how the URL changes.

    I disagree with Dennis in that “just about anything without a keyboard or touch screen” have a page limit size under 30kb. Not true. That’s mostly a problem with WAP only handsets. Most modern phones can handle XHTML. The RAZR isn’t a modern phone, it’s two years old and wasn’t even especially modern when it came out (outside of the design). The bigger problem isn’t page size but that it’s not valid XHTML. Validation is a lot more important on mobiles.


  10. Jon is right, running the Rivers through a transcoder like Google’s is an easy fix to the page size problem. But it’s an extra step. Less than 30% of US mobile users ever access the web from their phones. To drive widespread adoption by non-technical users things need to “just work” without extra steps.

    I stand by my statement that most mainstream phones can’t handle pages over 30KB. I’m talking about the subsidised “free” phones that the vast majority of the populace in this country are using and about the low end and second hand phones that prevail in the developing world. Most non-smartphones now do speak xhtml-mp but they don’t have the memory to process big pages.

    My suggestion is simply that smaller pages would make the Rivers accessible by a much higher percentage of world’s mobile users.


  11. I guess my opinion is that people who get the cheapest phone possible aren’t the ones in the demo that reads news while on the go. They aren’t likely to even have a data plan to be able to access pages (no matter how small they are).

    And there are *plenty* of phones without QWERTY/touchscreen that aren’t the low-tier.

    Top selling phones from

    1. Nokia 6682 (smartphone, fully web capable)
    2. Motorola RAZR V3m (CDMA, OpenWave browser, can handle most pages but might choke on the River since it’s not valid code)
    3. RAZR V3 (poor web support, only WAP)
    4. LG VX8300 (same as V3m web wise)
    5. Samsung D807 (web compatible, not sure the browser)

    While free phones do sell well, there are a lot of mid-tier phones being moved. All of those besides the V3 probably qualify as modern mid-tier handsets.


  12. Writly only alows you to post to one blog. It would be my blog editor of choice if only I could post to multiple blogs on multiple platforms.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: