Scripting News for 7/16/2007

Call out to the NY Times 

I’m a regular listener of the NY Times TechTalk podcast. It’s well done and usually interesting. The last few weeks they’ve had a segment on the iPhone, and last week reviewed some of the websites that work especially well on the device.

I was hoping they’d mention the nytimesriver site, after all, they write for the Times, and presumably read the Times, and since they all have iPhones, well I hoped they’d overlook any potential conflicts and tell their listeners who very likely read the Times too, and would want to on their mobile devices, including of course the iPhone. Alas, no mention.

I keep wondering how this great site can remain such a secret for so long. gave it tons of flow, but that’s dropping off now. I’ll keep thinking of ways of spreading the word.

OPML 2.0 spec work 

I’ve started to work on the final pass of the OPML 2.0 spec, in preparation for freezing it. The last changes were made on 3/1/06, well over a year ago. There haven’t been comments in months. People have been quietly deploying OPML 2.0, even though there’s a clear Do Not Deploy caveat at the top of the document.

Today I made a change to the explanation of the new-in-2 <ownerId> element. In the previous version it said the address had to point to a page with a form that allowed the reader to communicate with the author. It no longer says it must be a form. Further, in the previous version it didn’t say that it must be unique to the author. Now it does.

Please use the comment section on the annex site for feedback.

17 responses to this post.

  1. A compliment and a question for you on the site: First, I really love being able to see the stories update in real time. But I wish I could pull it in through my RSS reader on my Blackjack, because there are times when I want to blog something but can’t stop and do it right then and there. Are there plans to have this site available via RSS?


  2. What does “flatdown” mean? I know because I’ve implemented an OPML outliner before, and I had to figure it out, because you didn’t define the term before, and it’s still not defined in the 2.0 spec. But a spec should define new words it invents, especially when understanding them is crucial to implementing the spec properly.

    I don’t like the way the “flatdown” line count is dependent on the opened/closed state of the outline. That makes it inconvenient to represent, because the “flatdown” count changes when you open or close a node. And it makes it impossible to save the opened/closed state of sub-nodes whose parents are closed. It implies that when you close a node, all the child nodes are closed and their opened/closeed state is forgotten. So if you close the top level and open it back up again, the outline is not in the same state as it was before. It would be better to represent whether or not each node is opened or closed individually, instead of the “flatdown” scheme. But if you insist on using the “flatdown” scheme, then at least you should define the meaning of that term.

    Of course maybe I totally misunderstand what you mean by “flatdown”. But that’s why you should define what that term means. I’ve never heard anyone but you use that term.

    Here’s a link to somebody else who thinks “flatdown” is a confusing term, and doesn’t like how it works:

    “And it’s pretty bad. For example, to understand how to serialize node expansion states, you need to understand what “navigate flatdown X times and expand” means. And if it means what I think it does, then every time you expand or close (or move, add or delete) a node, you have to re-calculate the expansion states for the whole document below that node.”



  3. Also: What is the XML namespace of OPML 2.0? Or do you still not recognize XML namespaces as being important enough to use? You should read Don Box’s book, “Essential XML”, which explains them quite well, and makes the case for why they are important.

    That book is several years old, so there may be better references, but that’s where I learned about namespaces, and it was a great book. In the years that have passed since that book was published, XML namespaces have only become more important (especially for applications like OPML), as XML has become more widely used. If you insist on fighting against XML namespaces, you might as well be using YAML or JSON.



  4. Karoli, the site is generated from the Times’s RSS feeds, so if you subscribe to them you’ll get the same content that’s in the site. So no, I won’t be publishing any new feeds, no need to do it, the Times already does it.


  5. At the time I wrote 1.0 spec (where expansionState first appeared) I didn’t think it was necessary to define it because the only people who would care about the element (or so I thought at the time) were people who were writing outliners. Remember that OPML originated as the file format for an outliner. The only time you need to do anything with expansion state is when reading or writing an outline. I don’t understand what your friend’s issue is, and what he is recalculating every time something expands or collapses.

    About namespaces, what’s the problem? I say in the spec that you can include elements from other namespaces. I have applications where I use OPML as a namespace. What else would you like?


  6. Dave, as you know I’m not exactly the biggest fan of OPML, but you do have an opportunity here to significantly increase the potential utility of the format (in a “small pieces, loosely joined” fashion). I’d be happy to offer my help in getting OPML on the Semantic Web – there’s not actually a lot of effort needed, none for authors of OPML documents.

    The starting point would be to give OPML an XML Namespace, e.g.

    – there are immediate benefits – the version attribute and the element both become unnecessary, the namespace doc can say it all.

    More significantly, it will make OPML more of a first-class citizen of the web, in that all documents will be globally self-identifying through the namespace URI.

    Further down the line there’s a big opportunity for exploiting what Tim Berners-Lee calls Linked Data. In essence it’s just about making it easier for people/agents to follow their nose through links on the (Semantic) Web.

    By the addition of a (non-visible) attribute in the namespace document for OPML 2.0, all individual OPML 2.0 files can be *transparently* interpreted as RDF, using the GRDDL mechanism. In other words, the author of an OPML document doesn’t have to do anything extra, but their data is automatically on the Semantic Web.

    Aside from the fact their data would be usable in a *much* bigger range of tools, there’s a more direct benefit for users of OPML itself from having an OPML/RDF mapping – it can work both ways. Data expressed in the RDF model could be serialised as OPML (I’d be willing to provide example code for this).

    The specific addition to the namespace document would look something like this:

    As well as rdf2opml material, I’d be happy to provide the necessary grddl-profile.html doc, along with the XSLT needed for opml2rdf (the initial aim of both I imagine would be to cover your examples).


  7. Oops, it ate my markup. Try again:


    The starting point would be to give OPML an XML Namespace, e.g.

    <opml xmlns=”″>

    – there are immediate benefits – the version attribute and the <docs> element both become unnecessary, the namespace doc can say it all.


    The specific addition to the namespace document would look something like this:

    <head profile=””>


  8. RE: OPML 2.0

    I have been using the power of OPML inclusion for quite a while now over at (every single folder in the whole network has a link to an OPML ‘2.0’ url) This has been fundamental to our ability to mobilize directories with incredible efficiency. In fact *any* platform with a browser.

    I thought I’d make a note here of how we are using some of the outline attributes:

    For an RSS feed we use:
    outline type=rss with xmlUrl pointing to the feed

    For a website link we use:
    outline type=link with htmlUrl pointing to the web url

    For an OPML inclusion we use:
    outline type=include with url pointing to the OPML url

    During OPML import, we do employ a certain amount of ‘fuzzy logic’ (though I hate to do that, as I have to do this a huge amount when dealing with parsing RSS 2.0) which, when the system imports OPML with a type=link attribute the system will go and check the url to see if it is OPML or HTML and treat it accordingly.

    I believe that OPML can pave the way for ‘Web 3.0’ and the Semantic Web.

    An analogy I use when explaining our system (codename ‘treedia’ – a tree of media) is to say : imagine web search being like standing at the door of a bookshop like Borders.

    We all stand at the door, seeing all the books strewn all over the floor, waiting for the nice friendly guy wearing the t-shirt with ‘search’ emblazoned on his chest – ask him for the book we want and he rushes off and comes back with a selection for us to choose from. We might then buy one. It;s the one we want. Cool.

    But: often I want to know WHERE it came from. What other books were near it. What was it’s relation to other books I like, or maybe have never even heard of. I like to discover.

    So, bookshops have shelves and great big signs above them displaying categories and collections of books about similar and related topics.

    I like to browse when I’m shopping. I like to discover new shelves and categories while on the way to the section I want. I don’t like it when shop assistants bug me. But I like it when my friends recommend books and tell me about things I like or have never heard of. I then like to go a seek out more ifo about these topics if it piques my interest.

    This also tells me a lot more about my friends.

    When I go a friends house, old or new, I always look at their music and book collections on their shelves. It tell me so much more about them which I might not have known otherwise. I like to know what makes people tick. And tock.

    OPML 2.0 inclusion gives us the ability to take away directions to just the shelves we like and not have to store the whole library/collection. It’s much more efficient.

    I could even create my own bookshelf where none of the shelves where my own. They could simple be a ‘shortcut’ to all the favourite shelves I have even found, yet I can reach up and grab a book just as if it was in my own living room. Whenever my friends puts a new book on their shelves, I’ll see them appear next time I look at my shelves. I can also see who else has the same book and on what shelf they put it on. This could go on an on. Pretty compelling!

    OPML 2.0 provides a way for us to CONNECT COLLECTIONS of the things we like. This is not just ‘Social Media’ or ‘Social Networking’ :

    It’s ‘Social Multimedia Networking’. And it’s media type agnostic. It could be a book, a piece of music, podcast, a document, a slice of code, a recipe. Anything.

    I heart OPML!! 🙂


  9. sorry to digress somewhat on that comment, but I have been bubbling away around for well over a year on OPML2.0, while building the architecture for, as well as a load of other systems I have cooking.

    Just to add that I also think OPML2.0 is perfect for threaded forums too. Imagine Twitter in ‘3d’ (not in a Second Life sense though) – I could tap in at the level of conversation/discussion I wanted, without having to access the whole thread. But I could follow it back up the path if I wanted.

    Also I have made up a few extra system specific attributes to our OPML at, especially the owner id of the folder alsong with some privacy flags.

    Here’s the OPML url for my collection of podcasts

    No reason why this shouldn’t be links to anything.

    It’s also worth pointing out that I also think OPML and RSS could also be used to document every software api there is. They’re usually a tree structure. OPML would provide the navigation and each function/verb would have an RSS feed where the header describes the mane (title) and also how the software vendor expects it to be used (description) – then each item below would be user generated, showing new ideas and ways to use it, with smippets of code. Each item could also then have it’s own comment thread (more opml.. mmmm)

    Pretty darm powerful if you ask me, and it’s SO easy to build cool user interfaces to display, contribute and learn. Very easy to parse if the spec is sensible (naturally ;p).

    OPML plus RSS to the power of USERS is a PLATFORM.


  10. Dave, I asked two straightforward questions, but you still haven’t answered either:

    1) What does “flatdown” mean?

    2) What is the XML namespace of OPML 2.0?

    Instead, you answered these questions:

    1) Why didn’t you define “flatdown” in either of the specs? (Your answer doesn’t answer why you still haven’t defined “flatdown” after I asked.)

    2) Can you put XML that do define a namespace into OPML?

    Seriously, why do you think the word “flatdown” doesn’t require a definition? You said in the spec “I try to write specs in plain English, for intelligent people using common sense.” Do you think that everyone is supposed to use their common sense to reverse-engineer what “flatdown” means? Can you please address the problems I see with my interpretation of the word, or at least correct my interpretation? Why can’t an OPML file save the open/closed state of nodes whose parents are closed?

    And why haven’t you defined a namespace for OPML itself? That’s a totally different question than “Can you put elements with other namespaces into OPML?” I don’t understand what you mean by “I have applications where I use OPML as a namespace.”. A namespace is a unique URI that identifies the XML application. How can you use OPML itself as a namespace? Do you mean you literally use the four character string “OPML” as a namespace, like xmlns:opml=”OPML”? If so, then that’s the answer to my question. If not, then what else I would like to know is “What is the URI of the OPML 2.0 namespace?”, just like I asked the first time.



  11. Kosso, if you’re only working on Web 3.0, then you’re 7 versions behind! Ever since Tim O’Reilly trademarked Web 2.0, we’ve moved onto Web 11.0. That should give Tim enough time to trademark Web 3.0, Web 4.0, Web 5.0, Web 6.0, Web 7.0, Web 8.0, Web 9.0, and Web 10.0, defining them to mean whatever he wants, and have all of his proprietary conferences about the latests fads, while the rest of us are busy working on Web 11.0. But please don’t tell Tim that the rest of the world has moved on, because we would prefer to keep the term Web 11.0 truly open and free.

    Never use an even numbered web.



  12. Don, you have a choice too — you can try to work with me, or you can act like a federal prosecutor trying to convict me of some crime. In the last post, you chose the second approach.

    I just started working again on this spec, so maybe that explains why I haven’t dealt with every issue you or anyone else wants me to. Frankly this is why I avoided working on this for so long. Whenever I try to do some work here, I get people acting like federal prosecutors with me. If I don’t do anything, they leave me alone.

    Anyway here’s the document that uses OPML 2.0 as a namespace.

    As far as actually defining flatdown, that’s not something I would attempt casually or in a message in a discussion forum. Can you imagine how you would react if I made a MISTAKE in defining it? I think I would be subject to the death penalty!! 🙂


  13. Dave, it looks like WordPress just ate my last comment. Sorry, I forgot to snag the error message first time I posted, but when I tried again it told me it was a duplicate.


  14. Dave, you’re getting way bnt out of shape about the new york times; you should know by now you have to pay them; or write the article yourself and pay them, before they will mention anything cool.

    why do you even read the times?

    i lived overseas for many years, it’s all propaganda; especially american news outlets…just take a look at the intelligence of the news programs…nancy grace anybody?

    go european, asian.


  15. Posted by Ben on July 17, 2007 at 12:20 pm


    RE: ” I hoped they’d overlook any potential conflicts.”

    What conflicts do you see NYT needing to overlook?

    The main one i presume is that both the nytimesriver list and the version of the articles they point to omit all advertising. Are there other conflicts? Would they prefer that point to the regular NYT page and not the “print ready” version of the page so that some ads would load?

    Is it the re-packaging of the feeds themselves? Or is there some other conflict more worrisome?

    Anyway, keep innovating, and thanks for sharing your ideas, passion, and code with us all these years.



  16. The conflicts go the other way — they would be conflicted in that they are promoting a product that they profit from. They write for the Times, and presumably are paid for that, and the content of are those articles.

    We could easily put ads on those pages, but there’s been no reason to (hardly any flow) and it wouldn’t be fair without a deal with the Times (it’s their content after all).


  17. Posted by Bruce D'Arcus on July 17, 2007 at 6:19 pm

    Dave: “About namespaces, what’s the problem? I say in the spec that you can include elements from other namespaces. I have applications where I use OPML as a namespace. What else would you like?”

    Define this URI as the namespace for OPML 2. Using namespaces can’t be optional; either OPML 2 has a namespace, or it doesn’t.

    Also, I only quickly looked at the document for the updated spec, but it preserves what I consider a big mistake in the first version: encoding text in an attribute. Perhaps you can consider fixing that?


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: