Changes to the OPML 2.0 spec 
Here’s a list of the changes I made in the last couple of days to the draft OPML 2.0 spc. The changes were in response to comments here in July 2007. Included are notes on the specific suggestions, including ones I didn’t understand or decided not to implement.
I provided in response to a request from Don Hopkins, a rigorous definition of flatdown. Actual C source code is provided. I also defined the other outline processing directions — up, down, left, right, flatup and nodirection, even though Don didn’t request those (and they’re not mentioned in the spec).
I uploaded a copy of the C source code of the OPML Editor in a form that will be better indexed by search engines, so that future queries about the internal workings of outliners can be addressed by searching the source. It’s licensed under the GPL, and build instructions using XCode are provided (as are build files for a variety of popular development environments).
I don’t generally support this technology, I wrote much of the code but it was a very long time ago, and my memory isn’t so good anymore. But surprisingly, a lot of it came back. 🙂
Don Park wants to extend OPML using a wiki. Not sure I understand how this works, but he says it doesn’t involve changing the spec (thanks!) so it’s no problem for me. I’ll watch and see how it develops.
PS: Possibly the coolest thing made possible by today’s changes is the ability to embed OPML 2.0 data in RSS 2.0 feeds.
OPML 2.0 spec, day 2
Another day with more work on the OPML 2.0 spec.
Yesterday I provided some examples of OPML files that use the category attribute.
A reminder to people working on OPML apps, I have a beta of a validator that tests against the 2.0 spec.
Here’s an example call, the validator being used to check the current OPML file for Scripting News.
Also I’ve seen some comments recently that say that the spec isn’t very good. If you have concerns about the spec, please let me know what they are, now, while errors and omissions can be fixed. Thanks.
RSS 2.0 with OPML 2.0
Here’s an RSS 2.0 feed with an item that contains several outlines, basically the show notes for a podcast.
You might have to View Source to see what’s going on.
It’s like chocolate and peanut butter. Both flavors are tasty, but when you put them together, it’s even yummier!
Scripting News Annex 
Posted by Mark Bernstein on August 28, 2007 at 9:27 am
The spec refers to “encoded HTML”, but I don’t immediately see an indication of HOW the HTML is to be encoded. This should be spelled out.
Posted by l.m.orchard on August 28, 2007 at 1:03 pm
From the change notes, RE: expansionState: “In memory, we maintain a boolean in every outline element which indicates if its expanded or not. However, it’s much more efficient to remember the expanded state in the file format in one concise if machine-oriented, optional element.”
My wishlist item is to see the per-element expansion state on outline nodes as an attribute. But, I think that’s vetoed in favor of backward compatibility.
It would make XSLT transforms so much easier, though, since the flatdown algorithm seems really painful in that setting. I think the issue for XSLT, basically, is that the XML file-format structure *is* the in-memory representation.
“I’m not going to comment on or explain changes that fall outside this scope, or would even cause breakage with 1.x, or unnecessarily complicate the format.”
But, yeah, at the end of the day I get that.
Posted by l.m.orchard on August 28, 2007 at 1:12 pm
Also, re: “encoded HTML”: Maybe a clarification here would be along the lines of “text attributes may contain HTML markup, escaped as per the XML 1.0 specification.” Kind of redundant, but that’s my interpretation.
Posted by Dave Winer on August 28, 2007 at 1:39 pm
Les, it would be easy enough to define an attribute in the spec that indicates whether an outline is expanded or not, but none of the outlines have it. Do you have an application that generates OPML that would output files with the attribute present? I understand what you say about XML being the internal representation for some apps. Do you have an outline displayer that would work that way? Willing to put together a proof of concept?
In other words adding something to the spec is easy, but what happens next?
Also, I agree that saying what “encoded” means is redundant. OPML is XML, and encoding is defined in the XML 1.0 spec.
http://www.w3.org/TR/REC-xml/#charencoding
Posted by Matt Terenzio on August 28, 2007 at 2:34 pm
I agree about the coolness factor of OPML embedded into RSS. In fact, My head is already spinning with ideas to use it for.
As always, widespread support will determine a lot. I hope it catches fire.
Posted by Mark Bernstein on August 28, 2007 at 3:42 pm
Dave: is it obvious from http://www.w3.org/TR/REC-xml/#charencoding whether HTML entities are re-escaped or not? The single vs double escaping of html entities is a headache!
Posted by Dave Winer on August 28, 2007 at 3:50 pm
Mark, what would you have the spec actually say?
Posted by Devin Coughlin on August 28, 2007 at 6:22 pm
I’m still confused about flatdown. Suppose you have an unexpanded outline element A with a child B and that A’s next sibling is C. Then, if you go one unit flatdown from A, are you at B or C?
(If it’s the latter, then you would never be able to specify the expansion state for any element which has an un-expanded ancestor, right?)
I think you should be more explicit about whether expansion state is relevant in your definition of flatdown.
I’m also worried about where you write “Imagine the outline displayed on a screen.” It seems to me that, since an included outline “expands in place,” you would have to know about the state of included outlines in order to “navigate flatdown X times.” This would make expansionState fragile. Some explicit guidance about what to do with flatdown and inclusions would be helpful, I think.
(I’m coming at this as someone with little prior experience with OPML, so please feel free to smack me down if I’m making erroneous assumptions.)
Posted by Dave Winer on August 28, 2007 at 7:27 pm
Devin, yes it is a function of what’s expanded, and I’m not sure, but I think inclusions could effect it, but it’s just a convenience, not anything that should be considered mission critical.
If you really want to understand it, download the OPML Editor, and start writing some scripts. The op.go verb takes a direction as a parameter, so you can put the cursor somewhere, do an op.go (flatdown, 1) and see where it takes you. The expand the item and see where it takes you.
http://docserver.scripting.com/op/go
This is really way off-topic for me, I only wrote this stuff because Don really wanted to get to the bottom of it, kind of urgently. I spent a whole morning putting that writeup together. I have to do other work tomorrow. I want to do some work with Flickr, and then with the aggregator.
Posted by Paul Montgomery on August 28, 2007 at 8:00 pm
It may be a little late for this, but I would really like an attribute which specifies time and length for nested outlines.
That example of OPML tags within an RSS feed which provide show notes for a podcast enclosure is great! I think it would also be very useful to allow for those show notes to refer to specific places within the MP3 file, rendered in HH:SS format or something similar, possibly with the length attribute as well.
This would enable rich media players of not only podcasts but video, PowerPoint-type presentations, X3D scenes, etc to use OPML as a way to navigate within the file to the places readers want by using it as a browsable table of contents.
I don’t know if that’s the sort of thing you want left to namespaces, Dave, but I think it’s central enough to the core functionality of OPML that I’d love to see it as part of the core spec.
Posted by Dave Winer on August 28, 2007 at 8:28 pm
Paul, first, I’m glad you’re excited about the idea of embedding OPML in RSS. I am too.
But it’s just one example of a combination that now can happen. I’ve only tried this once before, a couple of years ago, and I’ve never actually used it, so I don’t even begin to know how to address the functionality you talk about in your post. It’s the kind of thing namespaces were designed for, and it couldn’t be in the core spec at this point, not without delaying it for a long time and then who knows what will come of this idea. In other words it truly is an application of RSS and OPML, not part of either of the formats.
BTW, you could also include OPML elements in Atom, or FOAF, or the Semantic Web, or whatever people come up with next year.
Posted by Paul Montgomery on August 28, 2007 at 9:33 pm
Thanks Dave, I suspected that namespaces were the way to go. Perhaps it’s the sort of thing that Don Park’s wiki would be useful for.
Posted by michael on August 29, 2007 at 4:25 am
You might want o archive your Mac sources in a .zip instead of a .sit file.
Unless someone is an old school Mac developer, they may not have StuffIt around to extract the files.
Zip is pretty much the standard now for Macs. The Finder supports creating and extracting from zip archives now.
Posted by l.m.orchard on August 30, 2007 at 7:30 am
“Do you have an application that generates OPML that would output files with the attribute present?”
At present, no. Closest thing I have are outlines in XHTML format with compact=”compact” attributes on the list elements, as per the XOXO microformat.
I figured OPML could have a collapsed=”{yes,no}” attribute on outline nodes, defaulting to yes. That, of course, is not backward compatible with the expansionState attribute unless apps supporting a per-outline collapse state maintain the expansionState in tandem.
“I understand what you say about XML being the internal representation for some apps. Do you have an outline displayer that would work that way? Willing to put together a proof of concept?”
I have some XSLT laying around that turns OPML into XOXO-style HTML, and vice versa. I’d like to do some more hacking with it in this vein, but unfortunately I’d probably not get around to it for a few weeks since my headspace is pretty full right now. No biggie, I won’t cry if I don’t get a pony. 🙂