The mystique of desktop web servers

Q: What is a desktop web server?

I guess it should be no mystery, it’s a web server that runs on your desktop.

Q: What makes a desktop web server different from one that runs on a server?

Not much. You could run Apache on your desktop. In fact, if you’re using a Mac, you are. When the time came for Apple engineers to decide to turn off the server, they thought, what the heck, let’s leave it on and see what happens. Unless I’m missing something, so far not much has, but it could.

Now let’s zoom back to 1998. I want to explain a decision I made in January of that year. We had almost completed the port of Frontier from the Macintosh to Windows. I was using the Windows version. We still we had to recreate a networking layer that would do for us on Windows what Apple’s networking, AppleTalk, was doing on the Mac. We could have used Microsoft’s proprietary networking, it was called COM, but since I already saw myself as an Internet developer, instead I wanted to use the Internet to connect my computers.

This had several important advantages.

1. I could connect my Macs and Windows machines using the same protocol.

2. Since there is no platform vendor for the Internet, they couldn’t break us, like commercial platform vendors always do. Read the thread on Apple’s latest RSS crap to get an idea how that works. Even if they’re well-intentioned, they still don’t care about the little guys, you know, like you and me. ;->

3. I could connect to Unix machines for free.

4. I could connect to anything that comes along later.

5. I could connect computers that were very far apart, since the Internet is a world-wide network.

Those are all pretty compelling reasons, so I announced my intentions and went ahead and it happened incredibly quickly, we created a new networking layer, and not only was it quite useful for our users, it went on to become one of the biggest defacto standards of the Internet, almost as big as RSS, but not quite so visible (since it’s really plumbing and largely of interest to developers although users like what they do with it).

Anyway, there was another reason this approach was cool, because it built on XML and HTTP, we got a web server for free. And thus was born UserLand’s web server. It’s built into Frontier, Radio and the OPML Editor. Every time you install one of these programs, you get a web server. People use it all the time, but they’re probably not even thinking about it, and most of them aren’t even aware of it.

This really came home to me in a conversation with Mike Arrington yesterday. I was trying to explain the Aggregator API that was released yesterday. He didn’t understand that there was a web server in there, and that you could put an API on it even if it’s running on the desktop. Why not, a lot of APIs are running on the desktop (that’s largely what Windows is).

I told Mike that it didn’t matter where I ran the software. Most people ran it on the desktop, but you could put a copy of the OPML Editor on a server and access the aggregator from afar. I guess it’s a very new idea (even though it’s already 8 years old) that you can use the protocols of the Internet to connect two apps that run on the desktop, but you can, we do it all the time. It’s very general, and quite powerful, and flexible.

  The Internet is not just a world-wide network, it’s also a local area network, and it’s can be an inter-application communication protocol, for two programs running on the same machine. The same simple protocol can be used in all these contexts.

To prove the point, I put a copy of the NewsRiver aggregator on a server of mine in Massachusetts that I’ve never actually seen. It’s definitely not running on my desktop. Here’s a link to the aggregator running on that computer. Go ahead and try it out.

http://test27.newsriver.org:5337/newsRiver/

Note, this link may not work tomorrow or the day after. ;->

PS: Another example of desktop web servers, Google’s desktop search. You access it through a web browser, but the software is running on your computer. Ever stop and think how that happened? It’s so simple people don’t even notice.

PPS: I wrote a piece that attempted to define desktop web servers at the beginning of 2001. I thought that would be the year of DTWS’s. It wasn’t. We still haven’t had an explosion of creativity in this area, but I think there’s still reason to believe we’ll see it, eventually.

PPPS: I wrote a piece in 1998 that explained in fairly non-technical terms how APIs work. Anyone trying to understand the basic concepts of what we now call Web 2.0 should master these concepts or else you’re just talking marketing BS. Maybe you already understand APIs, if so, good for you! But if you’re still scratching your head, the concepts really aren’t all that hard.

15 responses to this post.

  1. In fact, if you’re using a Mac, you are. When the time came for Apple engineers to decide to turn off the server, they thought, what the heck, let’s leave it on and see what happens.

    Mac OS X ships with Apache installed, but turned off.

    Reply

  2. Ooops. Mea culpa.

    Reply

  3. Dave, Fractional horsepower! Always been a cool idea. Why haven’t we had the year of DTWS yet? Blame the telcos and cable companies. The pipes are too narrow. Even if remote access is only 10% of the appeal of DTWS right now, that’s still enough to hold back explosive growth (Spolsky likes to write about how a 10% product improvement can lead to 100% sales increase.)

    Reply

  4. I like to call it the personal hub server. In the era after the crash, post 2001, I’d thought that such servers would gain much more in popularity, so that you could push compute to the edges. But I guess the economy didnt tank as much and moore kept going strong so that the costs of running a central server came down.

    The notion though that we are connected all the time is so much an early adopter bay area/cambridge notion. Which is why I think this renewal in ‘web2.0′ VC funding is likely to lead to an equal number of corpses. Lets get this clear: a vast majority of people have NO INTEREST in being connected all the time. I regularly spend weekends off the grid when I go climbing, and I’m on almost always otherwise. A lot of people I know still use there computers for their papers, their research, games, all offline. A larger number spend more time on their x-boxes.

    When you write software thats unavailable to these folks, you’ve missed a huge market. Something like writely for example, whats the point in having it on a far away server. Have it local. Its all very nice to mash up web services but 127.0.0.1 is always faster and makes sense for a lot of such services. The folks I work it arent the unwashed masses either, the market we arent even talking about. These are folks who have access online, but arent tethered to their computers like we are. We need an offline mode for data creation, and collaboration. Thats one reason why email clients are still so popular, and word and excel rule.

    And RSS would seem like such a key place where there would be large synergies between things on the desktop. My browser knows I read Dave’s sites quite religiously. A smart aggregator would ‘in the background’ podcatch-cache all his sites on my desktop. And with search built into all of linux mac and windows soon, there is a ton of info an aggregator could use to make smart caching desicions. Just as podcasting enables me to listen to interesting ‘shows’ on the go, so does a personal hub server for interesting articles for news, and maybe even in the future, for collaboration.

    Reply

  5. Posted by scott on January 12, 2006 at 9:51 am

    > because it built on XML and HTTP

    Today you are praising the the work of W3G and the IETF while yesterday you called them lame and the source of abysmal failures. WTF? I guess it all depends on the agenda du jour.

    Reply

  6. […] Being serious, though, this is very helpful for a post I am working on. I think Dave is onto something really big. […]

    Reply

  7. Posted by calvin on January 12, 2006 at 1:13 pm

    dave, good stuff. I do this too with apache/mysql and my own software. but I think the real power won’t be seen until ipv6. when you can walk around and have it always accessible when it’s connected, then we’ll see some interesting apps. right now you have to use a mobile dns service to keep your server online as much as possible as you carry it around in your bag.

    soon, I hope my little sidekick will start being a server always on, with me wherever I go and serving up the pics and posts I store on the sidekick, instead of piping them to my own blog (http://thearthole.net/arthole). I think that the real power of all this happens when ipv6 gets adopted.

    Reply

  8. Posted by rob rowntree on January 12, 2006 at 6:41 pm

    as long as your building an “heavy” api , why not go soap?
    it would be alot more useful to bind the entire interface to http rather than exposing it as xml-rpc.

    im not aware that you need xml-rpc in order to run pub/sub or to expose some server side interfaces. There are so many edge devices coming along that will have feed readers and that will not have xml-rpc client libraries that its a no brainer to make the decision to bind your api to http. An example, the sony psp.

    Reply

  9. Isn’t there a continuing problem of the lack of reliable IP addesses for desktop machines?

    We’ve had succesful servers on the desktop for a while … in Napster and Skype and other P2P applications. But they all work by adding an extra layer of addressing so that machines with varying IP addresses can be found reliably.

    Reply

  10. […] —on the mystique of desktop web servers. I hope for a great deal more development in this area because it goes to one of my chief concerns in the Google powered (not to say dominated) Web 2.0 world. That is, I own my stuff. Stuff here is defined as anything that I’ve created, be they words, pictures, audio, music, video, art (profane and otherwise), architecture, movie scripts, novels, novellae, essays, drawings, characatures, cartoons, readings, spoken word rap (for you Futurama fans out there); to paraphrase Justice Stewart, I know what’s mine. This wouldn’t seem like such an issue, but sitting where I am today I see a lot of companies that would like to own what I’ve created. Perhaps they don’t know it yet, but that’s what they (more specifically, the lawyers they employ) want. […]

    Reply

  11. I remember seeing a web server for Symbian based mobile (cell) phones a couple of years ago and I was blown away by the idea. I’m not sure if it was developed further since but because mobile phones are practically always-on and always connected it seemed to me that they were the perfect place for a web server.

    Reply

  12. Our relatively new Web 2.5 Blog is dedicated to the concept of web server apps on mobile devices, like wi-fi handhelds, flash drives, and laptops. The idea is to create “always-on-you” web services, which hit local screens. See link above…

    Reply

  13. Posted by Tim Barker on January 18, 2006 at 10:30 am

    What i find interesting with this approach is that you can blur the lines between being desktop-centric and web-centric very easily.

    Take Google Desktop as an example: You search locally and it gives you the same web experience as searching on www. It then makes it really easy for users to transition between the desktop search and web. If you look at their competitors (e.g. copernic), they don’t have a web-approach to this. For users, keeping the same mental-model for searching, regardless of what you’re searching, is a big win.

    So Dave, can you recommend any good desktop web-servers can i can embed in my next application? Apache is way too big to download/embed. I’ve seen a few small-footprint webservers listed sourceforge but it’s always hard to judge adoption. There must be some tiny http servers. Something similar to what sleepycat do for embedded databases.

    Reply

  14. […] First off I get a lot of offline questions about my role here at Erati, and with BlogMedia. So let’s set the snarkiness down for just a moment and talk about what goes on inside my brain. And a little about my day to day operations. My plans for world domination, and eventual control of the universe. Oh ok, just the blogosphere. No truly its the universe. Personally I am involved with blog networks for the enjoyment in it. And to take a crash course in starting my own business. Life is full of mistakes, and lessons to be learned. One thing I learned is never go into a bar, with a crate of dynamite strapped to a camels back. It’s just not recommended. Currently I wish I could devote myself full time to just this here blogging stuff, but bills have to be paid, fun has to be had, and Girlspoke needs attention, campaigning to make sure they damn well win that bloggy. Go vote everyone. . Because I’m actually going to be at SXSW so it would be badass to help accept that award as part of the whole Spoke Empire. Here’s an average day in the life of a blog executive. Well mine anyways. I wake up after 20 minutes of sleep, and work for a few hours, sleep again for a few more. Then sleep again. I then get back to work. And sleep some more. Do you see the pattern. It’s called Polyphasic Sleep. It’s the only way to get things done. I then fire up the whole old RSS Reader, and churn around looking for good content. I also spend 10% of my day thinking of ways to create new stuff that will be better than anything anyone else on the universe is doing. I then implement about 1% of that into a massive wordpress I run on my desktop. My desktop blog is about 100,000 pages and gets about a trillion mindviews a day by me trying to remember something. This is truly the best use of a wordpress IMO. I f you need help setting up your own Desktop WordPress check out UrbanGiraffe for help with this. Recently Dave Winer was talking about Web Servers running on your PC . Which is something I’ve been doing for awhile. It makes my life so much easier. I can blog schtuff to myself. Even have delicious links embedded on my desktop blog. I can talk all the trash I want and only I get to read it. I don’t even have to spell check. I can even use WordPress to manage my budget. And create a portal that aggregates all the site I enjoy reading in a way that I like and not the way anybody else wants me too. I’ve also begun experimenting with AirWRX a portable app server that rocks my world. Yes they have a blog too. And people wonder why I don’t accomplish anything during the day. To much experimenting with prototype software projects. […]

    Reply

  15. There are rumours of Radio’s demise in the face of WordPress. Can you help a new platform seeker decide which way to go?

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 60 other followers

%d bloggers like this: