Scripting News for 12/20/2007

Could S3 be an end-user product? 

I’ve not made much of a secret of the fact that I’ve been working on a new product, and am getting close to offering it as a public pre-alpha thing for Mac users only.

It’s fairly Flickr-centric, sucking photos down from Flickr in a variety of ways and pushing up photos in others. Like Radio 8 and Google Reader, it has the ability to maintain an output feed of stuff you want to pass on to friends and associates.

As I was developing this I wished that Flickr had the ability to store simple text files, because I needed a place to put an RSS 2.0 feed with enclosures on behalf of each user. Of course I used Amazon S3, but I had to implement my own lightweight identity system so that Juan couldn’t accidentally or intentionally replace Alice’s feed. If only every user had a place where they could store stuff that’s net accessible so that once and for all we could stop inventing new places.

I was inspired to write about this when I read an Uncov review of Pownce where they reminded me that they were reselling S3 storage at a big markup. What if the users had their own S3 storage that they paid for themselves?

Then it occurred to me to ask if people thought S3 could be an end-user thing. I’d like to find out, so if some non-technical users who have Amazon accounts would like to try setting up an S3 account, I’d be interested in hearing how it goes. Here’s an idea of how you get started.

1. You must already have an Amazon account. Nothing special about it, if you buy shirts or books or stereo equipment from Amazon, you use the same account for S3 storage. Already that’s pretty easy, millions of people have Amazon accounts, right?

2. Visit this page on Amazon, ignore all the stuff about objects and buckets. In the right margin is a big button that says “Sign up for this web service.” Click the button. A very familiar page appears, asking you to sign into your Amazon account as if you were going to buy something (you are!).

3. From there, I’m not sure what you see, because I have already enabled my account. But the end result of signing up is that you get two strings with weird names: Your Access Key ID and Your Secret Access Key. Any software that would save a document in S3-space on your behalf would need these two strange strings. In return each document would have a URL just like any other document on the web. Nothing strange about that.

4. You could also use the space to store stuff using an FTP-like app that runs in Firefox called S3 Organizer. It’s about as hard to use as the Mac Finder or the Windows Explorer, i.e. it’s no challenge for a moderately geeky user. The cool thing about it is that you’re able to share anything you upload into S3 space with anyone else. You can even use BitTorrent to access any file to save you bandwidth bills and distribute the load round the net. It’s all very easy to do, imho.

As a developer who has to pay for his users’ storage needs I would very much like to see users learn how to use S3 to store their stuff, so I can focus on writing software and fixing bugs instead of paying to store your stuff.

I’m with Rex 

Of course Apple is fascist scum for shutting down Think Secret.

Rex Hammock said the one thing that I as a Mac user have to say about the news.

“There’s nothing positive about this settlement for my side.”

Amen brother. I keep thinking “Someday Apple is going to regret that they took their customers so much for granted.” But I know better. I used to say that, and then I switched to Windows in 1997 (that’ll show em!) only to switch back in 2005. Every day I think of a new excuses to waste spend more money on Apple hardware. Apple doesn’t pay for it, we do.

But we can hate them for what they did to Think Secret, as if they care, but we do anyway.

Scripting News for 12/19/2007

CES, here we come! 

Thanks to PodTech, I’m going to CES again this year, and I’d like to see whatever it is that I should see. Suggestions please, in the comments.

I love devices that can be used for podcasting, for example something that fits in your pocket, has a battery, and wifi, and either is programmable by developers or includes a podcatcher. Is this the year of the podcatcher breakthrough?

I’m riding down on the PodTech bus with Scoble and ValerieWag and probably a lot of other coooolio bloggas (Marquis de Canta?). This time I hope to arrange my press pass in advance, and I want to meet lots of vendors who can send me review units through the year, so I don’t have to pay for all the stuff I review. (More likely so I can save my money for Uncle Steve.)

And if you’re a blogguh and you’ll be there you might want to check out the BlogHaus that PodTech is hosting 24-by-7 at CES at the Bellagio.

Let’s hope it’s not too commercial, and we can have some good meetings and schmoozes and get some great work done.

Alternate theory: If they want to be overly commercial, go for it, then every blogger should get a free 1TB Seagate drive.

Frozen peas 

Boobs on Ice: What’s with the peas?

RSS makes Google Zeitgeist this year 

This year RSS was the third-most asked-about “what is” term on Google, after love and autism.

Parking meter 

Scripting News for 12/18/2007

Does Twitter do enough? 

Evan Williams, the Blogger guy and Twitter co-founder, gave a talk at LeWeb3 about keeping software small, and how sometimes you can create a product by removing features from an existing product. He showed how Twitter is less than Blogger, no titles, comments, templates, etc. It’s almost nothing compared to Blogger, but we’re using it and liking it.

It’s not a new story. When I was coming of age in computer science, the newest computers were minicomputers. They were called that because they were smaller and did less than the mainframe computers that came before. They were followed by microcomputers which did even less and were a lot more popular than minicomputers (which of course were more popular than mainframes).

Scaling things down can make them more useful. But it’s a paradox because once a feature is in a product you can’t take it out or the users will complain so loudly that you put it back in right away. I know, I tried, a number of times to back out of features that I thought of better ways to do. You can always add features to products, it will make the existing users happy. But it often comes at a cost of making the product more complicated for first-time users, and they don’t have a voice, they can’t complain, they just go somewhere else, usually quietly.

So Evan has a point. Software design, if you’re creating wholly new products, is like haiku. Find the smallest subset of a mature product that will attract people and ship it.

But there are certainly features they could add to Twitter that would have no impact on the steepness of the learning curve (i.e. how easy it is for a new user to get started). For example users are good at skipping over prefs they don’t understand. But you have to think carefully about what the default should be, so there’s no penalty for not caring.

Also features that only appear in the API have no cost in complexity of the user interface. They might make it possible for a developer to build a new product on top of the existing one. Since the user of the base product can’t see the feature, it can’t make it harder to learn. An example — Flickr lets you build an RSS feed of recent pictures that have a certain tag, say snowstorm. It’s nice to have, but only if it doesn’t get in the way of other more basic features.

Some users say they don’t want new features, but I bet most of them would be very happy to use a new feature that made Twitter more fun or useful. And there are alot of users who don’t say anything about it, and don’t think much about it. Most people aren’t interested in theories about why products catch on, they like it or they don’t, and don’t know why they do or don’t.

It’s always good to ask questions about why things work, but if I could offer the Twitter folk any advice, I’d say don’t hesitate too much to put in new features that will make users happy. Ultimately users like new features in products they use a lot. There’s a reason why products tend to bloat over time, it’s because users demand it. The trick is to not compromise too much on ease of learning.

Scripting News for 12/17/2007

A flashconf on fair use? 

There’s been a mostly fantastic discussion about fair use in this neighborhood for the last few days.

It started when a photograph of Lane Hartwell’s was used in a video spoof of the Billy Joel song “We Didn’t Start the Fire.”

The first I heard of this was in a Twitter post of hers where she said she was turning off access to her entire Flickr collection because this picture was used without permission. A series of communications with the people who did the video resulted in the video being taken down.

Later Mike Arrington, who is a lawyer, wrote a piece saying she didn’t have right on her side, and that the video’s use of her picture was probably fair use. I found Mike’s piece compelling. Others took offense. It thought it was a useful part of the discussion.

I understand Ms Hartwell’s point of view. I hate it when people copy a whole post of mine and paste it into theirs. But then I grab bits of images and put them on my blog and people rarely complain. The blogosphere is built on being loose about copyright and fair use.

I’m doing a deal with a content company and all these issues are coming up. We haven’t been able to write a contract that covers all the things they want covered and make it possible for me to do what I need to do, and they want my product to work. It’s a real mess we’re in. Bloggers are supposed to be radicals when it comes to fair use and copyright, but that generalization doesn’t work with many creative people. Hartwell’s position in some ways is like the RIAA or MPAA, who bloggers often dismiss as clueless. How can we have it both ways? How can some defend her position yet not defend the entertainment industry?

There’s a lot to discuss here, and a lot of the discussion on the blogs has been informative and respectful. Not all of it, but to an unusually high degree.

So, I am interested in doing an in person “flash conference” on the subject of fair use in a few weeks.

I’d say next week if it weren’t Christmas.

Most conferences are so boring. I want to do a conf on a hot subject when it’s still hot in the blogosphere. This may be a good subject for such a quickly organized conference.

What do you think of the flash conference idea for this??

How spam will likely enter the Twitter community 

I think I know how spammers are going to enter the Twitter world. It’ll come in the form of replies, which basically function like email. You can direct a message with a url to anyone as long as you know their username.

Here’s a screen shot that illustrates spam being sent to a hypothetical user. I didn’t send it of course.

Another problem, the destination of the url is likely masked through the use of a shortener so the user could be clicking through to some really nasty place, with no way of knowing in advance that’s where they’re going. (Such messages probably wouldn’t alert you in advance that they’re about meds or poker or sex.)

The press and bloggers will run stories saying “Spam Comes to Twitter” and they’ll be right, even though it won’t be the main part of Twitter. Users will expect the company to do something about it, but I don’t see what they can do other than eliminate the feature. Users will certainly want the ability to completely opt-out of replies.

PS: I received direct messages saying that the JetBlue account is spam (screen shot), but it is not spam, it’s commercial information. Big diff. I would have to opt-in to see these messages in my stream. And if I got tired of it, I could opt-out. Spam is stuff that intrudes that you can’t easily turn off.

To celebrate the 10th anniversary of blogging 

Here’s a zip archive containing the source of the last 10 years of Scripting News.

Since Scripting News existed before blogs were invented, I went ahead and included the stuff that I blogged before there were blogs.

I hope this isn’t too confusing!

PS: Scott Karp asks if blogs can do journalism. Try this question. Can journalists do journalism? At best they seem to be able to copy each other, so mistakes propogate.

We’ve made so many accomplishments, both before and after the coining of the term, Karp for example starts with VIgnette. In 1997 if you told someone the functions of Vignette could be provided to millions of people virtually for free they wouldn’t have believed you. (This is factual btw, I did, and wasn’t believed.)

They also thought syndication would be done by the big publishing companies, something unweildy called ICE. We thought it should be simpler so that anyone could support it on both ends, and we won. The journalists have no record of this probably because they believed the big companies behind ICE and ignored the low-tech stuff. Jorn Barger used my software to do his “web log” — why isn’t that part of the story? Well it isn’t if all you think is important is the choosing of the name.

Scripting News is 10.71110623 years old today 

Here’s a formula that calculates how many years old Scripting News is on any given day.

double (clock.now () - date (”4/1/97″)) / (60*60*24*365.25)

The answer is: 10.71110623.

It counts the number of days since its inception and divides by the number of days in a year. It accounts for leap years, assuming there is an extra 0.25 days each year.

Scripting News for 12/16/2007

Bathtime in Clerkenwell 

Click the pic to play.

Scripting News for 12/15/2007

Okay let’s get viral! 

Fred Wilson did it, I had to do it tooooo.

In NYC, the Second Ave Deli is coming back 

Ed Levine writes a remembrance of the newest deli to open in NYC, with the same cast and food, at a new location, uptown (on 33rd St) and not on Second Ave.

New York sighs in relief. So does every deli fan in the rest of the U.S. and the world.

That seals it. I’m headed back to NYC before the end of the year.

PS: A funny thing happened when I entered Second Avenue Deli into Google Maps.

Amazon removes the database scaling wall 

When Amazon introduced S3 in March 2006 I knew I would use it and I was sure a lot of other developers would. I saw it as a solution to a problem we all have — storage that scales up when needed, and scales down when not. Otherwise we all have to buy as much bandwidth as we need in peak periods. With S3, you pay for what you use. It makes storage for Internet services more rational. Later they did the same for processors and queuing. And a couple of days ago they announced a lightweight scalable database, using the same on-demand philosophy and simple architecture and API. It’s going to be a huge hit and forever change the way apps are developed for the Internet.

I was explaining the significance of this to Scoble on the phone this morning. It’s worth repeating here.

When I developed Frontier in the late 80s and early 90s my target platform was a modern desktop computer, a few megabytes of RAM, a half-gig of disk, a few megahertz CPU. A system capable of running Quark XPress, Hypercard or Filemaker. It would be used to develop apps that would drive desktop publishing. Later, it was used to generate static websites, then a demonstration of democracy (a multi-author ultra-simple CMS), then news sites, which became weblogs, then blogs, and editthispage.com, Manila, weblogs.com, and that’s when scaling became an issue. (Later we side-stepped the scaling issue by moving most of the processing to the desktop with Radio 8.)

As we approached then cracked ease of use in web authoring, scaling became an issue, then the issue.

A Manila server would work fine for a few thousand sites, but after that it would bog down because the architecture couldn’t escape the confines of a single machine it was designed for in the 80s. (Before you say it’s obsolete, there still are a lot of apps for single machines. Perl, Python, JavaScript and Java share the same design philosophy.)

Same with weblogs.com. It worked great when there were a few thousand blogs. Once we hit 50K or so, we had to come up with a new design. Eventually we were tracking a couple million, and Frontier was hopelessy outclassed by the size of the problem.

If only Amazon’s database had been there, both Manila and weblogs.com could have been redesigned to keep up. It would have been a huge programming task for Manila, but it would have made it economically possible.

Today, when a company raises VC, it’s probably because their app has achieved a certain amount of success and to get to the next level of users they need to spend serious money on infrastruture. There’s a serious economic and human wall here. You need to buy hardware and find the people who know how to make a database scale. The latter is the hard problem, the people are scarce and the big companies are bidding up the price for their time. Now Amazon is willing to sell you that, to turn this scarce thing into a commodity, at what likely is a very reasonable price. (Haven’t had time to analyze this yet, but the other services are.) Key point, the wall is gone, replaced with a ramp. If you coded your database in Amazon to begin with you will never see the wall. As you need more capacity you have to do nothing, other than pay your bill.

Further, the design of Amazon’s database is remarkably like the internal data structures of modern programming languages. Very much like a hash or a dictionary (what Perl and Python call these structures) or Frontier’s tables, but unlike them, you can have multiple values with the same name. In this way it’s like XML. I imagine all languages have had to accomodate this feature of XML (we did in Frontier), so they should all map pretty well on Amazon’s structure. This was gutsy, and I think smart.

They’re going down a road we went down with XML-RPC and then SOAP. There may be some bumps along the way but there are no dead-ends, no deal-stoppers. All major environments can be adapted to work with this data structure, unless I’m missing something (standard disclaimers apply).

Their move makes many things possible. As I said earlier, if it existed when we had to scale weblogs.com, we would certainly have used it. One could build an open identity system on it, probably in an afternoon, it would be perfect for that. A Twitter-like messaging system, again, would be easy. It’s amazing that Microsoft and Google are sitting by and letting Amazon take all this ground in developer-land without even a hint of a response. It seems likely they have something in the works. Let’s hope there’s some compatibility.

Twitter takes a break, we’re awake, and wondering… 

There’s a big yellow bar on the Twitter home page today saying it will be down for maintenence betw 10AM and 10PM today. I haven’t heard any grumbling about this, but it’s worth a bit of a grumble.

What other basic form of communication goes down for 12 hours at a time?

What if the web went offline for 12 hours at a time? It’s unthinkable, because the web is built on the Internet and is decentralized and redundant. A single router or server can go down for a few hours, days or forever, and the web keeps working.

Same with the phone network. Imagine if all the cell phones and land lines went down for scheduled maintenence for 12 hours. Again, it’s unthinkable.

Even when there’s a good excuse like a big snowstorm in the east, when the airline system goes down for 12 hours, a lot of people are upset, and it never happens as a scheduled thing.

If Gmail started having twelve-hour planned outages, as much as I like Gmail, I’d switch. I can’t be without email for any extended period of time.

Okay, let’s give the guys at Twitter credit — they stopped being flip about Twitter taking naps or showers. No one likes jokes when a line of communication is down. Now I’d like them to take another step.

Explain to us what these long outages are for. I can take a guess — something about the database needs changing, and all the data in all the files must be processed to implement the change. Any updates made while such a process is running would be lost, so the server must be shut off. But this is just a guess.

Another guess — maybe they’ve hired a scaling expert who needs to make one final major adjustment before these outages are a thing of the past? No one would want to make such a promise, that’s offering too much temptation to Dr Murphy, but that would be good news. Maybe Twitter is getting on to solid ground, finally. If so, I’d like to know.

Meanwhile it’s fairly amazing that there isn’t a viable Twitter clone out there yet, one that does exactly what Twitter does, and runs all its applications.

I’d also like to see something much more decentralized, based on static files, available to any Twitter-like system. It doesn’t seem that far out of reach. With all the scaling troubles Twitter has had it’s surprising that there haven’t yet been any entrepreneurs willing to enter the space to compete with Twitter.

Users and developers are learning first-hand why centralized systems are so fragile. I’m sure they’re doing a heroic job at Twitter, the best they can with what they have, but it’s not good enough when the service takes a 12-hour break while many of the humans that depend on it are awake and working.

Scripting News for 12/14/2007

It can only mean one thing 

My black MacBook, purchased in May of last year, a day after the product was announced and a day after the computer it replaced, a white G4 iBook, died — died itself earlier today. There was an evil clicking sound coming from the back. The spinning rainbow cursor. Reboot it to see a disk with a flashing question mark.

The death of the black MacBook can only mean that there’s a new sexy Apple laptop coming soon. Somewhere between a fat version of the iPhone and a Sony Vaio. Hope it has a real keyboard, not a virtual one.

Amazon’s database 

A note from Jeff Barr that Amazon has announced the database companion to S3.

No doubt I’m going to use it.

I signed up. It’s not open yet.

Scripting News for 12/13/2007

U.S. Blues 

I’m Uncle Sam/that’s who I am
Been hidin’ out/in a rock and roll band
Shake the hand that shook the hand
Of P.T. Barnum/and Charlie Chan

Glad to be back home in the good ole U.S. of A.

Scripting News for 12/12/2007

Best wishes to Scoble 

I was in the audience yesterday when the one dramatic moment of the conference took place. Scoble was on stage. Mike Arrington was sitting to my right in the front row of the second section (on the right). Mike showed me a piece he was writing, and I gulped. It said Scoble was leaving Podtech and probably going to FastCompany to start a TV network for them. Since Scoble is my friend, I knew that there was some truth to this, but was disappointed to see it was coming out, esp at a moment when Scoble wouldn’t have a moment to think and consult with friends before formulating a response. HIs laptop screen was being projected as we watched him edit his comment on TechCrunch. At one point Mike asked if I thought he knew we could see what he was typing. I didn’t think he did.

Anyway, Scoble lets it all hang out. And somehow he gets away with it. What would kill most people just stings him, and he smiles through it all, in his bumbling Scoble-like way, and it always amazes me how he makes lemonade out of the lemons. This isn’t the optimal way to announce you’re leaving a company, and his deal with FC isn’t final yet. But somehow I think he’ll navigate this transition and come out in a better place after the dust settles.

Bebo has the right idea 

I don’t know what Bebo is, I guess it’s a social network, they say it’s #3 in the USA, #1 in the UK, but they just did something that’s pretty likely to work, if it’s technically possible. They’re cloning Facebook so that their service will run Facebook apps. What this means, if they can pull it off, is that they won’t have to fight to get support from developers. That’s a big deal.

Google could have done this with OpenSocial. Watching the panel assembled by Marc Canter at LeWeb3, I was reminded of every tech conference I’ve been to for almost 30 years. Some big company sitting in the center, and lots of smaller ones sucking up to them, not daring to say what’s obvious, that the big company is only interested in limiting the growth of an upstart (not present on stage of course). That’s where the Fear comes in Fear, Uncertainty and Doubt. We don’t say anything because we’re too scared to. Even the outspoken Marc Canter, who’s smart and has been around this block many times, doesn’t dare say what’s obvious.

Now Bebo decides to get in lockstep with Facebook, not Google. Of course! Facebook has the juice. But this is a dangerous place to be if Facebook doesn’t want them there, and being a big company, they probably don’t. The only value of having clones to FB is that it negates the threat from Google. Sometimes having clones is a good thing, but usually the clones take over the market. Witness MSIE and Netscape, and then Firefox. Compaq et al and IBM. dBASE and Fox. It’s possible to depose an installed leader if it’s possible to clone them. As a user and developer, I’d like to see FB be open to cloning. If I were a FB shareholder I’m not so sure.

One constituency that’s sure to like the existence of clones are Facebook developers. Without choice in platform vendors, they have nowhere to go when the sole vendor decides to take over their market. With a viable alternative, unless FB is incredibly aggressive and builds its competitive features so they can run on competitive platforms, at least developers will have a place to run their apps when FB encroaches.

Platforms are a game, like Risk, with rules and strategies. Google did not play the game wisely with OpenSocial. I chalk this up to inexperience on the part of the strategists. So far Facebook has been doing what’s needed to keep its dominant position. Bebo deciding to clone the Facebook API ratifies that position, it’s a gutsy move, but the best one available to them, and to other would-be Facebook competitors.

PS: The news is slow to reach Europe. Facebook said yesterday that they support what Bebo is doing. Very enthusiastically. That should be the end of OpenSocial.

Why I use Twitter 

I use it because it helps me keep in touch with people I want to be in touch with, without taking very much time. It serves a function that the links on Scripting News used to serve, but that was just a one-way thing. Now I get links and ideas from other people. It’s an equalizer, a playing field leveler. It’s useful the same way a cell phone is useful. Sure some conversations on cell phones might seem dull. So hang up. But don’t get rid of the phone (and certainly don’t make general statements about phones based on some people’s conversations).

Arc de Triomphe 

As the cab was heading back to my hotel it turned down an avenue and there was the Arc. I grabbed the camera and got a movie as we approached the circle around it.

Future-safe archives, day 2 

Phil Wolff on things for bloggers to do before they die.

Scripting News for 12/11/2007

Was TechMeme hacked? 

Here’s a screen shot of the HTML source.

Looks like it’s trying to serve some kind of ActiveX control.

Happy to be using a Mac.

PS: Scoble just showed up and TM works on his cell phone so there’s something weird going on.