Scripting News for 1/24/2007

Friday in SF, the Mobile Identity Workshop led by Doc Searls.  


Not sure if this requires disclosure or not, but I decided it’s better to disclose than not.

On Sunday, I added a command to the robots.txt file for that tells the TechMeme robot to not read this site. I was told by Gabe Rivera, the author of TechMeme, that this would have the effect I desired.

I found that when I’d link to an article here that was in its domain, that very quickly that link would appear on TechMeme. Recently, a link to an old article here spawned a link on TechMeme within minutes of its appearance on Scripting News. At the time it was the only item on this site. If a human being were echoing my links so quickly, I’d ask them to stop, if I felt they would respect the request.

I’ve been discussing this on and off, privately, with Gabe, for almost a year. I’m a regular reader of TechMeme, and I plan to continue to read it. There is nothing to stop TechMeme from pointing to articles here on Scripting News, but it’ll have to find out about the articles from pointers on other sites.

I’ve not decided if the ban is permanent or not. I want to try it this way and see what happens. It’s a way for me to learn how TechMeme works, and now that you know, for you to learn too. And it gives people more of an incentive to read this site, or subscribe to it, an incentive you may not have felt before, since so many of my links were turning into TechMeme items. Also it may appeared that I was just replaying TechMeme items here.

I don’t want this move to in any way reflect on the Gabe’s professionalism or ethics. TechMeme is a marvel, and very useful, and he’s a very respected member of the tech blogging community, deservedly so.

I care about quality more than quantity 

Mike Arrington: “It’s rare for a blogger to take action to reduce her/his own influence, but that is exactly what Dave has done.”

That’s what it looks like to Mike, and I respect that, but that’s not the way I look at it.

There are a lot of things I could do to have more influence, but it might not be the influence I want.

Before I put the robots.txt block in, I basically could, unilaterally, put someone else’s ideas onto TechMeme, but I couldn’t get my own ideas there. And because people had less incentive to come to Scripting News, they weren’t reading the things I really care about.

I made a similar choice a few years back, when I quit Wired. Believe me, my flow went waaaay down that day. But my freedom went way up. I’m not getting anything like that kind of lift, but Mike is right, I don’t care about influence as a quantity.

Washington politics 

Politics is all of a sudden interesting. The Congress is organizing across (as opposed to along) party lines. Yesterday on NewsHour, senior Republican Senator John Warner from Virginia sounded more like the opposition than a member of the same party as the President. It’s a miracle to see discourse come back to national politics. We are still in a very dark period, but it’s getting brighter. And once again the brilliance and luck that’s designed into our system is saving our ass. John Adams and Thomas Jefforson hated each other after George Washington left office peacefully and Jefferson’s cousin John Marshall had guts. We’re living in a country whose political system is largely formed around the personalities of these four men.

Throughout the President’s speech last night he expressed an idea that surely the founders, all of them, would have objected to — the idea that the people in the room were doing The People’s Work. No, that’s not the idea. The idea is much heavier than that. The people in the room are the people. That’s why the House is re-elected every two years, and that’s why the President has to come to Congress to get approval to go to war, and that’s why, when Warner reminds the interviewer that Congress is the equal of the Executive branch, she can be forgiven for needing a reminder, because for the last six years, Congress has not been doing its job. It thought it was here to serve the President. That is even further from the intent of the founders.

No matter, the founders win this one. Right now the Congress is not only serving the interests of the people, they are acting as the people, and that of course is good. That there are so many candidates for President is a sign that the system is working. Everyone should have the guts to think their ideas as worthy of discourse. This is where the philosophy of blogging and the philosophy of the US are totally in synch.

Jim Webb, the other Senator from Virginia, a Democrat, gave an absolutely stunning rebuttal to the President. I stood up and cheered, tears running down my cheeks. This is the kind of person that the founders imagined would be our leader. By the end of the speech I found myself hoping that Webb runs for President, although I think it unlikely that he will. But he would make a good President. I haven’t felt that about anyone in a very long time.

And also yesterday, lost in the State of the Union shuffle, were the initial revelations in the Scooter Libby trial. Fascinating, unexpected. The White House itself is becoming a circular firing squad. First the Vice-President is implicated by the prosecution, then Karl Rove is implicated by the defense. Who knew what when? Well it turns out there was a coverup. No major surprise, but it looks like it’s going to come out before Bush leaves office. What then? Will there be an impeachment? In the atmosphere in Washington today, it’s hard to imagine that the President wouldn’t be taken to court for his crimes. And for the President, the court is the House of Representatives.

Only one thing can save the President, it’s unmentionable, but it was mentioned in a CNN interview this morning with David Gurgen from Davos. Maybe the high mountain air made him say something that you wouldn’t say with your feet on the ground in the US. The one thing that could save the President, he said, is a catastrophe on the scale of 9/11/01.

Yes, politics has gotten interesting, once again.

Not sure if that’s good or not. πŸ™‚

State of Wikipedia 

More often than not, Wikipedia is the top result in Google searches, and more often than not, given a choice I’ll point to the Wikipedia page as the definitive source, without knowing whether the text was written by an impartial third party with good information, or someone else; while I know that in areas where I have expertis, the Wikipedia pages are the result of “edit wars” between partisans, trolls and the people being written about. It’s hard for me to know, for example, when reading a biography of Augustus Caesar (I’m a fan of the HBO series Rome) if it is the result of the same conflicted process.

Microsoft is the latest to fall in the trap. They are criticized by Jimmy Wales, who has no credibility in this area, having been caught editing his own biography, removing mention of his collaborator. Wales is aware of the basic flaw in Wikipedia, his actions indicate that, yet he isn’t above criticizing Microsoft for trying to hire someone to do what he did for himself. To be clear, what Microsoft did is absolutely wrong. If that practice were to escalate (and who knows that it hasn’t) then Wikipedia would just reflect the views of rich corporations and individuals. The biography of Bill Gates would talk glowingly of his philanthropy, and downplay (or omit) his conviction for antitrust.

To me, in areas outside my expertise, it seems that Wikipedia is an excellent source of information. But that’s the problem. In areas that I know better, I can see its flaws. I play by the rules and don’t fix the mistakes. That leaves it to the trolls to write the story. Somehow we have to resolve this. And Wales should recuse himself from being the judge in these matters.

22 responses to this post.

  1. Dave,

    Not so much to support Microsoft as to advance another POV, I think you may be unduly hard on the company for its Wikipedia efforts.

    Think about your own (bizarre) experience with WP–it sometimes seems useful to correct errors and set the record straight.

    I also think about my experience. The organization for which I work is sometimes controversial, and sometimes people write things that are plainly false or at least slanted. The first, I correct from time to time, occasionally just striking text. We’re not in cahoots with the Klan, for example (which wouldn’t even make sense, anyway). The latter I try to address with more information, noting facts that cut the other way or that the point is not so clear-cut. Of course, I am paid by my organization, but not specifically to edit Wikipedia.

    I think the key is openness: be clear about your affiliation, document assertions, remove content only when it is unarguably wrong or irrelevant, and explain changes. But there’s no reason why a person or organization should tie its hands from addressing inaccuracies or adding fruitfully to discussion. In some cases, individuals will get paid to do that. So what?

    Microsoft may has stepped over the line a bit–it should have been more open. But it may be facing some undue flack over its Open XML format, which, despite its shortcoming, is undoubtedly an improvement over its predecessor. The company’s impulse is understandable, but its means of execution were a bit off.


  2. I said they were wrong, and that’s what I think.

    Re Wikipedia, if it says something about you or a competitor that you don’t like you don’t have a choice but to take it. It doesn’t matter how much money you have to hire people to fix things. And come on, they told the guy to say what he thinks, but they’re customers, and somehow the guy is supposed to be unbiased? Heh. Good one. (Slapping my knee in appreciation of the good joke.)

    Let others figure out how to put a process in place that makes Wikipedia more accurate, or alternatively, given enough time the world will catch up with Wikipedia and realize that it often can’t be trusted. That’s where I’m at.

    I heard a story about the early days of eBay, about how all disputes were arbitrated by a user in Vermont or Maine, and that worked, but nowadays they need to hire former FBI agents to investigate crime on eBay. After a while, the utopian spirit fades, and the online communities become like the rest of the world. So of course its inevitable that rich companies are going to buy good publicity on Wikipedia, the same way big companies throw their weight around in RSS-land, and pollut the namespace with all kinds of poorly understood and duplicate stuff, and file patents for ridiculous supposed inventions. That doesn’t mean they should be immune to criticism for doing that.

    No, I wasn’t “too” hard on them, I said what I thought and that’s how it works.


  3. I agree with you absolutely about the Jim Webb response. The most encouraging nine minutes in US politics in a long time.

    I’m not sure, however, I agree with your view of the Founders’ intent. Congress isn’t the people, it’s the representative of the people. That’s an important distinction. We don’t have direct democracy, as in Pericles’ Athens (if you were male and free). We elect representatives.

    In a healthy democracy, that sometimes means that our representatives make decisions that are unpopular. Throughout Europe, for example, capital punishment is outlawed (in fact, you can’t become a member country of the European Union unless you get rid of the death penalty). In most cases, when legislatures did that, the death penalty was still favored by a significant majority. But the representatives made a wise choice.

    If Congress is merely the people, then we should decide everything by plebiscite. I don’t think that would be good.


  4. Very nice encapsulation of the SOTU Dave. (At some point it occurred to me that SOTU and STFU are remarkably close, united by the letters US to a T.)

    I don’t understand Gergen’s reasoning (which isn’t new), though. For several reasons:

    1. Another 9/11 would obviously be on the Bush watch. He and his policies would be responsible for it, i.e., it would confirm criticism that he is inciting more terror. And all the Homeland Security stuff would be shown to be bogus.

    2. It would mean he’s failed, after all this hoo-hah, to protect the U.S. from terrorism, and that the terrorists are indeed winning.

    3. We’ve *already had* another 9/11. It was called Katrina. Its unpleasantness was not even mentioned, undoubtedly for that very reason. Bush failed then, and another 9/11 would just be a reminder of more failures.


  5. Hi Dave. Two remarks…

    First, in my recollection, we hadn’t been talking about this before. You did raise the issue of exclusion last year, but didn’t mention why you were interested in this. Unless I’m missing some email or something…

    Second, I don’t know if the robot.txt will have the effect you desire. Specifically, I said “that prevents it from getting anything from your domain of course”. If that’s what you desire, then it will have the effect you desire. πŸ™‚


  6. Posted by Anton2000 on January 24, 2007 at 12:22 pm

    Sorry girls, what a waste of time


  7. Re: Microsoft/Wikipedia: There’s a post on slashdot (currently in the Microsoft/Wikipedia branch) from Doug Mahugh, who clains to be responsible for contacting the Australian about Wikipedia. He states that he had simply floated the idea to Rick to blog/comment on and he has not been paid.

    Re: Techmeme robots: Also, glad to see some effort going towards cleaning Techmeme from simple posts which link to the main story/thread. It seems that when a big story breaks, all an ‘a-lister’ needs to do is simply mention the link and lo’, there they are in the Techmeme thread – linking to an item with little or no substance over the actual story. but naturally, this can generate alot of traffic for the linker.
    I suppose this is where robots and human differ πŸ˜‰


  8. “It’s hard for me to know, for example, when reading a biography of Augustus Caesar [in Wikipedia]… if it is the result of the same conflicted process.”

    Of course it is, Dave.

    The problem is, that’s true of any reference work, or history, or what-have-you.

    The thing that Wikipedia does very well is be transparent about these conflicts. But it’s only the access to editing articles where Wikipedia differs from more traditional works. The same trolling, squatting, back-biting, etc. goes on in any editing process.

    This has always been my objection to your critique of Wikipedia. Perhaps I’m reading more in than is there, but I keep seeing an assumption that somehow these issues are unique to Wikipedia, and not so common as to be nearly universal. (If you like Rome, read the Anecdota by Procopius sometime.)


  9. Dang it, Hal. You beat me to it.
    The vision of Wikipedia isn’t to somehow improve the editing process, it’s to make it transparent and freely accessible to anyone. It really is as much an institution-building idea as it is a application model — free information, without registration or discrete identification.
    Wikipedia is not under attack so much that the community behind it now needs to back down from its ideals, the media just likes to act like it is.
    Do check out his TEDTalks speech if anyone hasn’t already:


  10. Posted by billg on January 24, 2007 at 3:35 pm

    For me, the transparency of Wikipedia adds little, if anything, to its trustworthiness. It probably detracts from it, in fact.

    A reader’s assessment of a publication’s accuracy is a complex process, usually rooted in how that publication’s statement stand up against other already trust sources and real life.

    Readers are aware that an editorial process is at work behind the scenes, just as they are that there’s a culinary process behind the scenes at their favorite restaurants. But, just as a blow-by-blow description of who’s arguing with who in the kitchen won’t change the taste of the food, documenting the editorial process won’t change the accuracy of the final product.

    It may change how that final product is perceived, just as watching what’s going on in the kitchen may change how you perceive tonight’s dinner. But, in either case, that insight has no impact on the final product itself.

    The kitchen analogy only goes so far, however, because Wikipedia is like a restaurant with an endless supply of cooks. When you decide to trust a restaurant, you’re really saying that you trust those specific people who are running the place.

    You can’t establish that kind of relationship with Wikipedia because, at least potentially, the door into the editorial staff is a revolving door. The fact that there’s a window in that door really isn’t that useful.


  11. Posted by Diego on January 24, 2007 at 4:04 pm

    “Re: Microsoft/Wikipedia: There’s a post on slashdot (currently in the Microsoft/Wikipedia branch) from Doug Mahugh, who clains to be responsible for contacting the Australian about Wikipedia. He states that he had simply floated the idea to Rick to blog/comment on and he has not been paid.”

    Isn’t that sort of like being arrested for solicitation? Sure, you didn’t go all the way, but you wished to do so. πŸ™‚


  12. Diego : I have no idea (maybe) what you mean by that comment, but in response to those above (your post somehow inspired this reply) πŸ™‚

    The Bible itself in open for ‘confusion via translation’ – The only way it (or any other religious ‘testament’) can be truly understood is to read it in the language it was written in. Translation will never, ever provide a TRUE translation of the original author – especially if that person is thousands of years gone.

    The same goes for ALL regligious ‘documents’ which so many people live by.

    These days – in 2007 – I KNOW the world is round. The other thing I KNOW is that humans should not kill other humans – that’s about it. It’s so easy to understand, yet so few ‘get it’. They ‘do’ kill eachother. Idiots. Stop it. Now.

    Anger and misunderstanding can drive many ‘stupid’ emotions, which drive people to do many ridiculous things. But only but learning from those MISTAKES can we ever have a chance on this simple planet.

    sheesh. Heavy πŸ˜‰ mov along now…. πŸ˜‰


  13. Dave, a technical question. You said: “There is nothing to stop TechMeme from pointing to articles here on Scripting News, but it’ll have to find out about the articles from pointers on other sites.”

    Well yes, there is. The robots.txt file change you made means that the Techmeme crawler will not hit your site at all, and it needs to do that to figure out what the headline and first X words of your entry are to be able to display those. It can’t just use the URL. By doing this, you are banning yourself completely from Techmeme, not just from the “discussion” part of secondary links, but the headlines as well.

    Is that what you wanted? Maybe what you wanted was to be eligible for headlines but not for the discussion. In that case, instead of amending your robots.txt, what you are after is adding the rel=nofollow attribute to all your outgoing links… as Wikipedia has just done.


  14. Paul : Essentially this where human intervention differs from ‘scripted/robtic’ action.

    I remember when Yahoo was a directory of great websites, chosen by humans FOR humans. It was great.

    But then, either the humans were usurped by ‘scripts’ which could do a ‘better’job via clever algorithms or we really dont seem to ‘care’ about ‘humans’ any more.

    Trouble is, when ‘people’ contribute to a system like that, we eventually come across those with their own ‘mission’ at heart. Conflicts of interest. Wikipedia is rife with it. The world is rife with it. Religion itself is rife with it.

    What people will ‘take as gospel’ is the problem.

    Right? πŸ™‚

    Wikipedia is a great research tool – and can be deeper or close to the truth – but whatever it is, it can help those ‘seeking truth and facts’.

    This IS(can be) the so-called ‘wisdom of crowds’

    What’s needed is a big fat truth/fact (potential) disclaimer to encourage those who seek the truth to look further than that website to seek it out – then add that knowledge for those who seek the same answer, after you went there.

    I have alot more to say that I think about this, but not here – Dave’s comment threads are not the place. Unless he says it’s fun to read – but I doubt it ;p

    These threads pass so quickly. As do we.


  15. kosso: I have no special insight into the mind of Gabe Rivera, but I assume he and his Wazzup crawler are going to follow the rules of robots.txt like every other search engine.


  16. Yeah man (Paul) – I think that’s the trouble. We either write scripts to do cool stuff ( I do visit TM daily) or we end up with real people, with real issues/conflicts who are happy to bend the truth at the expense of those who purely seek the true truth.

    I think TM *needs* human intervention – but I dont think users will digg it, so to speak πŸ˜‰

    But I do believe that ‘the truth will out’ – and I have no idea, why, in this day and age, it takes so long to transfer feelings to people through text, when those feelings are so simple. So logical, at times, too.

    I really do wonder what the world would be like, if we could read the public diaries (ie:’blogs’) of the ancient people – especially the Egyptians and Pharaohs – There was such a HUGE amount of mathmatical understanding then, that surely, there must have been ‘logic’.

    Would they have *any* influence. Of course they would. So many of our human faith systems are based on ancient texts. Whether you like if or not, everything we say now on the web will be ‘available for indexing’ by all the search engines of the future.

    I really think the ancient races would have ‘got it’. But the thing is – could *anyone* ‘do’ it? I doubt it. Despite the mindnumbing intelligence there must have been 2000 or so years before some guy called Jesus popped up (so-called year-zero), there must have been (and still is) huge ignorance in what was (is) good for ‘the people’.

    sorry for the rant – I blame brain-washing US television in hotels for days πŸ˜‰


  17. Surely they will adhere to certain instructions provided by the robots.txt ‘standard format’ – will is is ‘good’ and unbiased?

    The best thing about the web (for most of us) is we have the opportunity to state our opinions and share knowledge and facts.

    I don’t know of any robots which really understand the difference between real discourse and simple echo/linkage.

    Tricky job for Gabe, though eh? πŸ˜‰ As long as over 90 per cent of links are worthwhile, then I can live with that – but wouldn’t it be cool if one day, there was a truly unbiased human opinion behind the source of facts and whether or not sharing it was worthwhile? Making it easier for people to grok without digging deeper?

    Ain’t it sad that it’ll probably never happen? πŸ˜‰ At least, not one that a machine script could ever understand. πŸ™‚


  18. Pass de kutchie ‘pon de lef’ hand side.


  19. Paul, that was not my intention, and I think you’re right, because yesterday it seems TM should have pointed to one of my pieces since it was pointed to by other sites.

    I’m doing what Gabe told me to do in April of last year. As far as I know there is no other opt-out mechanism that’s more specific.

    What I want is for TM to not count a pointer from Scripting News to another article. I’ve tried to explain it as clearly as I can. I’ve also sent Gabe a couple of emails about it, and would be happy to email with him privately to be sure.

    It’s someting very much like rel=”nofollow” btw,


  20. Yes, I think rel=nofollow is what you’re after then. Unless you can successfully lobby Gabe to modify his scraper and/or algorithm to discount your outbound links without you fixing your own site.

    The difference is between removing the “vote” that your link represents from just TechMeme’s algorithm, or to other search engines’ algorithms, i.e. Google’s PageRank. If you want to remove your linklove from Techmeme, why not remove it from Google et al as well?

    But that is a whole other bag of monkeys, since you leave yourself open to the same criticisms as Wikipedia got in the last couple of days…


  21. I don’t care if people criticize. If they do it fairly it’s just an expression of a point of view. If they make it personal or try to punish me somehow for disclosing, then I think it reflects poorly on them. I could have just said nothing and I doubt if anyone would have noticed.

    If Gabe wishes to provide a more specific way of expressing this intent, and it’s not too difficult to implement on my side, I’ll do it.


  22. Posted by Diego on January 25, 2007 at 3:04 pm

    Hey Kosso…

    “Diego : I have no idea (maybe) what you mean by that comment, but in response to those above (your post somehow inspired this reply) :)”

    Oh, maybe I took your comment prior to mine the wrong way. I read it as you saying, OK Microsoft talked to someone about changing some Wiki entries for money, but in the end nothing happened and no money was given. Sort of suggesting it’s not that bad because no money changed hands. That’s why I said isn’t that like propositioning a hooker, but you get picked up by the cops. You still had intent, even though you didn’t go through with it. You still wished the transaction to have gone through. You being the collective you and not you persoanlly. Don’t want any trouble here. πŸ™‚

    Anyway…. hehe I digress. Maybe I misunderstood, and that’s cool. I never had any problem with what you said. πŸ™‚


Leave a Reply to Paul Montgomery Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: