I'm concerned about conferences with serious gender imbalances, much like Kathy Sierra and Shelley Powers with their recent posts on ETech. However, it's not ETech I'm thinking of.

To put it bluntly, a knitting conference can be hostile to men. The interaction style is touchy-feely -- women walk right up to strangers and fondle their knitted garments, invading personal space. Although some have said that men aren't discouraged from attending and in fact receive positive attention for being there, this can be of the form of "How nice it is to have men attending", "Is this knitting book for your mother?" (ref) and "I'm sure your wife will love this yarn.

All the instructors at knitting conferences are female, and one wonders if the mostly-female program committee could have something to do with that. In the expos or markets, one finds patterns for shawls, purses and female garments. Although some of the market vendors are male, they are clearly there as "booth bunnies", to attract women to look at the yarn. Some of the male vendors are even pressured to wear demeaning and ridiculous knitted vests.

What can we do about it? Well, we can be more equal in our language, for a start. Articles like this, although mentioning a few knitting men, are given titles like "The yarn is flying as more women discover the joys of knitting." Vendors like Habu textiles are taking a step in the right direction, offering stainless steel yarn, which is sure to appeal to men. Joe, a male knitter, has some other great ideas, such as making the act of knitting into more of a competition.

Researching Bali, found this:

The once beautiful Bale’s and other buildings have long since fell into a state of disrepair. It was as if the villagers had ‘locked-on’ to the ring of the till from the tourist dollar. This was more than evident when I had a short but sharp conversation with one of the souvenir sellers:

Tourist: I was here 18 years ago and I am surprised at the change in the village.

Seller: (Abrupt tone) What’s wrong with the village?. Now I have a motorbike. We have TV and electricity and... a phone!

It was my point precisely! Too many cultures in this world are decimated by the tourist dollar. Not only that, the intrusion of the modern world has an overall affect upon the social, religious and political aspects of that society. So much so that it literally forces the entire socio-cultural structure to undergo a metamorphosis in order for adaptation. Having said that, tourism is good for the state of the economy in any country. However, when it has a serious affect on the core culture of that country, then it becomes disadvantageous.

Much as I'd love to visit other countries and have them be all picturesque and unique, I can't begrudge a Balinese his motorbike, TV, electricity and phone. I wouldn't give up my conveniences to live in the style of my grandmother, nor would I appreciate pressure to maintain the religion of my ancestors. A tourist like this one would have to stop being a tourist (being one of those intrusions), go back to the country of his own ancestors, give up his own conveniences etc. for me to take those sentiments seriously.

Philippe pointed me to David Berlind's list of Technology's top ten failures. Two of them are directly about calendar standards, asking for the ability to schedule meetings and share calendars. Another two are email related. Good news, David, we're working on the calendaring standards, and calendar and email implementation issues for you, already. Sorry I can't -- personally -- help you with phones and printing problems just yet.

Who is your favorite Philosopher?

Ekr points out that the question "Who's your favorite philosopher" does not have an analogue in all fields. In highly technical fields, you wouldn't ask "Who's your favorite cryptographer" or "Who's your favorite biologist". OTOH it's common for musicians to talk about their favorite pianist or favorite composer, for painters to have a favorite painter, and for writers to be asked about their favorite writer (it's a formula interview question particularly for writers). So I guess that means that in not only asking the question but insisting on it as a valid or important question, David Brooks is implying that philosophers are like artists, their work subject to aesthetic judgements, preferences and stylistic likes and dislikes. I'm not sure that's what he intended in bringing attention to the question.

There's lots of action in the online calendar world. A fair amount of it gets brought to my attention, which is lots of fun.

• Ted pointed me to Mosuki, and I hope to meet some of the creators shortly. It's more oriented towards personal calendars than the other sites but it does allow you to share.
  
• I met Brian Dear a few months ago and heard about EVDB. Now that it's announced, I can talk to others about it too. Scraping sites for event and venue information sounds like a really valuable service.
• Kragen Sitaker showed me a del.icio.us-like calendar site that seemed pretty cool. Rather than try to scrape every page, his demo let the user pick certain text, send it to the demo site and it would turn that text into an event (pulling out date, time and location if it could) on your calendar.
• Brian posts about Upcoming, a public events site with feeds to let you know about event categories you're interested in, and a way to add events you want to your personal calendar. It has a HTTP/iCalendar interface that allows users to synch into iCal. According to del.icio.us, upcoming is the biggest new thing with hundreds of bookmarks.
• So far, Trumba's newly announced OneCalendar gets a rather less enthusiastic response. Trumba consists of several ex-Visio guys according to their press release, and like EVDB they have high-powered investors so expect some noise. Already you can synch with Outlook which has got to be a highly desired feature.
  
• Of course, we are working on some of the same stuff too, only with a federated server approach. When you share your calendar with Chandler you can share it on any WebDAV or CalDAV server (see also recent article). We're working on a WebUI for such shared calendars so that the calendar owner or their friends can view the shared calendar or individual events just by going to the URL in the browser. We may do tagging just like everybody else, too.

So far I don't use any of these, there isn't quite enough "there" there, whether it's users I want to share with or features I need. My calendar is in iCal, stored locally, and I don't even share it with anybody, though I do share one other person's iCal calendar. But I don't think that isolation will last long given the exploding options.

I wonder what Yahoo, Google and Microsoft are going to do in this space? Yahoo has more of the world's calendar info than any other site at this moment, because of its excellent support for group calendars. What will Yahoo do with that data? If it does not expand its features soon, possibly allowing users to synch up calendar data on the site with their client software, Yahoo will soon see its calendar data decay and vanish. As for the two gorillas, I really don't have any idea whether they will buy, innovate, or crush.

In response to my last post, Mark pointed out that .NET's HTTPClient supports pipelining. Heikki said that Mozilla put in support for this but it isn't turned on normally because so few servers support it. Furthermore, Heikki reports that tests showed only a 7% performance increase when Mozilla uses pipelining. The Mozilla FAQ is a good start for information on this though it doesn't have the data.

The application I had in mind (Chandler) does more synchronization than browsing over HTTP/WebDAV. For example, if the user decides to work offline, and while offline moves a bunch of resources from one collection to another, then goes online again, Chandler would have to issue one MOVE request for each moved resource. With pipelining, Chandler could theoretically fire each request off and wait to start seeing the responses come back in order over the same connection. The only thing limiting throughput is the bandwidth and MOVE requests and responses don't take up that much. Without pipelining, Chandler has to wait for each response before sending the next request. Now this work is limited by latency, multiplied by the number of resources being moved.

Still, I don't consider this proof that pipelining would be useful. It would be great to see more data one of these days -- assuming, of course, that there really is a performance problem in the first place.

Are there any open source HTTP client libraries that support pipelining? Are there any open source libraries that solve an isomorphic problem with a different protocol? So far I'm not turning up anything but this is the kind of thing that is a little hard to Google for.

At the gym last night I was lifting weights and a man started this converstaion.

Man: "What are you going to do with all those muscles? Beat up the boys?"
Me: "Yup."
Man: "Well then, stay away from me.

What am I supposed to make of that? Seriously, I'm asking here.

... So this hat doesn't fit me. The question is, would it fit a guy with a size 7 3/4 head? Joe, you might or might not want to click through :)

There are a whole bunch of "Working Group last calls" going on for email-related technology. Roughly, these are calls for reviews of Internet-Drafts before they are submitted to become standards. I called for reviews for an IMAP Extensions working group document which adds annotations (like properties, or metadata) to emails in an IMAP server. The Lemonade WG has issued last calls on these documents (all last calls on the mailing list and the documents themselves are linked from the Lemonade charter page):

• the Catenate proposal, used to allow clients to more efficiently work on draft messages stored on an IMAP server
• URLAUTH, which lets one user show an attachment to another user without sending them the whole attachment (basically I give you authorization to view my copy on my server)
• Server-to-server requirements for email event notification systems -- intended to allow voice mail and email servers from different vendors work together more smoothly
• The BURL draft, allowing clients to forward an email without downloading and uploading it again as email systems currently require.
  

There are also upcoming improvements to the performance and extensibility of the LIST command which is the heart of IMAP, and ongoing improvements to access control, both coming to a head in IMAPEXT. If you're interested in email protocol and developing standards for those, this is an exciting time for that. I guess that means about 7 of us globally are excited right now :)

I attended a knitting convention this weekend, and learned how to do japanese short rows, double crochet (which is like a cross between knitting and crochet), and twined knitting which is a three centuries old Swedish technique a.k.a. "Tvåändsstickning". The marketplace was also amazing and very tempting and dangerous to my wallet.

The "gallery" at the Stitches conference is very ad-hoc -- it's on the backs of the attendees. Here's a few pics of people I saw.

It's starting to become well-known that well-published suicides inspire more suicides. There's an explanation on everything2.com, but I'm dissatisfied with the references to heroic portrayal as the explanation for the phenomenon. The Wikipedia entry is much more complete, but it still explains copycat suicides by refering to the media "glorifying the deceased." This "reward" theory is simple--that the media encourages copycat suicides by providing attention and glory as a reward for suicide. The standard sources also explain that media coverage may provoke admiration of the deceased in eulogizing him, and by explaining the causes of the suicide and thus implying that suicide is an understandable or normal response to those causal events.

The reward theory has been debunked somewhat in Cialdini's book Influence,
the Power of Persuasion. Some brief reasons to doubt the reward theory:

• Even consistently negative publicity encourages copycat suicides
• Copycat suicide demographics are surprisingly similar to the original suicide (e.g. 35-year old women don't tend to see Kurt Cobain's suicide as one to copy no matter how much glory is supplied)
• Deaths which are not publicized as suicides but which might have been (car and plane accidents) provoke a rash of similar deaths even though there's no glorifying of suicide.
  

Cialdini's model to explain copycat suicide is more sophisticated than the simple "reward" model -- Cialdini calls the effect "social proof" and it goes beyond just suicide to explain how people copy the actions of those they consider similar enough to be role models. So even if the suicide is criticized unanimously, someone who feels similar (same preference in music, perhaps) will find actions to emulate. In fact, the criticism from the dis-similar people may strengthen the bond felt towards the role model. The bond may be strengthened by the copycat feeling that society is against both himself and the original suicide and provoke feelings of solidarity. Thus, a society that reacts to some unusual person's suicide by criticizing their unusual taste (goth clothes, punk music, or playing dungeons and dragons) may in fact create a self-fulfilling prophecy in calling that taste dangerous and then see a spate of suicides in a population sharing that taste. There's also some evidence (again you can reference Cialdini who discusses Goethe and his hero suicide Werther) that even fictional suicides inspire copycats.

Dorothy Parker seems to have intuitively understood this. I recently read Bobbed Hair and Bathtub Gin, a look at literary culture in the Twenties by examining the lives of Dorothy Parker, Edna St. Vincent Millay, Edna Ferber and Zelda Fitzgerald. Dorothy attempted suicide several times in her life, each time via a different approach, and damaged her own health and reputation. In her short story "Big Blonde", Dorothy wrote:

There was no settled, shocked moment when she first thought of killing herself; it seemed to her as if the idea had always been with her. She pounced upon all the accounts of suicides in the newspapers. There was an epidemic of self-killings -- or maybe it was just that she searched for the stories of them so eagerly that she found many. To read of them roused reassurance in her; she felt a cozy solidarity with the big company of the voluntary dead.

The subtlety of the copycat suicide effect probably means there's no easy solution. It doesn't work to have the media to portray suicides negatively -- that doesn't prevent the social proof phenomenon and may even strengthen it. We probably don't want to instruct the media to entirely suppress news of suicides. Even if we did that, there's still fiction and possibly other art (music lyrics?) -- and even if we censor art there's still deaths-which-might-have-been-suicides.

Despite these subtleties, most journalism codes around discussing suicide in the media focus on the glorification aspect. E.g.

a licensee must not broadcast a program which depicts suicide favourably or as a means of achieving a desired result (ref)

The WHO report from 2000 also talks of glorification and acceptance of suicide as an understandable response, but it goes even further, stating that "certain types of coverage may help to prevent imitation of the suicidal behaviour" (p 6). However, I've not yet seen evidence of that, and I worry that is merely wishful thinking. If Cialdini's model is closer to being correct then the very type of coverage that the WHO report suggests may do more to encourage copycat suicides by mourning the deceased, providing details of their life and families, and providing "risk indicators and warning signs" which can trigger the role model effect.

Update: I've never updated a Wikipedia page before, but this seemed a good time to try. I added the paragraph on Cialdini's social proof model.

I got annoyed on the drive to work today, listening to NPR's Morning Edition program. The show was covering California's new display hardware tax and recycling program. I guess "coverage" always means looking for flaws, but this was silly.

Here's the facts: California is adding an electronics recycling fee, which retailers apply based on size of display, from $6 to $10 per display. This includes laptops, monitors and TVs.

Morning Edition added the factoid that California pays recyclers $0.48 per pound to process the displays, to chew them up and sorting the resulting bits into plastic, glass, etc. From this factoid and the fee rates, Morning Edition concluded that the tax would certainly not pay for disposal. They gave an example of a 53 lb monitor, which would be taxed at $10 at time of purchase but would cost $25.44 to recycle.

With the information provided so far in the program, I could see immediately several flaws in their argument. First, not all monitors are that heavy. People buy 5-10 pound laptops, pay $6 or $8 fee at purchase time, and those only cost $2.40 to $4.80 to dispose of. Second, there's time-value of money: the $10 paid for the 53 lb monitor is paid years before that monitor has to be recycled, and if the state saved that money it could accrue interest.

Even if you say that this year's recycle fee income should cover this year's disposal costs (rather than allocate the fee to one item over its life) it's entirely possible that the program does pay for display recycling. It works if people buy more lightweight displays (for which they overpay on the recycle fee) than heavy displays (for which they underpay). At the same time, it also works if people buy more displays per year than they dispose of -- also a reasonable possibility.

This kind of thing in the media bothers me more and more. It makes me think "How stupid do they think I am?"

Today was the first Interop event for CalDAV. Sadly, I am not there, having cancelled a trip due to a cold. But I'm excited to hear about preliminary successes. It's only been a couple months since people started developing code to this spec, so this is great progress!

I went off on a tangent today and wondered how many translations of Marot's poem "A une Damoyselle Malade" could be found on the Web. I thought there would be more because there are nearly a hundred in Hofstadter's book "Le Ton Beau de Marot" alone, and he encourages readers to write their own. Anyway, here's the list I found, using the first line rather than a title to refer to them because that's what I found most memorable:

• Petter Hesselberg wrote Min Konfekt in Norwegian and provided an English gloss of that
• Mark Irons wrote Little Mole
• Craig Clark wrote Precious Girl
• David Short wrote Sweet Jeannou
• Melanie Mitchell wrote To my sweet (or in her more wordy version, "To be a joy forever to my sweet")
• Enno de Witt wrote Teder Wicht in Dutch (gloss, anyone?)
• Kat Walsh wrote Sweetest Rose.

Rather than provide my own translation, I wondered what Babelfish would provide. To get a decent approximation I had to remove line breaks (format as a paragraph), replace abbreviations with fully-spelled-out words, and modernize a couple words. My input:

Ma mignonne, je vous donne le bon jour; Le séjour, ce est prison. Guérison recouvrez, puis ouvrez votre porte et que on sorte vite, car Clément le vous commande. Va, friande de ta bouche, qui se couche en danger pour manger confitures; si tu dures trop malade couleur fade tu prendras et perdras le embonpoint. Dieu te donne santé bonne, ma mignonne.

Babelfish spit out (with me re-adding the line breaks as with the original):

My nice,
I give you
the good day;
The stay,
it is prison.
Cure
cover,
then open
your door
and that one left
quickly,
because Clément
orders you.
Goes, fond of delicacies
of your mouth,
which lies down
in danger
to eat
jams;
if you hard
too sick
insipid color
you will take
and lose
the plumpness.
God gives you
good health,
my nice.

Not too bad, really, though I had to trick Babelfish to get it to be even that good.

It seems that to keep an eye on terrorists and other dangerous people, the government could be more creative in outsourcing the hard work of keeping an eye on citizens. Leave it to an expert -- Santa Claus. He'd just have to expand his "watch list" from kids to include adults as well. Then we could update the famous song and and still keep the exact same Christmas spirit:

Don't drive like a jerk,
Or be late paying fees,
Steal Post-Its from work
Or share mp3s,
Santa Claus is coming to town.

He's making a list,
Of those who can't fly,
With guys who made jokes,
In the security line,
Santa Claus is coming to town.

He knows if you've been cheating,
On your taxes or your wife,
He knows if you've been smoking pot,
Three strikes and you're in for life.

Anyway, happy holiday season.

Jon Udell just posted his wishlist for a calendar protocol (comparing it to WebDAV-only and to traditional calendar-only protocols), a list that's rather close to explaining why I'm working on CalDAV and what I want it to do. (Jeffrey Harris pointed this out to me and even pinged Jon about it, thanks Jeffrey)

A couple new items in my "stuff I made" pages, both the knitting page and the general page: a crocheted purse (almost clutch size) and knitted lace pillowcase edgings.

I'm hiring again at OSAF -- this position is for our very first server developer. I'm looking for somebody who can lead the charge, crank out code, and make the server do us proud.

Data modeling is hard. Some loosely correlated thoughts and links:

• Model-driven architecture is rigid, at least with the tools as we know them today.
• RDF has a simple basic model but leads to very complex structures, as Adam Bosworth explains.
• Pictures express complex relationships relatively readably, like the picture in this paper. Unfortunately we need to translate the pictures into text in order to use these in software and network protocols.
• The more complex your picture is, the more unreadable your text is.
  
• Text has to be relatively flat to be readable.
  
• References in data formats are like "goto" jumps in programming -- you lose context.
  
• Maybe if data modelers put a little more thought into flattening their models we'd find them easier to use? This may make the models seem less "rich" but "KISS" is good too.
  

I don't know how true all of this is, but I'm learning.

The "relatively flat" observation seems to hold at least some validity in data formats, programs and even books. Experienced programmers, with the help of good indenting, can see quickly that they're within an 'else' statement inside a loop inside another loop inside an 'if' statement, but even experienced programmers screw this up sometimes (and even more experienced programmers flatten out the code by delegating some reasonable piece off to another method). Books are better if there's no more than three (maybe four) layers -- chapter, section, sub-section, and even this much organization requires human-readable text to link from one section to another and summarize what a bunch of sections are going to say.

There's a higher quality of homeless people in Palo Alto. I just saw a hand-lettered sign outside of Whole Foods, explaining that the homeless of Palo Alto need food donated in the holiday season (and a very shifty looking guy collecting food on their behalf). Among the list of requested foods was "organic turkey". No cheap turkey for the Palo Alto homeless, please!

Timboy has a good medium-sized post on why meetings suck yet they're indispensible.

A while back I posted on honesty in journalistic bias. This week TechCentralStation has a longer essay about why we might see more openness around bias and why that's a fine thing.

Being aware of bias is something I agree with, but I do worry about blinkered views of the world. Too many people reading some highly biased source will simply not read any opposing source, or do so with only mockery in mind. We've got plenty of polarization, thank-you. So my preferred model is journalists who say "Here is my natural bias, and here is me being as unbiased as I can be in covering this topic, through rigorous reasoning and discourse with others who disagree with me."

Writing protocol standards is hard work, harder than writing specifications, although they are similar tasks. One of the reasons is that you have to describe the protocol in sufficient detail that somebody who wasn't involved in the process and has different software experience (different features, different user interactions, different architecture, different platform or different programming languages) can still implement the standard and interoperate with other implementors. (Actually it's so hard to do this that no standard gets it "right". At the IETF we're well aware that we do successive approximations, first doing internet-drafts and then doing RFCs at different stages of maturity. ) But we can at least try to do it right, and a proper effort requires a lot of effort including:

• A description of the model
• Implementation requirements
• Examples of protocol usage
• Definitions/schemas

Often these will seem redundant with each other but they're all important.

The model

The model is key for first-time readers and for people who need to know something shallow about the protocol. There are different kinds of models that are important for protocols, and some of them are described (and examples given) in one of Ekr's works-in-progress:

• The protocol messaging model. Do messages have headers and bodies, or do they have XML element containers? Does the server respond to messages in the same connection? In a fixed order? Can the server originate messages?
  
• The protocol state machine. Are there different states (e.g. pre-handshake, pre-authentication, and main state)?
• The protocol's data model. What data types are there and what relationship do they have to each other -- folders and messages and flags (IMAP), or collections, resources and properties (WebDAV)?
  
• The addressing model, which is almost part of the data model. In SIMPLE you can address other people whereas in XMPP you can address not only human actors but specific software instances running on behalf of those humans. And not to be speciesist, non-humans as well.

There's probably other kinds of models but I've seen examples where each of these could have been explained better. It took me a while to understand IMAP annotations because I didn't factor in the part of the model where each annotation might have several different values depending on the authentication id used to access the value.

The model is important not just for first-time readers and shallow users but also later on for deep users who want to extend the protocol. HTTP has been extended in many ways by people unfamiliar with the way the model is supposed to work. For example, HTTP normally uses the Content-Type to declare the type of the message body, just as one would expect from a concept borrowed from MIME and a messaging system. However, one extension to HTTP (now part of HTTP 1.1 or RFC2616) breaks that model by applying an encoding to the body and that encoding is specified in a different header. So if that feature is used the Content-Type no longer strictly works that way. RFC 3229 moves further away from the MIME-like model as it extends HTTP -- it defines an alternative model, where the Content-Type refers to the type of the resource that is addressed. So now of course there's a schism in the HTTP community about which is the best model to proceed with, to the point of having academic papers written about the alternative models. More clarity about the model in the first place would have helped not only first-time readers of the HTTP spec but also might have helped have fewer problems with these extensions.

Finally, a clear model helps implementors remember and understand each of the requirements. Humans have trouble fitting a bald list of requirements into some memorable pattern, so give implementors a mental model (or several) and they'll do so much faster, with less confusion and mistakes.

Requirements

The requirements are deeply important, as much so as the model. At the IETF we place so much importance on the wording of requirements that we have a whole standard describing the wording of requirements. Why?

First, models can be interpreted differently by different people. This can happen very easily. IMAPv4 was originally defined in RFC 1730 and there was a lot of text about the model, particularly the different states. However a lot of people implemented the details differently and RFC2060 had to get more specific. Finally, RFC 3501 revised RFC 2060, and most of the changes made in RFC3501 were simply clarifying what the consequences of the model were for various cases -- because implementors made different assumptions, came to different conclusions, and argued persistently about the validity of their incompatible conclusions. Chris Newman explained this to me today when the topic of models + requirements came up, and he should know -- he authored/edited RFC 3501.

Second, a model explains how things fit together, whereas requirements explain what an implementation must do. Implementors are human and operating under different pressures, so it is easy for implementors to read a lot of flexibility into the model and the examples. Clients want to believe that servers will do things similarly (makes their logic easier) so they tend to assume that is the case. So when things are flexible, they must be explained to be so, to encourage client implementors to account for differences. E.g. RFC 3501 says

"Server implementations are permitted to "hide" otherwise accessible mailboxes from the wildcard characters, by preventing certain characters or names from matching a wildcard in certain situations."

When things aren't flexible, the document needs to say so so that implementors aren't given any wiggle room or room for confusion. In RFC3501 we see

The STATUS command MUST NOT be used as a "check for new messages in the selected mailbox" operation

This text is much stronger than saying that the "STATUS command requests the status of the indicated mailbox" (that sentence is also in RFC3051). It's even stronger than saying that the STATUS command isn't intended as a way to check for new messages. (It might be even clearer to say that "client implementations MUST NOT use the STATUS command..." but this is good enough.) IETF standards-writers and implementors have learned painfully that they need to use well-defined terms in attention-getting ALL CAPS in order to get implementors not to misunderstand wilfully or accidentally, whether something is a requirement.

A few more reasons why requirements are needed:

• Requirements often add more detail than the model should hold. Since the model should be high-level and readably concise, it can't be expected to define all behaviors.
• Sometimes requirements are examples of the conclusions that somebody would draw if they fully understood the model and all its implications. These have to be complete, however, not only selected examples, because no two people have the same full understanding of the model and all its implications. The requirements help people go back to the model and understand it the same way.
• Human readers need repetition in order to understand things. Sometimes the requirements restate the model in a different form, and that's fine. When essay writers want their audience to understand they say what they're going to say, say it, then say what they said. We can make our standards more interoperable by balancing that approach with our typical engineering love of elegance through avoiding redundancy. Humans aren't computers, so the engineering avoidance of redundancy in code isn't fully applicable to human-readable text.

Examples

Examples are, thankfully, better understood. It's pretty rare to see a protocol go to RFC without a few good examples. Readers expect and demand them (more so than the model or requirements) because we know from reading many kinds of technical documents how useful examples are. I hope I don't need to justify this too much, in fact I find I need to do the opposite and remind people that examples do not replace requirements or models. Implementors need examples to understand the requirements and models but they can easily draw conclusions from examples that are counter to the requirements and don't fit in the model. When a specification has an inconsistency between a requirement and an example, trust most developers to implement to match the example, not the requirement.

Definitions/Schemas

Definitions and schemas also tend not to need much justification in a techie crowd. We're attracted by the idea of having absolute certainty about what's valid by trusting a program to compare an example to a definition or schema and validate it. So once again, I have a caveat to offer rather than a justification: make sure that definitions or schemas are put in context very carefully. Can an implementor use the schema to validate incoming XML and reject anything that doesn't match the schema? Probably not, or else it would be impossible to extend the protocol. Early WebDAV implementors built XML schema validators into their servers and rejected client requests that extended the protocol in minor ways that should have been compatible, so I'm taking this lesson from actual experience.

---

I certainly can't say that when I'm a protocol author, I succeed in doing all of this. But after eight years reviewing and implementing good and bad protocol specifications, I'm beginning to see what works.

Comments welcome.

Today I managed to explain (better than I've ever explained before) a few principles in the design of a network system. I use a client/server network system although you can generalize this to P2P easily. This is the diagram I drew on the whiteboard.



If it's hard to grok this completely abstractly, an IMAP client/server are good to mentally plug in. There are so many different IMAP clients and servers and they all have different APIs, storage models, and internal data models. By "data model" I mean data structures or object models, including caching and relationships between things. So if your Java code instantiates a MailMessage object which has a link to a EmailAddress instance for the 'From' field, that's all part of the internal model. The protocol's data model is similar: in IMAP there are folders and mail messages, mail messages have headers, one of which is the From header, and so on.

So I intended this diagram to convey a whole lot of stuff.

The protocol is the most inflexible part of this system. If you've got any interoperability at all with your protocol, even between unsynchronized releases of client and server, then your protocol is the most fixed element in the system. People constantly use new clients to talk to old servers, and old clients to talk to new servers, which means that even when new clients talk to new servers you're likely using an old protocol. Since your protocol is the hardest thing to change, both its syntax and its data model, it had better be extensible, basic and support many usage models.

The internal data model is the most flexible part of this system. APIs and protocols must continue to be supported across releases. Storage formats are easier to change than APIs but often require upgrade steps. Thus, the internal model is actually the easiest thing to change. Doesn't that mean that it's less important to get that right, because it can be tweaked and refactored to do new things or benefit from new knowledge? Yet many architects focus deeply on the internal model, spending much more time getting it right than the API or the protocol.

Client, server and protocol data models and content models diverge. Many architects design a networked system that starts with the same data model on the client and server and thus naturally they want the same data model expressed in the protocol. But these diverge naturally, sometimes even before Server 1.0 and Client 1.0 ship. For example the implementors of Server 1.0 discover that they need to cache certain strings together for scalability and subtly the data model on the server begins to change. Be aware from the beginning that this will happen. It's not a bad thing. It may even be a good thing to allow the server to be fast and the client to innovate on features.

Practice information hiding at the dotted lines. These are the places to really focus on modularization. Many software developers already understand that your API shouldn't be tied directly through to your storage model and this principle can easily be extended to the protocol modules. I've written bad code that tied too much to the protocol so I'm guilty of that one myself. It seems that unless there's a good reason, the protocol implementation shouldn't be tied directly to the storage model (the implementation should instead retain the freedom to change how things are stored without rewriting everything). It might not be so bad to tie the protocol to the API, i.e., by implementing the protocol logic using only the API. That way, any internal changes that leave the API unchanged, also leave the protocol unchanged. But that isn't always the best choice -- sometimes the protocol support needs access to internals and you don't want to complicate the API too much just to make the protocol fast.

Corollary: Use the best technology and architecture choice for each component independently. Because your client model will diverge from your protocol model and that one from the server model, data model consistency is not a good reason to use the exact same table structure or even the same database software on the client and server. (There may be other good reasons like expertise). Don't try to create the same indexes; the client and the server data access patterns will also diverge if they're even the same to begin with. Don't try to recreate the same caches. Send your server and client teams to different countries to work, maybe! That way the protocol becomes one of the most important ways they client and server teams communicate and they can make fewer hidden assumptions about how the code on the other side works (but they will make some anyway which will bite you in the ass).

Standard protocols and proprietary protocols aren't much different. If the protocol data model and client and server protocol naturally diverge, then even if your system starts out with highly similar models by implementing a proprietary protocol, that advantage erodes and becomes a disadvantage, hindering extensibility. OTOH if you start out implementing a standard protocol and enforcing good separation between the data models, this is a good long-term strategy. You know from the start that there will be translation between the data models -- every protocol message that comes in will have to result in objects or data structures being instantiated in the internal format, and every protocol message that goes out is a tranformation from internal objects or data structures. So that translation layer is solid from the beginning. Furthermore, if the system is using a proven protocol, the extensibility and performance features are likely to be better than one can easily design from scratch.

Protocol syntax isn't very important as long as it's extensible. Translating between models that are different is harder than translating between different syntaxes. It's like translating a business course from American into Chinese -- the language is the easy part, the culture and environment are so different that you can easily mean something you didn't intend to mean. So it's not the end of the world if the syntax is header fields or XML documents, as long as there's a clear way to extend either one. The extensibility is key so that as the clients and servers evolve they're not totally hamstrung by an inflexible protocol.

Whew. That's asking a lot of a l'il ol' whiteboard sketch. Comments welcome.

I have to say, I love a good Fisking, or to Canadianize that, a Frumming. On TCS, Radley Balko takes on David Frum's National Review column on obesity and taxes. It's a good read in its entirety, but I thought it would be fun to summarize anyway, to show how each argument was demolished.

Frum argues:

1. Canadians are less obese than Americans
2. Portion sizes are smaller in Canada than in US.
3. It's because Canadians are less wealthy that portion sizes are smaller.
4. Smaller portions lead to less obesity.
5. Obesity leads to health care costs.
   
6. Making sodas more expensive (by taxation) will cause lower consumption of sodas (conclusion: also reduce obesity, also reduce health care costs).

Without facts, one might follow that logic. Balko, however, demolishes one after another of these, showing how much of a house of cards that logic was:

1. Canadians are similarly obese to Americans and Frum's evidence was only anecdotal.
2. Portion sizes are similar and Frum's evidence was only anecdotal.
3. Since portion sizes aren't smaller in Canada, wealth isn't a factor in portion sizes (at least the wealth difference between CA/US doesn't matter to that). Also note that total consumption of caloric sodas has been steady for decades as Canadians have gotten significantly richer (and soda cheaper).
   
4. This one requires more data to completely demolish, but the evidence that total consumption of caloric sodas has been steady for decades does cast doubt on the idea that smaller cans of sodas will reduce consumption.
   
5. There's more evidence that poor fitness (sedentary lifestyles) has a much greater health care cost than obesity.
6. Such a small increase in price of soda is unlikely to change consumption, given that consumption has been steady for decades as soda production has gotten cheaper and people richer.
   

Balko doesn't point this out, but soda doesn't appear to be very price sensitive. Rather than buying cheap store-brand sodas the market overwhelmingly goes to the higher-priced image brands.

I've enjoyed working at OSAF for the past eight months and I'm starting to understand why. We're nice but we're not losers.

We shipped a minor release this month - the 0.4 release of Chandler. The release date of October 26 was picked months ago, as was a candidate set of workflows and other tasks to achieve. We met our release date, partly by making some minor cuts to the feature set (though we still reached the overall goal of having something experimentally usable or demoable), partly by working professionally towards common goals. Even better, we did it without panic, without yelling at each other, without having sales presell the release (heh heh -- no sales). We didn't make developers stay all night. We didn't have unpleasant meetings. We didn't demonize anybody for their bug counts. And we still managed to release on a schedule.

I don't want to get too deep into how we did it (one could write a book), but it had a lot to do with honesty and trust. Maybe when the stakes and egos are high it's too easy to fool oneself into believing ridiculous schedules. I'd like to think our transparency (it's all out there on the wiki helped us be honest with ourselves and communicate potential problems early.

It's nice to have confirmation that it's possible to reach high goals and still be sane.

My local security expert can be opinionated and frank, sometimes.

Me: "So have you ever tried getting IPSEC working?"

Him: "I'd rather have a prostate exam."

Making wedding stuff

I had a fun time making stuff for my friend's wedding a couple weeks ago. Now the pictures have come out.

I made silk shawls for the bridesmaids. Lessons learned:

• I couldn't iron a double-folded-over edging into silk organza. Pin a lot.


• Sewing raw silk organza edges into a folded border is hard. I bought black silk thread so the seam wouldn't show much, and then held the organza very taut, front to back, as I fed the pinned edge through the machine. Still, the tension in the seam pulled the organza together a bit, until I ironed the *&$! out of it.


• Use a ton of water to iron silk charmeuse. But do iron, because it's worth it to get the piping straight.


• Silk must be carefully pinned, particularly when you're sewing 7 layers together. Even so, I caught excess organza in the seam a couple times.


• Use the right sewing needle to go through those 7 layers. When I used the needle I normally use for quilting, it made a scary "thunk" sound.

Sorry, no pics of the finished shawls at this point. They ended up black silk organza bodies, with charcoal charmeuse endings (folded over the raw organza edges) and purple charmeuse piping.

I was hanging around doing nothing on the wedding day and suddenly (but willingly) pressed into service making the ball for the flower girl to carry. The materials: a styrofoam ball, ribbon, and green "toothpicks" with wire already wrapped around the non-pointy end. The idea, as I saw it (but I Am Not A Florist) was to wrap the free end of the wire around a flower stem, then put the pointy end of the toothpick into the ball. Repeat until covered. Sounds simple, right? Lessons learned:

• Plan ahead. Will you be putting leaves on the ball along with the flowers? Put them on first, dummy, or they cover up the flowers.


• Pick flowers with strong stems. Flowers with weak structure will be soooo frustrating.


• Wrap several flowers/leaves together onto the same toothpick. You probably don't have enough toothpicks, plus it goes faster.


• Don't try to push the toothpicks in too far. Otherwise you push the wire or flower right off the toothpick just as it gets buried in styrofoam.


• Attach the ribbon handle as strongly as you can -- it's going to be swung around by a 2.5 year old (cute Ava). Very firmly attach the ribbon to two toothpicks and stick them in at different angles so the tension doesn't pull them straight out.


• Don't fret, there is no way to hold it without crushing some blooms or tearing off some petals now and then. After failed attempts at wrangling it with wire handles which backfired and tore off flowers, I just used the fingertips of one hand to support it.


• Finally, don't worry. Everybody will love it anyway.

Actually, it turned out quite pretty. It's by no means the prettiest thing in this photo, but it worked. Many people helped out a lot and it was a beautiful wedding with a sense of cooperation. For example -- credits to Cheng for the photos I used here.

Joe and St. Peter wrote a WebDAV-related IETF Internet-Draft recently. In fact, it combines two technologies I admire, WebDAV and XMPP, in a way that uses each technology precisely for what it's good for (WebDAV to store application data, XMPP to route application-specific messages).

I talked to a bunch of people at Educause about this yesterday, and they were excited about this and other new WebDAV features because many universities are betting on WebDAV. University of Memphis reports two services that people can't live without: email, and now WebDAV. Since WebDAV is such a flexible repository technology it's hard to tell a potential customer (like Memphis 2.5 years ago, when I worked at Xythos) what it will give them. But give it to users, and they will figure it out. Looks like the Atompub people are storing blog postings on a Web server, which students increasingly can do now via their WebDAV accounts [1]. And of course the universities are also keen on storing calendar data in WebDAV accounts. More application data in fewer server repositories means lower administration costs for these universities.

With all this increased excitement, you'd think I could get more people to show up at WebDAV Working Group meetings or contribute on the mailing list. The next meeting is in Washington DC, in the second week of November. Want to join in? It's easy -- all you have to do is... join in.

---

[1] I realize that although giving students WebDAV accounts is a little new, giving them Web accounts, or Web space in their Unix accounts, isn't new. However, in some colleges students had to request this service, or know enough Unix to be able to author the content. Plus raw HTTP doesn't support multi-user applications very well. WebDAV extends the usefulness and usability of personal Web space.

This is why I like cats.

I was trying to figure out if anybody was thinking of implementing CalDAV in Slide, so naturally I googled 'caldav slide'. No direct answers, but I did notice a LaughingMeme blog post from last spring that I'm sorry I missed, commenting on CalDAV. I agree with the gist of his caveats, and we're trying to address those, I think.

I can't vouch for the accuracy of this description of Canadian conversational styles (link via Julie), but it rings a bell -- whether because I'm Canadian, female or both, I don't know. Surprisingly, the "weaving" style can sometimes be seen as more aggressive, because it involves pushing conversational pieces into pauses. My American SO sometimes thinks I'm interrupting him. Well, I am interrupting if you assume there are rules about waiting until somebody is done speechifying even in one-on-one conversation, but the interjections are meant to build the conversation, not to be rude to him.

Recipe for Ginger ice cream -- sugar and no-sugar options

Ingredients:

• 1 cup half and half cream
• 1/4 cup grated fresh ginger root
• 1 cup heavy whipping cream, whipped to soft peaks
• 1 cup ginger ale or diet ginger ale
• 1/2 to 3/4 cup sugar or Splenda
• 1 T. lemon juice

Soak the grated ginger root in half and half cream. Put this and the ginger ale and whipping cream in the fridge to chill for an hour or so. Use a fine sieve to take the fresh ginger out of the cream. Stir the ginger-flavored cream together with the whipped cream, ginger ale, sugar/Splenda and lemon juice. Immediately put in your ice cream maker etc..

Mmm! I imagine for those who love ginger and can also eat sugar, candied ginger would be a nice addition to the recipe.

How thrilling! After only two years, the XMPP WG has produced RFCs.

• RFC3920, Core protocol (framework for applications like notifications, as well as instant messaging and presence)


• RFC3921, Instant messaging and presence over the core protocol


• RFC3922, how to map XMPP presence information and instant messaging to the same common interchange format that SIMPLE instant messaging can map to


• RFC3923, requirements for end-to-end security of instant messages even when routed over more than one protocol

The work began earlier than two years ago, of course. Jabber was designed while Jeremy Miller was waiting for the IETF to come up with a standard IM protocol (and taking too long). When the IETF process still appeared stalled, Jeremy and others proposed that the IETF could standardize Jabber, by creating a working group to design a revision to Jabber that met the IETF standards for internationalization, security, XML namespace usage and a few other things. A BOF meeting was held in July 2002, very eventful and entertaining as IETF meetings go. The WG was formed, and I was added as a co-chair to help Pete Resnick, on October 31 2002. Peter Saint-Andre authored *all* of our internet drafts, doing a huge amount of difficult work in an extremely timely, professional and skilled manner. Whew! It's been fun!

Via Who Throws a Shoe, I learn that Mel Brooks has called Toronto's bagels "mushy". I can comment on this vital issue. Although I've never lived quite in Toronto, I've lived nearby and visit frequently. Also I've sampled Montreal bagels, New York bagels and many instances of second class bagels from around the continent.

You see, only Montreal and New York bagels are really the original ethnic bagel, made by Jewish people for Jewish communities with the same unadulterated ingredients (slightly different for these two cities, I believe) and quirky bagel-making techniques (including boiling the rings before baking in hot ovens, and who knows what other black magic). That may be because Montreal and New York have long had large communities dominated by Jewish culture, and the ability to affect bagel tastes throughout the rest of the city.

In the rest of Canada/US, bagels are poor imitations made with standard baking equipment and therefore with modified recipes. Donut stores that decide to add bagels to their menu often create pseudo-bagels that can be made with their existing equipment. The result is topologically the same as a bagel but otherwise sub-par -- a mainstream "bagel" is merely a bun with a hole. Even steaming equipment produces a milquetoast substitute. It's just not the same, but most North Americans don't know the difference or don't care. Perhaps some soft-gummed softheads even prefer the less chewy mainstream bagel.

So what's Toronto's story? It's mixed. Some parts of Toronto are like the rest of the wasteland outside Montreal and New York, as epitomized by the ever-present donut stores which make such great Canadian donuts (more on this another time) but such humdrum purported bagels. However, there is a strong Jewish community in some parts of Toronto (such as North York). So in these communities, if you know where to shop, you can find real boiled-and-baked bagels, real chewy and dense and shiny on the outside.

Mel, I'm sympathetic, but you should know better than to buy a bagel just anywhere.

Another stab (like mine last spring) at why there seems to be a disproportionate fear for economics and economists: Arnold Kling puts forth a few possible reasons in his TCS article. He mentions the math barrier, but I still think that disliking the conclusions of economic thinking is one of the biggest reasons -- and can't be dealt with simply by teaching economics without math.

Another knitting project completed: a fuzzy baby cardigan.



Also see Ekr's post describing our weekend camping trip which was our toughest trip yet but I feel surprisingly good only a day later.

I can't resist exposing unintended consequences of well-meaning social programs. Here's a new one (to me) related to the minimum wage: by having a minimum wage, a state blocks entry to the workforce to certain very low-wage and entry-level workers, and this affects the stepping-stone process by which low-wage workers move up the chain to earn higher wages. In other words, a minimum wage hurts a state's rate of moderate wage employment later on. A great explanation at Marginal Revolution.

Open source and open standards

This is part two of a rumination on the trendy adjective "open", exploring why open source and open standards don't go together as well as one might think.

The phrase "open standard" is more loosely defined than "open source", although Bruce Perens is attempting another definition. Microsoft calls a protocol an open standard if the specification is publicly readable or if Microsoft allows or licenses others to legally implement it. However, Microsoft typically retains change control to its so-called open standards. Sun also maintains change control over its major standards and may not even have non-discriminatory licensing for implementations, but claims their standards are more open than Microsoft's because you can "check out" of the standard (ref). The war between Microsoft and Sun over who has more open standards is laughable.

I think these are the main criteria to judge how open a standard is:

• Is published and available at low or zero cost
• Can be implemented by anybody freely or with non-discriminatory licensing
• Can be modified by a group with either completely open membership (IETF) or one that does not discriminate in membership applications (W3C).
• Can be implemented interoperably on a wide variety of platforms and with a wide variety of technologies. (E.g. HTTP can be implemented in C, Java or most other languages, on Windows, Unix or just about any modern operating system.)
  

Anyway, for now let's call standards whose development is open to public contributions "public standards".

Why would an open source effort fail to support public standards, or fail to be interoperable? A lot of possible reasons here, many of which add up to either a high cost which open source developers can find difficult to bear, or lower benefit than a commercial product would gain from doing standards work.

• It can be expensive to learn how to implement the standard. Books, workshops and seminars can help the developer learn the standard, or participation in a standards group which may require airfare or meeting fees to do effectively. Writing a from-scratch implementation of a public standard is not usually easy.


• Developers may take shortcuts reading, understanding and implementing standards, trying to get something useful working fast rather than make the full investment. There may be certain features required by the standard but which aren't necessary for the product to work in the developer's main use cases. The shortcuts lead to less-interoperable implementations. Closed-source projects may take shortcuts too, but there are pressures to fix the holes quickly in order to claim compatibility, prove interoperability, and sell to customers.


• Vanity, confidence, or ego: it's fun and impressive to design a protocol from scratch. An open source developer who is doing the whole project for fun may find the entertainment value more important than a paid developer.


• Commercial products must have the credibility of being supported for the long run, and in planning for the long run, developers have other tradeoffs to make. For example, a protocol designed in-house for a specific purpose tends to have design errors which make it harder to maintain in the long run. There are a lot of subtle design points and extra effort involved in making a protocol extensible and flexible. Later on, continuing to support the protocol with newer versions of the software may come to be quite a pain. If they're wise and can afford to do so, developers of long-lived software make the decision to invest early in supporting a standard protocol rather than invent their own with unknown problems.


• What if the developer can put in the effort to implement the public standard, but it's not quite ideal for the developer's purposes? It's possible for the developer to influence the public standard, but contributing to standards takes a long time and a lot of effort. If the existing standards don't quite match up to the requirements, an open source developer may not have the resources or time to help it change. Thus, the developer may choose to only partly implement the standard, or implement undocumented extensions, at a detriment to interoperability.


• There's the assumption in open source that because you can read the source, the source is the specification. Why should an open source developer take the time to publish a specification for the protocol when you can just read the source to see how it works? So therefore when open source developers do extend or invent a protocol, the resulting protocol often remains undocumented, which isn't conducive to interoperability.


• Interoperability testing can be expensive. It might require revenues of some kind to be able to afford interoperability testing. And if you can't even afford interoperability testing, it's harder to justify implementing the interoperable standard.
  

Although it seems a little rude because Subethaedit is a wonderful tool, I'll pick on Subethaedit for a minute. It's free, it's open source, it has a relatively open process, and it even supports some public standards. It uses Rendezvous (an Apple preview of Zeroconf, which is a public standard in development) and BEEP (an IETF public standard). However there is also a non-public protocol used over BEEP to communicate around the actual changes to the communally edited document. Thus, it would be a challenge for somebody to write another client that interoperated with Subethaedit.

Sadly, many open source projects (as well as many closed source) use the phrase "supports open standards" or "based on open standards" as if it were pixie dust, automatically conferring open interoperability. That's not the case. An open source project can easily fail to be interoperable with other projects, just by innovating new features without rigorously documenting and designing protocols and protocol extensions.

Some open source projects overcome all these hurdles and support public standards, which I, personally, am very grateful for. Once in a while open source developers actually design a new protocol which is interoperable, flexible and unique enough to become a widely supported public standard, and that too deserves kudos.

Open as in source, or Open as in process?

A long-standing terminology issue in free software is whether it's "free as in speech, or free as in beer" (ref). A related confusion arises with the adjective "open". The term "Open Source" is most authoritatively defined by the Open Source Initiative. Other phrases like Open Standard and openness (as a way of operating an organization) are more loosely used. With efforts like Open Content and Open Law (not to mention OpenCola), openness is clearly in vogue now.

These "opens" don't always go together, despite common assumptions. There's an automatic assumption that these are all meritorious and that anybody with the merit of doing open source software also has the merits of following open standards and an open process.

Open source and open process

Since "openness" is too vague, I'll take a stab instead at considering "open process". The phrase is relative: a more open process is one which is more transparent, less restricted in participation, and less obscure. For example anybody may search the Mozilla bug database and see bug status (transparent) and anybody may submit a bug (unrestricted participation). In addition the bug database is easy to find and there are instructions for use (not obscure).

Why would an open source effort fail to have an open process? Simple -- a process has a cost that must offset its benefits, and an open process can have a higher cost than a closed process. A small group of programmers (one to three, maybe four) don't get much benefit from formal processes, let alone formally open processes. The way to contribute to an open source project, or how submissions get accepted, or what feature get added, may happen in a very opaque way; decisions may be made by one person or made in private conversations among a small group.

Contrast that to a company where there are very likely to be customer requests and some standard process to handle them. The process may very likely be closed and private, but it's also quite possible for a company to have an open process to discuss feature inclusion. For example, at Xythos, we started a customer forum where customers could get together and discuss the merits of various feature requests. The final decision was up to us but the process was not entirely closed. Some open source projects don't even have a forum for discussing feature requests, or a public list of feature requests.

Of course, open source efforts are probably more likely overall to have open processes. Sites like Sourceforge contribute to this synergy by opening up bug tracking and discussion forums in a cheap way -- open source contributors don't have to make a huge investment to make the development process more open as well.

At OSAF we put a lot of effort into ensuring that our processes are transparent, participatory, and not obscure. It's been estimated that one part of that alone, the weekly IRC chats, consumes nearly one morning per developer per week -- that's enormous. Documenting everything we say in in-person meetings and keeping public Wiki pages up to date are other major costs. Obviously we hope that doing the "openness" thing confers benefits in the long run to the larger community, but at such a high cost, it's obviously not a cost every open source group can bear.

Part II of this post, discussing the weaker synergy between open source and open standards, will come this weekend I hope.

The CalSch Working Group is being closed. I believe this is a good thing -- IETF working groups which stick around for years and years tend to shrink and get dysfunctional. Closing the group clears the slate.

This doesn't mean that no standards work is going to get done in calendaring. In fact, it seems quite the opposite since in the past six months there's been quite a surge of interest in calendaring features and calendar interoperability. The new lists I mentioned in a previous post have seen a fair bit of traffic. The access protocol discussion list (ietf-caldav@osafoundation.org )has 96 members and 72 posts in under a month. The other list discussing bringing the calendar format standards to a new level of RFC (ietf-calsify@osafoundation.org), has 85 members and 154 posts in the same time period. I've talked to a number of different companies, organizations and even venture capitalists about calendaring standards action, and there's an interesting roundtable coming up in Montreal.

Since I have been dragging people into discussing calendaring standards for a year now, and published a calendaring standard proposal eight months ago, I feel like my timing, for once, is right. Maybe I'm one of the butterflies furiously flapping my wings to create a bit of wind-speed out there today.

I read Scott Rosenberg's post today about extraordinary attempts to avoid appearance of journalistic bias, then got to discuss it with him today too. I noted that the problem seems to be a bit worse in the US. In Canada and the UK, I think newspapers may be a little happier to be known to have a bias. Not the Globe and Mail, perhaps, which takes a high-horse approach to politics and society despite clearly having biases at times, but at least the Sun. And in the UK I've heard papers are more likely to take sides and I know the Economist sometimes bluntly admits its bias. In the US, it seems, papers go to greater lengths (including forbidding attendance at benefit concerts) to avoid being seen as a mouthpiece for either the Democrats or the Replublicans.

One theory for the extra attempts to appear neutral: since there's only two real US parties, any admitted bias is tantamount to admitting that the paper favours one of them. In Canada politics may not be quite so partisan or polarized, or at least haven't been for so long. A Canadian paper with a slight conservative bent could be a PC mouthpiece, or a Reform mouthpiece, or neither, so at least there's confusion about which party it's supposed to be the mouthpiece for. But I'm not sure this theory holds up because certainly Canadian politics can be partisan (and nasty), and UK politics may be polarized much like American.

Another theory is that since Americans are generally so quick to come up with conspiracy theories, Americans are therefore quick to assume that politicians are somehow controlling journalistic output. Therefore a paper must appear especially untainted to avoid being written off as government-controlled propaganda.

I'm not terribly happy with either of these theories, and maybe I'm wrong that this is uniquely American, so feel free to chime in.

Wikis suck for specs

At OSAF, unfortunately, we don't have very good specs. We have great designs, but it's really hard to go from the designs to know what exactly works and doesn't work, what's implemented and what isn't, and how the non-GUI choices were made, at any given release. It's hard for a developer to pick a feature and see how to implement it, because the developer could either see too much (the wiki pages or fragments that show the bigger context for the design, the aspects of the feature that we can't do today) or too little (the wiki pages or fragments that seemed irrelevant but were vital). Part of the problem may be that we didn't have anybody on staff whose job it was to write specifications, like program managers at Microsoft do. But part of the problem is certainly tools.

OSAF uses a Wiki for most of our current documentation, from proposals to user documentation to meeting notes to personal jottings. The wiki is very fluid and chaotic and not very authoritative. It seems a wiki can be made to be more authoritative -- the wikipedia does this by formalizing around a single kind of content -- but we can't easily do that because our wiki already contains such divergent content. Besides, a wiki isn't an ideal environment for long documents, for printing, for version control, or for template usage.

Length: Specs are ideally longer than the average wiki page. To fully describe a feature so that the different contributors actually agree (rather than just appear to agree), to document the decisions made, and to make the feature testable and documentable, might require a 30 page specification for a 1-dev-month feature. Note that for a long document readers require a table of contents -- ideally clickable if you're reading it on a computer, with page numbering if it's printed.

Printing: You really want to be able to print specs, even today. I'm tempted to bring printed specs to a meeting and ban laptops to get them reviewed properly. Reviewers can mark up paper. Implementors can glance between paper and screen (and write on the paper) as they work.

Version control: Specs should be versioned and checked in and out and maintained in a versioned repository, just like code. Some wiki software does this but it's not very good yet -- I couldn't view a version of this page from before the picture was added. HTML doesn't version very well in general partly because images are external. And if you break wiki pages into smaller-than-30-page chunks for readability, then the versioning gets worse.

Templates: For template support, I miss Word. When I worked at Microsoft, there were Word wizards who made beautiful, useful templates for many purposes, from expense reports to specifications. As a program manager, I wrote many specifications, and I found that a template somewhat like this one kick-started the process. Word also allows templates to contain "click to fill in" fields (like author, product, release, team information; spec status, implementation status) including support for drop-down boxes to limit selection to a few appropriate choices. My favourite spec template had sections for overview, administrative UI, setup concerns, test considerations, rough documentation plan, and internationalization issues. Each of these sections was a reminder that the feature being specified might have impact on other groups. When I joined a group that didn't have a template I made or modified one to start with because I found them so useful. The IETF has specification templates and they make for some consistency between vastly different protocols and authors. A familiar template vastly improves readability.

Am I missing something that would solve this problem in HTML or in another open format? Is there some easy-to-use tool or set of tools to solve this problem? I know I could write my own software to do this, and it wouldn't even have to be terribly complex -- no worse than the XML2RFC tool which produces IETF specifications matching the template. But that's harder both to write the template (and software to generate the document) and for the author who has to get the angly brackets right.

[Tangentially related: According to Mark's amusing post, as a spec reader, I'm a moron. I don't understand a spec merely by reading it through. After I've implemented a spec or tried to explain it to somebody else, or extend it or edit it, I begin to understand it.]

[UPDATE: Ekr and Sheila point out that a Wiki also sucks for entering text. It's not WYSIWYG, so the process of updating and viewing and correcting pages can be slow. Also you've got to be online to change a page. There's no way to synchronize parts of the Wiki for offline changes other than manual painful silliness. Ekr objected to the whole comparison in the first place. He said that evaluating a Wiki for writing specs was like evaluating a bunsen burner for baking bread.]

When I lived in Seattle I volunteered with a group that ran conferences for middle and high-school girls to learn about careers. The conferences were arranged so that each girl could pick a slate of careers (3 or 4 ) and go to workshops where there was supposed to be hands-on practice. For a lot of careers -- vet, doctor -- it's easy to bring in actual real tools and let the girls use them, so those make for easy workshops. It's not so easy for the software industry, however, as I learned when I tried to increase the participation from women at my company.

The obvious approach to teaching girls what it might be like to be a developer is to put them at a computer and show them code. Unfortunately that requires having access to a computer lab at the conference location and setting up all those computers with the same tools. I only saw this done once -- Shaula Massena ran a brilliant workshop where girls used Visual Basic to construct a little prank program they could install on Dad or Mom's PC at home. The program simly popped up a dialog saying click here to dismiss, only when you click "here" the dialog jumped somewhere else on the screen :) Shaula sent the girls home with floppies containing their program executable, very cool.

I eventually helped program managers, testers and tech writers come up with workshop plans that didn't require a computer lab and software installation. For testers, we brought some home appliances in -- mixer, hair dryer, toaster -- explained how to write a test plan, and asked the girls to execute the test plan. They explored edge cases like "what happens when you press the toast button when you're already toasting". The girls also always had fun criticizing the design and usability of the devices, which we encouraged.

For tech writers, the workshop involve writing instructions and then watch somebody follow those instructions to see how challenging it is to write clear instructions. I brought a pile of coloured paper and asked each girl to make a paper airplane (or other origami thing) and then on another piece of paper explain how to make the same airplane. Then the girls exchanged instructions and tried to follow somebody else's instructions. At the end we compared results between the original and the one made by following instructions. Here, fun was had throwing planes and decorating and naming them as well as constructing them. Some girls decorated their instructions pages too -- budding Web designers.

Finally, for program/product managers, my favourite workshop was "Design your own cell phone". I focused the discussion of features by introducing the concept of a target audience and a limited budget. What features are so important for your target audience that you just have to have them? The girls split into teams and came up with lovely ideas. Often, of course, the target audience was "teenage girls" and one group came up with a "privacy phone" that would stop your bratty brother from hearing any of your conversations or messages. But one group targetted old people and thought what it would take to handle a user with poor eyesight and hearing. And another group targeted young kids (or really, concerned parents of young kids) and designed a brightly-coloured egg-shaped phone, wearable on a lanyard around the neck, with only three programmed "send call" buttons so that the kid could only call appropriate authority figures to arrange pick up time, report an emergency, and so on. The girls thought that the phone should also have toy-like features so that the kid would have something to play with besides calling Mom, Dad and Emergency, so they thought there could be a couple more buttons to play ringtones or record and playback thoughts.

For six years I've thought that the kidphone would in fact make a cool product. Finally I find validation via Gizmodo: the MYMO phone for kids. I should have hired that team of girls and founded a company.

Everybody is knitting ponchos this summer, so I had to as well.

If this background material is familiar to you, scroll to the bottom for the call to arms.

I've been following calendaring standards for six years now. In that time, iCalendar has become a fairly widely accepted standard for expressing events, todos and other calendar information in a MIME text document. Many calendaring applications can import iCalendar files, or export to them, or generate an email with an invitation formatted as an iCalendar document. However, there are some problems with iCalendar's complexity, particularly in expressing recurrances, and the companion standards for conveying iCalendar files (iMIP and iTIP) have their own problems.

Calendar interoperability testing has happened sporadically in these years. The first calendar interoperability event was held in 2000, a virtual event in 2002 and another in-person event this summer at Berkeley, organized by the newly revitalized CalConnect consortium. Still, interoperability hasn't improved as much as we'd like because at some point you need to go back and fix the standard.

Also during these six years, the CalSch working group has worked and bickered over the proposal to standardize how clients access a calendar stored on a server -- this protocol, CAP, would be to calendars what IMAP is to email folders. I've never liked the design of CAP, down to the very core model of how it works. So six months ago I made a counter-proposal, CalDAV, and threw it out there as an internet-draft. Finally I'm getting more than just nibbles on CalDAV, in fact several groups have told me their intent to implement CalDAV starting this year. And at the same time, other folks are getting invigorated about revising iCalendar and bringing it to draft standard.

This is all great news. Calendaring interoperability has languished except for that burst of productivity back in 1998. People are locked into one calendar application depending on what server technology they have available, since there's no common calendar access standard. Invitations work, kinda, but in practice the problems with recurrances mean that people must follow up the machine-readable text with human-readable text in case a mistake was made between two different vendors' software.

Good news, but nowhere near done yet -- this is just the beginning. Now we need volunteers. We need people to write specification text, review text, and manage the issues. We need people simply to pay attention to the work being done and provide their experience, or simply opinions, to bring issues to resolution.

Here's where to start -- two new mailing lists, one to take iCalendar to draft standard, one to develop and standardize CalDAV. Tell your friends! Subscribe to one, subscribe to both! We've got good work to do and good people to do it but we need your help.

Tim Bray, a veteran of many other Internet/engineering communities, is a newcomer to the IETF, and was slightly disturbed by some of what he saw. With open mind, he attended IETF meetings, both a few official in-person meetings and the meeting Saint Peter calls the "undernet" of the plenary, the badattitude jabber room (named after a tradition of corporate criticism).

Like Saint Peter, I'm not concerned about what Tim Bray calls the "severe angst" that appeared in badattitude. I think when you invite a group of people together to complain and rant, they do so, attempting to be humorous and entertain each other, at the expense of reflecting the entire reality and balance of their opinions. In fact, several of us have noticed that ever since badattitude started existing during IETF plenaries, there's much fewer dissent voiced at the microphone. Badattitude only has about 60 people and there can be 600 in the actual plenary room so it's hard to believe that allowing 10% of the people to let off steam can directly reduce the microphone usage by a noticeable amount (clearly more than 10%). Theories include:

• Uncorrelated -- people stopped complaining during open-microphone for other reasons.
• Tipping point -- the few people whose frustration was reduced through badattitude, and didn't go to the microphone, brought microphone usage below some tipping point, influencing others not to complain as well.
• The Squawker Squad -- the possibility that the people I know happen to be those most likely to complain, and by drawing them into badattitude I induced a significant percentage of the bellyachers to give their opinions elsewhere.

The first theory is the most likely but I prefer the last (/me grins).

I was supposed to have a full transcript of the badattitude jabber room due to the automatic logging functionality of the jabber client, but nitro ate the chat log. Really -- it's gone, except for the first five minutes. Honest. Anyway, some folks logged in under their real names and might not like it published.

WebDAV and Wiki get compared frequently. They're often in frameworks that solve the same problem, but they play different roles within those frameworks.

A Wiki is a Web site running Web application code that allows people to browse Web pages, easily add new Web pages, and easily edit any Web page through Web forms (the Web application code powers those Web forms and handles their input). Originally Wiki described the first such application but the word has spread and similar applications that run completely different code are also called Wikis.

WebDAV is an IETF Proposed Standard, RFC2518, describing a set of extensions to the HTTP protocol. These extensions allow a client application (or other software agent) to create new HTTP resources more easily, manage them, find them without following hyperlinks, and change them in a way that multiple authors can work together simultaneously.

Wiki uses HTTP forms to transport requests	WebDAV extends HTTP by defining new methods.
Wiki is a UI.	WebDAV is completely orthogonal to a UI.
There exist a number of different Wikis, implemented in different languages, supporting different features.	There are a lot of different WebDAV servers as well as WebDAV clients. The clients can theoretically all work against any server although in practice there are a few outliers (special purpose clients or servers) that don't interoperate so well with the main group.
Wiki is a thin-client application, where any browser can be the client because the smarts are on the server (the code running the Web forms).	WebDAV was created to allow rich clients to author Web resources. A WebDAV client is definitely the entity in control -- but you have to download or find WebDAV client software. Luckily, these clients ship now in all Windows and Mac OS's.
Wiki is a generic name that seems to be used if something is similar to original Wiki	WebDAV is a set of requirements, so a WebDAV implementation can be tested for compliance.
You'd probably not call something a Wiki, if it didn't enable multiple users and allow easy creation of new pages, and result in a Web site collection of human readable pages.	WebDAV can be used to do synchronize with a source code repository, or to change configuration files, or to access a calendar server, or many more things that you would never call "shared authoring".
It's easy to extend (a) Wiki, just code in the new feature -- e.g. search.	It's hard to extend the WebDAV standard, it requires IETF drafts, reviews, and many revisions in the IETF doc format. It can take years -- e.g. SEARCH, begun around 1998. OTOH it's easy to extend a WebDAV server -- e.g. Xythos added anonymous permission tickets to the WFS server -- but no other clients are likely to interoperate with a custom feature like this for a while.

Anyway, in this case, the different roles mean that Wiki and WebDAV are not at all incompatible, and to compare their advantages and disadvantages as if they are both apples can be misleading. Brad, whom I met at a party last week, realizes this: he added WebDAV support to an existing Wiki codebase. It allows another way to add pages to a Wiki or to modify them, and can interact completely smoothly with the existing WebUI. You'd think somebody would have done this before now, but I can't see that anybody has. Way to go Brad.

[UPDATE Aug 9: clarified "extend webdav" to differentiate between extending the standard, and extending a server]