Another Outlook connector link related to yesterday's post: caldavxp is Mark Slater's project to develop a CalDAV plugin (technically, a message store provider) for Outlook. I learned today some of the many, many ways one can extend Outlook, it gets pretty complicated I take it.

At next week's CalConnect meeting in Utah (brrrr... but thanks Novell for hosting), I suspect one of the topics will be whether it's possible to write some kind of plugin for Outlook so that it can talk to calendar servers other than Exchange. Naturally one obvious approach is to write a generic CalDAV plugin for Outlook, and this kind of thing would make it much easier for many organizations to migrate to a standards-based approach.

The technical barriers are daunting. Outlook doesn't seem to have been designed for this kind of plugin at all. Such a plugin might have to use undocumented APIs even to work at all, and then naturally have lots of code to handle changes in those APIs from release to release. Some embryonic work is in progress at OpenConnector if anybody wants to jump right in, or I can put people in touch with each other so email me. At this point, amongst the group of people I've been talking to, we're not even convinced we have a decent approach outlined (so input from MAPI experts is particularly welcome) let alone how much work it would be. But we're starting to talk.

We're getting very, very close with CalDAV. Draft version 09 is submitted and I'm sure it will be posted soon. There's not a lot of open issues remaining (one is how to use ETags, which is the topic of lively discussions in both the CalDAV and the WebDAV mailing lists). Interoperability is pretty good already -- I got a great reception at ApacheCon when I demo'ed Chandler publishing a new event to a calendar shared on Cosmo, and then used Sunbird to refresh the shared calendar and see the new event (note that Chandler 0.6 was just released so you can do this too). We'll have one more interoperability event in January before likely finishing the draft and submitting it. We plan to do pseudo-last-calls in WebDAV and CALSIFY WGs but if you've been waiting to read the draft, it's great to get comments even before last-calls.

Tom Evslin posts with several reasons why Web sites don't provide APIs -- and yet predicts that many more will provide APIs in the future. I can add one more reason, and that's the risk that with an API, somebody would build an application that presented the data through an alternative UI, and the site would lose eyeballs. For example, if Yahoo Group calendars could be sucked down in iCalendar format, people would probably visit the site less often and receive fewer ad impressions.

Still, I agree with Tom that despite all these forces against opening up APIs,
there are even stronger forces for having APIs -- competitive advantage. Some company hoping to compete at lower cost will provide the API and try to make up the revenue in other ways or simply survive with less ad revenue. If the service is more valuable with the API people will move to that service. I hope that in a year or few, people won't stand for a calendar Web site that doesn't let them use a standard API to have direct access to their own calendar data.

Favorite Songs of Stalkers

6. "I Will Follow You", by Ricky Nelson

"I will follow you
Follow you wherever you may go
There isn't an ocean too deep
A mountain so high it can keep me away"

5. "I will Follow", by U2

If you walkaway, walkaway
I walkaway, walkaway...I will follow

4. "The Power of Love", by Air Supply

Even though there may be times
It seems I'm far away
Never wonder where I am
'Cause I am always by your side

3. "I'll Drive All night" by Celine Dion

So just remember -
I'm gonna make you mine
I'm gonna drive all night
till the morning light
I'm gonna roll till dawn
with the windows down
and the radio on
You know I'd drive all night
just to hold you tight

2. "Nothing Can Keep Me From You" by Kiss

Wherever you are, that's where I'm gonna be
No matter how far, you'll never be that far from me
Some how I would find you, move heaven and earth to be by your side
Oh, I'd walk, this world to walk, beside you

1. "I'll Be Watching You" by Sting

Every breath you take
And every move you make
Every bond you break
Every step you take
I'll be watching you

0. Christmas Bonus Stalker Song:

He sees you when you're sleeping
He knows when you're awake
He knows if you've been bad or good
So be good for goodness sake!
O! You better watch out!
You better not cry.
Better not pout, I'm telling you why.
Santa Claus is coming to town.

I posted to a couple IETF lists about yubnub.org, a nice productivity hack -- a command-line for the Web. Since I'm always looking up IETF WG charters, internet-drafts and RFCs, I found the existing 'rfc' command and added two more.

To jump to an RFC: rfc xxxx

e.g. 'rfc 2822' (don't forget space)

Internet-Draft Database Search: (shows list with filename substring match): ids keyword

e.g. 'ids dusseault' to find draft-dusseault-caldav-08, but not draft-ietf-webdav-rfc2518bis because 'dusseault' isn't in the title of that one

To jump to a WG charter: wg wgname

e.g. 'wg imapext'

Now Dan Gurney has extended the 'rfc' command so you can use text as well as numbers. You don't have to remember that iTIP is RFC2447, just type 'rfc itip' in the yubnub.org interface, and see results. Thanks Dan!

BTW I personally setup yubnub to work on the address line or search box in Firefox so I don't even go to yubnub before typing my command. One less step on the way to what I need!

For those who follow my crafty pursuits, I've just posted pictures in the gallery of the mini-quilt I finished this weekend. Well it's more of a wall hanging made with quilting (no piecing).

Also if you saw the Butterfly Quilt for Ally, posted earlier on the gallery, well everybody loves it and wants to be my niece too. I don't have an opinion from my real niece (she's only one year old, after all, and extraordinarily easily bored) but her parents love the quilt. Whoo hoo!

It's been a busy few weeks, which is why I'm only now posting about another IETF meeting although it happened Nov 7. The meeting was the XML-PATCH-OPS BOF, and here are the official minutes.

The SIMPLE WG has been working on some HTTP extensions and using XML in order to allow instant messaging clients to interoperably edit buddy lists (stored on the IM server) and other configuration data. Special functionality to modify/retrieve XML stored on an HTTP server is rampant these days, so it seemed like a good idea to consider general mechanisms, rather than only design mechanisms limited to SIMPLE use cases. So Jari Urpalainen has been working on a general XML diff, or patch algorithm -- like Unix diff files, only specialized for XML (operations that can add or remove branches from the XML tree structure, rather than operations on lines as in text diffs).

Once the SIMPLE WG was potentially working on such general mechanisms, it seemed like a good idea to hold a BOF (Birds Of a Feather) meeting to see if there were general use cases and find or identify other potential IETF participants. Some places where we thought we'd see interest:

• WebDAV allows authors to collaborate on documents stored on HTTP servers. Sometimes these documents are quite large and it would be useful to be able to upload changes without sending the entire file again. In fact, Adobe engineers have talked to me about this -- some of their WebDAV functionality is intentionally designed to limit the number of times large files are exchanged between client and server, so that the user isn't constantly waiting for slow uploads or downloads. Obviously an XML patch format only works if the document is in XML, but some Adobe tools do support XML formats (e.g. InDesign). Another piece to this puzzle is the HTTP PATCH operation I've proposed, an idea I intend to come back to shortly particularly if I get any help (hint, hint).
  
• The NETCONF WG is pursuing ways to interoperably configure network devices and has also settled on using XML and HTTP. They've got very similar problems of wanting to make small changes to large data sets.
  
• Large Web pages in XHTML could be edited using an XML diff format to upload only changes.
• Large Web pages in XHTML could be downloaded faster using RFC3229 and an XML diff format. A text diff is used today but an XML diff format could be even more efficient, particularly for...
• Blog feeds. Today, a blog feed can be a large XML file, in Atom or RSS format. Today, if the ETag or Last-Modified timestamp of the blog feed changes, the newsreader client downloads the entire file. Similarly, to add a single new post to the feed, blog editing tools may have to upload a new feed file (unless the server does this magically somehow). This is really just a special case of the general "large files being shared" case, but since blogging generates so much traffic it seemed worth mentioning.
  

Anyway, even with all these potential use cases, the only people who came to the BOF meeting with definite interest were SIMPLE WG participants. There wasn't even enough interest from outside SIMPLE to merit a separate mailing list, let alone to add non-SIMPLE requirements or
to form a separate effort. So the work proceeds on the SIMPLE mailing list. Still, I plan to keep up with Jari's work and possibly help him generalize it further -- for example, we may add the ability to make changes to text values of XML elements without replacing the entire text value.

Note that there exist other XML diff formats, but none of them are standardized. Microsoft's got one, the W3C has tackled this both for rdf and more generally (though the W3C didn't have any guidance for the IETF when we asked about this BOF), and it's been the subject of several theses: treepatch, diffxml and a survey.

In the past the IETF tried to discuss in a green-field manner how to internationalize email addresses and other email headers. None of the many options seemed really low-risk, however, and those discussions didn't get very far -- rat-holes being very frequent, and people having very different notions about what a solution could look like and what the outcomes and side effects might be.

The BOF I attended last week, however, was much more focused: it was about whether or not the IETF should charter a group to work on a specific experimental solution for internationalizing email addresses as outlined in several draft documents. That solution focuses on SMTP, so that "consenting adults" in an environment that wants to do i18n addresses may do so, and SMTP agents have a reasonable way to deal with these addresses when communicating to the wider world. It's a somewhat transitional approach, acknowledging that existing Mail User Agents (MUAs) and Mail Transport Agents (MTAs) won't immediately be able to handle these.

The discussion was mostly abour risks and unknowns. Some of the risks:

• It's not known how (and when, and by whom) IMAP would be updated to handle i18n addresses, and how clean that could be.

• It's not known how POP would be updated to handle i18n addresses (Chris Newman stepped into the line of fire here)

• It could be difficult to manage VCards, iCalendar objects, and Web pages where mailto URLs and email addresses appear. Some of these support i18n, some don't; even ones that do may have incompatible representations.
  

• Although i18n addresses are supposed to remain within these groups of consenting adults and be transformed before transmission to non-i18n MTAs, it's not known to what extent these would actually leak out.

• When these do leak out, the user experience of email users with non-i18n MUAs could be unsatisfactory.

• There will probably be serious difficulties when non-i18n MUAs are used to try address mail with i18n addresses -- it's possible that sometimes only an i18n address is known and the user can't figure out how to enter it or isn't allowed to by their software.

Normally the IETF is rather risk averse when there are so many unknowns, particularly when 'Net fragmentation might occur. In the past the IETF has gone around and around trying to get more certainty before even chartering a working group. This time, however, since there have been so many discussions and stalled related efforts before, the attendees took a leap into the unknown and approved the working group -- unanimously, I believe. Imagine Admiral Farragut's sailors taking a "hum" and deciding together to damn the torpedoes.

"Eye of Newt" Jello shots

Although it's been pointed out to me that doing jello shots at parties is more of a twenty-something activity than thirty-something, I don't care. I never did jello shots before so dammit, at our party this past weekend, I was determined to do jello shots.

The ostensible excuse was the Hallowe'en theme: orange jello shots with black "eye of newt" in the bottom. I used black tapioca pearls, the kind that are served in "pearl tea" which I love. I found packages of dried black pearls in an Asian grocery store and this worked well.

1 large package orange jello
2 c. boiling water
1 c. mandarin vodka (or other clear liquor)
1 c. cold water
1 pkg tapioca pearls (five servings)

Boil tapioca pearls -- I found it took longer than the advertised 5 minutes. Drain and rinse and distribute into little paper Dixie cups (I used about 20 but these were smallish shots). Mix boiling water and jello powder to dissolve then add liquor and cold water. Pour mixture into cups and put into the fridge for about an hour.

The paper cups are important because you have to squish the jello shot out into your mouth. Leaving the jello solid too long before serving means that the pearls get harder so don't prepare too far in advance.

Calendaring News: I haven't blogged about this in a while, so here's what's up.

• The CALSIFY WG met at the last IETF for the first time. We found authors to revise the iCalendar suite of standards (RFC2445, 2446 and 2447). We're meeting in November in Vancouver to discuss progress and open issues on these.
• The CalDAV proposal for standardizing calendar access (personal, group or public calendars) is going very well. We're very close to a draft we can last-call at the IETF. I'm proud to say that OSAF's Chandler and Cosmo both do some CalDAV and can test basic interoperability with other clients and servers like Sunbird and Oracle's server. I'm demoing some of this at Educause next week.
  
• We're starting to think about what extra metadata is needed to do public event calendars well -- for example, good location information. There's a technical committee within CalConnect talking about this, including EVDB/Eventful people.
• Another new CalConnect committee is discussing the application (and perhaps in some cases, adaptation) of these standards for use on mobile devices.
  

I can't write as humourously as Stitchy McYarnPants of the Museum of Kitschy Stitches (Vols. I, II, III, IV and V). I'm not as up-to-date as knittykitty of the the You Knit What blog. And even (or especially) my husband thinks I'm the last person to be an authority on this subject. However, I scanned some choice pattern pics from my small collection of "antique" (meaning, completely out of style) knitting and crafting books to give you: What Not to Knit.

Reasonable people adapt themselves to the world. Unreasonable people attempt to adapt the world to themselves. All progress, therefore, depends on unreasonable people.

George Bernard Shaw

Ekr explained the corrective eye surgery I just went through. Since I've been wearing glasses since I was five (and had limited use of contacts because of allergies) this perfect vision feels quite different to me -- perhaps my own self-image was more tied into the wearing of glasses than I had realized. Anyway I'm very happy and periodically almost giddy at this improvement.

Philippe said that the CalDAV panel at OSCon was full of very interested people. I'm quite gratified and quite sorry that I couldn't be there (at 6pm PST I was fast asleep recovering from meetings lasting from 9:00 am to 8:00 pm in Paris time yesterday).

Philippe also pointed to this blog commenting on OSCon, CalDAV and various calendar product and server providers.

My ideal cultural week (or so) in Paris
- 1 day in 7e, up Eiffel Tower to see whole city, then walk around Invalides and École Militaire (would have been even nicer to go into Army Museum)
- 1 day bike tour of Giverny and Monet's garden, being restored and regrown the way Monet grew it in order to paint it. Interior of Monet's house also fantastic due to the dozens of Japanese prints that influenced his aesthetic.
- 1 day Louvre to focus on the grand gallery, 19th centure and famous works of art I hadn't seen yet. Favourites include Ingrès and David's portraits of women.
- 1 day Centre George Pompidou, include main collection and special expositions (this time: Africa Remix)
- 1 day Versailles, include Grand and Petit Trianon as well as main buildings by going for the day pass
- 1 day Musée D'Orsay -- no, make that two, there's so much to see here and I really wanted to go slowly due to my interest in the impressionists and symbolists
- 1 day seeing the Galliera de Mode (not very big) and, inspired, go clothes shopping
- See the end of the Tour de France -- bonus!
- Pop into Musée Cluny for medieval textiles, tapestries, roman baths, stained glass and other cool stuff
- Every day: tiny cups of espresso at street-side cafés, baguettes, cheese, olives, pastries, and sometimes crêpes, ice cream, mussels, frites, duck, foie gras, prosciutto and rocket (roquette) salad

What I still wish I could do:
- Musée Marmottan has still more Monet works (than can be seen at Giverny or Musée d'Orsay) and it's on Bois de Boulogne which features in so many history fact and fiction books
- Rivoli Museums
- Musical Instrument museum
- Chartres
- Orangerie museum in Tuileries, Tuileries gardens themselves
- Grand Palais and Petit Palais
- More time browsing books and shops on left bank

Project Management for Sailors

Have you ever heard project managers talk like this?

We decided to take a different tack with the rollout project, after we ran afoul of the ordering processes. Julie was swamped with acquisitions paperwork and helping Sam learn the ropes, so we deep-sixed the new hardware. Office scuttlebutt is that the boss wants to cut expenditures so it's easier to go with the flow on that one. The OS upgrade project had a minor hitch, but that project is on deck now -- all we have to do is tie up the loose ends and it'll be all sewn up. That will put us on an even keel for the rollout and if nobody rocks the boat the support team will stick with us until the bitter end.

Although this is fictional I've heard all these colloquialisms in the office, and probably only recognize them as nautical because of an interest in etymology and a couple years of sailing. A non-sailor or person for whom English wasn't their native language might have a hard time seeing where these come from (and thus, how to use them properly). Herewith, a guide.

To take a different tack is a change in direction while sailing. Tacking itself means to progress by changing directions frequently, like zigzagging. This is necessary to make progress into the wind: a boat can sail quite fast when it's pointed nearly into the wind, but it can't make progress directly into the wind. Thus the boat zigzags so that the wind comes over the port side first, then starboard, then port. Don't replace with the word "tact" as Michael Rubin did.

To run afoul is to hit a snag, a complication, particularly in laws, regulations, contracts or processes. On a sailing boat a fouled rope is one which is caught up in another rope or another piece of equipment. Fouled ropes can cause quite a tangle if sailors keep doing what they're doing.

Swamped is quite suggestive, as a swamped boat is literally filled with too much water. A swamped boat is not only heavier but also sits lower in the water and presents much more drag on the water, so no wonder a swamped boat moves so slowly.

Learning the ropes is exactly what a sailor does on a new ship. Sailing ships have very individualistic rigging, often the result of years of modifications and jury rigs.

Deep sixing is burying something in six fathoms of water (a fathom is six feet), deep enough for it to be gone.

Scuttlebutt was the term for ship-board gossip. A butt is a cask of a certain size. A scuttle is a small hatch on deck. Thus, the scuttlebutt is a butt lashed onto the deck near the scuttle. Often this butt contained fresh drinking water. How a propos that today this might also be called "water cooler talk".

Go with the flow is simple -- it can apply to the flow of a river but also to the tides. Leaving harbour when the flow (the tide) leaves is much easier than going against the tide.

Hitch is a specific kind of knot tied in the middle of a rope. When winching a rope, winding it or running it through a cleat, a hitch would temporarily stop progress. However, it's not as bad as a fouled line.

On deck meant something physically on the deck of the boat, the exterior top surface where people stand. This probably migrated first to baseball where the batter going next is said to be on deck.

Tying up the loose ends meant to literally clean up the long ends of lines (ropes) hanging off a rigging once it was rigged. Loose ends were dangerous on ship, causing fouled lines and getting in the way of sailors moving around. Naturally this was always the last step in rigging, part of doing the job well.

All sewn up probably refers to the shroud around a corpse prepared for burial at sea. A corpse ought to be weighted down by something like a cannonball. The cleanest way of doing this was to put the cannonball and the corpse together in a piece of sturdy sailcloth and sew up the edges. After being sewn up there was nothing left to do before burial.

Even keel is a ship's position. The keel is the center bottom line of the boat from front to back. A ship with an uneven keel, dipping into the water more at either bow or stern, was probably badly loaded. An uneven keel meant that the ship wouldn't sail as efficiently because it would not present an optimal profile for water resistance. More generally it simply means going smoothly, steadily, without waves rocking the boat.

Rock the boat is too simple to need much explanation...

Bitter end is a very specific end of a line (rope) -- the end that goes around the bitt, a kind of deck post. I guess sailors would be told to pull on or coil the rope until they reached the bitter end.

(Sources: take another tack, deep six, scuttlebutt, all sewn up; also other pages on same sites)

IMG_3437.JPG

IMG_3437.JPG
Originally uploaded by hamstermacaroni.

Mimi is the master of taking friends shots including herself. Or at least so I conclude based on a sample of one photo. This is me and Mimi at a Giants game a couple weeks ago.

This posting has also been a test of the flickr photo-blogging tools. They're impressively easy to use, at least with a blogger blog.

Kim Cameron is working on the "Laws of Identity" and discussing them on his blog. Kim defends the choice of the word "Laws" quite strongly:

I tried to explain that the laws are not what Bob Blakley calls "desiderata" - things that we would like to see. They are the objective characteristics of an enduring identity system at Internet scale.

Oh, but how often we turn out to be wrong about these seemingly objective characteristics. Kim's Law 5 on Pluralism of Operators and Technologies understands that there will be more than just one identity authority, but wishfully states that there must be one "encapsulating protocol (a way of agreeing on and transporting things)". This reminds me of a heated BOF three years ago at the IETF for a WG I ended up being heavily involved in, chartered to take the Jabber protocol and standardize it as XMPP. At the time there was already another IETF WG doing instant messaging, and a friend of mine got up at the microphone, objecting to the formation of the WG, clearly quite upset, saying:

But you can't have two instant messaging protocols! That's like -- that's -- that's like having two IPs!

Of course the irony is that the IPv6 WG was probably right down the hall working out how to co-exist with IPv4 during an extended, possibly interminable transition period. And today we have both XMPP and SIMPLE and gateways between instant messaging protocols -- not just two, but probably more like eight (including MSN, AOL and other pre-existing systems). Much as we'd like computer systems to be simple, considerations of backward compatibility and competition between aesthetic models, among other things, keep things exciting.

I finished a knitting project in March, a purple shawl in Old Shale lace pattern. I just got around to posting the pics this weekend.

America is full of fat people, right? Well, I've been subject to the same delusion. I think it took hold firmly in 1993, when I drove from Waterloo, Ontario, to Milwaukee, for a few days at GenCon, and for some reason was struck by seeing all the fat people in the streets of Milwaukee. Was that simply observer bias? Who knows. Now I live in California and it's very clear to me that Californians don't tend to be obese, but I still had the meme in my head that "Americans are fat" even if California big cities might be some fitness-oriented exception.

Well, I'm encouraged to hear I might be wrong. It seems this is a common myth, and that other well-off countries have similar weight profiles, as explained by Paul Campos in a TCS interview (I keep reading TCS for exactly this kind of myth-busting material though many articles are more boring). I wish the article provided links supporting the claims, because there were a few quite interesting tidbits from the article:

• The CDC had previously estimated 400,000 deaths from obesity in 2004 but has recently had to revise that figure significantly downward (following CDC links, I see the surgeon general now says an "estimated 300,000 deaths[/year] may be attributable to obesity").
  
• The classification of "overweight" in children is defined as the heaviest 15% of children for a particular age cohort. That would mean nearly a million children are labelled overweight because they're defined that way. According to that methodology, 15% of children in a third-world, famine-wracked country would also be overweight. I can't find evidence at CDC of this methodology but there was a study showing that 16% of teens had been found to be overweight [*].
  
• There's no strong evidence that Type 2 diabetes -- one of the diseases justifying the labeling of obesity as an epidemic -- has increased, let alone having increased due to weight factors.
• The governor of Arkansas has written legislation proposing that body weight index be part of kids' report cards. If I were in Arkansas I would write him personally to indicate my disgust at such misguided and potentially harmful interference in the family.
  

One nasty statistic I picked up myself from the surgeon general's note is that "Overweight adolescents have a 70% chance of becoming overweight or obese adults". Yikes, oh no! But combine that with the claim that 60% of adults are overweight or obese, and a sensible person will see that so-called overweight adolescents would have only a slightly higher (if measurable) tendency than non-overweight adolescents to become overweight adults. Sigh.

I completed another long-term project this weekend -- though it was supposed to take only a couple months, I dawdled for over a year. It's a silk-mohair cardign, pattern and yarn chosen to show off some really marvelous Toulouse-Lautrec buttons I found in Vienna. More details and pictures can be found on my knitting gallery.

One of the projects I've been very involved in at OSAF has been the Cosmo project, an effort to take existing open-source WebDAV server technology and turn it into an easy-to-install, easy-to-administer "sharing server" for use by Chandler -- but also supporting standards that make it easy for any client to share too. Last week, Mitch did a demo with Cosmo in the background for the first time and it went smoothly. Thanks BCM!

I have completed a big long-term hobby project, and given it away (I think it was quite a surprise) so now I can reveal what I was working on: a king-size Amish-inspired wedding quilt. It's at the top of my crafts gallery right now if you want to take a look.

I needed to cancel a credit card I wasn't using, so I called up their toll-free line and tried to navigate their voice-activated automated menu. The automated system repeated all the choices ad nauseum, misheard me, and of course didn't have an option for "Cancel this goddamn card". I tried to say "Cancel card" and "Cancel" hoping that would either trigger some bonus feature or get me out of the system and talking to a human. I tried to say "Talk to representative". At some point the system offered the option of "Help me with something else" but when I said that it misunderstood me. Finally the system gave up on me -- literally said it couldn't seem to help me -- and that it would transfer me to a live person (thank-you).

I got some satisfaction, however, when they asked me why I was cancelling their card. "Because of the automated voice assistance system", I said -- and this was typed in without comment by the operator. I only felt the teensiest bit bad for lying.

I'm concerned about conferences with serious gender imbalances, much like Kathy Sierra and Shelley Powers with their recent posts on ETech. However, it's not ETech I'm thinking of.

To put it bluntly, a knitting conference can be hostile to men. The interaction style is touchy-feely -- women walk right up to strangers and fondle their knitted garments, invading personal space. Although some have said that men aren't discouraged from attending and in fact receive positive attention for being there, this can be of the form of "How nice it is to have men attending", "Is this knitting book for your mother?" (ref) and "I'm sure your wife will love this yarn.

All the instructors at knitting conferences are female, and one wonders if the mostly-female program committee could have something to do with that. In the expos or markets, one finds patterns for shawls, purses and female garments. Although some of the market vendors are male, they are clearly there as "booth bunnies", to attract women to look at the yarn. Some of the male vendors are even pressured to wear demeaning and ridiculous knitted vests.

What can we do about it? Well, we can be more equal in our language, for a start. Articles like this, although mentioning a few knitting men, are given titles like "The yarn is flying as more women discover the joys of knitting." Vendors like Habu textiles are taking a step in the right direction, offering stainless steel yarn, which is sure to appeal to men. Joe, a male knitter, has some other great ideas, such as making the act of knitting into more of a competition.

Researching Bali, found this:

The once beautiful Bale’s and other buildings have long since fell into a state of disrepair. It was as if the villagers had ‘locked-on’ to the ring of the till from the tourist dollar. This was more than evident when I had a short but sharp conversation with one of the souvenir sellers:

Tourist: I was here 18 years ago and I am surprised at the change in the village.

Seller: (Abrupt tone) What’s wrong with the village?. Now I have a motorbike. We have TV and electricity and... a phone!

It was my point precisely! Too many cultures in this world are decimated by the tourist dollar. Not only that, the intrusion of the modern world has an overall affect upon the social, religious and political aspects of that society. So much so that it literally forces the entire socio-cultural structure to undergo a metamorphosis in order for adaptation. Having said that, tourism is good for the state of the economy in any country. However, when it has a serious affect on the core culture of that country, then it becomes disadvantageous.

Much as I'd love to visit other countries and have them be all picturesque and unique, I can't begrudge a Balinese his motorbike, TV, electricity and phone. I wouldn't give up my conveniences to live in the style of my grandmother, nor would I appreciate pressure to maintain the religion of my ancestors. A tourist like this one would have to stop being a tourist (being one of those intrusions), go back to the country of his own ancestors, give up his own conveniences etc. for me to take those sentiments seriously.

Philippe pointed me to David Berlind's list of Technology's top ten failures. Two of them are directly about calendar standards, asking for the ability to schedule meetings and share calendars. Another two are email related. Good news, David, we're working on the calendaring standards, and calendar and email implementation issues for you, already. Sorry I can't -- personally -- help you with phones and printing problems just yet.

Who is your favorite Philosopher?

Ekr points out that the question "Who's your favorite philosopher" does not have an analogue in all fields. In highly technical fields, you wouldn't ask "Who's your favorite cryptographer" or "Who's your favorite biologist". OTOH it's common for musicians to talk about their favorite pianist or favorite composer, for painters to have a favorite painter, and for writers to be asked about their favorite writer (it's a formula interview question particularly for writers). So I guess that means that in not only asking the question but insisting on it as a valid or important question, David Brooks is implying that philosophers are like artists, their work subject to aesthetic judgements, preferences and stylistic likes and dislikes. I'm not sure that's what he intended in bringing attention to the question.

There's lots of action in the online calendar world. A fair amount of it gets brought to my attention, which is lots of fun.

• Ted pointed me to Mosuki, and I hope to meet some of the creators shortly. It's more oriented towards personal calendars than the other sites but it does allow you to share.
  
• I met Brian Dear a few months ago and heard about EVDB. Now that it's announced, I can talk to others about it too. Scraping sites for event and venue information sounds like a really valuable service.
• Kragen Sitaker showed me a del.icio.us-like calendar site that seemed pretty cool. Rather than try to scrape every page, his demo let the user pick certain text, send it to the demo site and it would turn that text into an event (pulling out date, time and location if it could) on your calendar.
• Brian posts about Upcoming, a public events site with feeds to let you know about event categories you're interested in, and a way to add events you want to your personal calendar. It has a HTTP/iCalendar interface that allows users to synch into iCal. According to del.icio.us, upcoming is the biggest new thing with hundreds of bookmarks.
• So far, Trumba's newly announced OneCalendar gets a rather less enthusiastic response. Trumba consists of several ex-Visio guys according to their press release, and like EVDB they have high-powered investors so expect some noise. Already you can synch with Outlook which has got to be a highly desired feature.
  
• Of course, we are working on some of the same stuff too, only with a federated server approach. When you share your calendar with Chandler you can share it on any WebDAV or CalDAV server (see also recent article). We're working on a WebUI for such shared calendars so that the calendar owner or their friends can view the shared calendar or individual events just by going to the URL in the browser. We may do tagging just like everybody else, too.

So far I don't use any of these, there isn't quite enough "there" there, whether it's users I want to share with or features I need. My calendar is in iCal, stored locally, and I don't even share it with anybody, though I do share one other person's iCal calendar. But I don't think that isolation will last long given the exploding options.

I wonder what Yahoo, Google and Microsoft are going to do in this space? Yahoo has more of the world's calendar info than any other site at this moment, because of its excellent support for group calendars. What will Yahoo do with that data? If it does not expand its features soon, possibly allowing users to synch up calendar data on the site with their client software, Yahoo will soon see its calendar data decay and vanish. As for the two gorillas, I really don't have any idea whether they will buy, innovate, or crush.

In response to my last post, Mark pointed out that .NET's HTTPClient supports pipelining. Heikki said that Mozilla put in support for this but it isn't turned on normally because so few servers support it. Furthermore, Heikki reports that tests showed only a 7% performance increase when Mozilla uses pipelining. The Mozilla FAQ is a good start for information on this though it doesn't have the data.

The application I had in mind (Chandler) does more synchronization than browsing over HTTP/WebDAV. For example, if the user decides to work offline, and while offline moves a bunch of resources from one collection to another, then goes online again, Chandler would have to issue one MOVE request for each moved resource. With pipelining, Chandler could theoretically fire each request off and wait to start seeing the responses come back in order over the same connection. The only thing limiting throughput is the bandwidth and MOVE requests and responses don't take up that much. Without pipelining, Chandler has to wait for each response before sending the next request. Now this work is limited by latency, multiplied by the number of resources being moved.

Still, I don't consider this proof that pipelining would be useful. It would be great to see more data one of these days -- assuming, of course, that there really is a performance problem in the first place.

Are there any open source HTTP client libraries that support pipelining? Are there any open source libraries that solve an isomorphic problem with a different protocol? So far I'm not turning up anything but this is the kind of thing that is a little hard to Google for.

At the gym last night I was lifting weights and a man started this converstaion.

Man: "What are you going to do with all those muscles? Beat up the boys?"
Me: "Yup."
Man: "Well then, stay away from me.

What am I supposed to make of that? Seriously, I'm asking here.

... So this hat doesn't fit me. The question is, would it fit a guy with a size 7 3/4 head? Joe, you might or might not want to click through :)

There are a whole bunch of "Working Group last calls" going on for email-related technology. Roughly, these are calls for reviews of Internet-Drafts before they are submitted to become standards. I called for reviews for an IMAP Extensions working group document which adds annotations (like properties, or metadata) to emails in an IMAP server. The Lemonade WG has issued last calls on these documents (all last calls on the mailing list and the documents themselves are linked from the Lemonade charter page):

• the Catenate proposal, used to allow clients to more efficiently work on draft messages stored on an IMAP server
• URLAUTH, which lets one user show an attachment to another user without sending them the whole attachment (basically I give you authorization to view my copy on my server)
• Server-to-server requirements for email event notification systems -- intended to allow voice mail and email servers from different vendors work together more smoothly
• The BURL draft, allowing clients to forward an email without downloading and uploading it again as email systems currently require.
  

There are also upcoming improvements to the performance and extensibility of the LIST command which is the heart of IMAP, and ongoing improvements to access control, both coming to a head in IMAPEXT. If you're interested in email protocol and developing standards for those, this is an exciting time for that. I guess that means about 7 of us globally are excited right now :)

I attended a knitting convention this weekend, and learned how to do japanese short rows, double crochet (which is like a cross between knitting and crochet), and twined knitting which is a three centuries old Swedish technique a.k.a. "Tvåändsstickning". The marketplace was also amazing and very tempting and dangerous to my wallet.

The "gallery" at the Stitches conference is very ad-hoc -- it's on the backs of the attendees. Here's a few pics of people I saw.

It's starting to become well-known that well-published suicides inspire more suicides. There's an explanation on everything2.com, but I'm dissatisfied with the references to heroic portrayal as the explanation for the phenomenon. The Wikipedia entry is much more complete, but it still explains copycat suicides by refering to the media "glorifying the deceased." This "reward" theory is simple--that the media encourages copycat suicides by providing attention and glory as a reward for suicide. The standard sources also explain that media coverage may provoke admiration of the deceased in eulogizing him, and by explaining the causes of the suicide and thus implying that suicide is an understandable or normal response to those causal events.

The reward theory has been debunked somewhat in Cialdini's book Influence,
the Power of Persuasion. Some brief reasons to doubt the reward theory:

• Even consistently negative publicity encourages copycat suicides
• Copycat suicide demographics are surprisingly similar to the original suicide (e.g. 35-year old women don't tend to see Kurt Cobain's suicide as one to copy no matter how much glory is supplied)
• Deaths which are not publicized as suicides but which might have been (car and plane accidents) provoke a rash of similar deaths even though there's no glorifying of suicide.
  

Cialdini's model to explain copycat suicide is more sophisticated than the simple "reward" model -- Cialdini calls the effect "social proof" and it goes beyond just suicide to explain how people copy the actions of those they consider similar enough to be role models. So even if the suicide is criticized unanimously, someone who feels similar (same preference in music, perhaps) will find actions to emulate. In fact, the criticism from the dis-similar people may strengthen the bond felt towards the role model. The bond may be strengthened by the copycat feeling that society is against both himself and the original suicide and provoke feelings of solidarity. Thus, a society that reacts to some unusual person's suicide by criticizing their unusual taste (goth clothes, punk music, or playing dungeons and dragons) may in fact create a self-fulfilling prophecy in calling that taste dangerous and then see a spate of suicides in a population sharing that taste. There's also some evidence (again you can reference Cialdini who discusses Goethe and his hero suicide Werther) that even fictional suicides inspire copycats.

Dorothy Parker seems to have intuitively understood this. I recently read Bobbed Hair and Bathtub Gin, a look at literary culture in the Twenties by examining the lives of Dorothy Parker, Edna St. Vincent Millay, Edna Ferber and Zelda Fitzgerald. Dorothy attempted suicide several times in her life, each time via a different approach, and damaged her own health and reputation. In her short story "Big Blonde", Dorothy wrote:

There was no settled, shocked moment when she first thought of killing herself; it seemed to her as if the idea had always been with her. She pounced upon all the accounts of suicides in the newspapers. There was an epidemic of self-killings -- or maybe it was just that she searched for the stories of them so eagerly that she found many. To read of them roused reassurance in her; she felt a cozy solidarity with the big company of the voluntary dead.

The subtlety of the copycat suicide effect probably means there's no easy solution. It doesn't work to have the media to portray suicides negatively -- that doesn't prevent the social proof phenomenon and may even strengthen it. We probably don't want to instruct the media to entirely suppress news of suicides. Even if we did that, there's still fiction and possibly other art (music lyrics?) -- and even if we censor art there's still deaths-which-might-have-been-suicides.

Despite these subtleties, most journalism codes around discussing suicide in the media focus on the glorification aspect. E.g.

a licensee must not broadcast a program which depicts suicide favourably or as a means of achieving a desired result (ref)

The WHO report from 2000 also talks of glorification and acceptance of suicide as an understandable response, but it goes even further, stating that "certain types of coverage may help to prevent imitation of the suicidal behaviour" (p 6). However, I've not yet seen evidence of that, and I worry that is merely wishful thinking. If Cialdini's model is closer to being correct then the very type of coverage that the WHO report suggests may do more to encourage copycat suicides by mourning the deceased, providing details of their life and families, and providing "risk indicators and warning signs" which can trigger the role model effect.

Update: I've never updated a Wikipedia page before, but this seemed a good time to try. I added the paragraph on Cialdini's social proof model.

I got annoyed on the drive to work today, listening to NPR's Morning Edition program. The show was covering California's new display hardware tax and recycling program. I guess "coverage" always means looking for flaws, but this was silly.

Here's the facts: California is adding an electronics recycling fee, which retailers apply based on size of display, from $6 to $10 per display. This includes laptops, monitors and TVs.

Morning Edition added the factoid that California pays recyclers $0.48 per pound to process the displays, to chew them up and sorting the resulting bits into plastic, glass, etc. From this factoid and the fee rates, Morning Edition concluded that the tax would certainly not pay for disposal. They gave an example of a 53 lb monitor, which would be taxed at $10 at time of purchase but would cost $25.44 to recycle.

With the information provided so far in the program, I could see immediately several flaws in their argument. First, not all monitors are that heavy. People buy 5-10 pound laptops, pay $6 or $8 fee at purchase time, and those only cost $2.40 to $4.80 to dispose of. Second, there's time-value of money: the $10 paid for the 53 lb monitor is paid years before that monitor has to be recycled, and if the state saved that money it could accrue interest.

Even if you say that this year's recycle fee income should cover this year's disposal costs (rather than allocate the fee to one item over its life) it's entirely possible that the program does pay for display recycling. It works if people buy more lightweight displays (for which they overpay on the recycle fee) than heavy displays (for which they underpay). At the same time, it also works if people buy more displays per year than they dispose of -- also a reasonable possibility.

This kind of thing in the media bothers me more and more. It makes me think "How stupid do they think I am?"

Today was the first Interop event for CalDAV. Sadly, I am not there, having cancelled a trip due to a cold. But I'm excited to hear about preliminary successes. It's only been a couple months since people started developing code to this spec, so this is great progress!

I went off on a tangent today and wondered how many translations of Marot's poem "A une Damoyselle Malade" could be found on the Web. I thought there would be more because there are nearly a hundred in Hofstadter's book "Le Ton Beau de Marot" alone, and he encourages readers to write their own. Anyway, here's the list I found, using the first line rather than a title to refer to them because that's what I found most memorable:

• Petter Hesselberg wrote Min Konfekt in Norwegian and provided an English gloss of that
• Mark Irons wrote Little Mole
• Craig Clark wrote Precious Girl
• David Short wrote Sweet Jeannou
• Melanie Mitchell wrote To my sweet (or in her more wordy version, "To be a joy forever to my sweet")
• Enno de Witt wrote Teder Wicht in Dutch (gloss, anyone?)
• Kat Walsh wrote Sweetest Rose.

Rather than provide my own translation, I wondered what Babelfish would provide. To get a decent approximation I had to remove line breaks (format as a paragraph), replace abbreviations with fully-spelled-out words, and modernize a couple words. My input:

Ma mignonne, je vous donne le bon jour; Le séjour, ce est prison. Guérison recouvrez, puis ouvrez votre porte et que on sorte vite, car Clément le vous commande. Va, friande de ta bouche, qui se couche en danger pour manger confitures; si tu dures trop malade couleur fade tu prendras et perdras le embonpoint. Dieu te donne santé bonne, ma mignonne.

Babelfish spit out (with me re-adding the line breaks as with the original):

My nice,
I give you
the good day;
The stay,
it is prison.
Cure
cover,
then open
your door
and that one left
quickly,
because Clément
orders you.
Goes, fond of delicacies
of your mouth,
which lies down
in danger
to eat
jams;
if you hard
too sick
insipid color
you will take
and lose
the plumpness.
God gives you
good health,
my nice.

Not too bad, really, though I had to trick Babelfish to get it to be even that good.

It seems that to keep an eye on terrorists and other dangerous people, the government could be more creative in outsourcing the hard work of keeping an eye on citizens. Leave it to an expert -- Santa Claus. He'd just have to expand his "watch list" from kids to include adults as well. Then we could update the famous song and and still keep the exact same Christmas spirit:

Don't drive like a jerk,
Or be late paying fees,
Steal Post-Its from work
Or share mp3s,
Santa Claus is coming to town.

He's making a list,
Of those who can't fly,
With guys who made jokes,
In the security line,
Santa Claus is coming to town.

He knows if you've been cheating,
On your taxes or your wife,
He knows if you've been smoking pot,
Three strikes and you're in for life.

Anyway, happy holiday season.

Jon Udell just posted his wishlist for a calendar protocol (comparing it to WebDAV-only and to traditional calendar-only protocols), a list that's rather close to explaining why I'm working on CalDAV and what I want it to do. (Jeffrey Harris pointed this out to me and even pinged Jon about it, thanks Jeffrey)

A couple new items in my "stuff I made" pages, both the knitting page and the general page: a crocheted purse (almost clutch size) and knitted lace pillowcase edgings.

I'm hiring again at OSAF -- this position is for our very first server developer. I'm looking for somebody who can lead the charge, crank out code, and make the server do us proud.

Data modeling is hard. Some loosely correlated thoughts and links:

• Model-driven architecture is rigid, at least with the tools as we know them today.
• RDF has a simple basic model but leads to very complex structures, as Adam Bosworth explains.
• Pictures express complex relationships relatively readably, like the picture in this paper. Unfortunately we need to translate the pictures into text in order to use these in software and network protocols.
• The more complex your picture is, the more unreadable your text is.
  
• Text has to be relatively flat to be readable.
  
• References in data formats are like "goto" jumps in programming -- you lose context.
  
• Maybe if data modelers put a little more thought into flattening their models we'd find them easier to use? This may make the models seem less "rich" but "KISS" is good too.
  

I don't know how true all of this is, but I'm learning.

The "relatively flat" observation seems to hold at least some validity in data formats, programs and even books. Experienced programmers, with the help of good indenting, can see quickly that they're within an 'else' statement inside a loop inside another loop inside an 'if' statement, but even experienced programmers screw this up sometimes (and even more experienced programmers flatten out the code by delegating some reasonable piece off to another method). Books are better if there's no more than three (maybe four) layers -- chapter, section, sub-section, and even this much organization requires human-readable text to link from one section to another and summarize what a bunch of sections are going to say.

There's a higher quality of homeless people in Palo Alto. I just saw a hand-lettered sign outside of Whole Foods, explaining that the homeless of Palo Alto need food donated in the holiday season (and a very shifty looking guy collecting food on their behalf). Among the list of requested foods was "organic turkey". No cheap turkey for the Palo Alto homeless, please!

Timboy has a good medium-sized post on why meetings suck yet they're indispensible.

A while back I posted on honesty in journalistic bias. This week TechCentralStation has a longer essay about why we might see more openness around bias and why that's a fine thing.

Being aware of bias is something I agree with, but I do worry about blinkered views of the world. Too many people reading some highly biased source will simply not read any opposing source, or do so with only mockery in mind. We've got plenty of polarization, thank-you. So my preferred model is journalists who say "Here is my natural bias, and here is me being as unbiased as I can be in covering this topic, through rigorous reasoning and discourse with others who disagree with me."

Writing protocol standards is hard work, harder than writing specifications, although they are similar tasks. One of the reasons is that you have to describe the protocol in sufficient detail that somebody who wasn't involved in the process and has different software experience (different features, different user interactions, different architecture, different platform or different programming languages) can still implement the standard and interoperate with other implementors. (Actually it's so hard to do this that no standard gets it "right". At the IETF we're well aware that we do successive approximations, first doing internet-drafts and then doing RFCs at different stages of maturity. ) But we can at least try to do it right, and a proper effort requires a lot of effort including:

• A description of the model
• Implementation requirements
• Examples of protocol usage
• Definitions/schemas

Often these will seem redundant with each other but they're all important.

The model

The model is key for first-time readers and for people who need to know something shallow about the protocol. There are different kinds of models that are important for protocols, and some of them are described (and examples given) in one of Ekr's works-in-progress:

• The protocol messaging model. Do messages have headers and bodies, or do they have XML element containers? Does the server respond to messages in the same connection? In a fixed order? Can the server originate messages?
  
• The protocol state machine. Are there different states (e.g. pre-handshake, pre-authentication, and main state)?
• The protocol's data model. What data types are there and what relationship do they have to each other -- folders and messages and flags (IMAP), or collections, resources and properties (WebDAV)?
  
• The addressing model, which is almost part of the data model. In SIMPLE you can address other people whereas in XMPP you can address not only human actors but specific software instances running on behalf of those humans. And not to be speciesist, non-humans as well.

There's probably other kinds of models but I've seen examples where each of these could have been explained better. It took me a while to understand IMAP annotations because I didn't factor in the part of the model where each annotation might have several different values depending on the authentication id used to access the value.

The model is important not just for first-time readers and shallow users but also later on for deep users who want to extend the protocol. HTTP has been extended in many ways by people unfamiliar with the way the model is supposed to work. For example, HTTP normally uses the Content-Type to declare the type of the message body, just as one would expect from a concept borrowed from MIME and a messaging system. However, one extension to HTTP (now part of HTTP 1.1 or RFC2616) breaks that model by applying an encoding to the body and that encoding is specified in a different header. So if that feature is used the Content-Type no longer strictly works that way. RFC 3229 moves further away from the MIME-like model as it extends HTTP -- it defines an alternative model, where the Content-Type refers to the type of the resource that is addressed. So now of course there's a schism in the HTTP community about which is the best model to proceed with, to the point of having academic papers written about the alternative models. More clarity about the model in the first place would have helped not only first-time readers of the HTTP spec but also might have helped have fewer problems with these extensions.

Finally, a clear model helps implementors remember and understand each of the requirements. Humans have trouble fitting a bald list of requirements into some memorable pattern, so give implementors a mental model (or several) and they'll do so much faster, with less confusion and mistakes.

Requirements

The requirements are deeply important, as much so as the model. At the IETF we place so much importance on the wording of requirements that we have a whole standard describing the wording of requirements. Why?

First, models can be interpreted differently by different people. This can happen very easily. IMAPv4 was originally defined in RFC 1730 and there was a lot of text about the model, particularly the different states. However a lot of people implemented the details differently and RFC2060 had to get more specific. Finally, RFC 3501 revised RFC 2060, and most of the changes made in RFC3501 were simply clarifying what the consequences of the model were for various cases -- because implementors made different assumptions, came to different conclusions, and argued persistently about the validity of their incompatible conclusions. Chris Newman explained this to me today when the topic of models + requirements came up, and he should know -- he authored/edited RFC 3501.

Second, a model explains how things fit together, whereas requirements explain what an implementation must do. Implementors are human and operating under different pressures, so it is easy for implementors to read a lot of flexibility into the model and the examples. Clients want to believe that servers will do things similarly (makes their logic easier) so they tend to assume that is the case. So when things are flexible, they must be explained to be so, to encourage client implementors to account for differences. E.g. RFC 3501 says

"Server implementations are permitted to "hide" otherwise accessible mailboxes from the wildcard characters, by preventing certain characters or names from matching a wildcard in certain situations."

When things aren't flexible, the document needs to say so so that implementors aren't given any wiggle room or room for confusion. In RFC3501 we see

The STATUS command MUST NOT be used as a "check for new messages in the selected mailbox" operation

This text is much stronger than saying that the "STATUS command requests the status of the indicated mailbox" (that sentence is also in RFC3051). It's even stronger than saying that the STATUS command isn't intended as a way to check for new messages. (It might be even clearer to say that "client implementations MUST NOT use the STATUS command..." but this is good enough.) IETF standards-writers and implementors have learned painfully that they need to use well-defined terms in attention-getting ALL CAPS in order to get implementors not to misunderstand wilfully or accidentally, whether something is a requirement.

A few more reasons why requirements are needed:

• Requirements often add more detail than the model should hold. Since the model should be high-level and readably concise, it can't be expected to define all behaviors.
• Sometimes requirements are examples of the conclusions that somebody would draw if they fully understood the model and all its implications. These have to be complete, however, not only selected examples, because no two people have the same full understanding of the model and all its implications. The requirements help people go back to the model and understand it the same way.
• Human readers need repetition in order to understand things. Sometimes the requirements restate the model in a different form, and that's fine. When essay writers want their audience to understand they say what they're going to say, say it, then say what they said. We can make our standards more interoperable by balancing that approach with our typical engineering love of elegance through avoiding redundancy. Humans aren't computers, so the engineering avoidance of redundancy in code isn't fully applicable to human-readable text.

Examples

Examples are, thankfully, better understood. It's pretty rare to see a protocol go to RFC without a few good examples. Readers expect and demand them (more so than the model or requirements) because we know from reading many kinds of technical documents how useful examples are. I hope I don't need to justify this too much, in fact I find I need to do the opposite and remind people that examples do not replace requirements or models. Implementors need examples to understand the requirements and models but they can easily draw conclusions from examples that are counter to the requirements and don't fit in the model. When a specification has an inconsistency between a requirement and an example, trust most developers to implement to match the example, not the requirement.

Definitions/Schemas

Definitions and schemas also tend not to need much justification in a techie crowd. We're attracted by the idea of having absolute certainty about what's valid by trusting a program to compare an example to a definition or schema and validate it. So once again, I have a caveat to offer rather than a justification: make sure that definitions or schemas are put in context very carefully. Can an implementor use the schema to validate incoming XML and reject anything that doesn't match the schema? Probably not, or else it would be impossible to extend the protocol. Early WebDAV implementors built XML schema validators into their servers and rejected client requests that extended the protocol in minor ways that should have been compatible, so I'm taking this lesson from actual experience.

---

I certainly can't say that when I'm a protocol author, I succeed in doing all of this. But after eight years reviewing and implementing good and bad protocol specifications, I'm beginning to see what works.

Comments welcome.

Today I managed to explain (better than I've ever explained before) a few principles in the design of a network system. I use a client/server network system although you can generalize this to P2P easily. This is the diagram I drew on the whiteboard.



If it's hard to grok this completely abstractly, an IMAP client/server are good to mentally plug in. There are so many different IMAP clients and servers and they all have different APIs, storage models, and internal data models. By "data model" I mean data structures or object models, including caching and relationships between things. So if your Java code instantiates a MailMessage object which has a link to a EmailAddress instance for the 'From' field, that's all part of the internal model. The protocol's data model is similar: in IMAP there are folders and mail messages, mail messages have headers, one of which is the From header, and so on.

So I intended this diagram to convey a whole lot of stuff.

The protocol is the most inflexible part of this system. If you've got any interoperability at all with your protocol, even between unsynchronized releases of client and server, then your protocol is the most fixed element in the system. People constantly use new clients to talk to old servers, and old clients to talk to new servers, which means that even when new clients talk to new servers you're likely using an old protocol. Since your protocol is the hardest thing to change, both its syntax and its data model, it had better be extensible, basic and support many usage models.

The internal data model is the most flexible part of this system. APIs and protocols must continue to be supported across releases. Storage formats are easier to change than APIs but often require upgrade steps. Thus, the internal model is actually the easiest thing to change. Doesn't that mean that it's less important to get that right, because it can be tweaked and refactored to do new things or benefit from new knowledge? Yet many architects focus deeply on the internal model, spending much more time getting it right than the API or the protocol.

Client, server and protocol data models and content models diverge. Many architects design a networked system that starts with the same data model on the client and server and thus naturally they want the same data model expressed in the protocol. But these diverge naturally, sometimes even before Server 1.0 and Client 1.0 ship. For example the implementors of Server 1.0 discover that they need to cache certain strings together for scalability and subtly the data model on the server begins to change. Be aware from the beginning that this will happen. It's not a bad thing. It may even be a good thing to allow the server to be fast and the client to innovate on features.

Practice information hiding at the dotted lines. These are the places to really focus on modularization. Many software developers already understand that your API shouldn't be tied directly through to your storage model and this principle can easily be extended to the protocol modules. I've written bad code that tied too much to the protocol so I'm guilty of that one myself. It seems that unless there's a good reason, the protocol implementation shouldn't be tied directly to the storage model (the implementation should instead retain the freedom to change how things are stored without rewriting everything). It might not be so bad to tie the protocol to the API, i.e., by implementing the protocol logic using only the API. That way, any internal changes that leave the API unchanged, also leave the protocol unchanged. But that isn't always the best choice -- sometimes the protocol support needs access to internals and you don't want to complicate the API too much just to make the protocol fast.

Corollary: Use the best technology and architecture choice for each component independently. Because your client model will diverge from your protocol model and that one from the server model, data model consistency is not a good reason to use the exact same table structure or even the same database software on the client and server. (There may be other good reasons like expertise). Don't try to create the same indexes; the client and the server data access patterns will also diverge if they're even the same to begin with. Don't try to recreate the same caches. Send your server and client teams to different countries to work, maybe! That way the protocol becomes one of the most important ways they client and server teams communicate and they can make fewer hidden assumptions about how the code on the other side works (but they will make some anyway which will bite you in the ass).

Standard protocols and proprietary protocols aren't much different. If the protocol data model and client and server protocol naturally diverge, then even if your system starts out with highly similar models by implementing a proprietary protocol, that advantage erodes and becomes a disadvantage, hindering extensibility. OTOH if you start out implementing a standard protocol and enforcing good separation between the data models, this is a good long-term strategy. You know from the start that there will be translation between the data models -- every protocol message that comes in will have to result in objects or data structures being instantiated in the internal format, and every protocol message that goes out is a tranformation from internal objects or data structures. So that translation layer is solid from the beginning. Furthermore, if the system is using a proven protocol, the extensibility and performance features are likely to be better than one can easily design from scratch.

Protocol syntax isn't very important as long as it's extensible. Translating between models that are different is harder than translating between different syntaxes. It's like translating a business course from American into Chinese -- the language is the easy part, the culture and environment are so different that you can easily mean something you didn't intend to mean. So it's not the end of the world if the syntax is header fields or XML documents, as long as there's a clear way to extend either one. The extensibility is key so that as the clients and servers evolve they're not totally hamstrung by an inflexible protocol.

Whew. That's asking a lot of a l'il ol' whiteboard sketch. Comments welcome.

I have to say, I love a good Fisking, or to Canadianize that, a Frumming. On TCS, Radley Balko takes on David Frum's National Review column on obesity and taxes. It's a good read in its entirety, but I thought it would be fun to summarize anyway, to show how each argument was demolished.

Frum argues:

1. Canadians are less obese than Americans
2. Portion sizes are smaller in Canada than in US.
3. It's because Canadians are less wealthy that portion sizes are smaller.
4. Smaller portions lead to less obesity.
5. Obesity leads to health care costs.
   
6. Making sodas more expensive (by taxation) will cause lower consumption of sodas (conclusion: also reduce obesity, also reduce health care costs).

Without facts, one might follow that logic. Balko, however, demolishes one after another of these, showing how much of a house of cards that logic was:

1. Canadians are similarly obese to Americans and Frum's evidence was only anecdotal.
2. Portion sizes are similar and Frum's evidence was only anecdotal.
3. Since portion sizes aren't smaller in Canada, wealth isn't a factor in portion sizes (at least the wealth difference between CA/US doesn't matter to that). Also note that total consumption of caloric sodas has been steady for decades as Canadians have gotten significantly richer (and soda cheaper).
   
4. This one requires more data to completely demolish, but the evidence that total consumption of caloric sodas has been steady for decades does cast doubt on the idea that smaller cans of sodas will reduce consumption.
   
5. There's more evidence that poor fitness (sedentary lifestyles) has a much greater health care cost than obesity.
6. Such a small increase in price of soda is unlikely to change consumption, given that consumption has been steady for decades as soda production has gotten cheaper and people richer.
   

Balko doesn't point this out, but soda doesn't appear to be very price sensitive. Rather than buying cheap store-brand sodas the market overwhelmingly goes to the higher-priced image brands.

I've enjoyed working at OSAF for the past eight months and I'm starting to understand why. We're nice but we're not losers.

We shipped a minor release this month - the 0.4 release of Chandler. The release date of October 26 was picked months ago, as was a candidate set of workflows and other tasks to achieve. We met our release date, partly by making some minor cuts to the feature set (though we still reached the overall goal of having something experimentally usable or demoable), partly by working professionally towards common goals. Even better, we did it without panic, without yelling at each other, without having sales presell the release (heh heh -- no sales). We didn't make developers stay all night. We didn't have unpleasant meetings. We didn't demonize anybody for their bug counts. And we still managed to release on a schedule.

I don't want to get too deep into how we did it (one could write a book), but it had a lot to do with honesty and trust. Maybe when the stakes and egos are high it's too easy to fool oneself into believing ridiculous schedules. I'd like to think our transparency (it's all out there on the wiki helped us be honest with ourselves and communicate potential problems early.

It's nice to have confirmation that it's possible to reach high goals and still be sane.

My local security expert can be opinionated and frank, sometimes.

Me: "So have you ever tried getting IPSEC working?"

Him: "I'd rather have a prostate exam."

Making wedding stuff

I had a fun time making stuff for my friend's wedding a couple weeks ago. Now the pictures have come out.

I made silk shawls for the bridesmaids. Lessons learned:

• I couldn't iron a double-folded-over edging into silk organza. Pin a lot.


• Sewing raw silk organza edges into a folded border is hard. I bought black silk thread so the seam wouldn't show much, and then held the organza very taut, front to back, as I fed the pinned edge through the machine. Still, the tension in the seam pulled the organza together a bit, until I ironed the *&$! out of it.


• Use a ton of water to iron silk charmeuse. But do iron, because it's worth it to get the piping straight.


• Silk must be carefully pinned, particularly when you're sewing 7 layers together. Even so, I caught excess organza in the seam a couple times.


• Use the right sewing needle to go through those 7 layers. When I used the needle I normally use for quilting, it made a scary "thunk" sound.

Sorry, no pics of the finished shawls at this point. They ended up black silk organza bodies, with charcoal charmeuse endings (folded over the raw organza edges) and purple charmeuse piping.

I was hanging around doing nothing on the wedding day and suddenly (but willingly) pressed into service making the ball for the flower girl to carry. The materials: a styrofoam ball, ribbon, and green "toothpicks" with wire already wrapped around the non-pointy end. The idea, as I saw it (but I Am Not A Florist) was to wrap the free end of the wire around a flower stem, then put the pointy end of the toothpick into the ball. Repeat until covered. Sounds simple, right? Lessons learned:

• Plan ahead. Will you be putting leaves on the ball along with the flowers? Put them on first, dummy, or they cover up the flowers.


• Pick flowers with strong stems. Flowers with weak structure will be soooo frustrating.


• Wrap several flowers/leaves together onto the same toothpick. You probably don't have enough toothpicks, plus it goes faster.


• Don't try to push the toothpicks in too far. Otherwise you push the wire or flower right off the toothpick just as it gets buried in styrofoam.


• Attach the ribbon handle as strongly as you can -- it's going to be swung around by a 2.5 year old (cute Ava). Very firmly attach the ribbon to two toothpicks and stick them in at different angles so the tension doesn't pull them straight out.


• Don't fret, there is no way to hold it without crushing some blooms or tearing off some petals now and then. After failed attempts at wrangling it with wire handles which backfired and tore off flowers, I just used the fingertips of one hand to support it.


• Finally, don't worry. Everybody will love it anyway.

Actually, it turned out quite pretty. It's by no means the prettiest thing in this photo, but it worked. Many people helped out a lot and it was a beautiful wedding with a sense of cooperation. For example -- credits to Cheng for the photos I used here.

Joe and St. Peter wrote a WebDAV-related IETF Internet-Draft recently. In fact, it combines two technologies I admire, WebDAV and XMPP, in a way that uses each technology precisely for what it's good for (WebDAV to store application data, XMPP to route application-specific messages).

I talked to a bunch of people at Educause about this yesterday, and they were excited about this and other new WebDAV features because many universities are betting on WebDAV. University of Memphis reports two services that people can't live without: email, and now WebDAV. Since WebDAV is such a flexible repository technology it's hard to tell a potential customer (like Memphis 2.5 years ago, when I worked at Xythos) what it will give them. But give it to users, and they will figure it out. Looks like the Atompub people are storing blog postings on a Web server, which students increasingly can do now via their WebDAV accounts [1]. And of course the universities are also keen on storing calendar data in WebDAV accounts. More application data in fewer server repositories means lower administration costs for these universities.

With all this increased excitement, you'd think I could get more people to show up at WebDAV Working Group meetings or contribute on the mailing list. The next meeting is in Washington DC, in the second week of November. Want to join in? It's easy -- all you have to do is... join in.

---

[1] I realize that although giving students WebDAV accounts is a little new, giving them Web accounts, or Web space in their Unix accounts, isn't new. However, in some colleges students had to request this service, or know enough Unix to be able to author the content. Plus raw HTTP doesn't support multi-user applications very well. WebDAV extends the usefulness and usability of personal Web space.

This is why I like cats.

I was trying to figure out if anybody was thinking of implementing CalDAV in Slide, so naturally I googled 'caldav slide'. No direct answers, but I did notice a LaughingMeme blog post from last spring that I'm sorry I missed, commenting on CalDAV. I agree with the gist of his caveats, and we're trying to address those, I think.

I can't vouch for the accuracy of this description of Canadian conversational styles (link via Julie), but it rings a bell -- whether because I'm Canadian, female or both, I don't know. Surprisingly, the "weaving" style can sometimes be seen as more aggressive, because it involves pushing conversational pieces into pauses. My American SO sometimes thinks I'm interrupting him. Well, I am interrupting if you assume there are rules about waiting until somebody is done speechifying even in one-on-one conversation, but the interjections are meant to build the conversation, not to be rude to him.

Recipe for Ginger ice cream -- sugar and no-sugar options

Ingredients:

• 1 cup half and half cream
• 1/4 cup grated fresh ginger root
• 1 cup heavy whipping cream, whipped to soft peaks
• 1 cup ginger ale or diet ginger ale
• 1/2 to 3/4 cup sugar or Splenda
• 1 T. lemon juice

Soak the grated ginger root in half and half cream. Put this and the ginger ale and whipping cream in the fridge to chill for an hour or so. Use a fine sieve to take the fresh ginger out of the cream. Stir the ginger-flavored cream together with the whipped cream, ginger ale, sugar/Splenda and lemon juice. Immediately put in your ice cream maker etc..

Mmm! I imagine for those who love ginger and can also eat sugar, candied ginger would be a nice addition to the recipe.

How thrilling! After only two years, the XMPP WG has produced RFCs.

• RFC3920, Core protocol (framework for applications like notifications, as well as instant messaging and presence)


• RFC3921, Instant messaging and presence over the core protocol


• RFC3922, how to map XMPP presence information and instant messaging to the same common interchange format that SIMPLE instant messaging can map to


• RFC3923, requirements for end-to-end security of instant messages even when routed over more than one protocol

The work began earlier than two years ago, of course. Jabber was designed while Jeremy Miller was waiting for the IETF to come up with a standard IM protocol (and taking too long). When the IETF process still appeared stalled, Jeremy and others proposed that the IETF could standardize Jabber, by creating a working group to design a revision to Jabber that met the IETF standards for internationalization, security, XML namespace usage and a few other things. A BOF meeting was held in July 2002, very eventful and entertaining as IETF meetings go. The WG was formed, and I was added as a co-chair to help Pete Resnick, on October 31 2002. Peter Saint-Andre authored *all* of our internet drafts, doing a huge amount of difficult work in an extremely timely, professional and skilled manner. Whew! It's been fun!