A few weeks ago, I took a class with Kathy Zimmerman at Stitches West. Kathy designs beautiful sweaters, and cabling is definitely her signature design element. She's also the author (star?) of an instructional DVD on cabling, and owns a yarn store in Ligonier PA.

Kathy showed off some of her sweaters, which was a class unto itself on proper finishing techniques. She introduced cabling charting, which I already love, and got us started customizing charts. I was inspired by a couple of her cables and immediately started sketching freehand, then translated into this:

Then we spent the rest of the class swatching our designs. I knit the bottom chart of of handspun three-ply from Frank the sheep

and the top one out of white worsted Plymouth Yarn Encore, which shows off cables nicely.

We also sketched out a plan for an entire sweater based on the brown sample! I don't know yet if that's what I'll make from that yarn when I'm done spinning it, but possibly. I'll need to check how much yardage I have when I'm done.

Jodi Green posted a link to TVO Archives. That is, the archives of public television in Ontario -- a personal wayback machine. I found "The Polka-dot Door", one episode, but more than enough to remind me of watching that show when I was six. Yikes. Today that show seems surreal and creepy.

Some of the interviews and forum shows are interesting to me now. There is an interview with Margaret Atwood at age 36 (she says "I'm old..."), where she comments a lot on how she is perceived and received by the public.

• "If they're afraid of successful women, you're viewed as a witch. If they view you as a mother figure, they want you to solve their problems."
• "I was talking about economics and politics, and we all know that girls aren't supposed to think about those things, much less talk about them."

Twenty years later interview on the publication of Oryx and Crake, she talks more about literature and politics than on perceptions of herself:

• "You do get these ideas for books that come more or less complete, you just work them out."
• "You can't tell authors what they need to do." on whether authors should be told to discuss politics
• "America is our big neighbour. And if they go down, we go down. So of course we worry. If a person blows you up on some street or other, he isn't going to see your teeny-weeny maple leaf."
• "I got tired of people saying, why do you always write about women." (on Jimmy, main character of Oryx and Crake)

Also an interview with Mordechai Richler, 1994, about his book "This Year on Jerusalem", where he talks about his changing opinions on Zionism from 1943 through to the date of the interview.

• "I'm saying the Israeli's invoke the dead for mean political purposes."
• "I don't think most Israelis want to be on the West Bank as an army of occupation fighting Arab kids. It's very embarrassing."

And more cool stuff, like a discussion of the letters between Pierre Trudeau and Marshal McLuhan, comparing them to a Philosopher King and Court Philosopher, or a collation of opinions on selling movies, with Sydney Pollack, Robert Altman, John Milius and Roger Corman.

Apple, I am incredibly frustrated.

I used to have an iPhoto plugin that published photos to Flickr. When I last updated iPhoto, I was initially happy to see that iPhoto seemed to have the same functionality built-in. I created an album that shared photos to Flickr and used it for a few months, dragging new photos in when I wanted them uploaded. Then I needed to move stuff to a new laptop so I went and archived years-old photos and deleted iPhoto albums I didn't need.

Two weeks later, I find missing photos in my Flickr photo stream, and broken links on Ravelry. Slowly I piece it together: when I cleaned out iPhoto on the old laptop, it silently deleted Flickr photos. Gone are the uploaded versions, titles, tags, descriptions I wrote, comments other people left, and counts of views. I can recover the photos themselves but I can't recover metadata. Most difficult to fix are broken links on other sites.

How could the designers have possibly thought it would be a good feature to silently delete published photos that have metadata of their own? Do the words "Share" and "Publish" not imply putting the photos out there? I tried this again and the words "Synch" and "delete" are not in the iPhoto UI -- the fact that I granted iPhoto permission to delete photos does not imply that deleting an album will delete them online. After all, deleting an album in iPhoto does not delete the photo in iPhoto, so the UI has trained me to think of albums as selections of photos, not containers of photos.

Apple, you had the chance to fix this. When I go looking for support for this, I see posts from obviously distressed and surprised users about thousands of photos deleted, and workarounds posted in response. Surely this allowed you to fix the problem within the last year? You could have issued an update to Flickr with the feature "Saves you from unknowingly destroying personal information online", but you did not. I rather expect this kind of behavior from you Apple, your pride in infallible UI design is well-known, but that does not stop me from being disappointed.

Flickr, I'm disappointed in you too. I know this is not your fault, but many postings in your forums have asked you to disable iPhoto's permission to delete. You could have done that or made mass-deletes "soft-deletes" or asked the flickr account holder to confirm the loss of metadata. You, too, had the chance to save users from losing image metadata.

I've noticed an interesting link between art and software, specifically a parallel to the Scrum process and the existence of Certified Scrum Masters: there are ZenTangles with a specific process for creating art or doodles, and Certified ZenTangle Teachers.

Besides the terminology, both ideas are about regulating and confining what are normally considered open-ended, unbounded problem areas. In ZenTangles, there are arbitrary rules: how many strings to start your ZenTangle with, precisely what fine-tipped pen to use, and the most emphasized restriction, the size of paper is 3.5" by 3.5". Scrum often fetishizes details like planning poker, estimate roshambo, and post-it notes (full disclosure: I've participated deeply in the fetishization of stickie planning).

Then, both approaches have the practitioner routinely break down work into small manageable units and focus on completing units. Breaking down even further, Scrum has recipes for small parts of scrum (like this recipe for a retrospective) and ZenTangles have recipes for line and space doodles. Finally, both approaches celebrate the fact that the "finished" output (a sprint, interval or doodle) is small and done quickly.

Are Zentangles art? Does Scrum produce good design? When do you follow the limitations and when do you depart from them?

Updated: here's my first zentangle, doodled while I was writing this post! Not on 3.5 square paper, nor with the right pen, nor did I follow the "string" process first -- I was always bad at staying within creative bounds.

I downloaded and played Civ V last night. I watched through the high-quality animated movie intro, and noticed how carefully the old man, the hut, and the young man were culturally mixed. Their skin was brown, but could have been from suntans and wear/age. The old man's hair was grey, the young man's hair was hidden. The clothing (including a turban-derived headgear on the young man) was carefully invented and mixed inspirations from different cultures, as far as I could tell. I am convinced this introduction was designed to be culturally neutral, as if the old man and his son could have been the early leaders of any of the cultures used in Civilization.

So it finally struck me: it's all male. They couldn't make it gender-neutral.

The original Civilization had 13 male leaders and one female (Elisabeth I was the default leader of the English, though you could type in "Henry VIII" or "Tony Blair" or "Rowan Atkinson" if you felt like it.) I didn't really notice at the time. Civ II had a female and a male leader for each civilization, which was in-your-face obvious gender equality. It meant that some famous women got exposure (like Catherine for the Russians) but others were pretty unknown (I'm thinking Nazca for the Aztecs, and Ishtar for the Babylonians -- and can you even guess which Civ had "Bortei" as its female leader?)

This kind of public, forced gender equality was what finally made me notice once in a while what role females were expected to play in video games. Like being rescued in Super Mario, rather than being the rescuer. Back when I was just a teenage gamer, I didn't ever notice. I was oblivious. It still can take me a while before I notice, like I spent all that time noticing the careful cultural neutrality in Civ V before it finally hit me there was no attempt at gender neutrality in that introduction.

My point? Given how blunt and ham-handed the stereotyped roles for females in video games are, and how long it took me to start noticing, even though I am a woman, is an indication of how poor I am (and I know most people are) at even noticing gender bias.

I swatched for the Prince of Wales fair isle sweater recreation project (last post) before I even received the yarn I ordered. I used two skeins of Elemental Affects. I also decided to try mixing handspun with commercial fair isle yarn:



The smaller swatch is the one that includes handspun, pale blue, dark blue and dark green, along with the natural light and dark yarns used in the gauge swatch. I learned that it looks nice even when the yarn is not perfect. I also improved the tension on the floats.

Then I received the Jamison and Smith yarn from England:



First I noticed that the natural colour I'd hoped to use for the background was grey, and not at all fawn or beige. That's too bad -- the fawn colour was what I was aiming for. I didn't order enough of the fawny and beige colours to use much of any of them, although I did steal some "sh. FC45" from a Jamison and Smith kit I purchased in the same order. Since I don't want to order just a couple more balls, I'm going to alternate just one or two rows of the background colours at a time.

Since I wanted to use some handspun anyway, I blended an interesting roving with white to get another pale yarn option:



This looked really light when it was in a rolag, but on the bobbin and in the finished yarn, it got darker.



It's clearly the darkest of the light yarns:



It's also very close to the same darkness as the orange I need to use as a "dark" colour. The Prince of Wales painting has bright orange cartouche shapes on a fawn background as one of the peerie patterns, so the orange has to be considered a dark or pattern colour.

I wasn't getting far with using markers to choose colours, so I went to embroidery.



This worked really great. I was able to improvise a little as I went, removing two rows from the charted pattern to make it slightly smaller and make the big X's more visible. Click through to the Flickr image to see my notes on how I modified the embroidered sample as I went along. I also viewed the sample in black and white - I think the contrast is OK overall even though, or perhaps because, the contrast is not even. The orange peerie will draw attention because it's bright orange, the zigzag peerie will draw attention because it's got the darkest yarns, and the big motif will draw attention because it's biggest.

Next step, I believe, is casting on. Perhaps I should do another sample using the real yarns and testing the border ribbing, but I think it will be OK if I don't.

Hang on folks, I'm taking this blog on a sharp shift of topic.

I decided a long time ago to knit a traditional fair isle sweater for a friend who's a sharp, if dated, dresser. I'm finally getting around to it: I purchased Alice Starmore's book of Fair Isle Knitting along with another couple of books, and found the next piece of inspiration I needed:



This is HRH the Prince of Wales in about 1921, painted by John St. Helier Lander. The prince is wearing a fair isle sweater. It's been recreated before, but I'm still enjoying using it as inspiration.

My friend lives in San Diego, so I need to make it a vest. My next step was to take his measurements:



And try to design the actual stitch pattern.



I wasn't very satisfied with the usefulness of using markers. It just feels so far from the way the yarn looks. At least I learned not to use very much light blue in the background, if any, because it makes the background much less fawn-coloured than it ought to be. I also found out how many rows the pattern would be, and at the row gauge I expected, calculated that I could get 5 vertical repeats of the pattern. That's not quite as much as I'd like so I may delete a couple rows (the top and bottom row of the large pattern.

I also ordered a bunch of Jamison and Smith yarn for the project. I ordered the Winterberry Ladies Slipover kit, as well as a bunch of extra pale colours. Oh and I ordered another kit to make for myself later.

As I waited for the Jamison and Smith to arrive, I purchased a couple natural-coloured balls of Elemental Affects yarn. There's not as many colours and certainly not as many heathered yarns, but if I need some I should be able to mix both brands. I used the Elemental Affects to make preliminary swatches, which I'll take pictures of for the next post.

Should a constrained device be a RESTful server or a client? I had been assuming server, and I think I can justify this, although I'm not saying that choosing to put the low-power device in the client role is wrong, because it may depend on constraints and use cases. I do think it's a bad idea to require both client and server roles in the more constrained devices so I'm treating this as an 'either' choice, not and/or. Here is my rough analysis, as part 3 in a series on designing a REST framework for constrained devices (parts 1 and 2).

Roles: Reactive, storing, resource owner

To begin with let's separate the three roles commonly included in the definition of a "server". The first role: "a server is a reactive process" [Andrews, 1991] while a client is a triggering process. The second role: the server stores authoritative versions of resources. Most client/server remote file systems have both these roles in the server: the server reacts to requests and enacts storage operations, while the client is the triggering process making requests, but also controls the namespace has constraints on resource state. These client/server file systems scale moderately well but are inflexible and hard to extend compared to HTTP. In REST, the server takes on a third role: the server manages the namespace and resource state.

It's not clear how much mix-and-matching can be done with these roles in practice. Would it work reasonably well to keep the resource and namespace ownership together with the storage, but to make that agent also be the triggering process instead of the reactive process? I don't know examples of that kind of system in practice, nor do I know how to analyze a theoretical system against fuzzy goals like "flexible" and "scalable". But in the discussion of which roles to assign to the low-power device, I try to keep the three roles separate in case that helps shine light on questions like "should the low-power device proactively send requests such as notifications".

Benefits to consider

1a. Continuity: A server in charge of its own storage, namespace and resource model can be installed and host the same resources, responding to requests at any time, for years without changes. The conditions of its use can even change within limits. A Web site that, when launched, handles a few browser requests a day for certain resources, can later have some "mashup" service querying those same resources at automated intervals and extracting the data. Automated clients don't have this ability to be used in different ways without changing their configuration or code. Applying this to our low-power example: a sensor can handle requests from COAP gateways during normal functioning and by laptop-based clients during configuration, testing or development of new applications, without needing to know why it is handling any request (modulo authorization [1]).

1b. Flexibility: The flip-side of continuity is that features can be added. A device can be upgraded without disrupting the operation of the other low-power devices around it, because an upgraded device can host the same resources as the original device, plus new resources.

2. Scaling: In previous posts, I talked about how scaling large is related to scaling small, due to the relationship between power and load. The flip-side of scaling up to a large load handled with a fixed amount of memory and processing power (the normal problem for HTTP servers), is scaling down the memory and processing power for a fixed load (the scaling problem for low-power devices). In the Web today, what scales better, HTTP servers or HTTP clients? It's well-known that HTTP servers scale well, but there's little concern for clients scaling. It is hard to write a program that load-tests HTTP servers by implementing many clients over many connections -- all too often, the load-testing client machine(s) run out of resources before the servers do.

In HTTP, the server is stateless (see where Roy's dissertation describes "Client Stateless Server" and "Layered Client Cache Stateless Server" for the fuller picture), but the client may not be. The client needs to figure out which servers to talk to, what resources to request from the server, how to interpret those resources and what to do next. In Web browsers, it's the human user who makes many of those choices and may have mental state. An automated client might well include a state machine as the client attempts to achieve a goal involving multiple resources and perhaps even multiple servers. At a minimum, the client knows "What page I'm on now" and "what context did I make a resource request in" as part of its state. In contrast to the client, the server can be stateless, reactive, and doesn't need to know who it's going to communicate with as long as they're authorized [1]. With good application design, a server ought to be implementable with very little use of dynamic memory since it is stateless.

3. Naturalness of resource model: the most natural thing seems to be to model sensor readings and the capabilities of the most constrained devices as resources. Further, if the sensor is used for many years without being upgraded, those resources can have extremely stable URIs. The most natural agent to own these resources is the sensor. This related to the flexibility benefit because of the naturalness of extending the resource model. A sensor v1 can have it resource with a set of readings, and a new device with additional readings can simply have additional resources. A multi-purpose sensor can have all the resources that each single-purpose sensor would have.

4. Naturalness of user model: User interfaces are on the side of the triggering process or client as the user initiates requests for information or changes of state, and their client sends that out. It will be very rare for a human to push a button or otherwise interact directly with a sensor and cause it to act as a client. Much more common will be human requests on a laptop or control panel, which can act as a client and send a request over the local network for sensor information.

---

To conclude, I think these roles and benefits fit together sensibly for a particular kind of constrained device. I am probably focusing too much on a particular idea of a constrained device and its use cases, so I'm happy to admit that this is not the answer for all devices and use cases and would be interested in hearing analysis about other cases. I'm also getting some good comments to the past two posts, so at some point I foresee a post in this series just to discuss the comments.

---

[1] Authorization does not necessarily break any of the assumptions of statelessness. For example, authorization decisions could be pushed down to a lower layer in the simplest cases: the authorization to communicate with a sensor is the same authorization needed to request its sensor data. In more complex cases, the authorization decisions can be explicitly made above the COAP layer but this is more code and storage on the low-power device. In any case, this investigation is starting to make me think that authorization should either be a network function or an application function, not a COAP function or at the resource transfer layer.

REST, with proper use of hypermedia, can be very appropriate for constrained devices. In my last post, I talked about how HTTP has a lot of cruft that could be removed if one were to design a HTTP-lite for constrained devices. The REST architectural style works for constrained devices a lot better than the HTTP syntax and HTTP feature list do. Naturally I'm working from Fielding's thesis, but some different points and emphases are warranted for the context of constrained devices interacting with automated agents, as contrasted to the user deciding which HTML link to click on to interact with a Web server. In order to dive into this, we'll need to really understand how REST requires using server-controlled hypermedia to advertise and navigate the server's resources.

REST involves

• Stateless design. That means no state machine thinking! The client can't assume server state. [1]
  
• A uniform interface on resources, similar to CRUD
  
• Caching support for resource representations
• Navigation to server-named resources via server-controlled hypermedia

The last point is often misunderstood. Despite the fact that we call HTTP client messages requests, it's far more common for programmers and engineers to treat them as commands or instructions (SOAP and RPC illustrate this very well). Many APIs designed to be used over the Web assume that the client decides what should be done and tells the server to do it. This isn't RESTful, and it's not just a quibble: putting the client in control of the interactions puts the client in control of the server's performance and ability to scale.

I didn't understand this when I first read Roy's thesis or when I first worked on WebDAV. I needed examples to make sense of the principle and its utility, particularly when the server does not use HTML and the client is not driven by an active user clicking on links. Thus, this post has examples of non-RESTful navigation and state discovery, as well as RESTful examples:

• The Twitter API: not RESTful in resource naming or discovery
• WebDAV: somewhat RESTful but made a few mistakes in extending HTTP
• Atom: quite RESTful application that sits very lightly and comfortably on HTTP

The Twitter API has fixed resources. Those resources have URIs state which are known to the client in advance. For example, the account profile update resource is named "http://api.twitter.com/1/account/update_profile.format". This limits the Twitter API's ability to rejigger its namespace to add features or to scale by separating resources along a different axis. Next, the client must know a set of permanently-named parameters to be used in a POST request to the URI, to update the profile on the account. The programming model is clearly that twitter client controls the Twitter server by sending commands over HTTP POST. The idea of CRUD is vaguely there (many of the API resources can be retrieved in full with GET, others updated with POST) but diluted by the invention of special update URLs. Caching is possible, at least.

WebDAV is an IETF standard that follows the idea of a limited set of consistent methods well, but doesn't do resource discovery by server-controlled hypermedia. WebDAV set out to provide more authoring functionality such as functions to organize resources and interact with a file system model, so that Webmasters and other Web content creators could have something more suited to Web development than FTP. The WebDAV designers created the PROPFIND request which works a lot like most remote file systems file queries work. In other words, it puts control over querying server resources in the hands of the client. The client determines the scope of the request and the list of properties to be returned, and in the first implementations of WebDAV, the server had little choice but to comply or fail entirely. WebDAV servers had lots of code to handle all the different possible variations on the PROPFIND request, parse its body, and scale differentially based on the different characteristics of the properties that might be requested. All the information to reply to any PROPFIND request that the client might construct, had to be available to any server that might have that request made. Of course, this approach seriously affected performance and scaling.

Atom Feeds was the first standard I saw that did user-free resource discovery via server-controlled hypermedia. An Atom feed is a document offered by the server in XML format, containing links in semantic markup. So, instead of querying the server for blog entries that match a certain pattern, the client simply asks for a feed document via GET. With Atom, the server can precompute or do lazy computation of feed documents, can cache them, can break them down when a feed gets large. This design leaves the scope and detail level of collection membership documents entirely in the servers hands.


More examples can be found at a site that classifies APIs along a continuum of RESTfulness. My quibble with that site is that not all APIs fall along that linear spectrum. WebDAV would have a green box for RESTful identification of resources and a green box for self-descriptive messages, but it would have mixed results for manipulating resources through representations, because collections are not manipulated through representations. It would also have mixed results for using hypermedia "as the engine of application state" because the client makes assumptions about giving resources URIs based on what collection they were put in.

Clearly I need to work on a third post in this series and perhaps a fourth, because I would still like to talk about why constrained devices should operate in the role of the server and provide documents similar to Atom feeds about their state and data. I also have thoughts about a hypothetical framework and how it could be applied in a specific type of constrained device in a way that may not need any dynamic application memory. Hope you can stand to wait.


[1]I think part of what took me so long to understand this was the amount of meaning packed into the word "state". Not only does this mean "what state is the server in, among the states in its state machine", but it also means "what are things named" and "what things exist". So unpacking the injunction that clients shouldn't presuppose state, also means that clients shouldn't presuppose names or existence of resources.

It's not entirely intuitive, but the same architectural style that allows Web servers to handle large scale usage and iterate so successfully, may also help constrained devices (like sub-10$-material-cost sensors) to handle small scale loads and power usage and avoid being upgraded for many years.

A lot of discussions of using REST in constrained devices have glossed over whether it's really interoperable HTTP that's being used (e.g. compressed HTTP) and whether the low-power device would be in the role of the client or the server. My opinion at this point is that for truly constrained devices, compressing HTTP itself would be the wrong choice, and that the device must be in the role of the server. The rest of this series explores why I believe that and should lay open the assumptions that might allow somebody to correct my information and revise my opinion.

The first part of the series attempts to explain is why HTTP is the wrong transfer protocol choice for some constrained devices. In theory it's a massively extensible protocol, with ways to extend the operation set, upgrade versions, add status responses, add headers, add resource content types and so on. Again in theory, almost all headers can be combined with each other and with different methods unless forbidden. In practice, the deployed base makes some HTTP extensions, and particularly the combinatorial expansion of extensions and optional features working together, quite difficult, and much special-case code must be written to handle the cases where widely-deployed software fails to implement some detail correctly. Here's a few examples of problems we've had over the years.

1. In developing WebDAV, we tried to use the standard "OPTIONS *" request (the special-case URI '*' means "give me the options or capabilities of the server"). However, we found that this is poorly implemented. Java servlets, for example, did not support the '*' URI. Other HTTP frameworks supported the '*' URI but did not allow the response to OPTIONS * to be extended by software extending the HTTP framework. Ugh.

2. HTTP has several different ways to indicate the content-length (historical reasons as well as for different use cases). One is the MIME type "multipart/byteranges". Another is chunked transfer encoding, where the server (and thus the client) does not need to know the total length until the last chunk is transferred. A third is "Content-Length". A fourth is for the server to terminate the connection, although this feature opens up the possibility of truncation attacks. Some of these methods work poorly with TCP connection continuation. Both clients and servers have to support almost all of these.

3. Both absolute and relative URIs are allowed in several locations, even though they're not a good idea in all locations. The resolution of a relative URI can be tricky and complicate client software. Implementations of this make it worse; e.g. the "Location" header was defined to allow only absolute URIs, but many implementations have been found which use their generic URI parsing or generating code to allow relative URIs in that field as well.

4. Parsing headers with quoting, separator (comma and semi-colon), whitespace and continuation rules is difficult. See http://greenbytes.de/tech/httpbis/issue-14.xhtml, http://greenbytes.de/tech/httpbis/issue-30.xhtml, http://greenbytes.de/tech/httpbis/issue-62.xhtml, http://greenbytes.de/tech/httpbis/issue-77.xhtml etc. Some headers defined in other specifications besides RFC2616 even used the syntax rules incorrectly and have to be special-cased.

5. The support for the Expect: header and 100 Continue response has never been good. In theory it's required, so there's no way for a client to ask if a server supports it, thus the client could end up waiting quite a while before giving up on the server initial response. Instead quite a few clients ignore the specification text and start sending their request bodies right after the headers including the "Expect" header, without waiting for the server 100 Continue response. This kind of feature also makes it harder to integrate authentication (what happens if the client uses this header when an authentication handshake needs to be initiated? ) and connection/transport features, as well as to implement intermediaries.

There are many more examples in a long issues list (http://greenbytes.de/tech/httpbis/index.xhtml), and some of the discussions of these issues get quite lengthy in considering how features are actually implemented and how they work combined with other features.

Developing a very simple HTTP client is simple, particularly if the client is talking to a single known HTTP server or is making only a small number of kinds of requests. Developing a limited-use HTTP server can be fairly simple too, although it gets to be significantly more complicated if the developer actually tries to handle all RFC2616 features and all unknown headers and methods properly. What turns out to be very hard is building an HTTP client library, server library or extensible server, because these are used in unexpected ways. It's easy for a developer using one of these libraries to break things, e.g. by manually adding a header that doesn't work with the headers added by the library. The library has to support using TLS and not; several kinds of authentication, several extensibility mechanisms and many failure modes.

The HTTP Components project at Apache talks about the many flaws and excessive simplicity of most HTTP client libraries, and states that "This is one reason why Jakarta, and other free and commercial vendors, have implemented independent HTTP clients". In other words, code re-use is seriously reduced by the way HTTP must be implemented. Some software companies are still selling HTTP client libraries to fill implementation gaps.

General-purpose HTTP servers -- ones that work for a large number of resources, a large number of requests, support TLS, content-negotiation, cache-control, redirect, authentication and other major features -- are even harder. When well implemented, HTTP server farms scale tremendously well. But much, much effort has gone into making those work.

When we look specifically at constrained devices, we see a much more limited set of use cases than the overall Web offers.

• Documents are not large! Thus, document range and response continuation features are not desired. Only one transfer-encoding should be necessary, and conditional request features won't be worthwhile.
• Documents from constrained devices are not intended for direct user consumption. There is no need for language and charset negotiation.
• Even negotiating content type may be rare. A constrained device will state what it supports and not negotiate.
• A constrained device will never act as a proxy, so need not support for Via headers and a bunch of status codes like 504. Further, if we define a non-HTTP, non-proxied transfer protocol that can be converted to HTTP at the first proxy step (or converted from HTTP at the last step), then the non-proxied transfer protocol doesn't need any proxy features at all.
• Redirects are not necessary or could at a minimum be drastically simplified, making most of the 300 status responses unnecessary.
• All of the 100 level informative status responses are unnecessary.
• The authorization model for constrained devices is quite different than the authentication model assumed in HTTP (and associated logic like the 401 status code behavior is therefore unnecessary). Access control will be simpler than in current Web servers.

Imagine instead of a messy HTTP client library or server stack, we had a protocol with about a third the features, less extreme extensibility and simplified parsing. If well-specified and interop-tested early and often, I imagine such a protocol could be implemented in framework form in a tenth the size of an HTTP server framework. In even more constrained cases, a very specific constrained transfer protocol implementation (e.g. one which supported a subset of CRUD operations and did no optional features) could be 1/100th the size of a simple HTTP Web server. That starts to be an appropriate size for burning to a chip and shipping in a $1.00 to $10.00 device.

I tried to get some sanity checking on my estimates, and it's not easy. A Web server can sometimes be a wrapper on a file system, in which case most of the logic involved comes "free" from the file system. For example, a simple implementation of one style of redirect might be a trivial number of lines of code if the underlying file system supports symlinks. Andrew McGregor pointed me to the Contiki operating system for embedded objects, which has a Web server already. So that's a proof that embedded and limited devices can sometimes do straight HTTP.

In sum, if interoperability, flexibility and executable size are all simultaneous concerns, HTTP as-is will pose problems. While it may not be necessary for all devices to use a more space-efficient REST transfer protocol than HTTP, it may be necessary for some. If it is necessary to do a new transfer protocol, it should be possible to design a protocol an order of magnitude smaller just by cutting features and unneeded options from HTTP; and possibly smaller yet by being very careful about parsing and limiting extensibility.

More later on REST itself scaling down.

A year in knitting: I knitted 6 miles of yarn in 2009.


Considering how much I spun in 2009, I'm pretty pleased. My stash definitely grew due to the spinning & gifts and purchases (5 miles purchased at Stitches West Feb 2009 alone, mostly not knit up yet).

1. Alpaca socks for Laurie (modeled), 2. fit of pathways 2 socks, 3. Zombiegurumi, 4. Vilai sock, 5. Prairie glass scarf, 6. Trekking sock, 7. Replacement travel shawl, 8. First handspun sock, 9. Amsterdam socks, 10. YAPS, 11. Panda wool retro rib, 12. Lavender shimmer scarf, 13. Petroglyph hat, 14. "Dove wing" icarus closeup, 15. Cowl from handspun., 16. pnw shawl on hangar, 17. zigzag fold, 18. Hawt sweater 1, 19. Malabrigo shawl.JPG, 20. Auction scarf 3 closeup

Defining a new URI or URN scheme properly turns out to be really difficult. I've been sponsoring drafts for such schemes for almost four years, and the same problems come up again and again. The major themes:

• Comparisons
• Uniqueness
• Stability

Comparisons

Comparing one URI to another turns out to be a commonly desired feature. Browsers look up cached pages based on URI comparison. If I click a link to bookmark in delicious.com, I'd like it to be bookmarked only once and if I've already bookmarked it, bring up that page so I can see what I tagged it with and when. Outside of http URIs, I'd still like to know if I'm already subscribed to an XMPP user, etc.

Comparison is harder than it sounds, but you already know that if you've dealt with any code requiring canonicalization, conversion or encoding. If a link in a Web page contains a space, at some point my browser has to convert that space to %20 to use in an HTTP request. Should the browser do that conversion before or after looking up the URL in the cache? Should delicious.com bookmark the URL with the space or the one that's used in HTTP requests? This is the tip of a very large iceberg that potentially includes all of internationalization and Unicode. Be very clear on what character set is used by each part of a URI, and if it's all ASCII, say so.

Case sensitivity is a frequent issue. Be very clear on which parts of the URI are case sensitive.

For new URIs, giving options makes the comparison job much harder. Let's say a new URI scheme needed to include a country designation: it seems nice to let users put a two-letter country code, a three-letter country code, a TLD or a OID in there. Only now one needs a horrid table to convert and compare these, and string comparison is no longer enough.

Optional syntaxes are similarly difficult; even allowing for '/' vs '\' can lead to error.

When the URI form is an alternate form for an identifier that already exists, now the URI may have to be comparable to something that's not a URI. For example, both IRI and URN forms exist for ISO OIDs. Don't they need to be compared to each other?

Can the URI form have query syntax? Is that part of the comparison or is that stripped off first? In HTTP URIs if I stripped off the query syntax I'd retrieve quite a different resource, but in some URI forms, the query syntax is used to carry information other than resource-identifying information. For example, can I compare two mailto URIs that have the same mail address, even if one of them has a query part with "?subject=Hey%20There"?

Can the URI form contain multiple values? The SMS URI definition had to include text on comparison when multiple SMS addresses were packed into the same URI. Does order matter to comparison?

Uniqueness

Frequently URI schemes need to avoid collisions, so that there isn't an attempt to give two different things the same identifier. The problem here is delegating the ability to create new URIs, while still avoiding collisions. The major fallback here is the DNS: URIs that contain a domain name, where the resource being named belongs to that domain, effectively delegate the uniqueness concern to that domain holder.

For example, we don't need to worry that "xmpp:lisa@jabber.org" will conflict with other resources, because the domain 'jabber.org' assigns usernames uniquely within that domain and prevents collisions.

The other main option is to use registries. For example, all OIDs, defined by ISO, use the ISO process to register numerical values and string values for use in the parts of an OID. Other times, IANA is the registry (e.g. for port numbers in HTTP URIs). If there is a new registry needed by the URI, this is more work and more to get right.

Some processes for ensuring uniqueness are quite heavyweight. Many IANA registries have processes which can take weeks or months to resolve. If the registrar is not IANA, who is going to actually run the registration process and under which rules? The OGC URN defined in RFC 5165 includes sub-namespaces issued by OGC itself . The first consequence of this is that the OGC organization must be referred to for any new OGC URN unless it explicitly delegates that part of the namespace. To reduce the burden of being a registrar in the case of non-permanent, test or experimental OGC URNs, the URN definition mentions the possibility of an experimental sub-namespace and the possibility of collisions within that namespace. Now implementors have to consider the possibility of leaked experimental names and dealing with collisions. The approval discussion of RFC5165 was lengthy, because of these nuances.


Stability

Some URIs need to refer to the same thing over only a short time, but typically the desired stable period is long or even longer. Domain names can be a problem here. Initially it might seem great to use HTTP URIs as XML namespaces, but consider whether the holder of the "example.org" domain will change over time, and whether the new holder will have the same policies regarding use and allocation of URIs in that namespace.

If a registry is used to achieve unique assignment, and the registrar is not IANA, then the stability of the registry must be considered. How long is the organization going to exist and maintain the registry publicly? We look for a public commitment, existing Web pages, a long-lived organization and so on. An explanation of the process and deciding factors for how names are assigned and how the organization ensures they are not reassigned, shows that they've thought about this commitment. See RFC 5328 for an example.


Random List of Gotchas

1. Does your URI scheme include use of fragment identifiers (like the #iri part of http://example.org/faq#iri)? Forget it; fragment identifiers relate to the media type of the *resource*, not the type of the URI. So if the URI "foo:bar:baz" retrieves a HTML page, then the fragment identifier would act like a HTML document fragment identifier.
2. ABNF is hard to get right. Get it reviewed by an expert. Use a ABNF generator or something like that to test your instincts. Refer to existing productions where possible. One common issue is to use a separator like "=" between two constructs, and then define one of those constructs in a way that it includes the separator character itself.
3. Another ABNF/syntax issue is to accidentally use a character that has an obscure meaning in URI syntax, or is simply reserved.
   
4. URIs that contain phone numbers include a whole barrel of troublesome monkeys. It's so hard to get telephone numbers right, with variable length and special encodings, '+' prefixes, dashes or spaces, and extension numbers, that it's worth trying very hard to use an existing phone-based URI instead of defining a new one.
5. If query parameters are used, can key values be extended? Can new key value be defined by anybody? Do they have global meaning (like mailto URIs) or purely local meaning (like HTTP URIs)?
6. The "community considerations" section required for URN registrations is frequently misunderstood. What the IETF looks for in this section is an indication that the work done to standardize this scheme and allocate a new scheme or URN type will add value; that there is benefit to the Internet community and not only to a private consortium or private company.
7. If any reviewer blithely says "whyever invent a new URI scheme, use HTTP for everything!" just ignore them until they provide actual reasoning for this proposal.
8. Embedding URIs within URIs, or any syntax that is infinitely extensible, is asking for trouble.

Final advice; provide more examples of actual URIs or URNs than you think people will need. Along with an example, explain how that example would be assigned, derived, and if applicable, dereferenced.

References

Here are the documents and registries that govern the registration and syntax of new URI schemes and new URNs.

• RFC3406:How to define a new URN namespace or "NID" or "Namespace Identifier".
• RFC2141: URN syntax, or the syntax of URIs that begin with 'urn:'.
• RFC3986: URI syntax, how to parse all URIs and URNs regardless of scheme.
• RFC4395: Guidelines and Registration Procedures for new URI schemes.
• IANA Scheme Registry: Existing registered URI schemes
  
• IANA URN-NID registry: Existing URN Namespace registrations
  

I remember reading Neal Stephenson's Snow Crash and his description of the Metaverse, his conception of virtual reality and online communication, thrilled me. I knew in many ways it was more realistic than Gibson's cyberspace. For instance, Stephenson described how people can choose their own avatars and it's a sign of a newbie or at least a non-programmer to have an "off-the-shelf" avatar, and indeed we see this in places from static online forums all the way to Second Life

One thing nagged at me back then: Stephenson realized that there's no reason not to teleport in virtual reality, but explained that the programming rules forbid it.

You can't just materialize anywhere in the Metaverse, like Captain Kirk beaming down from on high. This would be confusing and irritating to the people around you. It would break the metaphor... Once you have materialized in a Port, you can walk down the Street or hop on the monorail or whatever.

This is unrealistic in a virtual reality which is supposed to be the predominant way hackers like Hiro interact with each other online. Today, online gamers tolerate some limitations on teleporting in game environments like World of Warcraft or Puzzle Pirates, but even there, friction caused by do-nothing travel time is minimized. And in a more general communication milieu -- Web forums, Facebook, Twitter -- there isn't a single, limiting place. I can "be" on two forums at the same time on Ravelry, open two or more Facebook windows and chat with multiple people and I'm "there" with them all for some value of "there". Not only is there the ability to go immediately where I want to be in most online fora, but it doesn't even involve leaving the other "places" I already am.

Ok, here's another piece of the picture that didn't bother me in 1993 but does today:

Most avatars nowadays are anatomically correct, and naked as a babe when they are first created, so in any case, you have to make yourself decent before you emerge onto the Street... [Hiro sees] A liberal sprinkling of black-and-white people -- persons who are accessing the Metaverse through cheap public terminals, and who are rendered in jerky, grainy black-and-white.

This assumes an architecture where the client renders their own avatar. Even in that architecture, a proxy for a public terminal could render a classier avatar. Low-res displays would more likely affect the receiver than the sender -- somebody accessing the online universe through a poor public terminal might see every other avatar equally low-res, but their own avatar could still appear fantastic to people on good computers. It's complicated.

I guess the lessons are that today's online fora are less like the real-world than we could imagine fifteen years ago, and future online fora are less like the real-world than we are yet capable of imagining. We're still sending messages that look like paper mail and have envelope icons, and we still think of "bulletin boards" as a real model. We haven't integrated IM or twitter-like experiences fully into other experiences. Today, I'm downloading the Adium beta to see how twitter "what I'm doing" messages and community are integrated with IM and whether that improves on the old IM concept of presence in a significant way. Trivial interface changes in these sites and software can be significant in how people use them.

To borrow Ted's analogy when we touched on this over coffee today, we're in the same phase cinema was in when a movie camera was pointed at a stage, and a stage play acted upon it: the unique affordances of cinema weren't discovered immediately and are still being discovered even today. With online interaction, we're only beginning to discover how different it is from experience in the physically-limited real world.

I have a bunch of baking books, or cookbooks that include serious sections on baking. The Joy of Cooking and The New Best Recipe are my favourites by a long shot, and often I enjoy making the "best" scone even if the recipe is basically white sugar, white flour and a pound of butter.

However, sometimes I'm looking for a healthier scone, muffin or coffee cake -- something that I can eat for breakfast without too much guilt, or offer to health-conscious friends -- and I don't have resources that are just right for me. Ideally, a book on healthy baking would balance out a number of factors without being fanatical on any one of them:

• How is the whole grain content? Can some of the white flour be replaced with wheat, or can the recipe handle an optional addition of wheat germ, ground flax seeds, oats or so on?
• Can the sugar be cut down and/or replaced with honey or maple syrup?
• Can the fat be cut down without sacrificing moistness, shelf life, texture and flavour?
• Is the protein ratio good? Is substituting soy flour an option? Adding nuts?
• Are the ingredients readily available or can rare ingredients be optional?
• Is the taste pumped up? I eat less of pure, dark chocolate or tongue-tingling ginger sweets because my palate is satisfied earlier.
  

I understand some people get fanatical about one thing, just the sugar or fat or whole wheat content to a recipe, but I rather think balance is important and certainly taste is.

Along these lines, here's an adapted recipe for Mango Chutney coffee cake, derived from Light and Easy Baking. That book focuses only on reducing fat content, which I brought up a little again, but the pumped-up taste is there and the hot pepper is surprisingly good.

2-1/2 cups all-purpose flour
3 t. baking powder
1 t. salt
1/2 c. sugar
1/2 c. brown sugar
1 c. milk
1/3 c. canola oil
1 egg, slightly beaten
2 T. orange marmelade
1/3 c. raisins
3/4 c. chopped mango chutney
Additional pepper, cinnamon or cardamom, particularly if chutney is mild

Mix the dry ingredients together then mix the rest in. Bake in a loaf pan at 350 for 65 minutes.

Quick mommy blogging, just to say I'm still here.

A couple days ago I hear a scream and "Get it off me!" from the two year old in the next room. I go running. It's a piece of sticky fluff on his index finger from him poking under the furniture. Sarcasm kicks in but doesn't work:

Me: "Oh noes! It's a disaster!"

Him: "I has a zaster on my finger!!"

We had a really great IETF APPs area meeting today. We invited a whole bunch of people to talk about their topics and moved through the presentations quickly. These topics were:
- HTTP Resource Discovery
- Service/server Discovery
- Timezone publication
- SCRAM (Salted Challenge Response Authentication in SASL)
- Bayeux and cometd: JSON pubsub over HTTP
- BOSH for tunneling XMPP over HTTP
- rHTTP: reverse REST
- Analysis of all these server-initiated HTTP schemes plus Web Sockets
- Massive Multiparticipant Online Experience: the Overview

The slides are all already available here. Here's my favorite, a slide from Mark Lentczner's deck, for tying a whole bunch of things together. Even though Tufte would probably cry as Mark said.

I'm trying to read some code in Objective-C. This is hard, but it's solidifying my abstract understanding of programming languages. I'm not that hardcore a programmer, but I guess I've picked up a few things over the years (gawd that makes me sound old).

One of the neat things about Objective-C is that using a class or instance method involves sending a message. C++, in contrast, calls those methods. A C++ object has a fixed number of methods that can knowably be called. An Objective-C object might be able to handle arbitrary messages. This makes some things harder and somethings easier: polymorphism is easier; finding cases of using the wrong type of object or having a null object are harder to detect (must be done at runtime, not compile time).

The thing is, this is very familiar to me because this is how wire protocols work. In fact Objective-C has "protocols" which are interfaces, or a set of messages, that an object claims to be able to handle, so the terminology overlaps quite a bit. Anyway, in a wire protocol the client sends a message, and because anything can happen to that message, the client has to be able to handle a large number of outcomes. Polymorphism? You bet; a server that appears to be a HTTP server (implements the HTTP protocol) can also be a WebDAV server, a CalDAV server and an FTP server.

Designing protocols can be hard for people who think in terms of fixed interfaces à la C++. RPC-style protocols embody this thinking, making Remote Procedure Calls and expecting predictable, limited results. It makes more sense to me now, that RPC-style protocols are so brittle: designers and implementors are acting as if there's compile-time checking of the remote interface, whereas since the remote interface is on somebody else's computer that may have been upgraded or may just have a different implementation, of course there's no compile-time checking.

Mommy blogging today: Natasha said I should post about toddler sleeping stuff.

I have a kid that naps and goes to bed willingly and easily. Clearly there is a huge part of sheer luck in this because I've seen little correlation between loving, wise, firm parents and perfect kids, and I'm not always wise and firm. But we did luck out on a few things that have added to his personality to make for the easiest bedtimes ever.

1. We introduced an attachment toy early on. This toy, known here as "sleepy bear" because his eyes appear closed, is a blanket-with-head style minky toy.
2. We taught the sign for bear early on (bear hug yourself with arms crossed, and scratch your upper arms with each opposite hand) so he could ask for the toy pre-speech.
   

3. We attached a pacifier to sleepy bear with a folded strip of fabric. The fabric loop goes through the pacifier loop and around the pacifier, so it can come off for easy machine laundering of the toy.
4. We bought a second identical sleepy bear (and attached another fabric loop) when it became clear this was the favourite toy. Usually one remains hidden to be brought out very conveniently when there's contamination events or simply the bear has gotten too dirty from grubby hands.
   
5. Since sleepy bear is always "sleepy", he has to stay in the kid's bedroom most of the time. We make exceptions for when he's sick or at difficult times like coming home in the car after bedtime.
   

So now, when we say "It's bedtime" his response is (whining) "Nooooo...." but the next phase is "Let's go find sleepy bear" and he responds "OK" and follows us to the bedroom. Sleepy bear is closely associated with sleeping and triggers him to lie down and relax.

He is too young to say recognize fatigue and say "I'm tired, I'm ready for a little nap" but he's easily old enough to ask for bear. If we're at home and his hands are clean, we ask him to go into his bedroom and cuddle with bear until he's ready to come out without bear (and if he comes out with bear we bring him back to the bedroom and ask him to say bye to bear before he can go out and play again). So yesterday morning he did this and actually fell asleep, getting a bonus morning nap which he doesn't usually need any more.

When we want him to fall asleep in a new place (traveling or spending an evening at friends') we just bring sleepy bear. We pull out the toy and any old blanket, and that's enough for the kid to sleep in a new room fairly easily.

Finally, this somewhat limits pacifier use without ruling it out entirely. I don't care too deeply, but there's something annoying about a kid that talks through a pacifier all the time. Having the main pacifier attached to a toy limited to the bedroom means that most of the time when he's playing he doesn't have one. We do have a couple extras on leashes for carseat or stroller travel where it appears to seriously improve patience levels.


I don't mean to give advice because all parents are different and all kids are different, but I did agree it was worth explaining how this works for us. Good luck!

Messaging Architects put out a very nice press release about my joining the company. I used it as a bit of a soapbox to talk about what makes a fully Open Standard: free to read, free to implement and free to participate in.

I now work for Messaging Architects. I started a couple weeks ago but it's been busy; I traveled to Montreal last week to visit HQ and meet the management team.

It's going to be a fun job. The company is smallish (small enough for everybody to be on IM and see each other) but growing and building its product line. The M+Guardian product does spam control and other policy enforcement on email in transit, and I can definitely get behind spam control. The M+Archive is a bread-and-butter product for any company that has to follow regulations on email retention, which is a growing number, and I like the focus on swift retrieval. The M+NetMail email server was aquired from Novell a year ago and the company is now putting its stamp on the product (the team in Utah, who I met last October). In addition there's calendar integration, which you know I'm interested in, and possibly some file-sharing technology.

There's a lot of attention to customer needs at Messaging Architects, and a lot of enthusiasm and dedication. It's not hard for that to rub off on me even working from my own home! I'm in the midst of establishing a more fixed and attractive working spot at home, getting on IM with all my co-workers, joining regular meetings and getting the products running myself. Of course, I'm context swapping this new stuff with ongoing IETF work as I continue to handle the Applications Area Director responsibilities.

Thank-you to everybody who was looking out for me during the job hunt and if you're hunting, may you be as lucky as I.