tag:blogger.com,1999:blog-34549842024-03-07T00:00:45.063-08:00Not Invented HereUnknownnoreply@blogger.comBlogger610125tag:blogger.com,1999:blog-3454984.post-15374791031749870472016-05-12T10:25:00.001-07:002016-05-12T10:25:44.495-07:00Voting, consensus, and the tyranny of structurelessnessI am in a new organization with growing pains. We all want to work together but nobody knows how to make decisions in this organization. The funny and frustrating thing is that everybody is mature, experienced, smart and well-intentioned -- but their experience making group decisions is mostly limited to pre-existing institutions or small groups making simple decisions. In a medium sized group, members can do all the right things to propose a vision or launch a project, the things that would work in a pre-existing institution with processes for decision-making, but fail to close the deal.<br />
<br />
It's everybody's fault and nobody's. We are all hesitant to push forward decisions or to suggest processes that may seem like taking over or being bossy. We are seduced by the idea that we should just be able to agree, that we don't want to become "process-heavy". <br />
<br />
Many of us have worked in and been unconsciously formed by organizations where decision-making is disguised. I've worked in companies where many of the outward signs of consensus are there, where engineers suggest and criticize freely, but the team unconsciously looks to the boss for ratification of the plans that have emerged. Often the boss encourages one direction or another all along. Many engineers in these organizations claim they are consensus-based. They're not. <br />
<br />
Similarly, we often think of small groups picking a spot for dinner as a consensus process. What commonly happens is the instigator or organizer becomes the tacit decision-maker, making a choice based on input such as one diner's suggestion of French food. Tacitly, the person who hates French food knows to appeal to the organizer for a different decision. One person may opt out without crippling or destroying a complex undertaking with lots of individual investment. In small-group simple-issue decisions, the organizer may ratify a consensus, but it's not a consensus-based process, it's a single-decider process which happened to arrive at a consensus.<br />
<br />
In this new organization with growing pains, we don't know who to look to for official encouragement or ratification so proposals tend to wither. The members who listen to a proposal are participating in this withering. We listen politely and give feedback and ideas but we don't give the same kind of stamp of approval that a nod from a corporate manager would give. We don't help form or push the decision because we're too busy, or it might seem like taking over (being bossy again).<br />
<br />
After a while, we all finally agreed we need more process, but few of us have experience building that from the ground up without a boss. The idea of a consensus-based process is both appealing and scary. It's appealing because it seems like it would be lightweight process with few rules. It is scary because we fear that one person can block a consensus and the group would either fail or at least be injured by the bad feelings that would emerge. Both the appeal and the fear are based on deep misapprehensions.<br />
<br />
The <b>fear</b> of a consensus process failing is because only one of us has spent much time working in a true consensus process with enough structure to avoid destructive failures (a proposal may properly fail but the organization succeed; however if the organization fails that is destructive). The <b>appeal</b> of consensus as being a lightweight process is also wrong, because a working consensus-based system is a lot more work and process than most of us know. Some of the structure we would need in a medium-sized group, 10-50 (1):<br />
<br />
<ul>
<li>Define how to bring an idea to the group and get approval that it's worth putting effort into.</li>
<li>Define how the group demonstrates enough approval. Nods, hands up, hums? Approval votes? </li>
<li>Define how to put effort into an idea to make a specific proposal to get consensus on. </li>
<li>Define how a proposal is documented and put to the group, reviewed and commented on, for how long, and how to iterate versions of the proposal.</li>
<li>Define how somebody can raise a serious objection before a consensus call, to modify the proposal before it becomes too solidly formed.</li>
<li>Define who makes the consensus call, when, with how many members. Can subgroups make some kinds of decisions? How are agreement, indifference and objections registered? </li>
<li>Define what the consequences of a sustained objection are. If a full consensus is required, then how are hold-outs dealt with? Can hold-outs be kicked out? Does that require consensus?</li>
</ul>
<div>
In addition to structure, the group would need good consensus habits, such as showing encouragement of somebody with an idea that they should turn it into a proposal, and helping build the proposal; helping stay focused, helping objections be raised early and honestly. Without this much structure, good habits and shared practices, a consensus-based process without a tacit leader is not strong enough to avoid the <a href="http://www.jofreeman.com/joreen/tyranny.htm"><b>tyranny of structurelessness</b></a>. </div>
<div>
<br /></div>
<div>
Take a look how old that article is, and how resonant today. I really want to copy some points out of it: </div>
<div>
<ul>
<li>'The "structureless" rap group was an excellent means to consciousness-raising [paraphrased]. The looseness and informality of it encouraged participation in discussion, and its often supportive atmosphere elicited personal insight. If nothing more concrete than personal insight ever resulted from these groups, that did not much matter, because their purpose did not really extend beyond this.'</li>
<li>'There is no such thing as a structureless group. Any group of people of whatever nature that comes together for any length of time for any purpose will inevitably structure itself in some fashion'</li>
<li>'The inevitably elitist and exclusive nature of informal communication networks of friends is neither a new phenomenon characteristic of the women's movement nor a phenomenon new to women.'</li>
<li>'Structures of decision making are a boon to the overworked person'</li>
</ul>
</div>
<div>
Having arrived in our own generation and in our own group at some of these same realizations, we are now in the process of defining a voting and representative leadership process. Voting processes, like consensus processes, need definition -- but most of us are more experienced with voting so the habits and practices are already there. When we get over this process definition hump we'll be better able to figure out what we want to get done, and start getting stuff done.</div>
<br />
<br />
Footnotes:<br />
(1) I could say much more about rough consensus and large groups, and if I have any IETF readers of this blog left they'll know what I mean.<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-18312489588066180822015-06-22T15:13:00.002-07:002015-06-22T15:13:25.313-07:00Trello for agile developmentI've now observed a few development teams using Trello to manage their work. Trello feels great to use and it seems to promote creativity and participation, which is also great. There's some things it doesn't help as much, where personal discipline needs to be brought in and used consistently, if the team is to stay focused and continue making good prioritization decisions.<br />
<br />
<b>Making estimates</b>. While it's possible to install an extension to Trello to get estimates on development cards, it is not built in throughout the system. Personally, I never get involved in development work without estimating my tasks; I want the product owner people I work with to value my time. Estimates help them do so, and estimates also clarify conversations over features that have unnoticed ambiguity. Estimates are a critical communication tool for me.<br />
<br />
<b>Tracking velocity and releases. </b>Velocity helps teams predict and keep focused. If we'd like to release a beta in 4 weeks and our velocity is 12, and we have a lot of tickets estimated at 1, 2 and 3, one quickly sees how they add up to 2 weeks or 4 or 6. It's good to have the velocity and the estimates very well tracked and lined up against desired milestones. We all need help making sure things don't creep into the next four weeks unless they are really required to release the beta. With Trello, tracking velocity and comparing it to release dates needs to be done outside the tool.<br />
<br />
<b>Keeping focus on epics. </b>With agile development, it's easy to get distracted from an epic -- a larger feature that requires a bunch of things to come together before it can be released and useful. Small things come up and because they're small, the team may be attracted to just finishing those things off. How can you keep noticing when a task doesn't help meet an Epic goal? With Trello, it requires unusual discipline to keep a list of Epics cards sorted and make sure that every development card is tagged with the appropriate epic.<br />
<br />
Have I missed some critical features of Trello or patterns of use that would achieve these goals? I would be happy to learn I had, because it would allow me to advise teams using Trello currently.<br />
<br />
Personally, I like it when tools help provide discipline. I'm willing to live with less flexibility in return for help keeping disciplined. It's easier for a whole team to adapt to the discipline of a tool, than to have the project manager or other disciplined person always nagging the team to add the right label, to put in the estimate even though the software doesn't require it. My opinion isn't universal, but it is mine, and for those reasons I prefer Pivotal Tracker, or in a pinch I'll even use Jira/Greenhopper over Trello. Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-33309455122154627512015-04-27T14:20:00.005-07:002015-04-27T14:20:55.023-07:00Management of Risk in Software Testing<br />
I read a rather old book called "Technological Risk" while working on startup software test plans, and this is making me think of the tradeoffs in a very abstract manner.<br />
<br />
In the same way that there's a value for life whether one admits to putting a value on it or not, there's a value for bugs. When we invest in road infrastructure or cellphone usage laws to save lives, and whether we save lives in our own country or in a foreign country by sending aid, our personal and governmental expenditures put a value on life. <br />
<br />
Investing in finding bugs has a similar implicit value: is a given amount of testing worth the likelihood of finding a bug? Strict testing regimes that enforce a certain coverage at certain points in the release process have an attraction, but it should be recognized that a strict full test plan is not <i><b>better</b>, </i>it merely compensates for a lack of information. <br />
<br />
An ideal test process, from the perspective of maximizing effectiveness, would utilize as much information as possible about what could have gone wrong and what the costs of mistakes in various areas are. For example, in a product involving user login, making and seeing posts, and changing display timezone:<br />
<br />
<ul>
<li>if timezones are a new feature, the ideal test for this release would involve a lot of work on timezones, including changing system time and different cases of daylight savings boundaries.</li>
<li>If the timezone stuff hadn't changed in months, the ideal release acceptance test would only perform a sanity check on timezones, not go to extremes like changing system dates. </li>
<li>Every test pass would confirm user login even if nothing had changed there, because the cost of breaking login would be high. </li>
</ul>
<br />
An ideal test process with full balancing of risk and cost is not achievable in the real world, but it's very easy to do better than a strict test plan.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-55909225228932410592015-04-21T19:32:00.001-07:002015-04-21T19:32:05.757-07:00HTTP error responses in APIsI've seen a bunch of mis-use of HTTP status responses in HTTP/JSON or HTTP/XML APIs. It's not merely a matter of taste when interoperability can suffer, as when status responses are used by intermediaries or by client libraries, forcing implementors to override or work around those intermediaries or libraries. Or maybe I'm just unusually bugged by these things.<br />
<br />
Here are a couple gotchas I've seen more than once. Other opinions welcome.<br />
<br />
<b>Gotcha: Using 401 for 403 and vice versa</b><br />
<br />
401 means that the client is NOT authenticated, and in some client libraries and browsers, this could trigger an authentication request. The server is required in HTTP to send the authentication challenge header. If the client is authenticated but doesn't have the right permissions, use 403 instead. Or if the client is unauthenticated but it doesn't matter, because no authentication would make the request succeed, 403 is also valid.<br />
<br />
<b>Gotcha: </b><b>Using specific errors as general errors </b><br />
<br />
There are three errors whose English titles make them seem broadly useful: 406 "Not Acceptable", 412 "Precondition Failed", and 417 "Expectation Failed". If your API has conditions that need to be met before other parts of the API can be used, it seems natural to use 412, right? The problem is these three all are tied to specific HTTP features, and are almost never appropriate to the standard kind of HTTP API. <br />
<br />
<ul>
<li>"Not Acceptable" is tied to the Accept header. Only if the request failed because of the client's Accept header choice, might this be the right error to use. </li>
<li>"Precondition Failed" refers to precondition headers which include If, If-Match, If-None-Match, and If-Modified-Since. Use only if the request failed because of one of these precondition sent by the client. </li>
<li>"Expectation Failed" is the same and refers to the Expect header. </li>
</ul>
<br />
In addition, I've collected a bunch of useful habits around error response bodies.<br />
<br />
<b>Tip: Use a response body on certain error responses. </b><br />
<br />
It's perfectly legitimate to put a JSON body on a 4xx or 5xx response. The specification recommends it for some responses such as 409, where the error code indicates that the state of the resource makes the request fail, and it would sure help the client to know what the state of the resource is. <br />
<br />
<b>Tip: Include human-readable text</b><br />
<br />
Boy does this help client developers. <br />
<br />
<b>Tip: Include machine-parsable detail codes </b><br />
<br />
For example, if the API returns a number of 403 errors of different types, or 409 errors with different meanings, the client can handle those cases differently before even consulting the user. In the following example, the client could use the "uploadInProgress" key to know to automatically try again in 10 seconds if that's how the timeToWait was defined in the API documentation.<br />
<br />
<code> 409 Conflict<br />
Content-Type: application/JSON<br />
Content-Length: xxx<br />
<br />
{ "error": {<br />
"text": "This object cannot be updated because an upload is currently being processed. Try again in a few seconds.",<br />
"errorCode": "uploadInProgress",<br />
"timeToWait": 10<br />
}<br />
}</code>
<br />
<br />
Combining tip 2 and 3 produces a JSON response that has an envelope element "error" that can be extended to include more things, and extensibility is definitely good here.<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-34357970467884165842014-04-21T14:34:00.000-07:002014-04-21T14:34:21.622-07:00Dear applicantI wrote up a series of overly honest responses to job applicants a few years ago. Didn't send them, of course, it was purely therapeutic writing. Now that the job posting itself is stale and I don't even remember which company I was at when I wrote this, it's safe to post...<br />
<br />
<i>Dear applicant with broken links,</i><br />
<br />
Thanks for sending your resume in regards to our open position. Unfortunately, we're looking for the experience we asked for in the job description. I appreciated the personalized cover letter but it would have been even nicer if it weren't formatted so oddly in HTML.<br />
<br />
<i>Dear applicant with 20 jobs in the last four years,</i><br />
<br />
Thank-you for your interest in our position posted on craigslist. At least I think you're interested, but since you just cut and pasted your resume into email and didn't reformat it, nor did you add any personalized cover letter, I'm not sure I really feel the interest. You might like to know that your portfolio site has nothing on it but three static pages with a purple background. I'm sure that was an oversight?<br />
<br />
<i>Dear new-grad applicant,</i><br />
<br />
Thank-you for your interest in our position. I feel at this time we're looking for somebody who has the experience they claim -- your "ten years experience" summarized in your cover letter looks more like four small short-term, part-time contract jobs in the last four years when I look at your resume. Also, anybody who feels "proficiency with Microsoft Office Suite" is a skill worth listing for a technical position has too little of substance to say about their skills. <br />
<br />
I was intrigued to see that your portfolio site looks like it came from the nineties. Is Web 2.0 passé now, and retro static sites are the new white canvas? I find it fascinating that you recreated that limited, static feel with CSS and Javascript!<br />
<div>
<br /></div>
Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-3454984.post-53068206969694378952014-03-31T11:15:00.003-07:002014-03-31T11:15:56.814-07:00Streets are safer than everI recently looked up some data in response to a sad person thinking that "our streets" (the writer was writing from New York, but the perception is widespread) are less safe for kids than they were in her time. As long as I was doing the research I thought I should post it here too. TL;DR: The streets are safer than ever.<br />
<br />
<div style="background-color: white; margin-bottom: 0.75em; padding: 0px;">
<span style="font-family: inherit;">There’s more <em>relative</em> deaths due to accident and homicide than due to illness, but that’s because childhood deaths due to illnesses have plummeted. So that means a parent was right to be much more worried about kids dying from measles than guns in the early last century. But the risk is down for all causes, so today a parent ought to be much less worried overall, AND less worried about each individual cause.</span></div>
<div style="background-color: white; margin-bottom: 0.75em; padding: 0px;">
<span style="font-family: Arial, Helvetica, sans-serif; font-size: x-small;"><i>“For children older than 1 year of age, the overall decline in mortality during the 20th century has been spectacular. In 1900, more than 3 in 100 children died between their first and 20th birthday; today, less than 2 in 1000 die. At the beginning of the 20th century, the leading causes of child mortality were infectious diseases, including diarrheal diseases, diphtheria, measles, pneumonia and influenza, scarlet fever, tuberculosis, typhoid and paratyphoid fevers, and whooping cough. Between 1900 and 1998, the percentage of child deaths attributable to infectious diseases declined from 61.6% to 2%. Accidents accounted for 6.3% of child deaths in 1900, but 43.9% in 1998. <b>Between 1900 and 1998, the death rate from accidents, now usually called unintentional injuries, declined two-thirds</b>, from 47.5 to 15.9 deaths per 100 000.” from <a href="http://pediatrics.aappublications.org/content/106/6/1307" style="color: #666666;">Pediatrics journal</a></i></span></div>
<div style="background-color: white; margin-bottom: 0.75em; padding: 0px;">
<span style="font-family: inherit;"><a href="http://www.cdc.gov/injury/wisqars/pdf/10LCID_All_Deaths_By_Age_Group_2010-a.pdf" style="color: #666666;">The CDC data shows</a> that death by accidental injury is several times higher than death by homicide, for ALL age groups, even for 15-24 year olds. Accidental injury is mostly vehicular, so I tend to ask people if they drive on the freeway if they are worried about their kids playing outside or going trick-or-treating.</span></div>
<div style="background-color: white; margin-bottom: 0.75em; padding: 0px;">
<span style="font-family: inherit;">And in case one is worried about city streets, the drop in risk can't be attributed only to the suburbs and the country. Living in the city is now less dangerous. <a href="http://science.time.com/2013/07/23/in-town-versus-country-it-turns-out-that-cities-are-the-safest-places-to-live/" style="color: #666666;">This article</a> is about town vs country but also talks about overall safety in cities going way up. I looked up data for New York county, the densest county in NY State, compared to other NY counties using <a href="http://wonder.cdc.gov/">CDC Wonder</a>, and found that New York county had 22 deaths per year per 100,000 between the ages of 1 and 19. That's much closer to the lowest county, Westchester with 17 per 100,000, than to the highest, Sullivan county with 38.6 per 100,000. </span></div>
<div style="background-color: white; margin-bottom: 0.75em; padding: 0px;">
<br /></div>
Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-3454984.post-84573068328855479722014-03-10T18:36:00.000-07:002014-03-10T18:36:57.645-07:00How I hire engineers for startups<br />
I went looking for articles on how to interview programmers/engineers for startups. I didn't like much of what I found (I did like <a href="http://blog.eladgil.com/2010/02/hiring-first-5-engineers-what-sort-of.html">Elad Blog's take</a>), and none of them addressed engineering skills and new-technology choices the way I wanted, so wrote my own guide (and as usual, wrote too much). <i>My thanks to Mark Ferlatte and Rachel Gollub for reviewing a first draft of this. Mistakes remain mine</i>.<br />
<br />
<div class="p1">
My thesis is that a startup needs coders who are good engineers too. A startup that hires coders who are bad team players and engineers will quickly dig itself into a hole of technical debt. A startup is in a worse position than most other employers to teach engineering skills like technical teamwork, using specifications and unit testing; also it is in no position to spend 5 times as much time and money fixing bad code.</div>
<div class="p2">
<br /></div>
<div class="p1">
<b><i>1. How do you use specifications in your programming? Please expand...</i></b></div>
<div class="p2">
<br /></div>
<div class="p1">
Look for mid-range answers here. “Never” or "Specs are a waste of time" is bad — using specifications, wireframes or user stories is an important skill. An engineer who doesn’t know how to review and use a specification will at best put their own assumptions and that specification's errors into code, which will turn up as bugs later. </div>
<div class="p2">
<br /></div>
<div class="p1">
“I always need specs” may also be bad for a startup, unless the engineer takes some responsibility e.g. “If a spec doesn’t exist, I write down what I’m going to do and send it to the team." </div>
<div class="p2">
<br /></div>
<div class="p1">
A good engineer should also be able to talk about reviewing specs and improving them, and possibly even writing them or teaching product people how to write them. Look for information behind the quick answer, because companies and experiences vary hugely. A developer who can talk intelligently about how rapid iteration reduces need for elaborate specs sounds decent to me.</div>
<div class="p2">
<br /></div>
<div class="p1">
<b><i>2. How much do you rely on unit testing? When and how do you write unit tests? </i></b></div>
<div class="p2">
<br /></div>
<div class="p1">
Here the right point on the scale is at least 80% of the way towards “always”. Unit testing is an important skill and art and another one you don't have time to teach.</div>
<div class="p2">
<br /></div>
<div class="p1">
Excuses are not that impressive here. If a candidate said their past companies didn’t do unit testing I would wonder why the candidate didn’t learn and even promote unit testing on their own.</div>
<div class="p2">
<br /></div>
<div class="p2">
Good answers: “I rely on unit testing to help me think through a problem and I write unit tests sometimes before I write the function, but sometimes after, depending."</div>
<div class="p2">
<br /></div>
<div class="p1">
<b><i>3. When do you decide to use a new technology? Give an example of learning a new technology, promoting it in your team, and executing it. </i></b></div>
<div>
<br /></div>
<div class="p1">
Look for mid-range answers here too. Candidates should not be on the bitter bleeding edge of technology. There needs to be a drastic advantage to taking on the risk of brand-new technology. Ask how they justified picking up something untested, and how they assessed and minimized the risk. See my addendum below for examples of bleeding edge vs behind the curve.</div>
<div class="p2">
<br /></div>
<div class="p1">
On the other hand, candidates should be able to point to some new-ish technology (not just new to the candidate), year after year, that they learned and adopted. Try to find out if a candidate eagerly learned the technology or had to. Did the candidate choose and promote, or follow teammates’ beaten paths?</div>
<div class="p2">
<br /></div>
<div class="p1">
<b><i>4. What’s important to you in your work/employment? </i></b></div>
<div class="p2">
<br /></div>
<div class="p1">
Most answers are probably not critical hire/no hire factors here. However, you need to know what the candidate needs, in case 1 week later you’re saying “How do we convince Lisa to join us now” or in 6 months asking “Lisa is really awesome, how do we make sure we keep her?”. You must treat tech candidates' needs respectfully. <a href="http://startupboy.com/2011/12/13/why-you-cant-hire/">You need them more than they need you.</a> </div>
<div>
<br /></div>
<div class="p1">
That said, there are some bad answers for startups: “Working with rock stars” (or ninjas). “Stability”. “A mentor” (or a hands-on education in engineering). </div>
<div class="p2">
<br /></div>
<div class="p1">
Good answers: “Learning” (if self-directed). “taking on a variety of roles”. “Being very responsible for something”. “Moving fast” (if under control via unit tests, etc). “Being involved at the level of the business model.” “Having some non-trivial ownership in a venture. “ “Gain the experience to do my own startup.” “Be in a position to develop a great engineering culture." "Working with a good team." </div>
<div class="p3">
<br /></div>
<div class="p1">
<b><i>5. Can we see some samples of your code? (Alternatives: solve problem code on the whiteboard or join for a trial day coding on the project)</i></b></div>
<div class="p1">
<br /></div>
<div class="p1">
For this part, you need two or more really good <b>startup</b> engineers that are so good you CAN’T hire them to advise you. Have them do in-person or phone interviews or read source code to evaluate candidates. </div>
<div>
<br /></div>
<div class="p1">
If the candidate can’t point to some open source contributions, it may be possible to see some previous code under NDA; or the developer may have done some tutorial coding to learn some new system. Looking at source code is an effective use of time both for the candidate and for your startup. If this is not possible for the candidate, then a whiteboard or “homework” coding session is needed. </div>
<div class="p1">
<br /></div>
<div class="p1">
Another option is to have them work on a project for a day or two. Sometimes this can work and sometimes it can't. A programmer who can't or won't dedicate full days of programming to prove themselves to a startup (see above about you need her more than she needs you) may still be the right person to hire.</div>
<div class="p1">
<br /></div>
<div class="p1">
<b><i>What else to take notes on</i></b></div>
<div class="p1">
<br /></div>
<div class="p1">
While asking these questions, find out if the candidate is <b>nice and easy to get along with</b>. Startups are stressful enough already. Working with arrogant people makes startups worse. You need <b>good communicators</b>. Techies who assert and tell what to do may sound knowledgeable initially so dig deeper -- they need to be able to explain in detail how and why, too.</div>
<div class="p1">
<br /></div>
<div class="p1">
Try to find out if the candidate <b>learns fast</b>, both by asking about learning new technology, and ideally by asking making suggestions in the technical part of the interview (or after reviewing their code) that require the candidate to listen to another way of doing something and figure out how and why to do it that way. </div>
<div class="p1">
<br /></div>
<div class="p1">
The "new technologies" question should help you also answer: is a candidate <b>narrow or broad, a generalist or a specialist</b>. Rather than just hiring C++ engineers, startups need people who may write C++ and can deploy EC2 servers, do metrics in Python and debug some Javascript.</div>
<div class="p1">
<br /></div>
<div class="p1">
Try to find out if the candidate <b>can handle uncertainty and change</b>-- the spec question is a good time to address that as a side concern. </div>
<div class="p1">
<br /></div>
<div class="p1">
<br /></div>
<div class="p3">
<b><i><span class="s1">Addendum: </span><span class="s1">Examples of bleeding edge, cutting edge and behind the curve</span></i></b></div>
<div class="p3">
<br /></div>
<div class="p3">
Bleeding edge: Almost no startup in 2009-2010 needed to or could afford to build their entire codebase on Node.js that year. Don't hire the person who advocates this or something equivalent, especially if they haven't done it <i>themselves. (Not their team.)</i></div>
<div class="p3">
<br /></div>
<div class="p3">
Cutting edge: If the candidate learned Node.js in 2010, however, and decided that 2012 was the year to make it the basis of a product, and had a plan for hiring other engineers that know or want to know Node.js, that is a good sign. </div>
<div class="p3">
<br /></div>
<div class="p3">
Behind the curve: They should not still be promoting Java servlets for brand-new code-bases (at least not without Hibernate and Spring, and only if Java is already needed for the startup). Somebody whose only new technology learned in the last 5 years was Java servlets, because their manager assigned it to them, is not adopting new software technology at an appropriate pace or time.</div>
<div class="p3">
<br /></div>
<div class="p3">
You notice that my examples are older or heavily hedged. This is a really tricky area. I disagree with people I deeply respect about whether certain examples are a bad sign, a good sign, or "it depends" -- though I suspect if we talked about a specific candidate and what that candidate had said about this question we'd be able to come to a conclusion about the candidate, even if not about Java Servlets. Take really great notes so you can talk about the nuances with at least 2 of your tech advisors.</div>
<div class="p3">
<br /></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-27603843865927637602014-02-12T09:23:00.001-08:002014-02-12T09:23:29.877-08:00You must be this tall to ride the Elastic Beanstalk<div class="p1">
Elastic Beanstalk seems like it’s meant to allow a startup to easily deploy Web sites using the most common Web frameworks, and scale those sites gracefully with an integrated AWS scaling group. I’m judging it by that supposed purpose, and comparing it to Heroku.</div>
<div class="p1">
<br /></div>
<div class="p1">
Here's what we discovered using it in a startup. </div>
<div class="p2">
<br /></div>
<div class="p1">
<b>1. <i>Elastic Beanstalk deploys are inherently risky - delete working copy then cross your fingers</i></b></div>
<div class="p1">
<br /></div>
<div class="p1">
<span class="Apple-tab-span"> </span>EB documentation instructs users to push new code to an existing environment in order to deploy a new version. However, EB takes the site down in order to do this. If something goes wrong with the new version, the old version is gone. It should be possible to return to the previous version on the same beanstalk but it can get in a state where the <a href="https://forums.aws.amazon.com/thread.jspa?messageID=463388&tstart=0#">environment is corrupt</a>. Experienced users advise always creating a new environment, test it then redirect URLs, then throw the old environment out. </div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="Apple-tab-span"> </span>Compare to Heroku, where if the deploy fails, Heroku stays with the old working site, which is not replaced until the new site works. I also never experienced a failure to revert to the older version. </div>
<div class="p2">
<br /></div>
<div class="p1">
<b>2. <i>EB requires more manual or automation labour </i></b></div>
<div class="p1">
<br /></div>
<div class="p1">
<span class="Apple-tab-span"> </span>Because of EB’s risky deploys, best practice is commonly to create a new environment for each deploy, and test that environment before swapping the URLs with the running environment. This is fine, but as soon as one has a handful of environment variables and custom settings like VPC membership, the creation of that new environment needs to be automated in scripts and config files.</div>
<div class="p2">
<br /></div>
<div class="p1">
There are still risks. If one environment is not working, AWS refuses to swap the URLs! Creating a new environment reduces the chance that production would be pointing to a broken environment, but doesn’t completely eliminate it. If that happens, one has to use DNS to redirect the public URL to the working environment — because even if one deletes the broken environment, AWS doesn’t permit changing the URL of the working environment! </div>
<div class="p2">
<br /></div>
<div class="p1">
<span class="Apple-tab-span"> </span>Even if a startup does all this automation, compare to Heroku, where neither manual environment swapping nor custom automation are required. It's a good engineering practice to still automate source control checkout, tagging and deploy, but the deploy itself would be one line of the script.</div>
<div class="p2">
<br /></div>
<div class="p1">
<b><i>3. AWS support is not what it should be</i></b></div>
<div class="p1">
<br /></div>
<div class="p1">
<span class="Apple-tab-span"> </span>About half the time, AWS support was useless <b><i>or worse</i>. </b></div>
<div class="p1">
</div>
<ul>
<li>A common example of when it's useless is in asynchronous support, when the support representative asks repeatedly for information that was already given earlier in the conversation, or to try things which one has already tried (and mentioned). This happened to me several times.</li>
<li>An example of when it's worse than useless is when the support engineer suggests disruptive measures which do not help. Once when our production site was unreachable because we'd misconfigured VPC membership, the support engineer didn't help figure this out. Instead he had me repeatedly restart the environment in single instance mode then back to cluster mode; then restart the environment with a different machine image (AMI). It took hours and hours. He seemed knowledgeable but the things he knew were not the problem and he didn't seem to know how to confirm or rule out his hypotheses. </li>
</ul>
<div>
My experience with Heroku is that I need support less than 10% as often as I did with AWS EB. </div>
<div>
<br /></div>
<br />
<div class="p1">
<b><i>4. AWS documentation is not what it should be</i></b></div>
<div class="p1">
<b><br /></b></div>
<div class="p1">
If the service is complicated and doesn't always work the way one expects, then the documentation needs to be excellent. In the AWS documentation is that there's a big gap where the mid-level documentation should be. There is high-level "Here are the awesome features" documentation and low-level stuff like field names and config file formats. The missing parts are "Here's a mental model so you can understand what's happening" as well as "Here's an approach that we know works well for other users in this situation". </div>
<div class="p1">
<br /></div>
<div class="p1">
Without any documentation to supply good mental models of what's going on, or good practices for various common situations, it's really easy for inexperienced people to construct a setup that has unexpected drawbacks. </div>
<div class="p1">
<b><i><br /></i></b></div>
<div class="p1">
<b><i>Summary: Disappointed. </i></b></div>
<div class="p1">
<br /></div>
<div class="p1">
While EB is a definite improvement over running machines manually or on a hosted service at the level of EC2, I am disappointed that it is not as easy to use or as well-supported as Heroku. I have heard that with experienced, dedicated operations engineers, it's great -- but I also know of companies with experienced, dedicated operations engineers who run their own EC2 instances and manage scaling with auto-scaling groups, rather than let Elastic Beanstalk handle it for them.</div>
<div class="p1">
<br /></div>
<div class="p1">
I've heard the argument that AWS is cheaper than Heroku, but at a small scale, employing experienced AWS operations engineers is much more expensive. So is disappointing users and gaining AWS experience the hard way. If you must be on AWS eventually, do so when you're "tall enough" to really need the scale and hire the resources -- migrating between Heroku and EB itself is not that hard.</div>
Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-3454984.post-66628670201833411172013-12-11T14:51:00.000-08:002013-12-11T14:51:28.575-08:00Testing Rails apps when using DynamoDB<br />
Once we decided to use Dynamoid to replace ActiveRecord in our Rails project, we needed to figure out how to test it.<br />
<br />
First off, testing against Amazon's live DynamoDB is out of the question. I'm not going to be limited to network speeds to run unit tests, nor be required to have an Internet connection in the first place. Just a bad idea.<br />
<br />
Mocking out every 'save' and 'create' request on each Event and Plan interaction would have severely undermined the usefulness of any model tests and even many controller tests. When we mock a components behavior it's easy to make the wrong assumption about what it does. The mock is unlikely to catch things like suddenly an integer field is being added instead of a string even if that change would cause the real storage engine to fail. <br />
<br />
<a href="https://github.com/ananthakumaran/fake_dynamo">Fake_dynamo</a> is what I found to use. So far I'm really impressed by its accuracy in behaving the way Amazon DynamoDB does. I haven't found it to either raise false negatives (fails when the live service succeeds) or false positives (succeeds when the live service fails). Fake_dynamo can be used to run unit tests or just to run the Rails/DynamoDB system locally for development and ad-hoc testing. I installed the gem on the system. I did not include it in the Rails project because it's not called from anywhere in the project, it's run from the command-line to listen on a local port (default 4567).<br />
<br />
At this point it would be helpful to point out a command-line option I found useful:<br />
<br />
<code>
fake_dynamo -d fake_dynamo.fdb </code>
<br />
<br />
The -d option tells the gem which file to save to rather than the default. Unit tests can therefore use a different file than ad-hoc tests. This is nice because to do ad-hoc tests I might setup some long-lived data, whereas unit tests really only work well on a completely clean database rather than having dependencies between runs.<br />
<br />
So to clear out the database between runs I issue a network command to the local port to delete the whole database. I combine this in one line with running the tests:<br />
<br />
<code class="tr_bq">
curl -X DELETE localhost:4567 ; rake</code><br />
<br />
I should probably clear out the old db as part of the internal automation of the test run itself but haven't bothered yet.<br />
<br />
The last challenge was really how to do fixtures or factory data. I tested Rails fixtures, FactoryGirl and one other solution which I can't remember since this was over 2 months ago. Dynamoid, unfortunately, did not work with any of them. It turns out Dynamoid is missing too many ActiveRecord features to really work well with these systems yet. So for example<br />
<br />
<code class="tr_bq">
Attendee.delete(attendee_id)<br />
Attendee.find(attendee_id).delete
</code>
<br />
<br />
The first is supposed to work in ActiveRecord but doesn't work in Dynamoid; the second really does work in Dynamoid. Most of the time it's easy to replace the non-working syntax with one that works, but not when using helpers like fixtures/factory systems.
<br />
<br />
Summary: fake_dynamo good; Dynamoid shown to need more work. Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-3454984.post-30014974524297133982013-10-30T09:01:00.003-07:002013-10-30T09:01:43.550-07:00Using DynamoDB, work in progress<br />At work we're using Amazon Web Services' <a href="http://aws.amazon.com/dynamodb/">DynamoDB</a> for a backend. This is early days and a work in progress, but I thought I'd post about what we're doing so far because I've seen so little elsewhere about it.<br />
<br />
Our Web framework is Ruby on Rails. Rails is a system that favours convention over configuration. Most RoR developers use ActiveRecord, Rails' built-in system for object modeling and abstracting away SQL database access. If you <i>stay on the rails</i>, this works fantastically. Rails automates or partially automates many tasks and systems, from migrating your data when the model changes, to setting up unit tests that conveniently setup and instantiate the things you want to test. Building on top of this, many Ruby gems extend your Rails functionality in powerful ways (Web UI test automation, authentication and user management, access to social network sites).<br />
<br />
As soon as a project starts to diverge from Rails conventions, trouble begins. Trouble may be contained if the difference can be contained and made as conformant as possible to the default components. For example, when writing a API that serves RESTful JSON resources instead of HTML, it's best to figure out how to use views to serve the JSON in the same way that HTML views are generated (a topic of a <a href="http://nih.blogspot.com/2011_08_01_archive.html">few posts</a> I did a couple years ago).<br />
<br />
Which brings me to Dynamoid. Amazon's Ruby gem for access to DynamoDB is very basic and exposes DynamoDB architecture directly. That can be useful but it doesn't behave anything like ActiveRecord, and in order to use Rails' powerful tools and extensions, we need something that behaves as much like ActiveRecord as possible. The only ActiveRecord replacement for DynamoDB that I could find, that was at all active, was Dynamoid. So I'm pinning my hopes on it. AFAICT so far, it is incomplete but has "good bones". I've already fixed one tiny thing and submitted a pull request, and intend to continue contributing.<br />
<br />
Next post will be about testing in this setup.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-71212008361512276792013-10-14T16:56:00.004-07:002013-10-14T16:56:58.040-07:00Correctness impedes expressionIn kindergarten and grade one these days, teachers encourage kids to get their thoughts onto paper any old way. They don't explain how to spell every word and they certainly don't stop kids and correct their spelling. For a beginning writer, it will likely be all caps, and the teacher may not even suggest spaces between words. Here's my first grader's recent work:<br />
<br />
<blockquote class="tr_bq">
wanda-yi wan-t toa school </blockquote>
<blockquote class="tr_bq">
and i wat to room </blockquote>
<blockquote class="tr_bq">
four it was nis </blockquote>
<blockquote class="tr_bq">
iasst the tishr wat </blockquote>
<blockquote class="tr_bq">
harnamwas martha She </blockquote>
<blockquote class="tr_bq">
was nisdat war day sH- </blockquote>
<blockquote class="tr_bq">
Em</blockquote>
<br />
This means "One day I went to a school and I went to room four. It was nice. I asked the teacher what her name was, Martha. She was nice that(?) were day she (unfinished?)" Don't you love the phonetic "wandayi" for "One day I" ? I do. Note that the letter "I" is used for long sounds like in "nice", because that makes sense before one learns that that sound can be spelled many ways including "ie", "i?e", "aye", or "y". <br />
<br />
Okay, cuteness aside, I flashed to thinking about XCode while Martha explained why they teach writing this way: it's hard enough for a kid, writing slowly and awkwardly, to get three words out onto paper, let alone a whole page of writing. Many kids get intimidated by corrections and worry of mistakes. Instead of answers, she gives them a whole bunch of resources: try sounding it out, think of a similar word, try looking somewhere else in your own writing, see if the word is somewhere else in the room. Above all, she encourages practice and resourcefulness rather than perfection.<br />
<br />
Unlike Martha, XCode is like the stereotypical teacher from 60 years ago who would stand over you <b>constantly</b> and warn if she even thinks you're about to make a mistake. "That's wrong." "No, it's still wrong". "That's somewhat better, but still not good." "Now that's right, but now this is wrong."<br />
<br />
Maybe that's why I still use TextMate for Ruby. If the code doesn't have the right syntax, I'll learn about it later. (I write tests.) But for getting an algorithm out of my head and onto the screen, I much prefer not to be corrected and warned constantly while I'm doing it.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-23865608739970916582013-10-04T12:32:00.002-07:002013-10-04T12:32:58.639-07:00AWS Persistence for Core DataI like DynamoDB, and I like architecture that reduces the amount of backend engineering one needs to do in a company whose product is an app. So I was quite interested to investigate <a href="http://aws.amazon.com/articles/4435846131581972">AWS Persistence for Core Data</a> (APCD, for lack of a better short name) in practice.<br />
<br />
<b>APCD Overview according to me</b><br />
<br />
APCD is a framework you can install on an iOS app such that when the app wants to save an object to the cloud, it can set APCD to do so silently in the background. Not only does it save the object to the cloud, but changes made in the cloud can be magically synched back to the app's Core Data. There's a parallel framework for Android which is promising for supporting that platform with the same architecture.<br />
<br />
On the server end, if the server needs to do some logic based on client data, the server can access the DynamoDB tables and view or modify objects created by the applications. In theory one doesn't have to design a REST/other interface for synchronizing client data to the server or to other clients. That's a significant savings and acceleration of development, so we read up on APCD around the Web and implemented it. <br />
<br />
While there were a bunch of minor problems that we could have overcome, the primarily one was: nowhere does Amazon seem to document how to architect to use AWS Persistence, or explain <i>what is it for. </i>In the main article linked above, the sample code and objects are "Checkin" and "Location". But where's the context? Are these Checkin and Location objects in the same table? Is there one giant table for all data? Does each client have its own private table for a total of N tables? Or are there two tables? Or 2N? It really helps if new technology documentation includes some fully fleshed out applications to give context. Full source code isn't even what I'm talking about, but at least tell us what the application does, why it's architected the way it is use the new technology, and some other examples of what the new technology is for.<br />
<br />
<b>What I think APCD is for</b><br />
<br />
Well we recently put together a couple facts which suggest what APCD is for. <br />
<br />
<ul>
<li>You can't have more than 256 tables in a DynamoDB for an account, even when using APCD. This limitation is very relevant to architectural choices made with APCD.*</li>
<li>If an installed app has the key to access any part of a table, the app can access the whole table, all objects. There's no object-level permissions yet, and because the app access the data on DynamoDB through APCD, the server can't intercede to add permissions checking.</li>
</ul>
<div>
All right, so that tells us we can't architect the application so that each app instance saves its own table separate from other apps' tables. We run out of table space at 256 installed users if not sooner. It also tells us that if apps are going to share larger tables, the information in those tables has to be public information. </div>
<div>
<br /></div>
<div>
So that suggests to me that APCD is for apps to synchronize <b>shared public data</b>. For example, an application that crowd-sources information on safe, clean public bathrooms.</div>
<div>
<br /></div>
<div>
<b>How my sample app would work</b></div>
<div>
<br /></div>
<div>
The crowd-sourced bathroom app could have all the bathrooms' data objects in one big table, and each instance of the application can contribute a new bathroom data object or modify an existing one. A server can access the bathrooms data too, so Web developers could build a Web front-end that interoperates smoothly as long as the data model is stable. </div>
<div>
<br /></div>
<div>
Now to use the service, even if the whole dataset is too large to download and keep, an app could query for bathrooms within a range of X kilometers or in a city limit, and even synchronize local data for offline use. When the app boots up it doesn't have to download local bathroom data if it has done so before, instead APCD is supposed to fill in new data objects matching the query, and update the client with changes. </div>
<div>
<br /></div>
<div>
For security, we have to trust each app to identify users so we can identify and block simple bad actors (somebody using the app interface to insert false information), and we have to have some backup for dealing with the contingency where the app is completely hacked, its key is used to access the bathroom data, and somebody quite malicious destroys all the useful bathroom data. </div>
<div>
<br /></div>
<div>
<b>What we did</b></div>
<div>
<br /></div>
<div>
We ended up not using APCD because what we're building does not involve a shared public database. We have semi-private data objects shared among a small set of trusted users. Doing that with APCDs limitations seemed too far off APCD's garden path of easy progress.</div>
<div>
<br /></div>
<div>
Is there a better way to use APCD? </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
* Yes, you can have the 256 table count lifted, but not by much. Not, say, to 1 million. That's not how DynamoDB is architected to work well.</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-20858821444607113872013-09-26T14:50:00.000-07:002013-09-26T14:50:10.904-07:00Opportunities arising in fall 2013Working on a new project using cutting-edge AWS stuff and iOS 7, I note some opportunities.<br />
<br />
1. A really good Ruby library for working with AWS. <br />
<br />
Although Amazon really should hire more Rubyists, this could also be done by outsiders. AWS is powerful and Ruby is powerful. Hooking them together properly would be sooooo nice.<br />
<br />
2. An iOS module or framework for higher-level use of Amazon Persistence for Core Data (APCD)<br />
<br />
APCD is intriguing but Amazon has a lot more work to do. For example, objects can only have one properly-persisted relationship between them. You can't synch an Event object with an "organizer" relationship to Users as well as a "creator" relationship to Users. <br />
<br />
Whether Amazon does more work here or not, there's opportunities for people to build on top of this service, because it doesn't address problems like version skew between mobile app instances. For that matter, I'd like some explanation what it's <i>for</i> -- a real-time background table synch service is good for something but what exactly did the architects have in mind? Without knowing what it was built for and tested for, it's hard to know whether the service will work smoothly for the applications I'm thinking of.<br />
<br />
3. Documentation and examples for the new XCode unit testing<br />
<br />
There's vast amounts of information out there on unit testing with Java, Python and Ruby. There's blog posts upon blog posts of best practices, and many great questions and answers on Stack Overflow. But when it comes to XCode, I can't successfully google for "unit test method for filling in a text field in ios". Apple, why do you hate Web searches?<br />
<br />
Ok.<br />
<br />
Would somebody get on these please?<br />
<br />
Thank you.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-4923274627011299012013-09-26T14:01:00.001-07:002013-09-26T14:01:20.110-07:00Systems thinking vs algorithm thinking<br />
I was chatting with another programmer about our different styles. He's an incredible algorithm solver. He's done compression and encryption algorithms in school, and codecs and video processing and GUI animation effects since (he's 23). I tried to explain the kind of problem that I'm attracted to, which none of those are, and used the word "systems problems". <br />
<br />
"But isn't everything a system?". Only in the most trivial sense.<br />
<br />
What I was trying to distinguish by talking about <b>systems problems</b> and <b>systems thinking </b>in programming is modeling independent and interconnected agents. I'm not the only one with this kind of definition. In an interesting <a href="http://www.odi.org.uk/sites/odi.org.uk/files/odi-assets/publications-opinion-files/6485.pdf">publication on managing social change</a>, I saw the definition "Systems characterised by interconnected and interdependent elements and dimensions are a key starting point for understanding complexity science." Very close. Is there a better phrase for system-style solutions as opposed to algorithm-style solutions?<br />
<br />
Another way I explain this approach when I'm being self-mockingly post-modern is to say "I'm interested in the liminal spaces in computer architecture", which is an arty/jargony way of saying I'm interested in the interfaces between agents: APIs and protocols, typically. I also hear the words of a British-accented UWaterloo professor from 20 years ago, saying "Modularization!" and "Information hiding!" over and over. (Systems thinking supports object-oriented design.)<br />
<br />
I've worked with a ton of people who have the same mental models because I worked a lot in the IETF for ten years. It's a necessary part of communications security in particular, because in addition to thinking of each client and each server as ideal agents, one must always think of bad actors and flawed agents.<br />
<br />
Coming back to startups, I'm always surprised in a way when I talk to a programmer who doesn't design and implement protocols and APIs because they think so differently. It's more justifiably shocking when I meet people who know about and implement REST and aren't used to systems thinking!Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-78324823799109948202013-09-23T15:34:00.001-07:002013-09-23T15:34:09.631-07:00I'm reading <a href="http://www.goodreads.com/book/show/1210562.Dont_Make_Me_Think_And_Html_World_Wide_Web?from_search=true">Don't Make Me Think</a> by Steve Krug, and just read the section on home page messages. That's why I laughed out loud (surprising my cat) when I saw this home page:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibLOgrSPDiZykjo110PdREaMMpRKGm1AnfX8SUAjfAe1EpOA-w2BE3xeg1iQqU7dSRiHaQaL3H08gdo-qzP8xVvbv_LUW0DoeaEVxN_6U60mjwV9TtRstyNNBWvAw4LOGVsqBA/s1600/Screen+Shot+2013-09-23+at+3.25.10+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="187" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibLOgrSPDiZykjo110PdREaMMpRKGm1AnfX8SUAjfAe1EpOA-w2BE3xeg1iQqU7dSRiHaQaL3H08gdo-qzP8xVvbv_LUW0DoeaEVxN_6U60mjwV9TtRstyNNBWvAw4LOGVsqBA/s320/Screen+Shot+2013-09-23+at+3.25.10+PM.png" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Amazing, huh? It's got simple pricing! Free, pro or on-premise! What does it do? Not important!</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
If you scroll below the fold -- I swear this is exactly what I see above the fold my very first visit to the site -- there's a "Full Feature List". Now (and you really can't make this stuff up) I can see that Teambox supports</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ul>
<li>Capacity (this is a header)</li>
<li>Users</li>
<li>Projects</li>
<li>Storage</li>
<li>Organizations</li>
<li>Hosting</li>
<li>Support</li>
<li>Premium Features (this is the next header)</li>
<li>Group Chat</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk5eBEMKzdTpn0R0DfRUFvx-8v_ytBqOe3yM5fun5fUzMzjeGGKGBFD5c0atKEoAyurttrx1grxVS5pm3PQHFztmoWjHPUwubwTdfG7rOKtzIqhwGepLoXCTb8rI3S1Ezn6cjV/s1600/Screen+Shot+2013-09-23+at+3.31.08+PM.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgk5eBEMKzdTpn0R0DfRUFvx-8v_ytBqOe3yM5fun5fUzMzjeGGKGBFD5c0atKEoAyurttrx1grxVS5pm3PQHFztmoWjHPUwubwTdfG7rOKtzIqhwGepLoXCTb8rI3S1Ezn6cjV/s320/Screen+Shot+2013-09-23+at+3.31.08+PM.png" width="320" /></a></div>
<div>
<br /></div>
<div>
In other words, I still have no clue except that Group Chat is apparently a premium feature. Wow. At this point I don't even want to know. I feel like knowing what the service or product is would only detract from the delicious infinite possibilities of Teambox. I don't want the service, mind you -- I'm happy with the inscrutable mystery of Teambox. Some things are not meant to be known.</div>
Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-3454984.post-39162735550961557662013-09-20T12:18:00.003-07:002013-09-20T12:18:47.732-07:00Kevin Liddle makes <a href="http://blog.8thlight.com/kevin-liddle/2013/09/18/a-case-against-cucumber.html">a case against cucumber</a> in his blog post. Since he doesn't have comments, I'll basically comment here.<br />
<br />
I agree with Kevin that the idea that product managers will write cucumber tests is pretty weak. They might well read them and understand them however. <br />
<br />
I think, however, that the value of cucumber and its gherkin syntax are, as with many things, not exactly in the place where the designer thought the value would be. The value is in using <b>anything but ruby</b> to describe what you're trying to accomplish with ruby. Every so often I'll write or encounter a test written with the same narrow view with which the programmer wrote the code. Such a test verifies that the code reliably does the wrong thing in the larger sense. Using ruby to test ruby also encourages trivial tests where the implementation detail is verified, not the application logic (see <a href="http://blog.markrendle.net/2013/03/09/dont-unit-test-trivial-code/">Don't Unit Test Trivial Code</a> )<br />
<br />
My science-based (but not scientific) theory is based on how low-level thinking impedes high-level thinking. As an example, this happens when you're driving, thinking about something low-level like adding distances, and get distracted and miss your freeway exit. Cognitively, when you're in the middle of writing Ruby code and thinking about how to write Ruby, it's harder than normal to think "what should the code do". Switching to another language, gherkin or anything else, prompts the programmer to go meta. Going meta means repeatedly re-loading the mental model of how what i'm doing is fitting into a larger system and goals.<br />
<br />
This effect is known in a couple different fields:<br />
<br />
<ul>
<li> Learning math: "Good problem solvers possess metacognitive skill, the ability to monitor and assess their thinking" (ref <a href="http://www.ksde.org/LinkClick.aspx?fileticket=YWiW1OiEpDA%3D&">Support for Learning</a>)</li>
<li>Corporate strate<span style="font-family: inherit;">gy: "<span style="background-color: white; line-height: 19.192707061767578px;"><span style="font-family: inherit;">A strategic thinker has a mental model of the complete end-to-end system of value creation, his or her role within it" (wikipedia <a href="http://en.wikipedia.org/wiki/Strategic_thinking">Strategic THinking</a>)</span></span></span></li>
</ul>
<br />
So besides just changing to another language to avoid ruby-testing-ruby circularities, Gherkin is designed to make the programer think in terms of wants and fulfilling user expectations. The syntax "Given I am a new user, When I go to the home page, Then I should see the zero-content display" helps the developer pop from what she's trying to do to how she's trying to do it, and back, without losing the big picture.Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-48574121700962006812013-09-08T14:28:00.004-07:002013-09-08T14:28:45.353-07:00An information coordinator is useful <div>
<br /></div>
<div>
An information coordinator is a useful person, and after two years of working with a partner or alone, this weekend was a nice reminder of that.</div>
<div>
<br /></div>
<div>
The classroom camping trip was Saturday night, and since I haven't been working full time this summer, I volunteered to organize it. What I did:</div>
<div>
<ul>
<li>Printed out last year's camping duty list, announced that I was posting it two weeks ago.</li>
<li>Collected the duty list and sent out email for the last few needed duties.</li>
<li>Advised the shopping volunteer what to buy</li>
<li>Sent out a public email answering the questions individuals had asked</li>
<li>Kept track of which spots were free and directed arrivals (sounds like work, but wasn't really, more like socializing)</li>
<li>Triggered volunteers when to begin lighting the grill, and the campfire</li>
<li>Approved people's suggestions (people wanted to know if a suggestion would interfere with other plans so wanted some kind of coordination check, not really approval)</li>
</ul>
<div>
This didn't seem like work to me but it met with widespread gratitude. An organized point of information exchange is really what I was, timekeeper, and a maker of trivial decisions. Software projects call these people project managers, but they exist in all kinds of domains, sometimes with different names, sometimes with specialized expertise. Sometimes the project manager is just an organized person holding the clock, the task list, and the notebook.</div>
</div>
<div>
<br /></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-47033084288887542492013-08-19T10:22:00.003-07:002013-08-19T10:22:50.841-07:00Order of OperationsI recently taught my son to ride his bike without training wheels. He was very resistant and afraid of falling down. I discovered that the order of learning skills was very important. Before he could get confidence balancing and moving, he needed to be confident that he could brake and put his feet down at any time. This is a coordinated movement between hands and feet as well as body balance (which foot? which side? when?) so not as simple as it seems. <br />
<br />
Over twenty minutes on two days, we practiced braking dozens of times: I would hold his bike up while he put his feet on the pedals, help him move forward pushing the pedals, and then either tell him to brake, or let go and he would wobble and brake on his own. Eventually one time he forgot to brake and just kept going: breakthrough! So the order of learning was<br />
<br />
<ol>
<li>Learn how to stop</li>
<li>Learn how to go straight on his own</li>
<li>Learn how to turn</li>
<li>Learn how to start on his own</li>
</ol>
<div>
Despite having a sore lower back from holding up his bike so much, I thought this worked well. But what a strange order to learn in! Then I remembered how knitting is most often taught. The teacher will cast on a bunch of stitches and do a few rows, so that the knitter can (1) go straight, then learn to (2) turn at the end of the row, then (3) bind off at the end, and finally someday (4) cast on a new beginning.</div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-24000407083237534012013-08-14T09:50:00.001-07:002013-08-14T09:50:35.270-07:00UX for Lean Startups required reading<a href="http://www.goodreads.com/book/show/16128986-ux-for-lean-startups" style="float: left; padding-right: 20px;"><img alt="UX for Lean Startups: Faster, Smarter User Experience Research and Design" border="0" src="http://d202m5krfqbpi5.cloudfront.net/books/1356670411m/16128986.jpg" /></a><a href="http://www.goodreads.com/book/show/16128986-ux-for-lean-startups">UX for Lean Startups: Faster, Smarter User Experience Research and Design</a> by <a href="http://www.goodreads.com/author/show/381043.Laura_Klein">Laura Klein</a><br />
My rating: <a href="http://www.goodreads.com/review/show/625842738">5 of 5 stars</a><br /><br />
I loved Laura's book. As I read it I kept on putting it down thinking "I need to put this down and go follow her advice IMMEDIATELY" and then I would pick it up because I wanted to learn more and hear more of her voice. Since I know Laura I could hear her voice advising, explaining, and gently mocking commonly-held falsehoods. The tone combines with the topic matter to break down pre-conceptions, to convince and teach. <br /><br />Laura's advice is incredibly practical. Having just been through a startup I could immediately see what I could have applied, and working with other startups now I do get an opportunity to apply more ideas. Many ideas are only obvious in retrospect (like testing fake features when you're operating on a shoestring budget) and then even once the idea is obvious, there's great advice for making the most of the idea.<br />
<br /><br />
<a href="http://www.goodreads.com/review/list/1761246-milele">View all my reviews</a><br />
<div>
<br /></div>
Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-68862134786695201162013-08-04T10:29:00.001-07:002013-08-04T10:29:20.965-07:00Bleak House<a href="http://www.goodreads.com/book/show/31242.Bleak_House" style="float: left; padding-right: 20px;"><img alt="Bleak House" border="0" src="http://d202m5krfqbpi5.cloudfront.net/books/1280113147m/31242.jpg" /></a><a href="http://www.goodreads.com/book/show/31242.Bleak_House">Bleak House</a> by <a href="http://www.goodreads.com/author/show/239579.Charles_Dickens">Charles Dickens</a><br />
My rating: <a href="http://www.goodreads.com/review/show/670457610">5 of 5 stars</a><br /><br />
I love Bleak House. Dickens uses incredibly lush and complex metaphorical descriptions for London, estate houses, and especially, the Chancery Court. What makes this novel one of my favourite Dickens books is the sweet and good unreliable narrator, Esther Summerson. She is not perfect (which is even better than perfect). Her imperfections lie in her lack of self-confidence and too-trusting nature. Even as she tells in the nicest and most trusting tone about some other character's actions, the reader can tell that other character's truer nature. And eventually Esther comes around to the reader's opinion, saddened if the other character turns out to be less than her ideal. <br />
<br /><br />
<a href="http://www.goodreads.com/review/list/1761246-milele">View all my reviews</a>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3454984.post-51373683112059723672013-06-11T15:29:00.000-07:002013-06-11T15:29:25.163-07:00<span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">Two meetings was I invited to, </span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">And sorry I could not both attend, </span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">at two o'clock, what could I do,</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">the time was firm for each, I knew,</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">so I composed regrets to send.</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">I chose the meeting Paul had called </span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">because my thoughts were needed there,</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">and progress on V2 was stalled </span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">because the techs were so enthralled</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">with a framework with no market share.</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">And while Paul's engineers discussed</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">Their plans for building on Hadoop, </span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">And writing all their code in Rust, </span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">And if the crowdsource trend is bust;</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">I thought about that other group.</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">I'll have to read the notes they took,</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">No matter how obtuse or dense,</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">and at their data models look,</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">and find a time I can rebook,</span><br style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;" /><span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;">If I am to make a difference.</span><br />
<span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;"><br /></span>
<span style="background-color: white; color: #262626; font-family: arial, sans-serif; font-size: 12.727272033691406px; line-height: 15.994318008422852px;"><i>.... apologies to Robert Frost.</i></span>Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-66146762364207984782013-05-30T11:53:00.003-07:002013-05-30T11:53:57.263-07:00Modern Web architecture, cookie-cutter, view from the trenchesWeb architecture has come together quite nicely of late. The components work together better than ever before, and it's easier to put together a great Web site or service -- putting more emphasis on the design and user experience, of course, which remain important and hard.<br />
<br />
This list of steps is an overview of the cookie cutter approach. It works for most new Web sites and services. Many people I've worked with believe they have something different, a special wonderful unique thing that is not right for the cookie cutter approach, and to they extent that they're right, their differences should be minimized (focus on making your value differentiated, not your architecture). So, if you're getting started on a new site or even a new service within a service, try to put off being special and different as long and as far as you can. To minimize early development costs while remaining flexible and scalable (new hires as well as increased usage), try to make the default choice for every step below until it's proven you need to be different. <br />
<br />
I came up with this list by using these components myself in situations both early and at scale, by talking with lots of other developers about what actually works and is easy, and by keeping abreast of developments by following a few blogs and twitterers I admire.<br />
<ol>
<li>Pick a modern Web framework</li>
<br />
The popular favourites are Python and Django, and Ruby on Rails. If you are or have a technical person already, personal preference is a fine way to make this choice. These both have data abstractions, views, templates, lots of inherent flexibility, and a rich set of free extensions (e.g. Ruby 'gems'). If you're thinking about node.js, see "Qualms" below.<br />
<br />
<li>Pick a data store</li>
<br />
Hosted MySql or PostgreSQL both work great for a long time and a lot of scaling. Use hosted databases for online test, staging and production and either the same database or SQLite for development and local testing. It would be fine to just go with whatever is cheapest and easiest, which might mean making this decision together with "pick a hosting service" (E.g. Heroku uses PostgreSQL by default). NoSQL solutions go under "Qualms" below. <br />
<br />
<li>Pick a hosting service</li>
<br />
Look. Just don't run servers yourself, OK? Not until you're the size where your hosting costs annually are approaching the annual cost (roughly twice salary) of a dev/ops employee. Don't use EC2 either, that's still a lot of manual work. Startup efforts don't have the manpower or processes to handle an early new customer <b>and</b> a vulnerability disclosure somewhere in the Web stack, in the same week. So leave the operation and upkeep of the Web stack up to the hosting service. I like Heroku. <br />
<br />
<li>Design and implement landing page</li>
<br />
This is where you'll need to pick a CSS layout library like Skeleton. Use this library's columns and rows to lay out your pages.<br /><br />
<li>Plug in a user login module if you need login</li>
<br />
Either pick one that supports passwords, email verification, forgetting passwords etc, or pick one that supports using Facebook, Twitter, Google, etc identities. Plugging this in should be very quick if you're still on the cookie-cutter path. Bootstrappers can configure their service to send those validation and forgotten-password emails via a free GMail account initially. Django comes with authentication, whereas for Rails I've used both AuthLogic and Sorcery. <br /><br />
<li>Add JQuery to your Web site if you didn't already do it for the login feature</li>
<br />
A combination of other Javascript libraries could be used to replace JQuery, but JQuery does both common widgets and functions (like handsome buttons) and AJAX -- the AJAX part is necessary for accessing data from your service in the next two steps. <br /><br />
<li>Start with some kind of site template </li>
<ol><br /></ol>
You probably need Web page headers, footers, and common colours and fonts. A combination of Web framework page templates, including partial pages (e.g. the Web page footer is commonly a partial Web page that is included on every full page in your site) and CSS templates solve this. <br />
<br />
<li>Work out your core data model, using your Web framework and migrations</li>
<br />
Your core data model is that special wonderful thing you're building. It could be real estate listings, restaurant reviews, task lists or BBQ recipes. Frame it as objects and collections.<br />
<br />
<li>Expose your data as a Web API with JSON, and show in the Web page with AJAX. </li>
<br />
Read up on how your chosen Web framework supports REST and apply it to your data model. Read up on how your chosen Web framework supports AJAX. Often the way to expose the data model is a direct extension of your data model. However, sometimes it's driven more by the view. If the UI always shows collections of comments, you don't need a way to get just a single comment; you jump straight to "GET all the comments for this item." Use JQuery or another Javascript library to make those calls and load data dynamically into pages.<br />
<br />
This step and the previous step will happen over and over again as you add functionality. <br />
</ol>
<h3>
Additional steps</h3>
<ul>
<li>While in development, use free email; soon move to a hosted email provider.</li>
<br />
To begin with use your Web framework's default email manager (ActionMailer in Rails), and just use any IMAP account to send mail, such as a free GMail account. This is one of the things you'll need to upgrade sooner rather than later, by moving to a paid, hosted email provider. Sendgrid is working OK for me and took less than half a day to sign up, set up and switch all test/demo/production servers to. Sending SMS is similar -- in addition to choosing a SMS service you'll need to plug in a module.<br />
<br />
<li>When or if you need native mobile or 3rd party app support, firm up your Web API</li>
<br />
It's pretty cheap to get started with mobile functionality via HTTP/HTML/AJAX, at least for prototype, demo or even MVP. There are probably cases where a native app is needed right away, but that's not too common. The mobile client can use the same Web API that the javascript "client" uses, only it now becomes more important for that Web API to be stable, because a native mobile app or 3rd party app won't be updated at the same time as the Web site and API are updated. <br />
<br />
<li>When or if you need to do any authentication with other sites or services, use OAuth.</li>
<br />
OAuth comes into play for two situations: to authenticate users to another site such as Twitter, but also to authenticate apps or other sites that get special permissions on your service. Rather than cook up your own special key logic, just use OAuth, plugging in one of the libraries that already exist.<br />
<br />
<li>While you move along, let your Web framework help you as much as possible.</li>
<br />
Django and Rails both have a ton of useful stuff. The biggest one in my view is their database migrations assistance. Bundler, in Rails, is key to looking after all the gems you'll have after the above steps. Try to also be familiar with the scripts and scaffolds that allow one to build new functionality fast in the completely cookie-cutter format. </ul>
<ul><br />
<li>While you move along, start to add automated tests</li>
<br />
I add these right at the beginning. Selenium, and capybara let me automatically load Web pages from my site, fill in forms, press buttons, and do simple tests on the results. RSpec and its mocking features are critical both for unit tests and for mocking out dependencies. I run these myself and also, as soon as there are more than 1 developer, on a continuous integration server. I'm currently trying out Semaphore as my first hosted integration server but would still go to the trouble of running Jenkins on EC2 if Semaphore doesn't work out.</ul>
<h3>
Qualms</h3>
Node.js has great promise to unify Web programming under one language, bridging the divide between Web frontend javascript programmers and Web backend programmers using Ruby, Python or other. A lot of people I respect for being simultaneously practical, realistic and visionary, do like Node.js. My qualm is that there is no standard framework to use with node.js -- the level of services provided by Node.js is more primitive than that of Django or Rails, and thus additional components are needed.<br />
<br />
NoSQL is very powerful and often appropriate in large-scale Web sites and services. I've used it before. However, I have qualms suggesting a NoSQL solution at the outset, because the tools to hook frameworks and NoSQL together, as well as the tools to manage NoSQL systems, are not the default tools most familiar to Web developers, and not the most tested and evolved. That said, a startup with experience using NoSQL might, without my labeling them crazy, use hosted Mongo, Redis or CouchDB, provided they still abstract away the store from the Web framework's data model. In the long run this is something a hugely successful system probably needs to scale. A combination of SQL and NoSQL is especially powerful for large or complex services but if you're at the "pick a data store" step you're not there yet.<br />
<h3>
Summary</h3>
The subject matter or domain used to matter a lot when a startup began to build a Web site or service. These days, it doesn't seem to matter as much -- most startups I've seen can benefit from following most of these steps. It's a real benefit to keep the startup's special thing limited and cordoned off (think of it as making your specialness more <i>concentrated)</i>. It's easier to hire and onboard people, as well as to benefit from community advances, when the overall Web architecture is cookie cutter.<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-68837756136722870382013-05-16T14:45:00.002-07:002013-05-16T14:45:36.875-07:00Girls in gamesThe cultural gestalt these days seems to include more talk about women and video-games than ever before. Suddenly, women are <a href="http://www.theesa.com/facts/gameplayer.asp">47% of video-gamers</a> and game designers (and game advertisers) are still figuring out how to deal with that. If you haven't seen any of Anita Sarkeesian's <a href="http://www.youtube.com/feministfrequency">videos</a> before, they're really good. Her concepts help me understand a bunch of common patterns and gave me names to more easily identify them (the "smurfette effect" for example).<br />
<br />
Anita's concepts have not ruined my enjoyment of video games, but I may be getting more selective. We have a PS2 and have been playing some old games. One of them is <a href="http://www.okami-game.com/">Okami</a> which is an outstanding game in many ways: playable, artistic, incredibly rich and creative. In the game, the player's character is a wolf who begins a quest to regain divine powers the character once had, and rid the world of demons and darkness so it can bloom and the sun can shine again. When the world blooms it's simply gorgeous; all the art is painterly and beautiful.<br />
<br />
Although the wolf fights, and is the best fighter in the world, its gender is ambiguous. My son thought the wolf was a 'he', missing the repeated interactions when the wolf meets other minor gods and they always call the wolf "Amaterasu, mother of us all". Throughout the world there are reasonable ratios of male and female characters, adult and child and often the female is more powerful than the corresponding male (in the main city, the empress is clearly more important than the emperor). Both male and female characters are killed off, not just the females. Females need rescuing more than males but not exclusively. So there's more balance than normal.<br />
<br />
A few things grate only a tiny bit; the main non-player character who fights with a sword is alcoholic, lazy, scared and male, and goes off to save his brave, supportive lady love. The character who seems good but is secretly taken over by a demon is called "busty babe" over and over by the pixie (and she is pictured as having unreal boobs), and there are scenes about this 2-inch tall male pixie trying to literally get into her shirt. All in all, did not make me love the game any less -- the overall balance is so pleasing that an annoyingly-stereotyped interaction could be enjoyed as mild low humour.<br />
<br />
And then there's this kind of thing. *sigh* I thought the Internets were supposed to know all about me by now and target ads right at me. Clearly the Internets know I'm a gamer, but can't they tell I'm female?<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7Mred_QfkCJSEgmbzXHI2n3D0fMe6L8wX8TiQN4GYtV46QAm4iXm2gs06Pz8V6aI6QVep2eGxYXbfa_nftW0TMX_QQS-wY4vjZmS7Q0-8I7XP9kli268E9C3Ri_mJ9GclC8fN/s1600/Wartune+banner+ad.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7Mred_QfkCJSEgmbzXHI2n3D0fMe6L8wX8TiQN4GYtV46QAm4iXm2gs06Pz8V6aI6QVep2eGxYXbfa_nftW0TMX_QQS-wY4vjZmS7Q0-8I7XP9kli268E9C3Ri_mJ9GclC8fN/s320/Wartune+banner+ad.png" width="82" /></a></div>
<br />
<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-3454984.post-91142465382332976562012-06-25T18:27:00.000-07:002012-06-25T18:52:33.365-07:00Commentroversy, plus owls are the trendiest fowl.Commentroversy is a five-year-old word according to <a href="http://www.urbandictionary.com/define.php?term=commentroversy">Urban Dictionary</a>, but it was new to me, when I saw it in the comment section of Yarn Harlot's <a href="http://www.yarnharlot.ca/blog/archives/2012/06/21/now_that_you_ask.html">latest blog post</a>. And the storm in a teacup that has so many commentators making puns about stitches and needles? The USOC asked <a href="http://www.ravelry.com/">Ravelry</a> not to use the term "Ravelympics" to describe the event in which knitters set ambitious knitting goals during the real Olympics. Read the Yarn Harlot or <a href="http://www.slate.com/blogs/future_tense/2012/06/22/ravelry_olympics_usoc_apologizes_to_online_knitting_community_over_trademark_crackdown.html">Gawker</a>, I think the shitstorm is funnier than the issue itself and don't have much else to say.<br />
<br />
In unrelated news, I bought this <a href="http://www.topatoco.com/merchant.mvc?Screen=PROD&Product_Code=QC-YOLO">t-shirt</a> because it made me laugh. But then I had to try to explain it. Does it have something to do with YOLO being an acronym for "<a href="http://www.themaneater.com/stories/2012/4/3/yolo-youre-doing-it-wrong/">You Only Live Once</a>"? I do not know. But I did have a <a href="http://katedaviesdesigns.com/2008/11/22/owls/">suspicion</a> that <a href="http://www.etsy.com/listing/69650998">owls</a> <a href="http://www.etsy.com/listing/84443689">are</a> an <a href="http://www.etsy.com/listing/66006487">arty</a> <a href="http://www.etsy.com/listing/85881395">crafty</a> <a href="http://www.etsy.com/listing/100313914">Thing</a>. And I confirmed they are a Thing, without even counting <a href="http://www.etsy.com/listing/93981816">macrame owls</a>. You thought that ducks and chickens were overdone on home decorations? Owls are the trendiest fowl by far. In my totally scientific study based on Etsy search totals, here is the order of popularity of animals in kitsch:<br />
<br />
<a href="http://www.youtube.com/watch?v=0XM3vWJmpfo">Bird</a>: 294,000<br />
<a href="http://www.etsy.com/listing/83525218">Butterfly</a>: 144,000<br />
<a href="http://www.etsy.com/listing/96054470">Cat</a>*: 118,000 <br />
<a href="http://www.etsy.com/listing/66259375">Owl</a>: 112,000<br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/89610058">Dog</a>*: 110,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/99465460">Fish</a>: 67,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/85309873">Bear</a>: 63,000</span><br />
<a href="http://www.etsy.com/listing/83310060">Horse</a>: 56,000<br />
<a href="http://www.etsy.com/listing/94843948">Rabbit</a>: 49,000<br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/98375496">Mouse</a>**: 47,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/80311973">Monkey</a>: 43,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/84119060">Elephant</a>: 42,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/99829558/little-miss-busy-bee">Bee</a>: 32,000</span><br />
<a href="http://www.etsy.com/listing/96122677">Deer</a>: 25,000<br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/100824938">Frog</a>: 25,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/100702282">Ladybug</a>: 24,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/98292004">Turtle</a>: 23,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/100618746">Duck</a>: 22,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/90376657">Lion</a>: 20,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/86108402">Turkey</a>: 19,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/96780187">Pig</a>: 18,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/62488032">Fox</a>: 17,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/48445861">Chicken</a>: 17,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/102595982">Cow</a>: 17,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/89606404">Sheep</a>: 16,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/101894564">Sparrow</a>: 15,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/81840647">Wolf</a>: 13,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/55186525">Eagle</a>: 10,000</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/98785215">Crab</a>: 8,800</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/84826060">Panda</a>***: 8,200</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/99338103">Parrot</a>: 6,900</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/83370181">Beetle</a>: 5,800</span><br />
<span style="background-color: white;"><a href="http://www.etsy.com/listing/55677149">Lizard</a>: 5,700</span><br />
<br />
<a href="http://www.etsy.com/listing/97296557">Rat</a>: 4,400<br />
<a href="http://www.etsy.com/listing/99422116">Gorilla</a>****: 1,200<br />
<div>
<br /></div>
<br />
* I subtracted<span style="background-color: white;"> the hits for pet accessories, because there's handmade dog and cat collars, beds, and toys, which don't count as Cute Handmade [Animal] Themed Tchotchkes. If you just count housewares, for example, owls beat cats.</span><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">** Turns out, half the mouses on Etsy </span><span style="background-color: white;">are <a href="http://www.etsy.com/search/handmade?q=minnie%20mouse">Minnie</a></span><span style="background-color: white;">.</span><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">*** You think you're cute, panda? You're small-time, niche, pissant cute. You can't even reach the tail of a <a href="http://www.etsy.com/listing/99120807">stuffed owl</a>. I mean, really, you got beat out by crabs. Go home and eat shoots and leaves.</span><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">**** Disturbing.</span><br />
<span style="background-color: white;"><br /></span><br />
<span style="background-color: white;">***** Tiger and Zebra were omitted due to the hard-to-control-for number of results with tiger eye gems, tiger stripe and zebra stripe fabrics. ***** <-- self-referential footnote</span>Unknownnoreply@blogger.com1tag:blogger.com,1999:blog-3454984.post-89839060870426501012012-03-29T12:28:00.001-07:002012-03-29T12:28:27.172-07:00Better software development estimates<br />
Is it possible to estimate software tasks better? In certain circumstances, my experience tells me yes! It's not just a wish (/me waves to commenter on last post).<br />
<br />
First, to establish what kind of estimates I'm talking about. It's not possible to estimate anything past two months, in my experience. When I was working at Microsoft and heard lots of estimates from other teams, I realized that "<i>This software product will take us two years</i>" really meant "<i>We have no idea and we will never finish a product that recognizably matches our plan</i>". A one year estimate turn into a somewhat recognizable outcome in two or three years. Even a six month estimate, while it might turn into a year-long project fairly reliably, ends up finishing quite different tasks than the planned tasks which originally led to the six month estimate.<br />
<br />
Other things that make a difference in estimation accuracy:<br />
* <b>What language</b> is being used? Snags can be a bigger time dilator with a compiled or lower-level language like C, whereas hitting a snag in Python might not throw out the estimate that much.<br />
* Is something new being <b>integrated</b>? Any time a task involves compiling and linking a new library, or adding a new Ruby gem, I know the estimate is weaker.<br />
* <b>How</b> <b>routine</b> is it? Adding a new page to a Ruby on Rails project can be pretty damn predictable.<br />
* <b>How many people</b> are involved? Estimating a one-person task is way more accurate than estimating a team project. Even a one-person task that has an issue that requires another person's answer is less accurate than the task that can be done by one person independently.<br />
* How many other estimated tasks are <b>dependencies</b>? If you have a chain of tasks to estimate, a change in one task can throw out the others. <br />
<br />
And then there's <b>bias</b>. Have you ever noticed that some developers always estimate high and some always estimate low? It's intriguing, because even though they're consistently wrong, they are consistently wrong in the same direction. That means they are actually giving management good information, if management knows their biases. I once managed one optimist and one pessimist in the same team for over two years. The optimist always estimated about 1/3 of his actual time required. The pessimist always estimated about three times the actual time required. I would assign what looked like three months of work to the pessimist, and what looked like a week and a half of work to the optimist, and they would finish around the same time. <br />
<br />
One of the things I really do love about Agile (and Pivotal Tracker reifies this) is how it understands the above points. <br />
* <b>Task length</b>? Agile encourages developers to break work down into smaller tasks (no three-week tasks in any agile process I've ever seen)<br />
* <b>How many people</b> are involved? Agile encourages issues to be resolved before the estimate is even made, and it is designed for single-person tasks.<br />
* How many unfinished <b>dependencies</b> are there? Agile encourages planning with a shorter horizon, so the chain of dependencies is usually reduced.<br />
* Consistent <b>bias</b>? Agile tracks velocity, not accuracy, so a consistent bias is simply a constant part of a consistent velocity.<br />
<br />
With all that, one of the main things that intrigues me, and this is what I'm unpacking from my previous post, is whether better feedback would help developer estimates get even better than Agile already makes them. Agile does not measure time spent so it doesn't give developers feedback that would allow them to either fix a consistent overall bias, or to start to recognize tasks that need to be estimated a little higher.<br />Unknownnoreply@blogger.com2