Tuesday, August 30, 2011

My third post on doing RESTish JSON APIs in Rails is about testing the API. I'm assuming the network API should remain stable, not have fields added or removed without consideration (and ideally, documentation), especially if there are independently-written clients that use the server's API. So normal Rails "functional" testing is not enough here.

"Functional" testing is in scare quotes because it's got a specific meaning to Rails, not a generally-accepted meaning to programmers. It is how Rails defaults encourage programmers to test the actions of their controller classes, which is to say the results of their Web request handling (e.g. in the Rails guide, especially the section on testing views).

It gets pretty tedious to assert all the things are in a JSON response that should be:

assert_response :success
  data = JSON.parse(@response.body)["data"]
  assert_not_nil @data["comments"]
  assert_equal 1, @data["comments"].length
  assert_equal "The User", @data["comments"][0]["display_name"]
  assert_equal "the comment text", @data["comments"][0]["comment_text"]

Worse, this only tests that things that are tested for are there, not that new things don't creep in. It's pretty easy to modify code in a way that passes what looked like reasonable functional tests, but breaks the client.

While working on improving our testing for API responses, I've found a few things are important to me.
  • The tests should show what an entire response ought to look like (aids readability)
  • The tests should flag things that aren't supposed to be there as bugs
  • The test responses should be templates, so that variables can vary
  • The output of a test should show the difference, so I don't have to wade through pages of desired output and pages of actual output
It took us a while (thanks to Tom, Steve and others at work) to put all the pieces together, and there are probably more improvements to be made, but I'm happy enough with what we came up with to blog about it. We use
  • Cucumber to describe the request, the response status and the response body
  • Custom cucumber step definitions to let the step define the entire response body as a Ruby Hash
  • A custom template "Wildcard" object to let variables vary within the response body
  • Rails feature to compare hashes and find diff, to produce readable output
So now here's what one of our API response tests looks like!

Feature: Commenting
    Given the following post exists:
      | id | 1234 |
    And the following comment exists:
      | id | 882281 |
      | post_id | 1234 |
      | comment_text | "O gentle son, upon the heat and flame of thy distemper sprinkle cool patience." |
      | user_id | 101 |
  Scenario: Retrieve a comment 
    When I GET "/posts/1234/comments/882281" with a user session
    Then the response should have the following comment:
        "comment_text"=> "O gentle son, upon the heat and flame of thy distemper sprinkle cool patience.",
        "created_at"=> Wildcard.new(String),
        "updated_at"=> Wildcard.new(String),
        "creator" => {
          user_id => 101,
          handle => "gertrude",
          display_name => "Queen Gertrude",
          number_comments => Wildcard.new(Fixnum)
        "post_url" => Wildcard.new(String)

The Wildcard class looks like this:

class Wildcard
  def initialize(must_be_class=Object)
    @must_be_class = must_be_class
  def ==(other)
    other.class == @must_be_class

And the Cucumber step that implements "the response should have the following comment" looks like this:

Then /^the response should have the following ([^"]*):$/ do |element, string|
  the_hash = eval(string)
  response_data = JSON.parse(@response.body)["data"][element]
  assert_equal({}, response_data.diff(the_hash))

Note, what this means is that our API responses have "data" as the top element, with a "comment" element inside -- that part is asserted to be there, so that the contents of the comment can be compared with the Rails hash diff method. If the comment element in the response had an extra child, this would produce an error with the extra child in the diff. If the comment was missing a child, this would produce an error with the extra child in the diff. And if the comment has a child with a different value than what's in the template, this would produce an error with the returned value in the diff.

I realize this is not Behavior Driven Development (BDD). One can use Cucumber to do BDD, but the two are not inextricably linked. And our intent here is not to do BDD but to maintain a stable API. I think this works.

Monday, August 29, 2011

This is another post outlining "How I spent several days frustratingly getting something to work" so that "you don't have to". I think the theory behind these kinds of posts is the "you don't have to" part but the reality is "OMG I have to rant and boy it would help put this behind me if I could pretend there was some USEFUL lesson."

I already posted that I've been using Jenkins and RVM. The theory is great: Jenkins lets you set up lots of recurring jobs and have one dashboard to see that all of your software is basically working, while RVM allows you to choose a different Ruby version and gemset for each project that might need that.

The first Jenkins job I set up seemed to use the RVM Ruby-1.8.7 just fine, but when I tried to set up a second job that required Ruby-1.9.2, I never could get it working. First it tried to use the system ruby, which I blew away. Then I had hours tracking down references to the system ruby. The command "rvm use 1.9.2" appeared to work and chose the right rvm ruby directory. But even when it worked, when the very next command run by jenkins was "ruby -v", the machine either couldn't find the ruby install at all, or found the system one, but never the one just successfully activated by RVM. Even more frustrating, this worked at the command line, but not when running a job from Jenkins. (If anybody can explain how Jenkins runs jobs differently and what effect that has on context, I'd love to know -- having a model of what's going on would help.)

Finally I discovered "rvm-shell", which can force execution of a command within the context of what RVM set up. So now every line of the jenkins job script that uses ruby, bundle or rake, is wrapped in that:

rvm use 1.9.2@profiles
rvm-shell -c "ruby -v"
rvm-shell -c "bundle install"

rvm-shell -c "rake db:migrate"

# Finally, run your tests
rvm-shell -c "rake"

It worked. 56 is the magic number -- that is, build #56 was the first one to work on this project. Phew.

Tuesday, August 23, 2011

Many modern Web services or Web APIs use REST style with JSON request and response bodies. The first problem doing this with Rails is that Rails isn't entirely conducive to doing REST, which I already discussed, but the second problem is that Rails isn't very conducive to doing JSON in a way that keeps the API stable. As is normal with Rails, it gets the basics running really fast, but its default assumptions and "magic" utilities need to be bypassed for some purposes.

This seems to be the default approach -- just call Rails "as_json" or "to_json" methods on the model instance, which gets all of the model's database columns but none of the model's dynamic information. This example has an array of Message instances, and without any monkey-patching or overriding as_json, the effect of this will be to export all of the ActiveRecord fields of Message (but not other content such as attributes) as JSON elements. For example, if the Message model uses a "messages" table with a column called "title", this code would expose the value in that column as "title" in the JSON as well.

render :json => message_list.as_json

But if some of the database columns on the model are for internal use and not for the API, the API implementor needs to override the default behavior to include only the columns that are intended for the API:

# ... model for Message
def as_json(opts = {})
opts = {:only => [:title, :body, :thumbnail_url, :background_url, :action, :display_on_launch, :ok_button_string, :cancel_button_string]}

#... controller for Message
render :json => message_list.as_json

(One could also use "except" instead of "only" as an option for as_json, but then if a new column gets added to the Message model, it would automatically be added to the API whether you want it or not).

It gets even more complicated if some of the model's information needs to be converted, combined, or otherwise manipulated:

# ... model for Message
def as_json(opts = {})
# change id to be called message_id because existing clients use message_id
out = { 'message_id' => id.to_s(&:as_json) }

opts = {:only => [:title, :body, :thumbnail_url, :background_url, :action, :display_on_launch, :ok_button_string, :cancel_button_string]}

#... controller for Message
render :json => message_list.as_json

It starts to get ugly, and a lot of the ugliness lives in the Model, not the Controller... wait. Oops. In MVC, isn't this what the View is for?

Enter RJSON. This is a template type for Rails, by Tom Lieber. The Ruby hash (like a dictionary) is already very familiar to Ruby developers, so why not have the Ruby developer construct a hash in a template file. A RJSON template file named "messages.rjson" might look like this:

:messages => @messages.map do |msg|
:message_id => msg.id.to_s,
:title => msg.title,
:body => msg.body,
:thumbnail_url => msg.thumbnail_url,
:background_url => msg.background_url,
:action => msg.action,
:display_on_launch => msg.display_on_launch,
:ok_button_string => msg.ok_button_string,
:cancel_button_string => msg.cancel_button_string

And its controller looks like this, where @messages is the array placed in context for the view to use:

@messages = Message.find_all_by_user_id(user_id)
render :file => "messages"

This is simple, but even much more complicated examples remain readable. Partials can also be used; if some data structure gets re-used exactly the same way in more than one part of the API, that data structure can be a partial view. It's easy to see what the API should look like from the view, which is part of the point of views.

I recommend it, which is to say: by all means get your Web API prototyped quickly using as_json, but when you start to get serious about the API end-points and field names, documenting the API and keeping it stable, start using RJSON.

ETA: Tom pointed me to a recent gem published by somebody else with the same basic plan. It's also at github (same name, Jan de Poorter)

Wednesday, August 10, 2011

I've been setting up Jenkins to run Rails project unit and functional tests as builds.  This has been difficult, and 90% of the problems have been in getting different versions of various gems available.  RVM is supposed to help, as is Bundler.   I should have started out with a very methodical plan for giving access to the jenkins user, then having the jenkins user install rvm and create gemsets, then have the jenkins user install gems and run bundle install. Instead I got things working quickly with super-user permissions and then builds fail because it's hard to 'sudo' commands during an automated build.

Here's one problem running "bundle install" as the jenkins user:

Using rails (3.0.4)
Using right_http_connection (1.3.0) from git://github.com/rightscale/right_http_connection.git (at master) /usr/lib/ruby/1.8/open-uri.rb:32:in `initialize': Permission denied - right_http_connection-1.3.0.gem (Errno::EACCES)
from /usr/lib/ruby/1.8/open-uri.rb:32:in `open_uri_original_open'
from /usr/lib/ruby/1.8/open-uri.rb:32:in `open'
from /usr/local/lib/site_ruby/1.8/rubygems/builder.rb:73:in `write_package'
from /usr/local/lib/site_ruby/1.8/rubygems/builder.rb:38:in `build'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/source.rb:450:in `generate_bin'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/source.rb:450:in `chdir'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/source.rb:450:in `generate_bin'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/source.rb:559:in `install'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/installer.rb:58:in `run'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/rubygems_integration.rb:93:in `with_build_args'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/installer.rb:57:in `run'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/installer.rb:49:in `run'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/installer.rb:8:in `install'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/cli.rb:220:in `install'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/vendor/thor/task.rb:22:in `send'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/vendor/thor/task.rb:22:in `run'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/vendor/thor/invocation.rb:118:in `invoke_task'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/vendor/thor.rb:263:in `dispatch'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/lib/bundler/vendor/thor/base.rb:386:in `start'
from /usr/local/rvm/gems/ruby-1.8.7-p352@sonicnet/gems/bundler-1.0.17/bin/bundle:13
from /usr/bin/bundle:19:in `load'
from /usr/bin/bundle:19

After reading up on similar but different errors via Google, I concluded this was probably a local filesystem permission error.  I tried granting the jenkins user various permissions in /usr/lib/ruby and /usr/local/rvm/gems -- one misleading clue was that the "right_http_connection-1.3.0.gem" file in /usr/lib/ruby/1.8/gems/cache was the only gem in that directory with a different user/group than the other gems.  Fixing that didn't fix the problem, nor did deleting that gem file.  I then realized that bundler puts gems in a different place: .bundle in the project directory.  Oh, that's owned by root -- great, fix that.  Oops, didn't work.  Finally I deleted .bundle and ran bundle install again -- as the jenkins user -- and it worked.  

(This is probably boring to most readers but it's a small payment for all the other bloggers' posts I've read that gave me clues in various debugging dead-ends. )

Blog Archive

Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.