Blog article

Improvements to error responses in Rails 5 API mode

Jorge Bejar
Jan 12, 2016
12 min read

In previous posts, we demonstrated how to build an API application with Rails 5. Specifically, we covered integration with Backbone and Ember. At the time of writing of those posts, the only way to try out the all-new Rails API mode was by taking the code from the master branch, since Rails 5 had not yet been released. However, Rails-5-0-beta1 is available now, so you can easily start to build applications using Rails in its new API-flavored mode.

This Rails release also includes some changes related to the API mode that materialized in Rails a few weeks ago. The purpose of those changes is to improve error responses to JSON requests in Rails. We initiated a discussion about how Rails should handle and return responses in the proper format in the case of an error in processing a request, and then we finally came up with the necessary code changes in Rails.

Before digging into how error responses were improved, let’s recap how response format is determined in Rails.

Response format calculation in Rails

In Rails, the concept of format appears right in the first scaffold command. For example, if you run a simple bin/rails g scaffold post title, you can find references to response formats in the controller and also in the routes.

Part of the generated controller looks like the following:

class PostsController < ApplicationController

  ...

  # GET /posts
  # GET /posts.json
  def index
    @posts = Post.all
  end

  ...

  # POST /posts
  # POST /posts.json
  def create
    @post = Post.new(post_params)

    respond_to do |format|
      if @post.save
        format.html { redirect_to @post, notice: 'Post was successfully created.' }
        format.json { render :show, status: :created, location: @post }
      else
        format.html { render :new }
        format.json { render json: @post.errors, status: :unprocessable_entity }
      end
    end
  end
end

With Rails controllers, you have the ability to write code that can respond in different formats. By default, code generated by the scaffold command can respond in HTML or JSON formats. That is easy to see within the create action, but it also happens in the index action where the render is implicit (this action returns a list of posts and it’s rendered in the appropriate format).

Of course, there should be some sort of rule that determines which format is picked to respond to the request. In order to understand those rules, let’s now take a look at the routes generated by the scaffold.

The following is the output of bin/rake routes in our example application:

   Prefix Verb   URI Pattern               Controller#Action
    posts GET    /posts(.:format)          posts#index
          POST   /posts(.:format)          posts#create
 new_post GET    /posts/new(.:format)      posts#new
edit_post GET    /posts/:id/edit(.:format) posts#edit
     post GET    /posts/:id(.:format)      posts#show
          PATCH  /posts/:id(.:format)      posts#update
          PUT    /posts/:id(.:format)      posts#update
          DELETE /posts/:id(.:format)      posts#destroy

These routes are what you get when you have something like resources :post in your routes file. We can see that format here is inferred from a dynamic segment present in all the generated routes, and this segment matches the URL extension of the request. This is the most common way in Rails to determine the response format.

Therefore, a JSON response is generated if you issue a GET request to http:://mysite.com/posts.json and an HTML response is generated when the URL is http:://mysite.com/posts.html. In the case of the index action, it simply relies on the ability of Rails to render the list of posts in the format value inferred from the route path. When Rails is not able to infer any format (e.g. absence of extension in the path), or in the case of an unknown format, the response will be rendered in HTML format in a standard Rails application or JSON in an API-only application.

By using respond_to, like in the ‘create’ action of our PostController, you can be specific about the content of the response depending on the specified format. In this case, if the format is not provided, it will use the first format listed within the respond_to block. If there is no handler for the specified format, it will raise an ActionController::UnknownFormat exception (and it will result in a 406 Not Acceptable response error).

In general terms, this is a two-player game: We have a client that makes a request to our server and it could specify the expected response format, and we also have our application server that is able to respond in one or more formats. Therefore, the response format should be determined based on what was requested, but also consider the list of formats supported by the server. In Rails, adding the format in the URL is the most common way to specify the expected format of the response. This mechanism relies on how things are defined in the routes file and the controller code, as described above.

Defining the format with the HTTP Accept header

In the context of an API application, I personally prefer the use of the HTTP Accept header, which is also supported in Rails, since this is the way to specify the response format according to the HTTP protocol. The HTTP/1.1 RFC states:

The Accept request-header field can be used to specify certain media types which are acceptable for the response.

…

If no Accept header field is present, then it is assumed that the client accepts all media types. If an Accept header field is present, and if the server cannot send a response which is acceptable according to the combined Accept field value, then the server SHOULD send a 406 (not acceptable) response.

Please note that the term media type in the RFC is what we refer to as format in Rails.

Relying on the Accept header can be handy when the client code is another component such as an external application or the frontend part of a web application, given that you usually have control of the request headers. Hence, you can be explicit about the expected response format without being tied to the request path.

Responses when something goes wrong in the backend

Let’s get into the topic that motivates this post. One special case related to the responses returned by our Rails application is error responses, i.e. when some error occurs while the request is being processed. Rails can form error responses in HTML (using the error HTML files that can be found in the public folder), JSON or XML. An example of an error response for a JSON request might look like this:

{"status":"500","error":"Internal Server Error"}

In Rails 4.2.x, error responses are returned in the format that would be used for a successful response in almost all cases, if the application is running in production mode. This is not the case when the response format should be inferred from the request path and the error occurs before reaching the internal routing code in Rails. In these specific cases, the error response code is correct but the request body is empty.

There are probably just a few cases where the error occurs during execution in Rails middleware before the router is reached, but it is still an issue that can arise. In fact, the router code is executed after the entire middleware chain, right before the corresponding controller code is executed. For example, consider the following request to create a Post based on our PostController:

curl -H "Content-Type:application/json; charset=utf-8" -d '{"post": {"title":"My Post", "invalid value"}}' http://localhost:3000/post.json

Note that we are using the Content-Type header to indicate the format of the values sent in the request. Rails will attempt to parse the provided JSON value, but it will raise an ActionDispatch::ParamsParser::ParseError exception because the JSON is not well-formed (did you notice the invalid part in the example?). However, it fails to return an error response in JSON format because the code that parses the .json part of the request path is never executed. Instead, it just returns a 400 Bad Request response with an empty body. Adding an Accept header to ask for JSON responses, instead of relying on the .json extension in the URL, will cause an error response to be returned, again with the 400 Bad Request error code but now with a body in the expected format as well:

{"status":"400","error":"Bad Request"}

The difference can be very subtle, but if our client component processes anything further based on the response content in the case of errors, we may need to utilize the Accept header to be safe when using Rails 4.2 or previous versions.

Among the changes related to error responses in Rails, we added some code to respond with the format indicated in the request path when an exception is raised, before executing the router code. The approach is very simple: We retrieve the format from the extension of the URL by guessing that the format is the extension part of the request path.

The result of determining the format this way is not always equivalent to what is parsed by the router, as the :format fragment could theoretically be in any part of the path. However, we think that this method will work for all but a very few cases. In fact, I’ve never seen a Rails application with custom routes where the :format fragment is not the extension part of the URL.

You can see what was finally changed for Rails 5 by looking at the MimeNegotiation#format_from_path_extension method here. Also, you can learn more about the limitations of this approach when you have routes with constraints from this discussion.

Error responses in development mode

Up to now, we have discussed how error responses are formatted given the default production settings. In fact, there is a specific Rails configuration setting called action_controller.consider_all_requests_local that controls how errors are rendered. It’s set to false by default in config/environments/production.rb causing responses to present the error in a user-friendly way (generally, a user-oriented error message that doesn’t expose internal information about the exception).

In contrast, consider_all_requests_local is set to true by default in development mode, and it generates the standard Rails HTML error page for debugging purposes that displays the exception message, the stack trace and also the web console where we can execute code on the server, except for XHR requests where error responses are rendered in plain text. Therefore, getting an error response with JSON or other formats is not possible with the default settings in development mode using Rails 4.2.

With the arrival of the Rails API mode, we found that such error responses in development mode were not so convenient when working on a Rails API project. In many cases, you may want to receive error responses in the format expected by your client component that also include information about the error for debugging purposes. So, the development error responses would differentiate from production error responses in their content but not in the format used to render them.

We actually wanted developers to have the chance to decide how the error responses should be. We believe it would strongly depend on the interaction with the client component: If you’re working exclusively in the backend component, you would prefer to use the HTML error pages since you can better inspect the exception information in a formatted HTML page and take advantage of the web console. On the other hand, if your work integrates both sides, the Rails backend and the client, it’s preferable to receive JSON error responses, even in development, so that they go into the expected flow in the client component (where you probably already have a way to inspect and debug problems).

Well, Rails 5 includes some changes that could hopefully help in those cases. A new configuration key was added to control how errors are rendered in development: debug_exception_response_format. It should be defined in the config/environments/development.rb file. It accepts two values: :default to render errors in the “Rails 4.2 way” (HTML pages or text error responses for XHR requests) and :api to render error responses using the proper format.

If you create a new application with Rails 5, the debug_exception_response_format won’t be included in the config/environments/development.rb file, but it’s nice to know that its default value depends on the type of application you’ve created. If API mode is turned on, the default value for debug_exception_response_format is :api, and in the case of an API-only application, the default value is :default. Of course, you can use the debug_exception_response_format option to customize the error responses in development if the Rails defaults do not apply to your specific situation.

A collateral fix for web-console

When we were working on the implementation of all these changes to Rails error responses, we ran into some issues with the web-console gem. We found that the web-console browser code was appended to error responses even when those were rendered in JSON, turning them into unintelligible messes that included JSON, HTML and JS code all at the same time. 

Although the web-console was always checking if the response format was HTML before appending the console code, this issue occurred because Rails and web-console were using different approaches to deduce the response format. While Rails defines the response format based on the extension in the URL and the Accept headers in the request as explained above, web-console was relying on the request Content-Type header. It led to a pull request where we concluded that web-console should stop relying on the Content-Type header (since it must be used to determine the request body format rather than response format according to RFC 2616 section 7.2.1). Relying on the format inferred by the Rails code, and avoiding a separate logic for that in this gem, was good enough to decide whether the console should be appended or not.

Conclusions

Hope you now have a better understanding of format negotiation and error responses in Rails, and the differences between development and production mode. Also, we would like to hear about your experiences developing API-only Rails applications with the latest modifications to error responses in development mode. Please send us any feedback that could help to improve the API mode in Rails.

Finally, regarding the format negotiation in Rails, a discussion is currently taking place regarding the proper way to respond in cases where the expected format is not supported or is unknown. It could lead to new modifications or fixes that can slightly alter the behavior mentioned in the first part of this article.

Back to articles

Let’s build the next big thing together.

contact us