When good controllers go bad: getting started with Coach
Last editedJun 2024
Just over two years ago, Lawrence wrote about Coach, our open-source Ruby library which makes it easier to build robust, maintainable and well-tested APIs by replacing Rails controllers built with ActionController with chains of "middleware".
Since then, we’ve continued using Coach and we’ve no doubt that it has allowed us to move fast, write code that stands the test of time and maintain developer happiness.
In this post, we'll build a simple API using ActionController, discover the pain points, and then see how Coach can help.
When good controllers go bad
We can get the hang of how Coach works by writing a simple controller-based API, and then rewriting it using Coach. Let's build an API for viewing attendees for an event, with the attendee to be returned specified by its ID:
class AttendeesController < ApplicationController
rescue_from ActiveRecord::RecordNotFound,
with: :not_found_error
def show
attendee = Attendee.find(params[:id])
render json: attendee
end
private
def not_found_error
render json: { error: I18n.translate("not_found") },
status: 404
end
end
We'd have something like this in our config/routes.rb
, thereby exposing our new API at
/attendees/:id
:
resources :attendees, only: :show
Let's say we were going to add the ability to also update and delete attendees. These,
like our #show
action, would also have to load an attendee by its ID. To make our code
more
DRY (Don't Repeat Yourself),
we can abstract the loading of the resource away, using a before_action
to run some
shared code before several controller actions:
# We haven't written the #update and #delete actions, but we'd have something like this
before_action :find_attendee, only: [:show, :update, :delete]
def show
render json: @attendee
end
private
def find_attendee
@attendee = Attendee.find(params[:id])
end
That's not too painful. Let's say we now wanted to authenticate these API requests, with a user identifying themselves by providing an access token. We'd probably end up with something like this:
before_action :check_authorization_header
before_action :check_access_token
before_action :check_user_permissions
private
def check_authorization_header
header_value = request.headers["HTTP_AUTHORIZATION"]
return missing_access_token_error unless header_value.present?
token_type, token = header_value.split(" ", 2)
return missing_access_token_error unless token_type == "bearer"
@access_token = token
end
def check_access_token
@user = User.find_by(access_token: @access_token)
return invalid_access_token_error unless @user.present?
end
def check_user_permissions
unless @user.permissions.include?("attendees")
return invalid_permissions_error
end
end
def missing_access_token_error
render json: { error: I18n.translate("missing_access_token") },
status: 401
end
def invalid_access_token_error
render json: { error: I18n.translate("invalid_access_token") },
status: 401
end
def invalid_permissions_error
render json: { error: I18n.translate("invalid_permissions") },
status: 403
end
Now, as you can imagine, we'd probably want to authenticate all of our API this way. Things get a little painful when we want to share this authentication logic between controllers, and not just between actions in one controller.
We could put our new methods in ApplicationController
, or even in an
ActiveSupport::Concern
which we include
in each controller where we want it.
Whichever way we go, there are problems:
- It's hard to think of a good way to customise the permissions required on a per-action
basis (here, we check for the :attendees
permission, but what if we wanted to change
this per-controller or even per-action?)
- Testing our authentication logic by itself is difficult - if we extract it into a
ActiveSupport::Concern
, there are ways of testing, but it's painful to set up a
realistic state for our tests
- The amount of state we have - stored in instance variables - is growing and is
difficult to reason about or manage (for example, it's hard to be sure that the state
you're expecting is actually set and isn't just nil
)
Let's add some more complexity. Naturally, our event is very cosmopolitan, so we need to be able to return our errors in the user's language so they get the best experience. Let's add that:
around_action :with_locale
def with_locale
I18n.with_locale(request.headers["HTTP_ACCEPT_LANGUAGE"]) { yield }
end
As you can see, before too long:
- We have a bunch of state stored in instance variables (
@user
,@attendee
) which is difficult to reason about and track across the controller - We lack any good way to abstract away and reuse shared logic, let alone unit test it independently or configure it where required
- Our controller has lots of
before_action
s and evenaround_action
s which are hard to get your head around (not least because of theexcept
s andonly
s)
Here's a real-life example from one of our controllers, before we migrated it to Coach.
This is the logical conclusion of working with ActionController
where you're doing
something even mildly complex:
around_action :with_locale, except: [:cancel]
before_action :only_allow_html, only: [:authorize]
before_action :build_oauth_params_from_params, only: [:authorize]
before_action :build_oauth_params_from_session, except: %i[authorize cancel]
# You would shiver to see the number of instance variables set in here...
before_action :set_instance_variables
before_action :check_client_id
before_action :check_redirect_uri
before_action :check_scope
before_action :check_response_type
before_action :redirect_to_cancel, except: %i[authorize cancel]
before_action :persist_oauth_params_to_session, only: [:authorize]
before_action :set_permitted_params
We've put together an example project, where
you can see the
final version of the controller we built, AttendeesController
,
plus
tests.
How Coach can help
We've built a simple API and have seen the problems you can easily run into, even with very minimal complexity. Let's rewrite what we've just written using Coach, and see how it can help - we'll quickly see the benefits.
1. Rewriting our controller action as a single Coach::Middleware
Let's start by writing a single Coach::Middleware
that does everything we need for this
endpoint. After that, we'll begin pulling out reusable parts (e.g. authentication) and
learning how to test them independently, and then put them together to form our API.
We'll refer to this middleware, the "primary" or top-level one for the API endpoint, as
an action middleware to avoid things getting confusing when we build more middleware
later. In Coach, everything we build is a Coach::Middleware
, but middleware can depend
on other middleware, making it easy to compose more complex behaviours.
# This file is available in our demo project at https://github.com/gocardless/coach-demo
# as `app/routes/single_coach_middleware/attendees/show.rb`
module Routes
module SingleCoachMiddleware
module Attendees
class Show < Coach::Middleware
def call
I18n.with_locale(supplied_locale) do
unless access_token && bearer_token?
return missing_access_token_error
end
user = User.find_by(access_token: access_token)
# To send back an HTTP response, we just return a
# Rack-compliant response (an array of a status code,
# headers and body) from our #call method
return invalid_access_token_error unless user.present?
unless user.permissions.include?("attendees")
return invalid_permissions_error
end
attendee = Attendee.find_by(id: request.params["id"])
return not_found_error unless attendee.present?
[
200,
Constants::Api::RESPONSE_HEADERS,
[attendee.to_json]
]
end
end
private
def supplied_locale
request.headers["Accept-Language"]
end
def access_token
authorization_header&.split(" ", 2)&.second
end
def bearer_token?
authorization_header&.split(" ", 2)&.first == "Bearer"
end
def authorization_header
request.headers["Authorization"]
end
def invalid_access_token_error
[
401,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("invalid_access_token") }.to_json],
]
end
def missing_access_token_error
[
401,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("missing_access_token") }.to_json],
]
end
def invalid_permissions_error
[
403,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("invalid_permissions") }.to_json],
]
end
def not_found_error
[
404,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("not_found") }.to_json],
]
end
end
end
end
end
So we can see the benefits as clearly as possible, let's write some tests. We'll use FactoryGirl to set up our models:
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `spec/routes/single_coach_middleware/attendees/show_spec.rb`
RSpec.describe Routes::SingleCoachMiddleware::Attendees::Show do
subject(:instance) { described_class.new(context) }
let(:attendee) { FactoryGirl.create(:attendee) }
let(:user) { FactoryGirl.create(:user, :can_manage_attendees) }
let(:context) do
{
request: instance_double(
ActionDispatch::Request,
params: params,
headers: headers,
),
}
end
let(:params) { { "id" => attendee.id } }
let(:headers) { { "Authorization" => "Bearer #{user.access_token}" } }
describe "#call" do
it { is_expected.to respond_with_status(200) }
it do
is_expected.to respond_with_body_that_matches(attendee.to_json)
end
context "with an invalid access token" do
let(:headers) do
{ "Authorization" => "Bearer not_a_real_token" }
end
it { is_expected.to respond_with_status(401) }
it do
error = { error: "Invalid access token" }
is_expected.to respond_with_body_that_matches(error.to_json)
end
context "specifying a non-default language" do
before { headers["Accept-Language"] = "fr" }
it do
error = { error: "Jeton d'accès invalide" }
is_expected.to respond_with_body_that_matches(error.to_json)
end
end
end
context "with a user who doesn't have access" do
let(:user) { FactoryGirl.create(:user) }
it { is_expected.to respond_with_status(403) }
it do
error = { error: "Invalid permissions" }
is_expected.to respond_with_body_that_matches(error.to_json)
end
end
context "with a non-existent attendee" do
let(:params) { { id: "not_a_real_id" } }
it { is_expected.to respond_with_status(404) }
it do
error = { error: "Not found" }
is_expected.to respond_with_body_that_matches(error.to_json)
end
end
end
end
Finally, we can link our action up to the Rails router in config/routes.rb
like this:
router = Coach::Router.new(self)
router.draw(Routes::SingleCoachMiddleware::Attendees,
base: "single_coach_middleware/attendees",
actions: [:show])
2. Translating errors into the requested language
We now have a simple and fully-tested API. However, our middleware's #call
method has
a tonne of responsibilities, and we can't test anything in isolation or reuse our logic.
Let's start by extracting out the multi-language behaviour into a discrete
Coach::Middleware
, which our action will declare as a dependency:
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `app/middleware/translate.rb`
module Middleware
class Translate < Coach::Middleware
def call
I18n.with_locale(supplied_locale) { next_middleware.call }
end
private
def supplied_locale
request.headers["Accept-Language"]
end
end
end
Here, we define a #call
method as before - this is what all Coach::Middleware
look
like. The one thing that changes in this one is that we do a single, discrete job, and
then call next_middleware.call
when we're done, rather than returning a response. This
is similar to Rack middleware,
which are at the heart of how Rails itself works.
In this case, we open an I18n.with_locale
block, and then call our next middleware from
within that block. This means that our calls to I18n.translate
in our action will
return strings in the requested language.
We can unit test Middleware::Translate
independently of any specific action middleware,
like this:
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
`spec/middleware/translate_spec.rb`
RSpec.describe Middleware::Translate do
subject(:instance) { described_class.new(context, next_middleware) }
let(:context) do
{
request: instance_double(ActionDispatch::Request,
headers: headers)
}
end
let(:headers) { { "Accept-Language" => "fr" } }
context "with a language set" do
let(:next_middleware) { -> { expect(I18n.locale).to eq(:fr) } }
it "sets the language" do
# Our expectation is set in the `next_middleware` lambda, since
# the I18n.locale setting is only very short-lived and returns
# to normal before #call finishes
instance.call
end
end
context "with no language set" do
let(:headers) { {} }
let(:next_middleware) { -> { expect(I18n.locale).to eq(:en) } }
it "uses the default language" do
instance.call
end
end
end
How does this end up working? We can now start using this in Routes::Attendees::Show
,
which will make things much clearer:
module Routes
module SingleCoachMiddleware
module Attendees
class Show < Coach::Middleware
uses Middleware::Translate
def call
# We no longer need our `I18n.with_locale` block here
unless access_token && bearer_token?
return missing_access_token_error
end
user = User.find_by(access_token: access_token)
# To send back an HTTP response, we just return a
# Rack-compliant response (an array of a status code,
# headers and body) from our #call method
return invalid_access_token_error unless user.present?
unless user.permissions.include?("attendees")
return invalid_permissions_error
end
attendee = Attendee.find_by(id: request.params["id"])
return not_found_error unless attendee.present?
[200, Constants::Api::RESPONSE_HEADERS, [attendee.to_json]]
end
private
# We can now get rid of the #supplied_locale private method,
# since that is now managed by `Middleware::Translate` ✂️
# We'll hide the other unrelated private methods in this
# and future examples to keep things simple.
end
end
end
end
We call the uses
method in our action to declare the Translate
middleware as
a dependency. Middleware we declare with uses
get run in order before our
action. So Middleware::Translate
will get run first, and it is responsible for
calling our action, the next middleware in the chain (hence the
next_middleware.call
).
We've now extracted the ability to set the language into its own middleware and tested that behaviour on its own, so we can remove tests related to that from the action middleware's unit tests (which we wrote in step #1).
It's worth nothing that we may well want to add integration tests, exercising the route and all the middleware it depends on, on top of our unit tests - but even there we won't test every possible case, instead focusing on the most common paths, trusting our rigorously unit-tested middleware to work (a bit like we trust the tested behaviours of Rails to just work).
3. Authenticating with an access token
Next, let's try doing the same with our authentication functionality, splitting it out into a middleware:
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `app/middleware/authenticate.rb`
module Middleware
class Authenticate < Coach::Middleware
provides :user
def call
return missing_access_token_error unless access_token.present?
user = User.find_by(access_token: access_token)
# To send back an HTTP response, we just return a Rack-compliant
# response (an array of a status code, body and headers) from
# our #call method
return invalid_access_token_error unless user.present?
provide(user: user)
next_middleware.call
end
private
def access_token
bearer_token? && authorization_header&.split(" ", 2)&.second
end
def bearer_token?
authorization_header&.split(" ", 2)&.first == "Bearer"
end
def authorization_header
request.headers["Authorization"]
end
def invalid_access_token_error
[
401,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("invalid_access_token") }.to_json],
]
end
def missing_access_token_error
[
401,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("missing_access_token") }.to_json],
]
end
end
end
This middleware is similar to our last one, but with two major differences:
- In certain situations, it returns a HTTP response, rather than calling
next_middleware.call
. - We "provide" data. This replaces how we typically use instance variables in controllers. We define on the middleware class what data it can be expected to "provide", and then we "provide" it. This can be accessed by other middlewares, including our action, which "require" it.
We can access the user
value which was provided by Middleware::Authenticate
in our action. Explicitly defining these requires
and provides
allows Coach
to perform some helpful error checking. If we "require" a certain piece of
context, but omit the middleware which "provides" it, Coach will throw an
exception when the application boots.
Let's add some tests for our new middleware. Coach provides some helpful RSpec
matchers, like respond_with_status
, respond_with_body_that_matches
and provide
to
make it easy to write clean, readable tests. To make these available, you just need to
require 'coach/rspec'
(most likely in your spec/rails_helper.rb
file).
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `spec/middleware/authenticate_spec.rb`
RSpec.describe Middleware::Authenticate do
subject(:instance) { described_class.new(context, null_middleware) }
let(:context) do
{
request: instance_double(ActionDispatch::Request,
headers: headers),
}
end
let(:headers) do
{
"Authorization" => authorization_header,
}
end
context "with a valid Authorization header" do
let(:user) { FactoryGirl.create(:user) }
let(:authorization_header) { "Bearer #{user.access_token}" }
it { is_expected.to call_next_middleware }
it { is_expected.to provide(user: user) }
end
context "with an Authorization header with an invalid bearer token" do
let(:authorization_header) { "Bearer lol" }
it { is_expected.to respond_with_status(401) }
it do
is_expected.
to respond_with_body_that_matches(/Invalid access token/)
end
end
# We'll leave the rest of the tests out here to keep things from
# getting too long - they look roughly the same! ✂️
end
Let's start using our newly-provided user in our action:
module Routes
module SingleCoachMiddleware
module Attendees
class Show
uses Middleware::Translate
uses Middleware::Authenticate
# Make our user, provided by `Middleware::Authenticate`,
# available in our #call method
requires :user
def call
unless user.permissions.include?("attendees")
return invalid_permissions_error
end
attendee = Attendee.find_by(id: request.params["id"])
return not_found_error unless attendee.present?
[200, Constants::Api::RESPONSE_HEADERS, [attendee.to_json]]
end
private
# We can now get rid of our #access_token, #bearer_token?,
# #authorization_header, #invalid_access_token_error and
# #missing_access_token_error methods ✂️
end
end
end
end
4. Checking users' permissions
We're still checking the user's permissions in our action. Let's abstract that out and make it reusable, writing another middleware:
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `app/middleware/check_user_permissions.rb`
module Middleware
class CheckUserPermissions < Coach::Middleware
requires :user
def call
unless user.permissions.include?(scope)
return invalid_permissions_error
end
next_middleware.call
end
private
def scope
config.fetch(:scope).to_s
end
def invalid_permissions_error
[
403,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("invalid_permissions") }.to_json],
]
end
end
end
This middleware introduces two new things:
- Everything is a
Coach::Middleware
, so our middleware can require things (like our action requires the user provided byMiddleware::Authenticate
) as well as provide them - This middleware is configurable, referring to
config[:scope]
, making it more abstract and reusable
We won't write tests here for this new middleware to keep things short, but you can find the tests for this middleware and the next one in our sample project.
To provide configuration for a middleware, we just pass keyword arguments to
uses
:
module Routes
module SingleCoachMiddleware
module Attendees
class Show
uses Middleware::Translate
uses Middleware::Authenticate
uses Middleware::CheckUserPermissions, scope: :attendees
requires :user
def call
attendee = Attendee.find_by(id: request.params["id"])
return not_found_error unless attendee.present?
[200, Constants::Api::RESPONSE_HEADERS, [attendee.to_json]]
end
private
# We can now remove the #invalid_permissions_error private
# method ✂️
end
end
end
end
5. Loading the right record from the database
Finally, let's move our shared logic for finding the right record in the database to a middleware. Since this is likely to be a commonly-performed task with a range of models, we'll make it a bit more abstract, supporting any model:
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `app/middleware/get_model_by_id.rb`
module Middleware
class GetModelById < Coach::Middleware
provides :model
def call
model = model_class.find_by(id: id)
return not_found_error unless model.present?
provide(model: model)
next_middleware.call
end
private
def model_class
config.fetch(:model_class)
end
def id
request.params["id"]
end
def not_found_error
[
404,
Constants::Api::RESPONSE_HEADERS,
[{ error: I18n.translate("not_found") }.to_json],
]
end
end
end
Once again, our middleware is configurable, so we'll pass the model_class
keyword argument when we add it to the chain by calling uses
:
# This file is available in our demo project at https://github.com/gocardless/coach-demo
# as `app/routes/coach_middleware_chain/attendees/show.rb`. (We've changed its name,
# since this represents the final version.)
module Routes
module CoachMiddlewareChain
module Attendees
class Show < Coach::Middleware
uses Middleware::Translate
uses Middleware::Authenticate
uses Middleware::CheckUserPermissions,
required_permission: :attendees
uses Middleware::GetModelById, model_class: Attendee
requires :user, :model
def call
[200, Constants::Api::RESPONSE_HEADERS, [model.to_json]]
end
private
# We can now remove our last private method,
# #not_found_error ✂️
end
end
end
end
6. Writing a unit test for our action middleware
We've now moved from a complicated controller to a chain of modular Coach::Middleware
,
each with a single responsibility.
Our shared middleware are unit-tested, but we don't have any unit tests covering the action itself, or an integration test covering the whole flow from top to bottom. Let's add a unit test so we can be confident our API works as expected.
The unit test for our action will make sure that, using the inputs provided by our previous middleware, our action produces the right result. This looks very simple, with just a single example, since the complicated logic is handled by individually-tested middleware which have been abstracted away:
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `spec/routes/coach_middleware_chain/attendees/show_spec.rb`.
RSpec.describe Routes::CoachMiddlewareChain::Attendees::Show do
subject(:instance) { described_class.new(context) }
let(:attendee) { FactoryGirl.create(:attendee) }
let(:user) { FactoryGirl.create(:user, :can_manage_attendees) }
let(:context) do
{
request: instance_double(ActionDispatch::Request),
model: attendee,
user: user,
}
end
describe "#call" do
it { is_expected.to respond_with_status(200) }
it do
is_expected.to respond_with_body_that_matches(attendee.to_json)
end
end
end
7. Finishing off with an integration test
The integration test is a little more involved. Here, we'll simulate a real HTTP request coming in, and make sure that the whole chain of middleware does what we expect, returning the correct response.
We don't need to test every single case, since we should be able to trust the unit tests of our middleware, but here for completeness, we'll do a fairly thorough test. As you write further actions for your API, you'll develop an intuition and conventions to help you decide what to test.
# This file is available in our demo project at
# https://github.com/gocardless/coach-demo as
# `spec/requests/coach_middleware_chain/attendees/show_spec.rb`.
RSpec.describe "GET /single_coach_middleware/attendees/:id", type: :request do
let(:attendee) { FactoryGirl.create(:attendee) }
let(:user) { FactoryGirl.create(:user, :can_manage_attendees) }
let(:headers) do
{
"Accept" => "application/json",
"Authorization" => "Bearer #{user.access_token}",
}
end
it "returns a JSON representation of the attendee" do
get "/single_coach_middleware/attendees/#{attendee.id}",
headers: headers
expect(response.status).to eq(200)
expect(response.body).to eq(attendee.to_json)
end
context "requesting a non-existent ID" do
it "returns an error" do
get "/single_coach_middleware/attendees/123",
headers: headers
expect(response.status).to eq(404)
expect(response.body).to eq({ error: "Not found" }.to_json)
end
end
describe "authentication" do
context "with a user with insufficient permissions" do
let(:user) { FactoryGirl.create(:user) }
it "returns an error" do
get "/single_coach_middleware/attendees/#{attendee.id}",
headers: headers
expect(response.status).to eq(403)
expect(response.body).
to eq({ error: "Invalid permissions" }.to_json)
end
end
context "not providing an Authorization header" do
before { headers.delete("Authorization") }
it "returns an error" do
get "/single_coach_middleware/attendees/#{attendee.id}",
headers: headers
expect(response.status).to eq(401)
expect(response.body).
to eq({ error: "Missing access token" }.to_json)
end
end
context "providing an invalid access token" do
before { headers["Authorization"] = "Bearer lolwut" }
it "returns an error" do
get "/single_coach_middleware/attendees/#{attendee.id}",
headers: headers
expect(response.status).to eq(401)
expect(response.body).
to eq({ error: "Invalid access token" }.to_json)
end
context "requesting errors in French" do
before { headers["Accept-Language"] = "fr" }
it "returns an error in French" do
get "/single_coach_middleware/attendees/#{attendee.id}",
headers: headers
expect(response.status).to eq(401)
expect(response.body).
to eq({ error: "Jeton d'accès invalide" }.to_json)
end
end
end
end
end
Round-up
We've discovered how Coach can help us write modular, unit testable APIs that are easy to reason about.
We started with an ActionController
action, depending on a bunch of
before_action
s and even an around_action
, which was hard to test, made it
tricky to pull out abstractions and was confusing for the developer.
We rewrote that controller action as a single Coach action, encapsulated in a
Coach::Middleware
and called from our Rails router. We wrote a single unit
test for that action.
We then pulled our individual jobs that our action did into separate,
individually-tested Coach::Middleware
that our top-level "action" middleware
then depended on. We start with building Middleware::Translate
, then
Middleware::Authenticate
and then two configurable (and thus easily reusable)
middleware, Middleware::CheckUserPermissions
and Middleware::GetModelById
.
You can see all these steps, complete with tests, in our demo project.
Coach encourages patterns that have scaled well with our engineering team, while new joiners frequently say Coach is their favourite part of our codebase. We'd love to see other teams enjoy the benefits we've already gained from Coach.
Building a world-class payments API is at the heart of what we do at GoCardless, and Coach has been one of the key tools we’ve used (and built!) to maintain simplicity in our codebase as we strive towards this goal.