In my original post about zero downtime deployments I wanted to use puma. At the time of writing puma did not support zero downtime restarts: while the connection was kept alive, all workers were killed at once so no requests could be served until the new workers had fully started up.

This changed as of puma v2.0.0.b6. Now you can send SIGUSR1 to the puma master process and puma will phase out old workers while starting new workers one at a time.

Note that this process takes longer than unicorns SIGUSR2 + preload_app restarts because unicorn spawns your new workers all at the same time; this means that puma requires roughly n-times your apps launch time to complete a phased restart.

During your relaunch process you'll end up with workers running old and new code at the same time. Just make sure you don't break your old workers by running incompatible migrations immediatly :)

If you are using supervisord, foreman and mina, here's a short description on how I got it working:

Setup

First, you'll need to be running puma in clustered mode. In this example I'll spawn one master process and three worker processes:

# Procfile
app: puma -p 8619 --workers 3

The good thing about puma is that we do not need a wrapper like unicornherder to handle changes in PID since the master always stays around.

Running foreman export supervisord will leave use with something like this:

# /etc/supervisor/conf.d/app.conf
[program:app-1]
command=bundle exec puma -p 8619 --workers 3 --dir /home/app/current
autostart=true
autorestart=true
stopsignal=QUIT
stdout_logfile=/home/app/shared/log/website-1.log
stderr_logfile=/home/app/shared/log/website-1.error.log
user=app
directory=/home/app/current
environment=RAILS_ENV="production"

[group:app]
programs=app-1

As you can see I just generated a supervisord configuration which directly starts puma in clustered mode.

Note It's important that you add the --dir /path/to/current option, since puma won't pick up changes to your code base otherwise.

Deployment with mina

Assuming this is not our very first deployment, we need to restart workers using minas to :launch directive. To issue a phased restart we need to do the following:

1. finding pumas master pid by:

   • listing all ruby processes via ps -C ruby -F
   • greping for /puma (only the process spawned by supervisor will contain this line)
   • using awk to get the process id via awk {'print $2'}

2. sending SIGUSR1 to the puma master process to initiate a rolling restart

Minas deploy task looks like this:

task :deploy => :environment do
  deploy do
    invoke :'git:clone'
    invoke :'deploy:link_shared_paths'
    invoke :'bundle:install'
    invoke :'rails:assets_precompile'

    to :launch do
      queue %[kill -s SIGUSR1 $(ps -C ruby -F | grep '/puma' | awk {'print $2'})]
    end
  end
end

That's it.

Let's try out our setup using ab:

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking blog.nicolai86.eu (be patient).....done


Server Software:        nginx/1.2.4
Server Hostname:        blog.nicolai86.eu
Server Port:            80

Document Path:          /
Document Length:        10817 bytes

Concurrency Level:      6
Time taken for tests:   3.430 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      1163452 bytes
HTML transferred:       1096866 bytes
Requests per second:    29.15 [#/sec] (mean)
Time per request:       205.801 [ms] (mean)
Time per request:       34.300 [ms] (mean, across all concurrent requests)
Transfer rate:          331.25 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       56   60   3.1     58      73
Processing:   130  142   9.5    140     173
Waiting:       70   81   9.0     78     111
Total:        189  202  10.4    200     237

Percentage of the requests served within a certain time (ms)
  50%    200
  66%    203
  75%    206
  80%    209
  90%    218
  95%    225
  98%    230
  99%    237
 100%    237 (longest request)

As you can see - it works. Also note the log output when sending SIGUSR1 to puma:

started with pid 38012
[38012] Puma 2.0.0.b6 starting in cluster mode...
[38012] * Process workers: 3
[38012] * Min threads: 0, max threads: 16
[38012] * Environment: development
[38012] * Listening on tcp://0.0.0.0:8619
[38012] Use Ctrl-C to stop
[38012] - Worker 38016 booted, phase: 0
[38012] - Worker 38015 booted, phase: 0
[38012] - Worker 38017 booted, phase: 0
[38012] - Starting phased worker restart, phase: 1
[38012] - Stopping 38015 for phased upgrade...
[38012] - Worker 38119 booted, phase: 1
[38012] - Stopping 38016 for phased upgrade...
[38012] - Worker 38132 booted, phase: 1
[38012] - Stopping 38017 for phased upgrade...
[38012] - Worker 38146 booted, phase: 1

If you see similar output when sending SIGUSR1 your phased restarts using puma are working as expected!

Wrapping up:

• puma supports phased restarts in clustered mode since v2.0.0.b6
• using phased restarts we can achieve zero downtime deployments with puma
• phased restarts work by replacing workers one by one, which takes some time to complete if you are running many workers. But it also consumes less memory than having twice the number of workers running
• if a worker fails to start up puma master tries to restart the worker. Only if the new worker starts up successfully will puma replace the old worker.

That's it! Happy hacking!

Last week I solved a recurring problem in my job as a software developer: Combining different parts of a user interface which all need to change the URL parameters to work properly.

The Problem

Take for a example a dataset presented using a paginated table. The table can be ordered using any of the displayed columns and the dataset can be filtered using a form and some links.

The resulting URL could consist of the following parameters:

• pagination: a Hash containing page, the current page being displayed and per_page, the number of items on display
• order: attribute to order the data by
• direction: direction to order the data in
• filter: hash containing different filter options

Changing the current page, adjusting the filter or ordering should not change the remaining URL parameters - you don't want your users to always start on page one just because they changed the ordering in the table.

A Common Solution

In Ruby on Rails one can merge parts of the URL parameters using a Hash and passing the resulting Hash to any path helper, e.g.

root_path({ pagination: { page: 1 } }.merge({ order: "attribute" }))

will generate something like this

/dashboard?pagination[page]=1&order=attribute

Now this code is tedious to write and ugly to read. As you add more and more options to the URL this code gets more and more out of control.

Enter UrlPlumber

Instead of merging Hashs you specify the attribute you want to change using a keypath and a value. The other URL parameters will remain unchanged.

The above example could be rewritten as follows:

plumber = ::UrlPlumber::Plumber.new(params)
root_path(plumber.plumb("order", "attribute"))

Moving UrlPlumber into a simple helper function allows us to further simplify the code:

root_path(plumb("order", "attribute"))

This also works with arbitrary nesting; some examples:

plumb("table.pagination.page", 2) # => changes table[pagination][page] param to 2
plumb("pagination") # => removes pagination from the URL

UrlPlumber is designed for this common scenario. It works just as you might expect, and is completly tested. Integration in existing Ruby on Rails projects should be painless and simple.

Give it a try if you stumble across this problem one day :)

I've been playing around with IcedCoffeeScript for a while now and wanted an easy integration with Ruby on Rails Asset Pipeline.

The result is iced-rails. I decided to introduce the .iced file extension for IcedCoffeeScript, rather than overwriting .coffee even though IcedCoffeeScript is a superset of CoffeeScript. The Iced command line also defaults to .iced and I want to avoid surprises when using the gem.

IcedCoffeeScript generates continuations to avoid callback pyramids by introducing two new keywords, await and defer. You can find more details about IcedCoffeeScript here.

At the moment the Asset Pipeline integration has one caveat in that the iced boilerplate is generated per .iced file. Ideally it should be generated only once. I guess I'll add this feature in a later version.