Lately I've been spending a lot of time playing with Docker, particularly with Web UIs and "clustering" APIs. I've been using Nginx and uWSGI for most of my sites for quite some time now. My normal go-to for distributing load is with nginx's upstream directive.
This directive can be used to specify the address/socket of backend services that should handle the same kinds of requests. You can configure the load balancing pretty nicely right out of the box. However, when using Docker containers, you don't always know the exact IP for the container(s) powering your backend.
I played around with some fun ways to automatically update the nginx configuration and reload nginx each time a backend container appeared or disappeared. This was really, really cool to see in action (since I'd never attempted it before). But it seemed like there had to be a better way.
Mongrel2 came to mind. I've played with it in the past, and it seemed to handle my use cases quite nicely until I tried using it with VirtualBox's shared folders. At the time, it wasn't quite as flexible as nginx when it came to working with those shared folders (might still be the case). Anyway, the idea of having a single frontend that could seamlessly pass work along to any number of workers without being reconfigured and/or restarted seemed like the ideal solution.
As I was researching other Mongrel2-like solutions, I stumbled upon yet another mind-blowing feature tucked away in uWSGI: The uWSGI FastRouter.
This little gem makes it super easy to get the same sort of functionality that Mongrel2 offers. Basically, you create a single uWSGI app that will route requests to the appropriate workers based on the domain being requested. Workers can "subscribe" to that app to be added to the round-robin pool of available backends. Any given worker app can actually serve requests for more than one domain if you so desire.
On the nginx side of things, all you need to do is use something like uwsgi_pass with the router app's socket. That's it. You can then spawn thousands of worker apps without ever restarting nginx or the router app. Whoa.
So let's dig into an example. First, some prerequisites. I'm currently using:
- nginx 1.6.0
- uwsgi 2.0.4
- bottle 0.12.7
- Python 3.4.1
- Arch Linux
The first thing we want is that router app. Here's a uWSGI configuration file I'm using:
[uwsgi] plugins = fastrouter master = true shared-socket = 127.0.0.1:3031 fastrouter-subscription-server = 0.0.0.0:2626 fastrouter = =0 fastrouter-cheap = true vacuum = true # vim:ft=dosini et ts=2 sw=2 ai:
So, quick explanation of the interesting parts:
- shared-socket: we're setting up a shared socket on 127.0.0.1:3031. This is the socket that we'll use with nginx's uwsgi_pass directive, and it's also used for our fastrouter socket (=0 implies that we're using socket 0).
- fastrouter-subscription-server: this is how we make it possible for our worker apps to become candidates to serve requests.
- fastrouter-cheap: this disables the fastrouter when we have no subscribed workers. Supposedly, you can get the actual fastrouter app to also be a subscriber automatically, but I was unable to get this working properly.
Now let's look at a sample worker app configuration:
[uwsgi] plugins = python master = true processes = 2 threads = 4 heartbeat = 10 socket = 192.*:0 subscribe2 = server=127.0.0.1:2626,key=foo.com wsgi = app vacuum = true harakiri = 10 max-requests = 100 logformat = %(addr) - %(user) [%(ltime)] "%(method) %(uri) %(proto)" %(status) %(size) "%(referer)" "%(uagent)" # vim:ft=dosini et ts=2 sw=2 ai:
- socket: we're automatically allocating a socket on our NIC with an IP address that looks like 192.x.x.x. This whole syntax was a new discovery for me as part of this project! Neat stuff!!
- subscribe2: this is one of the ways that we can subscribe to our fastrouter. Based on the server=127.0.0.1:2626 bit, we're working on the assumption that the fastrouter and workers are all going to be running on the same host. The key=foo.com is how our router app knows which domain a worker will serve requests for.
- wsgi: our simple Bottle application.
Now let's look at our minimal Bottle application:
from bottle import route, default_app application = default_app() application.catchall = False @route('/') def index(): return 'Hello World!'
All very simple. The main thing to point out here is that we've imported the default_app function from bottle and use it to create an application instance that uWSGI's wsgi option will use automatically.
Finally, our nginx configuration:
daemon off; master_process on; worker_processes 1; pid nginx.pid; events { worker_connections 1024; } http { include /etc/nginx/mime.types; access_log ./access.log; error_log ./error.log; default_type application/octet-stream; gzip on; sendfile on; keepalive_timeout 65; server { listen 80 default; server_name localhost foo.com; location / { include /etc/nginx/uwsgi_params; uwsgi_pass 127.0.0.1:3031; } error_page 500 502 503 504 /50x.html; location = /50x.html { root /usr/share/nginx/html; } } } # vim:filetype=nginx:
Nothing too special about this configuration. The only thing to really point out is the uwsgi_pass with the same address we provided to our router's shared-socket option. Also note that it will bind to port 80 by default, so you'll need root access for nginx.
Now let's run it all! In different terminal windows, run each of the following commands:
sudo nginx -c nginx.conf -p $(pwd) uwsgi --ini router.ini uwsgi --ini worker.ini
If all goes well, you should see no output from the nginx command. The router app should have some output that looks something like this:
spawned uWSGI master process (pid: 4367) spawned uWSGI fastrouter 1 (pid: 4368) [uwsgi-subscription for pid 4368] new pool: foo.com (hash key: 11571) [uwsgi-subscription for pid 4368] foo.com => new node: :58743 [uWSGI fastrouter pid 4368] leaving cheap mode...
And your worker app should have output containing:
subscribing to server=127.0.0.1:2626,key=foo.com
For the purpose of this project, I quickly edited my /etc/hosts file to include foo.com as an alias for 127.0.0.1. Once you have something like that in place, you should be able to hit the nginx site and see requests logged in your worker app's terminal:
curl foo.com
The really cool part is when you spin up another worker (same command as before, since the port is automatically assigned). Again, there's no need to restart nginx nor the router app--the new worker will be detected automatically! After doing so, each request will be spread out across all of the subscribed workers.
Here's a quick video of all of this in action, complete with multiple worker apps subscribing to one router app. Pay close attention to the timestamps in the worker windows.
While this is all fine and dandy, there are a couple of things that seem like they should have better options. Namely, I'd like to get the single FastRouter+worker configuration working. I think it would also be nice to be able to use host names or DNS entries for the workers to know how to connect to the FastRouter instance. Any insight anyone can offer would be greatly appreciated! I know I'm just scratching the surface of this feature!
