I just wanted to let everyone know that I tried the method Rob Shultz suggested, and it works. In case it helps someone in the future, I'll sum up the problem and solution here.<div><br></div><div>My setup and requirements were as follows:<div>
<ul><li>custom CMS implemented with Rails</li><li>single app instance serves many websites</li><li>each website uses http and https</li><li>each website has an SSL certificate for its own domain, so each needs its own dedicated IP</li>
<li>hosting company policy limits me to five IP addresses per slice</li><li>each site has its own static file directory</li><li>CMS makes heavy use of caching (each site has its own cache directory)</li></ul></div><div>The problem was that in order to realize decent economies of scale, I needed to be able to host far more than five websites on my slice. In addition, for reasons of cost and convenience, I only wanted to maintain a single instance of the Rails app rather than running a copy of it on multiple servers. The IP address limit of five per slice was a killer.</div>
<div><br></div><div>The solution to the problem was to set up nginx on multiple front-end slices which would each proxy requests to nginx on a single back-end slice, which would itself then proxy those requests to the Rails app. I got this set up, and it worked great, except for one thing. Because the requests to the back-end nginx are always received from the front-end nginx over http (even when the client request to the front-end nginx is over https), the Rails app was unable to tell whether the client's original request was over http or https. (I couldn't simply proxy from the front-end nginx directly to the back-end Rails app due to the app's caching requirements.)</div>
<div><br></div><div>Per Rob's suggestion, I configured two server directives per site on the back-end nginx -- one for http and one for https. However, since the back-end nginx wouldn't be talking to clients directly, it does not need to listen on ports 80 or 443. Instead, it can listen on arbitrary port numbers as long as the front-end nginx instances know what those port numbers are. As such, the two server directives for the website <a href="http://one.com">one.com</a> on the back-end nginx can listen on ports 4000 (for http) and 4001 (for https). The server directives for <a href="http://one.com">one.com</a> on the front-end nginx will proxy requests for <a href="http://one.com">http://one.com</a> (port 80) to the back-end nginx on port 4000, and it will proxy requests for <a href="https://one.com">https://one.com</a> (port 443) to the back-end nginx on port 4001.</div>
<div><br></div><div>The significance of listening on multiple ports is that the back-end nginx can tell the Rails app that requests to port 4000 server were originally made over http and that requests to port 4001 were originally made over https. I'll attempt to illustrate here (this won't look right without a fixed-width font).</div>
<div><br></div><div><div><span class="Apple-style-span" style="font-family: 'courier new', monospace;"> front-end back-end</span></div><div><span class="Apple-style-span" style="font-family: 'courier new', monospace;"><a href="http://one.com">one.com</a> +----------+ +-----------+</span></div>
<div><span class="Apple-style-span" style="font-family: 'courier new', monospace;">request ---http---> | port 80 | ---port:4000---> | port 4000 | ---proto:http---> +-----------+</span></div><div><span class="Apple-style-span" style="font-family: 'courier new', monospace;"> \ | | | | | Rails app |</span></div>
<div><span class="Apple-style-span" style="font-family: 'courier new', monospace;"> \--https--> | port 443 | ---port:4001---> | port 4001 | ---proto:https--> +-----------+</span></div><div><span class="Apple-style-span" style="font-family: 'courier new', monospace;"> +----------+ +-----------+</span></div>
</div><div><br></div><div>Only one front-end slice is shown, and it is shown only for one site, but this should give you an idea of how this can be expanded. The front-end slice can (in my case) be expanded to host five sites, and more slices can be added as needed.</div>
<div><br></div><div>Of course, to ensure the back-end server isn't tricked, it's necessary to make sure the server is set up to listen on ports 4000 and 4001 only from the front-end servers.</div><div><br></div><div>
If anyone else runs into a similar IP limitation or has some other need to proxy http and https traffic through two instances of nginx, I hope this helps you out.</div><div><br></div><div>Nick</div><div><br></div><div><br>
</div><div><br><div class="gmail_quote">On Thu, Oct 30, 2008 at 4:05 PM, Nick Pearson <span dir="ltr"><<a href="mailto:nick.pearson@gmail.com">nick.pearson@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Yes, and that's my plan exactly. The only reason I need to listen on two separate ports for each site is that each site caches its content independently, which means that nginx has to be able to look for the cached content and server that up without ever touching Rails. So, for two sites to be able to each have a cached index.html file (as well as static image files), I have to have a site-specific path in each server directive.<br>
<br>For instance, consider the following:<br><br> server {<br> listen 80;<br> server_name <a href="http://site-a.com" target="_blank">site-a.com</a> *.<a href="http://site-a.com" target="_blank">site-a.com</a>; # needs to be site-specific<br>
root /var/www/site-a;<br> location / {<br> # serve static files<br> if (-f $document_root$uri.html) {<br> rewrite (.*) $1.html break;<br> break;<br> }<br> # serve cached pages directly<br>
if (-f $document_root/../../../current/tmp/cache/content/site-a/$uri.html) {<br> rewrite (.*) $document_root/../../../current/tmp/cache/content/site-a/$1.html break;<br> }<br> }<br> }<br><br>
I realize I could set the root to "/var/www" (and drop the "/site-a"), then use the $host or $http_host variable in my static/cache paths, but my CMS supports *.domain.com-style vhosts, which can't be represented on the file system. If I drop the *.domain.com-style vhost support, then I could have paths like /var/www/<a href="http://site-a.com" target="_blank">site-a.com</a> with symlinks pointing to it (like /var/www/<a href="http://www.site-a.com" target="_blank">www.site-a.com</a> -> /var/www/<a href="http://site-a.com" target="_blank">site-a.com</a>).<br>
<br>Even if I could figure out a good way to represent this on the file system, the CMS (and my nginx config for serving static and cached content) supports serving different files for a request to the same site based on the requested host. This is useful (and is actually being used) for a company with multiple locations that wants a site tailored to each location. For instance, when you request <a href="http://site-a.com" target="_blank">site-a.com</a>, you see the home page with the address and phone number for the company's primary location in the header. Requesting <a href="http://site-b.com" target="_blank">site-b.com</a> shows the exact same home page except that the header now has the address and phone number for the company's secondary location. Similarly, a slightly different logo image can be served for <a href="http://site-b.com" target="_blank">site-b.com</a>, even though both images are at /images/logo.gif. As such, simply symlinking /var/www/<a href="http://site-b.com" target="_blank">site-b.com</a> to point to /var/www/<a href="http://site-a.com" target="_blank">site-a.com</a> would break this functionality.<br>
<br>I still think the original solution will work -- I'll just have to have two server directives on the back-end nginx for each site (one for http, and one for https). This isn't a problem, as this is how it works now -- only now, the backend nginx uses server_name to choose the proper server directive whereas with the new solution it will use an internal IP and port number to do the same thing.<br>
<font color="#888888">
<br>Nick</font></blockquote></div></div></div>