So what I tried was separating it into two copies of nginx (both with the S3 patch): one that handles all of the local access, and that in the case where the file doesn't exist locally and needs to get sent to S3, it proxies it to the 2nd copy of nginx. *That* copy has the S3 forwarding, and with this setup, now it seems to work ok.<div>
<br></div><div>The next issue I have is that nginx seems to block and freeze on filesystem access if there are problems - we had a problem w/ one network mount and even w/ timeouts specified:</div><div><br></div><div><div>
sendfile<span class="Apple-tab-span" style="white-space:pre"> </span>on;</div><div> tcp_nopush<span class="Apple-tab-span" style="white-space:pre"> </span>on;</div><div> tcp_nodelay<span class="Apple-tab-span" style="white-space:pre"> </span>on;</div>
<div><br></div><div> client_body_timeout<span class="Apple-tab-span" style="white-space:pre"> </span>5;</div><div> client_header_timeout<span class="Apple-tab-span" style="white-space:pre"> </span>5;</div><div> send_timeout<span class="Apple-tab-span" style="white-space:pre"> </span>1;</div>
<div><br></div><div> # keepalive_timeout<span class="Apple-tab-span" style="white-space:pre"> </span>0;</div><div> # keepalive_timeout<span class="Apple-tab-span" style="white-space:pre"> </span>75 20;</div><div> keepalive_timeout<span class="Apple-tab-span" style="white-space:pre"> </span>5 5;</div>
</div><div><br></div><div><br></div><div>We still had nginx hang for a long time on an access to a filesystem that was having issues. (We're using glusterfs to mount remote volumes from our storage servers.) So we put in gluster timeouts, and that seems to time out the gluster volumes, but nginx still remains locked up for an unacceptable amount of time.</div>
<div><br></div><div>Is there a way to have nginx timeout the request (return a 503 or something, for instance) if it doesn't get a response from the filesystem in x seconds? Ideally, we'd like it to time out after just a second or two.</div>
<div><br></div><div>What I'm looking at now is having Varnish in front of nginx and seeing if I can get *that* timeout to work, but obviously this would be a lot cleaner with fewer layers. :)</div><div><br></div><div>
Thanks,</div><div>Nicholas</div><div><br></div><div><br><div><span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:medium;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div style="word-wrap:break-word">
<span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:medium;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div style="word-wrap:break-word">
<span style="border-collapse:separate;color:rgb(0, 0, 0);font-family:Helvetica;font-size:medium;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div style="word-wrap:break-word">
<div><table style="border-top:1px dashed rgb(204, 204, 204);border-bottom:1px dashed rgb(204, 204, 204);width:585px" cellpadding="5" width="100%"><tbody><tr><td><div style="font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:9px;color:rgb(153, 153, 153);line-height:9pt">
<div style="float:left"><strong style="color:rgb(255, 0, 0)">Nicholas Tang</strong><strong style="color:rgb(85, 85, 85);margin-left:3px;margin-right:3px">:</strong></div><div style="float:left;font-weight:bold;color:rgb(85, 85, 85);margin-right:3px">
VP, Dev Ops<br></div><br><div style="float:left;margin-right:5px"><a href="mailto:nicholas.tang@livestream.com" title="Email me" style="color:rgb(153, 153, 153);text-decoration:none" target="_blank">nicholas.tang@livestream.com</a></div>
<div style="float:left;font-weight:bold;color:rgb(204, 204, 204);margin-right:5px">|</div><div style="float:left;margin-right:5px">t: +1 (646) 495 9707</div><div style="float:left;font-weight:bold;color:rgb(204, 204, 204);margin-right:5px">
|</div><div style="float:left;margin-right:5px">m: +1 (347) 410 6066<br></div><div style="float:left;font-weight:bold;color:rgb(204, 204, 204);margin-right:3px">|</div><div style="float:left">111 8th Avenue, Floor 15, New York, NY 10011</div>
</div></td><td style="width:100px" width="100px"><a href="http://www.livestream.com/" title="Visit livestream.com" target="_blank"><img src="http://livestream-email.s3.amazonaws.com/signatures/ls-logo-100x25.jpg" alt="www.livestream.com" border="0" height="25" width="100"></a></td>
</tr></tbody></table></div></div></span></div></span></div></span> </div><br>
<br><br><div class="gmail_quote">On Mon, Dec 20, 2010 at 1:02 PM, Piotr Sikora <span dir="ltr"><<a href="mailto:piotr.sikora@frickle.com">piotr.sikora@frickle.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi,<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
We're running nginx 0.7.68 w/ the s3-proxy patch applied, and we're seeing reproducible segfaults when we hit (or miss) certain files:<br>
</blockquote>
<br></div>
Can you reproduce them without s3-proxy patch?<br>
<br>
Best regards,<br>
Piotr Sikora < <a href="mailto:piotr.sikora@frickle.com" target="_blank">piotr.sikora@frickle.com</a> ><br>
<br>
<br>
_______________________________________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
<a href="http://nginx.org/mailman/listinfo/nginx" target="_blank">http://nginx.org/mailman/listinfo/nginx</a><br>
</blockquote></div><br></div>