Hello Jeff...<br><br><div class="gmail_quote">On Fri, Jan 28, 2011 at 3:08 AM, Jeff Mitchell <span dir="ltr"><<a href="mailto:jeff@jefferai.org">jeff@jefferai.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hello,<br>
<br>
We have a particular application (gitweb) that performs a particular,<br>
extraordinarily slow function when the home page is loaded. As the<br>
number of repositories has increased, this has grown to take several<br>
*minutes* per page view (yes, ugh).<br>
<br>
To combat this, we tried setting up cache so that this only occurs once<br>
an hour, but it's still causing too much of a problem...both because it<br>
doesn't always last an hour (sometimes we see the delay happen again<br>
sooner) and because when it does expire, it simply takes too long (and<br>
if multiple clients connect at that same time making that request, it<br>
gets even worse).<br></blockquote><div><br>I think you look the wrong way at the problem you have. You should fix these issues you mention which are NOT the expected behavior before trying to fix your situation with your proposed patch of allowing a different time to a special IP and leave a bot requesting the page.<br>
<br>First problem: early renewal of page.<br><br>This is not a mistery why it happens: Either your server it's returning a header that tells nginx to refresh the cache, or you have a problem inside your configuration. If you tell nginx to update each 60 min, it will always respect that except backend server tells nginx to refresh the cache. <br>
<br>Second problem: multiple clientes renewing cache.<br><br>This can easily be fixed by using proxy_cache_stale, you can set it up to return users the last cached page while the backend ends updating the first request that expired. This means that after a requests pass to the backend, while that request does not complete, nginx will return the last cached (which already expired) version to any new request for that page.<br>
<br>If you fix those both problems then I think you won't need all that workaround you want to do.<br><br>I'm not nginx guru and maybe it's me looking the wrong way at your problem but in case it help this is how I dealt before with the same problem you described.<br>
<br></div>Good Luck!<br><br>Guzmán Brasó<br>--<br><a href="http://www.guruhub.com.uy">http://www.guruhub.com.uy</a> <br>+59898674020<br>Montevideo, Uruguay<br><br></div><br>