<div>Is there any tool to provide anti-multi-thread-spider function based on logs?</div>
<div>Some mad spiders drive my host to heavy load. :(<br><br> </div>
<div><span class="gmail_quote">2008/3/13, dika <<a href="mailto:andika@agrakom.com">andika@agrakom.com</a>>:</span>
<blockquote class="gmail_quote" style="PADDING-LEFT: 1ex; MARGIN: 0px 0px 0px 0.8ex; BORDER-LEFT: #ccc 1px solid">
<div text="#000000" bgcolor="#ffffff">Hi there,<br><br>Unfortunately nginx doesn't support piping log method inside it's config script, so you have to set it separately.<br>try this:<br><br>~ point your access/error log to a file, such as <i>logs/access.log</i> and <i>logs/error.log</i><br>
~ if <i>logs/access.log</i> and <i>logs/error.log</i> has been exist, delete them, and recreate as fifo's<br>~ <i>mkfifo logs/access.log logs/error.log</i><br>~ make a .sh files to put a cronolog script and piping, like:<br>
<br>#!/bin/bash<br><pre><code>(cat /usr/local/nginx/logs/access.log |/usr/local/sbin/cronolog -l /var/log/nginx/access.log \
/var/log/nginx/%Y/%m/%d/%H/access.log) &
(cat /usr/local/nginx/logs/error.log |/usr/local/sbin/cronolog -l /var/log/nginx/error.log \
/var/log/nginx/%Y/%m/%d/%H/error.log) &</code></pre>~ start/execute thus file whenever you start your nginx<br><br><small><small>Source and references:<br><a onclick="return top.js.OpenExtLink(window,event,this)" href="http://pjkh.com/articles/nginx-chronolog.html" target="_blank">http://pjkh.com/articles/nginx-chronolog.html</a></small></small><br>
<br><small><small>rgrds,</small></small><br>Andika<br><b><small>INDONESIAN</small></b>
<div><span class="e" id="q_118a6f794a7829de_1"><br><br>Corey Donohoe wrote:
<blockquote type="cite"><pre>We use that logrotate config on like 1500+ linux instances. It
works... really. I too have been burned by various things wrt log
rotation, but nginx shows nothing but love. ;)
On 3/12/08, Dave Cheney <a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:dave@cheney.net" target="_blank"><dave@cheney.net></a> wrote:
</pre>
<blockquote type="cite"><pre>Meh - i've had logrotate screw up on my too many times and delete a
log file that things are logging into.
On 13/03/2008, at 2:46 PM, Corey Donohoe wrote:
> Yo,
>
> We do something like this at Engine Yard.
>
> Daily nginx logrotation during non-peak hours:
>
> <a onclick="return top.js.OpenExtLink(window,event,this)" href="http://pastebin.com/f400c3e0e" target="_blank">http://pastebin.com/f400c3e0e</a>
>
> Daily log migration to a drive with more space:
>
> <a onclick="return top.js.OpenExtLink(window,event,this)" href="http://pastebin.com/f6bc64e94" target="_blank">http://pastebin.com/f6bc64e94</a>
>
> On 3/12/08, Dave Cheney <a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:dave@cheney.net" target="_blank"><dave@cheney.net></a> wrote:
>> Try something like this
>>
>
>
> --
> Corey Donohoe
> <a onclick="return top.js.OpenExtLink(window,event,this)" href="http://www.atmos.org/" target="_blank">http://www.atmos.org/</a>
> <a onclick="return top.js.OpenExtLink(window,event,this)" href="http://www.engineyard.com/" target="_blank">http://www.engineyard.com/</a>
>
</pre></blockquote><pre> </pre></blockquote><br></span></div></div></blockquote></div><br><br clear="all"><br>-- <br>bianbian coding life:<br><a href="http://bianbian.org">http://bianbian.org</a>