<br><br><div class="gmail_quote">On Thu, Dec 3, 2009 at 5:35 PM, agentzh <span dir="ltr"><<a href="mailto:agentzh@gmail.com">agentzh@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">On Fri, Dec 4, 2009 at 7:13 AM, Brian Pane <<a href="mailto:brianp@brianp.net">brianp@brianp.net</a>> wrote:<br>
> A follow-up question: When a body filter is processing buffers that<br>
> were produced by the proxy module (i.e., buffers containing an HTTP response<br>
> from a back-end server), is it safe to discard those buffers?<br>
<br>
</div>I think so :) My ngx_chunkin module's output body filter does<br>
something like that to simply disgard the contents of the 411 error<br>
page and I don't think an upstream module like ngx_proxy will have any<br>
difference here :)<br>
<div class="im"><br>
> Or is it necessary to<br>
> pass the buffers through to the standard filters?<br>
<br>
</div>Then your client will probably see those stuffs sooner or later, which<br>
is certainly not what you want :)<br>
<div class="im"><br>
> It's not clear to me<br>
> whether anything in the standard filter chain needs to see the buffers<br>
> containing the upstream response in order to do cleanup or flow control.<br>
<br>
</div>No, at least according to my knowledge :) Cleanup and flow control<br>
usually happens in places like ngx_http_finalize_request, or a<br>
read/write event handler. A notable exception might be bufs with the<br>
temporary field set to 1. These bufs might need to be freed sooner<br>
rather than later, but they're usually allocated in the current<br>
request's pool anyway, it may not be a very big issue.<br></blockquote><div><br></div><div>After an afternoon with gdb, I think I've found one dependency: before</div><div>throwing away the buffers, the body filter needs to set buf->pos=buf->last</div>
<div>for each one. This is due to the code at line ~201 of core/ngx_output_chain.c:</div><div><br></div><div><div> last = ctx->output_filter(ctx->filter_ctx, out);</div><div><br></div><div> if (last == NGX_ERROR || last == NGX_DONE) {</div>
<div> return last;</div><div> }</div><div><br></div><div> ngx_chain_update_chains(&ctx->free, &ctx->busy, &out, ctx->tag);</div><div> last_out = &out;</div><div><br>
</div><div>After the filter returns, that code still references the filter chain that was</div><div>passed into the filter (the variable "out"). It looks like the event loop</div><div>in ngx_event_pipe_write_to_downstream counts up the unsent bytes in</div>
<div>this chain, where unsent for each buf is computed as buf->last - buf->pos.</div><div>I don't yet understand all the logic that follows, but the end result was that</div><div>for large responses, I'd eventually cross a threshold where my filter kept</div>
<div>getting called over and over with no new data.</div><div><br></div><div>-Brian</div><div><br></div></div></div>