'Connection reset by peer' when replying before the end of POST data

'Connection reset by peer' when replying before the end of POST data

[Lunar]

Hi!

Short introduction: I am working on a simple "one-click" file
sharing web application specific focus on protecting users' privacy [1].
Thanks to Rainbows! it can work with big uploads without any request
buffering which is simply marvelous! :)

I am currently trying to implement a limit on the maximum uploaded file
size, not unlike what is already done by Rainbows::MaxBody.
Unfortunately, it looks like answering a request while the client is in
the middle of posting data is not supported that well by Rainbows!

Here is a minimal test case:

--- 8< --- config.ru ---------------------------------------------------

class InterruptTest
  def call(env)
    # HTTP 1.1 standard (and curl) needs this
    /\A100-continue\z/i =~ env['HTTP_EXPECT'] and return [ 100, {}, [] ]

    error = "Request entity too large!\n"
    env['rack.input'].read 1000
    Rainbows.sleep 1
    [ 403, { 'Content-Type' => 'text/plain' }, [ error ] ]
  end
end

run InterruptTest.new

--- >8 -----------------------------------------------------------------

--- 8< --- rainbows.conf -----------------------------------------------

Rainbows! do
  use :ThreadSpawn
  rewindable_input false
  client_max_body_size nil
end

--- >8 -----------------------------------------------------------------

I am starting Rainbows! with the following command-line:

    $ rainbows -E none -p 8081 -c rainbows.conf rackup.ru

And then, when asking curl (the `test` file is 7636 bytes long):

    $ curl -v -F "file=@test;type=text/plain" http://localhost:8081
    * About to connect() to localhost port 8081 (#0)
    *   Trying 127.0.0.1... connected
    * Connected to localhost (127.0.0.1) port 8081 (#0)
    > POST / HTTP/1.1
    > User-Agent: curl/7.21.0 (i486-pc-linux-gnu) libcurl/7.21.0
    > OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
    > Host: localhost:8081
    > Accept: */*
    > Content-Length: 7829
    > Expect: 100-continue
    > Content-Type: multipart/form-data;
    > boundary=----------------------------cd790f73307f
    > 
    < HTTP/1.1 100 Continue
    < HTTP/1.1 403 Forbidden
    < Date: Tue, 28 Feb 2012 21:20:15 GMT
    < Status: 403 Forbidden
    < Connection: close
    < Content-Type: text/plain
    < 
    Request entity too large!
    * Recv failure: Connection reset by peer
    * Closing connection #0
    curl: (56) Recv failure: Connection reset by peer

This "connection reset by peer" is annoying as it will result in Apache
stating "Bad gateway", or Firefox displaying "The connection was reset".

I believe Rainbows::MaxBody having the same issue, but I am not sure.
Also looking at the code, it looks like Rainbows::MaxBody trust the
Content-Length header and do not mesure the actual amount of bytes
received if the header is present. I believe Content-Length can be faked
by malicious clients, so it might be better to use limit_input! for
every connections.

In any cases, I would very much like to solve this issue, but I feel a
little bit lost on where to start.

My assumption was that other webservers were doing it right, otherwise
no one would ever see a 413 Request Entity Too Large message in a
browser. I see that Nginx has options related to SO_LINGER and that
Apache also mention "Lingering Close" when discussing how to close
connections. But I don't really know where to poke on Rainbows! to play
with these kinds of options. Anyway, this issue looks closeloy like the
one described in section 8 of
<http://ftp.ics.uci.edu/pub/ietf/http/draft-ietf-http-connection-00.txt>.

Thanks again for any help solving this. :)

[1] You can have a look at
    <https://git.codecoop.org/projects/coquelicot>, but there is not
    that much to see yet. The code using Rainbows! is not yet in a
    releasable state, but progress is steady, so expect some news
    later. :)
[2] <https://httpd.apache.org/docs/2.2/misc/perf-tuning.html>

Cheers,
-- 
Lunar
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Eric Wong]

Lunar <lunar@anargeek.net> wrote:
> Hi!
> 
> Short introduction: I am working on a simple "one-click" file
> sharing web application specific focus on protecting users' privacy [1].
> Thanks to Rainbows! it can work with big uploads without any request
> buffering which is simply marvelous! :)

Good to know, I also handle a lot of uploads (via PUT, though)

> I am currently trying to implement a limit on the maximum uploaded file
> size, not unlike what is already done by Rainbows::MaxBody.
> Unfortunately, it looks like answering a request while the client is in
> the middle of posting data is not supported that well by Rainbows!

Rainbows! really can't be "nice" to clients that try to abuse it.

> Here is a minimal test case:
> 
> --- 8< --- config.ru ---------------------------------------------------
> 
> class InterruptTest
>   def call(env)
>     # HTTP 1.1 standard (and curl) needs this

If you get a Content-Length, you can return something other than 100
if Content-Length is too much.

>     /\A100-continue\z/i =~ env['HTTP_EXPECT'] and return [ 100, {}, [] ]

>     error = "Request entity too large!\n"
>     env['rack.input'].read 1000

You can also drain the socket to avoid connection resets:

      input = env['rack.input']
      buf = "" # reuse buf to reduce GC pressure
      while input.read(1024, buf)
        # ignore buf, keep reading until nil
      end

>     Rainbows.sleep 1
>     [ 403, { 'Content-Type' => 'text/plain' }, [ error ] ]
>   end
> end
> 
> run InterruptTest.new
> 
> --- >8 -----------------------------------------------------------------
> 
> --- 8< --- rainbows.conf -----------------------------------------------
> 
> Rainbows! do
>   use :ThreadSpawn
>   rewindable_input false
>   client_max_body_size nil
> end
> 
> --- >8 -----------------------------------------------------------------
> 
> I am starting Rainbows! with the following command-line:
> 
>     $ rainbows -E none -p 8081 -c rainbows.conf rackup.ru
> 
> And then, when asking curl (the `test` file is 7636 bytes long):
> 
>     $ curl -v -F "file=@test;type=text/plain" http://localhost:8081
>     * About to connect() to localhost port 8081 (#0)
>     *   Trying 127.0.0.1... connected
>     * Connected to localhost (127.0.0.1) port 8081 (#0)
>     > POST / HTTP/1.1
>     > User-Agent: curl/7.21.0 (i486-pc-linux-gnu) libcurl/7.21.0
>     > OpenSSL/0.9.8o zlib/1.2.3.4 libidn/1.15 libssh2/1.2.6
>     > Host: localhost:8081
>     > Accept: */*
>     > Content-Length: 7829
>     > Expect: 100-continue
>     > Content-Type: multipart/form-data;
>     > boundary=----------------------------cd790f73307f
>     > 
>     < HTTP/1.1 100 Continue
>     < HTTP/1.1 403 Forbidden
>     < Date: Tue, 28 Feb 2012 21:20:15 GMT
>     < Status: 403 Forbidden
>     < Connection: close
>     < Content-Type: text/plain
>     < 
>     Request entity too large!
>     * Recv failure: Connection reset by peer
>     * Closing connection #0
>     curl: (56) Recv failure: Connection reset by peer
> 
> This "connection reset by peer" is annoying as it will result in Apache
> stating "Bad gateway", or Firefox displaying "The connection was reset".

> I believe Rainbows::MaxBody having the same issue, but I am not sure.

Yes MaxBody does have this issue.  It's better to save bandwidth than
show a nice error message if somebody is trying to attack you.
But you can implement the socket draining example above to get the nice
error message.

> Also looking at the code, it looks like Rainbows::MaxBody trust the
> Content-Length header and do not mesure the actual amount of bytes
> received if the header is present. I believe Content-Length can be faked
> by malicious clients, so it might be better to use limit_input! for
> every connections.

It only uses Content-Length as a hint.  Unicorn::StreamInput (the
env["rack.input"] object) won't read _past_ Content-Length so HTTP
pipelining can be used with uploads.

If it reads too little, then it'll just timeout eventually.

> In any cases, I would very much like to solve this issue, but I feel a
> little bit lost on where to start.
> 
> My assumption was that other webservers were doing it right, otherwise
> no one would ever see a 413 Request Entity Too Large message in a
> browser.

They probably drain the socket or wait a bit before closing the
connection.  Rack doesn't give developers much control over how the
socket is managed.

> I see that Nginx has options related to SO_LINGER and that
> Apache also mention "Lingering Close" when discussing how to close
> connections. But I don't really know where to poke on Rainbows! to play
> with these kinds of options. Anyway, this issue looks closeloy like the
> one described in section 8 of
> <http://ftp.ics.uci.edu/pub/ietf/http/draft-ietf-http-connection-00.txt>.

Also, neither Matz Ruby 1.8 nor 1.9 can do lingering close as IO#close
still holds onto the GVL (blocking all clients).

I tried to make close(2) release the GVL for 1.9.3, but it doesn't
interact well if another thread was operating on that IO object.

> Thanks again for any help solving this. :)
> 
> [1] You can have a look at
>     <https://git.codecoop.org/projects/coquelicot>, but there is not
>     that much to see yet. The code using Rainbows! is not yet in a
>     releasable state, but progress is steady, so expect some news
>     later. :)

Cool (especially that you host your own), I don't see an obvious way to
clone your repo and look at it, though :x
_______________________________________________
Rainbows! mailing list - rainbows-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Lunar]

Hi Eric,

Thanks for your quick answer!

Eric Wong wrote:
> Lunar <lunar-L8VBg9NU5dbk1uMJSBkQmQ@public.gmane.org> wrote:
> > I am currently trying to implement a limit on the maximum uploaded file
> > size, not unlike what is already done by Rainbows::MaxBody.
> > Unfortunately, it looks like answering a request while the client is in
> > the middle of posting data is not supported that well by Rainbows!
> [...]
> 
> You can also drain the socket to avoid connection resets:
> 
>       input = env['rack.input']
>       buf = "" # reuse buf to reduce GC pressure
>       while input.read(1024, buf)
>         # ignore buf, keep reading until nil
>       end

I thought about it, but I would rather not do that. Coquelicot is meant
as a small, self-contained application that can also be run at home
behind broadband connections with small bandwidth or quota. If users set
a file limit of 5 MiB, I would rather not drain the whole 700 MiB before
denying the request.
 
> > In any cases, I would very much like to solve this issue, but I feel a
> > little bit lost on where to start.
> > 
> > My assumption was that other webservers were doing it right, otherwise
> > no one would ever see a 413 Request Entity Too Large message in a
> > browser.
> 
> They probably drain the socket or wait a bit before closing the
> connection.  Rack doesn't give developers much control over how the
> socket is managed.

Quoting the "HTTP Connection Management" document [1] which I mentioned
previously: "Servers must therefore close each half of the connection
independently."

After digging some more at Nginx, it looks like Nginx is doing exactly
that. I also looked at Apache and the code responsible for half-closing
both side of the connection is very readable. Function is called
`ap_lingering_close()` in server/connection.c.

It looks like there is no way to call shutdown(2) in Ruby 1.8.7, but
from the documentation, Ruby 1.9.3 has IO#close_read and IO#close_write
method that will call shutdown(2) for each half of a socket.

I don't have Ruby 1.9.3 right now, but I can try to produce a test case
later demonstrating the differences between calling `@io.close` and
`@io.close_write; @io.close`.

In any case, I think it would worth to try to add such call sequences in
Rainbows! and see if it helps. If you can't give it a shot yourself,
could you point me to the relevant part of the code?

[1] <http://ftp.ics.uci.edu/pub/ietf/http/draft-ietf-http-connection-00.txt>

> Also, neither Matz Ruby 1.8 nor 1.9 can do lingering close as IO#close
> still holds onto the GVL (blocking all clients).
> 
> I tried to make close(2) release the GVL for 1.9.3, but it doesn't
> interact well if another thread was operating on that IO object.

I am not that versed in Ruby internal, but I fail to see how this could
be related to first closing the clients write side of the connection.
There should be a window between IO#close_write and IO#close in which in
GVL does not matter, does it?

> > [1] You can have a look at
> >     <https://git.codecoop.org/projects/coquelicot>, but there is not
> >     that much to see yet. The code using Rainbows! is not yet in a
> >     releasable state, but progress is steady, so expect some news
> >     later. :)
> 
> Cool (especially that you host your own), I don't see an obvious way to
> clone your repo and look at it, though :x

I will keep the list in touch when interesting meat is available. In the
meantime, you should be able to `git://git.codecoop.org/coquelicot.git`
(maybe in a few hours, it looks the public access got lost at some
point).

Cheers,
-- 
Lunar
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Eric Wong]

Lunar <lunar@anargeek.net> wrote:
> Quoting the "HTTP Connection Management" document [1] which I mentioned
> previously: "Servers must therefore close each half of the connection
> independently."
> 
> After digging some more at Nginx, it looks like Nginx is doing exactly
> that. I also looked at Apache and the code responsible for half-closing
> both side of the connection is very readable. Function is called
> `ap_lingering_close()` in server/connection.c.

It's very expensive to do what Apache does with threads+Rainbows!.  It
also appears that Apache and nginx continues to completely drain the
socket if the client is sending, too (which is your case).

I don't know if there's a way around draining the socket entirely to
avoid connection resets in the client :<

I'm more interested in keeping good clients going and not wasting server
resources dealing with clients I don't want in the first place.

> It looks like there is no way to call shutdown(2) in Ruby 1.8.7, but
> from the documentation, Ruby 1.9.3 has IO#close_read and IO#close_write
> method that will call shutdown(2) for each half of a socket.

There's a shutdown method in BasicSocket for 1.8

> In any case, I think it would worth to try to add such call sequences in
> Rainbows! and see if it helps. If you can't give it a shot yourself,
> could you point me to the relevant part of the code?

You could emulate lingering close with something like this:
(valid for ThreadSpawn/ThreadPool, not EM/Cool.io stuff)

diff --git a/lib/rainbows/client.rb b/lib/rainbows/client.rb
index b456eca..9d8256a 100644
--- a/lib/rainbows/client.rb
+++ b/lib/rainbows/client.rb
@@ -6,4 +6,16 @@ class Rainbows::Client < Kgio::Socket
   include Rainbows::ProcessClient
 
   alias write kgio_write
+
+  def close
+    close_write
+    buf = ""
+    begin
+      kgio_wait_readable(2)
+      buf = kgio_tryread(512, buf)
+    rescue
+      break
+    end while buf
+    super
+  end
 end

I'm not going to apply that to Rainbows!, it's needless
overhead/complexity for dealing with clients I want to reject
anyways (and you're still draining the client socket).

> [1] <http://ftp.ics.uci.edu/pub/ietf/http/draft-ietf-http-connection-00.txt>
> 
> > Also, neither Matz Ruby 1.8 nor 1.9 can do lingering close as IO#close
> > still holds onto the GVL (blocking all clients).
> > 
> > I tried to make close(2) release the GVL for 1.9.3, but it doesn't
> > interact well if another thread was operating on that IO object.
> 
> I am not that versed in Ruby internal, but I fail to see how this could
> be related to first closing the clients write side of the connection.
> There should be a window between IO#close_write and IO#close in which in
> GVL does not matter, does it?

Having the kernel handle lingering close() should be an "easier" version
of the above patch (but I think you still need to drain the input as
long as your client is sending).
_______________________________________________
Rainbows! mailing list - rainbows-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Lunar]

Eric Wong wrote:
> Lunar <lunar@anargeek.net> wrote:
> > Quoting the "HTTP Connection Management" document [1] which I mentioned
> > previously: "Servers must therefore close each half of the connection
> > independently."
> > 
> > After digging some more at Nginx, it looks like Nginx is doing exactly
> > that. I also looked at Apache and the code responsible for half-closing
> > both side of the connection is very readable. Function is called
> > `ap_lingering_close()` in server/connection.c.
> 
> It's very expensive to do what Apache does with threads+Rainbows!.  It
> also appears that Apache and nginx continues to completely drain the
> socket if the client is sending, too (which is your case).
> 
> I don't know if there's a way around draining the socket entirely to
> avoid connection resets in the client :<

With your proposed patch, everything is working as intended. At least, I
can confirm Firefox correctly display the error message sent by the
server and not the less understandable "Connection reset by peer". I can
also confirm that it does not send the complete file: it looks like it
stops sending as soon as either it notices the socket is closed on its
write part, or when the response arrives. I am satisfied enough not to
dive in libxpcom…

> You could emulate lingering close with something like this:
> (valid for ThreadSpawn/ThreadPool, not EM/Cool.io stuff)
> 
> diff --git a/lib/rainbows/client.rb b/lib/rainbows/client.rb
> index b456eca..9d8256a 100644
> --- a/lib/rainbows/client.rb
> +++ b/lib/rainbows/client.rb
> @@ -6,4 +6,16 @@ class Rainbows::Client < Kgio::Socket
>    include Rainbows::ProcessClient
>  
>    alias write kgio_write
> +
> +  def close
> +    close_write
> +    buf = ""
> +    begin
> +      kgio_wait_readable(2)
> +      buf = kgio_tryread(512, buf)
> +    rescue
> +      break
> +    end while buf
> +    super
> +  end
>  end
> 
> I'm not going to apply that to Rainbows!, it's needless
> overhead/complexity for dealing with clients I want to reject
> anyways (and you're still draining the client socket).

That was exactly what I was looking for. It even works fine on Ruby
1.8.7. I can include that code directly in Coquelicot (by reopening
Rainbows::Client to add this extra method) so my pet case is solved.

I could see it part of Rainbows!, as optional behaviour, off by default
if you'd prefer, but I am fine either way.

Thanks for your help! I will let you know as soon as enhanced Coquelicot
codebase using Rainbows! is clean enough for a wider exposure (and I
have been noticed that the public repository is now back).

Cheers,
-- 
Lunar
_______________________________________________
Rainbows! mailing list - rainbows-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Eric Wong]

Lunar <lunar@anargeek.net> wrote:
> With your proposed patch, everything is working as intended. At least, I
> can confirm Firefox correctly display the error message sent by the
> server and not the less understandable "Connection reset by peer". I can
> also confirm that it does not send the complete file: it looks like it
> stops sending as soon as either it notices the socket is closed on its
> write part, or when the response arrives. I am satisfied enough not to
> dive in libxpcom…

Cool, thanks for confirming.  It's good that your client knows to back
off.  My original patch isn't safe for bad clients that send
continously.  Below is a safer version of my original patch, can you
see if it works?

diff --git a/lib/rainbows/client.rb b/lib/rainbows/client.rb
index b456eca..1dcb6d4 100644
--- a/lib/rainbows/client.rb
+++ b/lib/rainbows/client.rb
@@ -6,4 +6,16 @@ class Rainbows::Client < Kgio::Socket
   include Rainbows::ProcessClient
 
   alias write kgio_write
+
+  def close
+    close_write
+    kgio_wait_readable(2)
+    buf = ""
+    case kgio_tryread(512, buf)
+    when nil, Symbol
+      break
+    end while true
+    ensure
+      super
+  end
 end

> I could see it part of Rainbows!, as optional behaviour, off by default
> if you'd prefer, but I am fine either way.

I'll probably call it "lingering_close" like nginx calls it.  Of course
this needs to be supported for EM/Cool.io, too.
_______________________________________________
Rainbows! mailing list - rainbows-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Lunar]

Eric Wong wrote:
> Lunar <lunar@anargeek.net> wrote:
> > With your proposed patch, everything is working as intended. At least, I
> > can confirm Firefox correctly display the error message sent by the
> > server and not the less understandable "Connection reset by peer". I can
> > also confirm that it does not send the complete file: it looks like it
> > stops sending as soon as either it notices the socket is closed on its
> > write part, or when the response arrives. I am satisfied enough not to
> > dive in libxpcom…
> 
> Cool, thanks for confirming.  It's good that your client knows to back
> off.

Actually, they don't, I was a little too quick to answer. I made some
more test, sending a 8.1 MB file over a Wi-Fi connection between two
systems. For each test, I created a pcap file by recording the HTTP
transmission.

curl is 7.21.2-4, and I am using:

    curl -v -F "file=@test-file.xcf;type=application/x-xcf" \
            -F "submit=submit" http://webserver/

epiphany is 2.30.6-1, using WebKit and IIRC, libsoup for its HTTP
connections. Iceweasel is version 3.5.16-12. w3m is 0.5.2-10. All coming
from Debian Squeeze.

Here's what's hapenning for nginx (0.7.67-3+squeeze1) when
`client_max_body` is set to `1m`:

  - curl: 413 displayed, 9.5K
  - w3m: 413 displayed, 8.8M
  - iceweasel: 413 displayed, 8.8M
  - epiphany: 413 displayed, 8.8M

For Apache (2.2.16-6+squeeze6), using `LimitRequestBody 1048576`:

  - curl: 413 displayed, 1.8K
  - w3m: 413 displayed, 8.8M
  - iceweasel: 413 displayed, 8.9M
  - epiphany: 413 displayed, 8.8M

So it looks usual HTTP web server and clients will drain all the POST
input in order to display the error message. *sigh*


Then I set up Rainbows! with the following (running latest gems on Ruby
1.8.7):

--- 8< --- rainbows.conf -----------------------------------------------
Rainbows! do
  use :ThreadSpawn
  rewindable_input false
end
--- >8 -----------------------------------------------------------------

--- 8< --- rackup.ru ---------------------------------------------------
run lambda do |env|
  while env['rack.input'].read; end;
  [ 200, { 'Content-Type' => 'text/plain' }, [ 'OK' ] ]
end
--- >8 -----------------------------------------------------------------

Same tests with this configuration:

  - curl: OK displayed, 9.5M
  - w3m: OK displayed, 9.5M
  - iceweasel: OK displayed, 9.5M
  - epiphany: OK displayed, 11M

If I add `client_max_body_size 1024 * 1024` in rainbows.conf:

  - curl: 413 displayed, 1.4K
  - w3m: 413 displayed, 11K
  - iceweasel: connection reset by peer, 21K
  - epiphany: connection terminated unexpectedly, 20K

> My original patch isn't safe for bad clients that send continously.
> Below is a safer version of my original patch, can you see if it
> works?
> 
> diff --git a/lib/rainbows/client.rb b/lib/rainbows/client.rb
> index b456eca..1dcb6d4 100644
> --- a/lib/rainbows/client.rb
> +++ b/lib/rainbows/client.rb
> @@ -6,4 +6,16 @@ class Rainbows::Client < Kgio::Socket
>    include Rainbows::ProcessClient
>  
>    alias write kgio_write
> +
> +  def close
> +    close_write
> +    kgio_wait_readable(2)
> +    buf = ""
> +    case kgio_tryread(512, buf)
> +    when nil, Symbol
> +      break
> +    end while true
> +    ensure
> +      super
> +  end
>  end

Using this patch (actually, defining Rainbows::Client#close in
rackup.ru), I get:

  - curl: 413 displayed, 1.5K
  - w3m: 413 displayed, 17K
  - iceweasel: connection reset by peer, 19K
  - epiphany: connection terminated unexpectedly, 16K

I have tried to come up with something closer to what other webservers
do, and it looks like it (but I might have overlooked something):

  def close
    close_write
    buf = ""
    loop do
      kgio_wait_readable(2)
      break unless kgio_tryread(512, buf)
    end
    ensure
      super
  end

Using this, I get the following results:

  - curl: 413 displayed, 1.4K
  - w3m: 413 displayed, 8.8M
  - iceweasel: 413 displayed, 8.8M
  - epiphany: 413 displayed, 8.8M


This whole issue leaves me severely disappointed. It looks like I either
have to waste much of my users time (by forcing them to upload the whole
file) or leave them clueless if the file is too big (by cutting the
connection early, resulting in a "reset by peer message"). What a
choice…

Hope that helps,
-- 
Lunar
_______________________________________________
Rainbows! mailing list - rainbows-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Lunar]

Lunar wrote:
> Using this, I get the following results:
> 
>   - curl: 413 displayed, 1.4K
>   - w3m: 413 displayed, 8.8M
>   - iceweasel: 413 displayed, 8.8M
>   - epiphany: 413 displayed, 8.8M
> 
> 
> This whole issue leaves me severely disappointed. It looks like I either
> have to waste much of my users time (by forcing them to upload the whole
> file) or leave them clueless if the file is too big (by cutting the
> connection early, resulting in a "reset by peer message"). What a
> choice…

Just for the record, reverse proxying this last iteration with Nginx
(with "proxy_buffering off;") or Apache (with
"SetEnv proxy-sendchunks 1") result in 413 displayed and full
transmission of the given file with all clients (curl included).

-- 
Lunar
_______________________________________________
Rainbows! mailing list - rainbows-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: 'Connection reset by peer' when replying before the end of POST data

[Eric Wong]

Lunar <lunar@anargeek.net> wrote:
> > This whole issue leaves me severely disappointed. It looks like I either
> > have to waste much of my users time (by forcing them to upload the whole
> > file) or leave them clueless if the file is too big (by cutting the
> > connection early, resulting in a "reset by peer message"). What a
> > choice…
> 
> Just for the record, reverse proxying this last iteration with Nginx
> (with "proxy_buffering off;") or Apache (with
> "SetEnv proxy-sendchunks 1") result in 413 displayed and full
> transmission of the given file with all clients (curl included).

Ouch :(

I think using 100-continue is the only halfway-viable choice.
Too bad it's:

  1) subject to race conditions
  2) can't help with chunked transfers
  3) not widely-supported (any widespread clients besides curl?)

Anyways, thanks for all your testing and reports!
_______________________________________________
Rainbows! mailing list - rainbows-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying