From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eric Wong Newsgroups: gmane.comp.lang.ruby.mongrel.devel Subject: Re: [RFC Mongrel2] simpler response API + updated HTTP parser Date: Wed, 7 Oct 2009 18:35:43 -0700 Message-ID: <20091008013542.GA12370@dcvr.yhbt.net> References: <20090912235729.GA9370@dcvr.yhbt.net> Reply-To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1254965815 12431 80.91.229.12 (8 Oct 2009 01:36:55 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 8 Oct 2009 01:36:55 +0000 (UTC) To: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Original-X-From: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Thu Oct 08 03:36:44 2009 Return-path: Envelope-to: gclrmd-mongrel-development@m.gmane.org Content-Disposition: inline In-Reply-To: <20090912235729.GA9370-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org> User-Agent: Mutt/1.5.18 (2008-05-17) X-BeenThere: mongrel-development-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Errors-To: mongrel-development-bounces-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org Xref: news.gmane.org gmane.comp.lang.ruby.mongrel.devel:155 Archived-At: Received: from rubyforge.org ([205.234.109.19]) by lo.gmane.org with esmtp (Exim 4.50) id 1Mvhvw-00046S-1L for gclrmd-mongrel-development@m.gmane.org; Thu, 08 Oct 2009 03:36:44 +0200 Received: from rubyforge.org (rubyforge.org [127.0.0.1]) by rubyforge.org (Postfix) with ESMTP id 7530E1858272; Wed, 7 Oct 2009 21:36:43 -0400 (EDT) Received: from dcvr.yhbt.net (dcvr.yhbt.net [64.71.152.64]) by rubyforge.org (Postfix) with ESMTP id 338531858272 for ; Wed, 7 Oct 2009 21:35:44 -0400 (EDT) Received: from localhost (unknown [12.186.229.34]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPSA id 82A5B1F92F; Thu, 8 Oct 2009 01:35:43 +0000 (UTC) List-Post: Eric Wong wrote: > Hi all, > > I've pushed out some changes based on fauna/master[1] to > git://git.bogomips.org/ur-mongrel that includes a good chunk of the > platform-independent stuff found in Unicorn. We hit one bug/weird-interaction with Rails in Unicorn so here's a fix I put it. Unfortunately the test cases for TeeInput in Unicorn currently rely on fork() + pipe() (it was just more natural for me to write), but if there's interest I could be persuaded to write a non-*nix version. This issue that could be arguably considered a bug in Rails: https://rails.lighthouseapp.com/projects/8994/tickets/3343 Just in case, I'm also asking for Rack to allow the readpartial method into the "rack.input" spec here: http://groups.google.com/group/rack-devel/browse_thread/thread/3dfccb68172a6ed6 >>>From 87254d37c519b63a1d39c938cd4a53b08e2a1065 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Wed, 7 Oct 2009 18:24:27 -0700 Subject: [PATCH] more-compatible TeeInput#read for POSTs with Content-Length There are existing applications and libraries that don't check the return value of env['rack.input'].read(length) (like Rails :x). Those applications became broken under the IO#readpartial semantics of TeeInput#read when handling larger request bodies. We'll preserve the IO#readpartial semantics _only_ when handling chunked requests (as long as Rack allows it, it's useful for real-time processing of audio/video streaming uploads, especially with Rainbows! and mobile clients) but use read-in-full semantics for TeeInput#read on requests with a known Content-Length. --- lib/mongrel/tee_input.rb | 43 +++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 41 insertions(+), 2 deletions(-) diff --git a/lib/mongrel/tee_input.rb b/lib/mongrel/tee_input.rb index 442c55a..3605e20 100644 --- a/lib/mongrel/tee_input.rb +++ b/lib/mongrel/tee_input.rb @@ -44,6 +44,26 @@ module Mongrel @size = tmp_size end + # call-seq: + # ios = env['rack.input'] + # ios.read([length [, buffer ]]) => string, buffer, or nil + # + # Reads at most length bytes from the I/O stream, or to the end of + # file if length is omitted or is nil. length must be a non-negative + # integer or nil. If the optional buffer argument is present, it + # must reference a String, which will receive the data. + # + # At end of file, it returns nil or "" depend on length. + # ios.read() and ios.read(nil) returns "". + # ios.read(length [, buffer]) returns nil. + # + # If the Content-Length of the HTTP request is known (as is the common + # case for POST requests), then ios.read(length [, buffer]) will block + # until the specified length is read (or it is the last chunk). + # Otherwise, for uncommon "Transfer-Encoding: chunked" requests, + # ios.read(length [, buffer]) will return immediately if there is + # any data and only block when nothing is available (providing + # IO#readpartial semantics). def read(*args) socket or return @tmp.read(*args) @@ -58,9 +78,9 @@ module Mongrel rv = args.shift || @buf2.dup diff = tmp_size - @tmp.pos if 0 == diff - tee(length, rv) + ensure_length(tee(length, rv), length) else - @tmp.read(diff > length ? length : diff, rv) + ensure_length(@tmp.read(diff > length ? length : diff, rv), length) end end end @@ -140,5 +160,24 @@ module Mongrel tmp end + # tee()s into +buf+ until it is of +length+ bytes (or until + # we've reached the Content-Length of the request body). + # Returns +buf+ (the exact object, not a duplicate) + # To continue supporting applications that need near-real-time + # streaming input bodies, this is a no-op for + # "Transfer-Encoding: chunked" requests. + def ensure_length(buf, length) + # @size is nil for chunked bodies, so we can't ensure length for those + # since they could be streaming bidirectionally and we don't want to + # block the caller in that case. + return buf if buf.nil? || @size.nil? + + while buf.size < length && @size != @tmp.pos + buf << tee(length - buf.size, @buf2) + end + + buf + end + end end -- Eric Wong