about summary refs log tree commit homepage
path: root/test/unit/test_http_parser.rb
DateCommit message (Collapse)
2016-10-20Add some tolerance (RFC2616 sec. 19.3)
Hi all. We're implementing client certificate authentication with nginx and unicorn.  Nginx configured in the following way: proxy_set_header X-SSL-Client-Cert $ssl_client_cert; When client submits certificate and nginx passes it to the unicorn, unicorn responds with 400 (Bad Request). This caused because nginx doesn't use "\r\n" they using just "\n" and multilne headers is failed to parse (I've added test). Accorording to RFC2616 section 19.3: https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3 "The line terminator for message-header fields is the sequence CRLF. However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR." CRLF changed to ("\r\n" | "\n") Github commit https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf176de05720f6be758f8b41b1f PS: Googling "nginx unicorn ssl_client_cert" shows the problem. 
2015-12-13http: TypedData C-API conversion
This provides some extra type safety if combined with other C extensions, as well as allowing us to account for memory usage of the HTTP parser in ObjectSpace. This requires Ruby 1.9.3+ and has remained a stable API since then. This will become officially supported when Ruby 2.3.0 is released later this month. This API has only been documented in doc/extension.rdoc (formerly README.EXT) in the Ruby source tree since April 2015, r50318
2015-02-06fix uninstalled testing and reduce require paths
This fixes a bug introduced in commit fe83ead4eae6f011fa15f506cd80cb4256813a92 (GNUmakefile: fix clean gem build + reduce build cruft) which broke clean Ruby installations without an existing unicorn gem installed :x
2014-08-18remove mongrel.rubyforge.org references
mongrel.rubyforge.org has been dead longer than rubyforge.org!
2013-10-26license: allow all future versions of the GNU GPL
There is currently no GPLv4, so this change has no effect at the moment. In case the GPLv4 arrives and I am not alive to approve/review it, the lesser of evils is have give blanket approval of all future GPL versions (as published by the FSF). The worse evil is to be stuck with a license which cannot guarantee the Free-ness of this project in the future. This unfortunately means the FSF can theoretically come out with license terms I do not agree with, but the GPLv2 and GPLv3 will always be an option to all users.
2012-11-13tests: remove assert_nothing_raised (part 2)
assert_nothing_raised ends up hiding errors and backtraces, making things harder to debug. Since Test::Unit already fails on uncaught exceptions, there is no need to assert on the lack of exceptions for a successful test run. This is a followup to commit 5acf5522295c947d3118926d1a1077007f615de9
2011-08-29add GPLv3 option to the license
Existing license terms (Ruby-specific) and GPLv2 remain in place, but GPLv3 is preferred as it helps with distribution of AGPLv3 code and is explicitly compatible with Apache License (v2.0). Many more reasons are documented by the FSF: https://www.gnu.org/licenses/quick-guide-gplv3.html http://gplv3.fsf.org/rms-why.html ref: http://thread.gmane.org/gmane.comp.lang.ruby.unicorn.general/933
2011-07-13http: reject non-LWS CTL chars (0..31 + 127) in field values
RFC 2616 doesn't appear to allow most CTL bytes even though Mongrel always did. Rack::Lint disallows 0..31, too, though we allow "\t" (HT, 09) since it's LWS and allowed by RFC 2616.
2011-06-17test_http_parser: fix for URI too long errors (#3)
The random garbage generator may occasionally generate URIs that are too long and cause the URI-specific error to be raised instead of the generic parser error we recently introduced. Follow-up-to: commit 742c4d77f179a757dbcb1fa350f9d75b757acfc7
2011-06-15test_http_parser: fix for URI too long errors (again)
The random garbage generator may occasionally generate URIs that are too long and cause the URI-specific error to be raised instead of the generic parser error we recently introduced. Follow-up-to: commit 742c4d77f179a757dbcb1fa350f9d75b757acfc7
2011-06-15test_http_parser: fix for URI too long errors
The random garbage generator may occasionally generate URIs that are too long and cause the URI-specific error to be raised instead of the generic parser error we recently introduced.
2011-05-23strip trailing and leading linear whitespace in headers
RFC 2616, section 4.2: > The field-content does not include any leading or trailing LWS: > linear white space occurring before the first non-whitespace > character of the field-value or after the last non-whitespace > character of the field-value. Such leading or trailing LWS MAY be > removed without changing the semantics of the field value. Any LWS > that occurs between field-content MAY be replaced with a single SP > before interpreting the field value or forwarding the message > downstream.
2011-04-13fix some 1.9.3dev warnings
2011-02-03test/unit: fix tests under Ruby 1.9.3dev
Ugh, one day I'll clean them up, one day...
2011-02-02http: parser handles IPv6 bracketed IP hostnames
Just in case we have people that don't use DNS, we can support folks who enter ugly IPv6 addresses... IPv6 uses brackets around the address to avoid confusing the colons used in the address with the colon used to denote the TCP port number in URIs.
2011-01-05http_parser: add clear method, deprecate reset
But allows small optimizations to be made to avoid constant/instance variable lookups later :)
2010-12-20http: update setting of "https" for rack.url_scheme
The first value of X-Forwarded-Proto in rack.url_scheme should be used as it can be chained. This header can be set multiple times via different proxies in the chain, but consider the first one to be valid. Additionally, respect X-Forwarded-SSL as it may be passed with the "on" flag instead of X-Forwarded-Proto. ref: rack commit 85ca454e6143a3081d90e4546ccad602a4c3ad2e and 35bb5ba6746b5d346de9202c004cc926039650c7
2010-11-06enable HTTP keepalive support for all methods
Yes, this means even POST/PUT bodies may be kept alive, but only if the body (and trailers) are fully-consumed.
2010-06-08test_http_parser: memory usage test for non-MRI
Non-MRI runtimes (like Rubinius) may implement garbage collection very differently than MRI.
2010-06-05File.readable? and File.writable? are more readable
...than "test ?r" and "test ?w" Not everybody comes from a Unix shell programming background, even though they *should* ;)
2010-05-07http: allow horizontal tab as leading whitespace in header values
This is allowed by RFC 2616, section 2.2, where spaces and horizontal tabs are counted as linear white space and linear white space (not just regular spaces) may prefix field-values (section 4.2). This has _not_ been a real issue in ~4 years of using this parser (starting with Mongrel) with clients in the wild. Thanks to Iñaki Baz Castillo for pointing this out.
2010-02-13http: fix memory leak exposed in concurrent servers
First off, this memory leak DOES NOT affect Unicorn itself. Unicorn allocates the HttpParser once and always reuses it in every sequential request. This leak affects applications which repeatedly allocate a new HTTP parser. Thus this bug affects _all_ deployments of Rainbows! and Zbatery. These servers allocate a new parser for every client connection. I misread the Data_Make_Struct/Data_Wrap_Struct documentation and ended up passing NULL as the "free" argument instead of -1, causing the memory to never be freed. From README.EXT in the MRI source which I misread: > The free argument is the function to free the pointer > allocation. If this is -1, the pointer will be just freed. > The functions mark and free will be called from garbage > collector.
2009-12-19http: allow userinfo component in absoluteURIs
This is not explicitly specified or listed as an example in in rfc2616. However, rfc2616 section 3.2.1 defers to rfc2396[1] for the definition of absolute URIs, so the userinfo component should be allowable, even if it does not make any sense. In the real world, previous versions of Mongrel used URI.parse() and thus allowed userinfo, so we also have precedence to allow userinfo to be compatible *in case* our interpretation of the RFCs is incorrect. This change is unfortunately needed because *occasionally* real clients rely on them. Reported-by: Scott Chacon [1] rfc3986 obsoletes rfc2396, but also includes userinfo
2009-09-08"encoding: binary" comments for all sources (1.9)
This ensures any string literals that pop up in *our* code will just be a bag of bytes. This shouldn't affect/fix/break existing apps in most cases, but most constants will always have the "correct" encoding (none!) to be consistent with HTTP/socket expectations. Since this comment affects things only on a per-source basis, it won't affect existing apps with the exception of strings we pass to the Rack application. This will eventually allow us to get rid of that Unicorn::Z constant, too.
2009-09-06http: ignore Host: continuation lines with absolute URIs
This probably doesn't affect anyone with HTTP/1.1, but future versions of HTTP will use absolute URIs and maybe we'll eventually get clients that (mistakenly) send us Host: headers along with absolute URIs.
2009-09-02http: SERVER_PROTOCOL matches HTTP_VERSION
And it'll default to HTTP/0.9 if HTTP_VERSION is not specified (as version-less HTTP requests imply HTTP/0.9.
2009-08-18http: support for multi-line HTTP headers
While I still consider pound to be irrelevant, but I still sometimes get hand-crafted HTTP requests that come in with multiline headers. Since these are part of the HTTP specs and not difficult to support, we might as well support them for the sake of completeness.
2009-08-15http: support for "Connection: keep-alive"
ab still sends this with HTTP/1.0 requests, which is unfortunate, but synthetic benchmarks are good for marketing purposes!
2009-08-11http: add "HttpParser#keepalive?" method
This should be used to detect if a request can really handle keepalives and pipelining. Currently, the rules are: 1. MUST be a GET or HEAD request 2. MUST be HTTP/1.1 3. MUST NOT have "Connection: close" set This also reduces the amount of garbage we create by globalizing constants and using them whenever possible.
2009-08-09Switch to Ragel/C-based chunk/trailer parser
This should be more robust, faster and easier to deal with than the ugly proof-of-concept regexp-based ones.
2009-06-10Optimize body-less GET/HEAD requests (again)
No point in making syscalls to deal with empty bodies. Reinstate usage of the NULL_IO object which allows us to avoid allocating new objects.
2009-04-21test: empty port test for absolute URIs
2009-04-21http11: support underscores in URI hostnames
They aren't common, but apparently there exist URLs with them, so we'll support them.
2009-04-21http11: rfc2616 handling of absolute URIs
We now parse the scheme, host and port from Absolute URIs and ignore them if the equivalents are specified in the other headers.
2009-04-21http11: make parser obey HTTP_HOST with empty port
This means "Host: foo-bar:" (trailing colon) will assume server_port is 80, not a blank string.
2009-04-21HttpParser: set QUERY_STRING for Rack-compliance
2009-04-21Move absolute URI parsing into HTTP parser
It's part of the HTTP/1.1 (rfc2616), so we might as well handle it in there and set PATH_INFO while we're at it. Also, make "OPTIONS *" test not fail Rack::Lint
2009-04-12test_http_parser: fix broken URL in comment
This was back when I did s/mongrel/Unicorn/g on the sources.
2009-03-29http11: use :http_body instead of "HTTP_BODY"
"HTTP_BODY" could conflict with a "Body:" HTTP header if there ever is one. Also, try to hide this body from the Rack environment before @app is called since it is only used by Unicorn internally.
2009-03-24simplify the HttpParser interface
This cuts the HttpParser interface down to #execute and #reset method. HttpParser#execute will return true if it completes and false if it is not. http->nread state is kept internally so we don't have to keep track of it in Ruby; removing one parameter from #execute. HttpParser#reset is unchanged. All errors are handled through exceptions anyways, so the HttpParser#error? method stopped being useful. Also added some more unit tests to the HttpParser since I know some folks are (rightfully) uncomfortable with changing stable C code. We now have tests for incremental parsing. In summary, we have: * more test cases * less C code * simpler interfaces * small performance improvement => win \o/
2009-03-21unicorn/http11: remove GATEWAY_INTERFACE
It's a CGI-ism and is not in the Rack spec, so don't bother.
2009-02-09s/Mongrel/Unicorn/g
Avoid conflicting with existing Mongrel libraries since we'll be incompatible and break things w/o disrupting Mongrel installations.
2009-01-31No more special params hash.
2008-07-15http_parser: restore r996 (accept '"', '<', and '>' characters in URLs)
This seems to have gotten accidentally dropped during a merge: r1020 Some broken web browsers don't properly escape ", <, and > characters in URLs, however these URLs to occasionally legitimate and sometimes show up. This patch was submitted by Eden Li here: http://rubyforge.org/pipermail/mongrel-users/2006-October/001845.html This patch was accepted by Zed Shaw here: http://rubyforge.org/pipermail/mongrel-users/2006-October/001847.html git-svn-id: svn+ssh://rubyforge.org/var/svn/mongrel/trunk@1031 19e92222-5c0b-0410-8929-a290d50e31e9
2008-07-15Remove nasty_pound_header test since we've always failed it
ref: http://thread.gmane.org/gmane.comp.lang.ruby.mongrel.devel/37/focus=45 We got 'pen' mixed up with 'pound' in that thread but the gist of it is still relevant: these nasty headers are irrelevant. 'pound' is the only thing that's known to send these nasty headers and I (Eric Wong) don't consider pound relevant at all. git-svn-id: svn+ssh://rubyforge.org/var/svn/mongrel/trunk@1030 19e92222-5c0b-0410-8929-a290d50e31e9
2008-03-24Backport Eric's changes to the http parser from trunk (Eric Wong).
Apply fix for Ragel 6 (Eric Wong, Ry Dahl). Two tests fail with the new parser (1 failed with the old parser). Needs investigation. Close #12 (mongrel_rails send_signal leaves a filehandle open until gc). Close #14 (mongrel_rails command line option --num-procs does not change the max number of procs). Close #15 (mongrel squashes helpful exception in register method). Close #16, XXX needs audit! (CGIWrapper "options" attr_reader has no corresponding @options variable). Close #20 (Mongrel doesn't erase temporary files during it's operation on windows). Close #19, XXX needs audit! (HttpResponse#reset does not properly reset HeaderOut). Close #22 (gem_plugin should load gems from Gem.path not Gem.dir). Close #23 (mongrel_cluster's mongrel_rails configuration option isn't fully respected). If I had git, being offline wouldn't have resulted in one massive commit. git-svn-id: svn+ssh://rubyforge.org/var/svn/mongrel/branches/stable_1-2@995 19e92222-5c0b-0410-8929-a290d50e31e9