| Date | Commit message (Collapse) |
|---|
|
The HTTP parser is fix for oddly-aligned reads of trailers (this
technically affects headers, too, but is highly unlikely due to
our non-support of slow clients). This allows our HTTP parser
to better support very slow clients when used by other servers
(like Rainbows!). Fortunately this bug does not appear to lead
to any invalid memory accesses (and potential arbitrary code
execution).
FreeBSD (and possibly other *BSDs) support is improved and and
all the test cases pass under FreeBSD 7.2. Various flavors of
GNU/Linux remains our primary platform for development and
production.
New features added include the "working_directory" directive in
the configurator . Even without specifying a
"working_directory", symlink-aware detection of the current path
no longer depends on /bin/sh so it should work out-of-the-box on
FreeBSD and Solaris and not just systems where /bin/sh is dash,
ksh93 or bash.
User-switching support is finally supported but only intended
for use in the after_fork hook of worker processes. Putting it
in the after_fork hook allows allows users to set things like
CPU affinity[1] on a per-worker basis before dropping
privileges. The master process retains all privileges it
started with.
The ENV["RACK_ENV"] (process-wide) environment variable is now
both read and set for `unicorn' in the same way RAILS_ENV is
used by `unicorn_rails'. This allows the Merb launcher to read
ENV["RACK_ENV"] in config.ru. Other web servers already set
this and there may be applications or libraries that already
rely on this de facto standard.
Eric Wong (26):
cleanup: avoid redundant error checks for fstat
test_helper: connect(2) may fail with EINVAL
GNUmakefile: fix non-portable tar(1) usage
tests: provide a pure Ruby setsid(8) equivalent
more portable symlink awareness for START_CTX[:cwd]
test_signals: avoid portability issues with fchmod(2)
cleanup error handling and make it less noisy
Do not override Dir.chdir in config files
configurator: add "working_directory" directive
configurator: working_directory is expanded
configurator: set ENV["PWD"] with working_directory, too
configurator: working_directory affects pid, std{err,out}_paths
configurator: update documentation for working_directory
TODO: remove working_directory bit, done
Util.reopen_logs: remove needless Range
worker: user/group switching for after_fork hooks
Fix autoload of Etc in Worker for Ruby 1.9
bin/unicorn: allow RACK_ENV to be passed from parent
tests for RACK_ENV preservation
http: allow headers/trailers to be written byte-wise
http: extra test for bytewise chunked bodies
tee_input: do not clobber trailer buffer on partial uploads
test_exec: ensure master is killed after test
Util::tmpio returns a TmpIO that responds to #size
TODO: remove user-switching bit, done
unicorn 0.94.0
Wayne Larsen (1):
bin/unicorn: set ENV["RACK_ENV"] on startup
[1] - Unicorn does not support CPU affinity directly, but it is
possible to load code that allows it inside after_fork hooks,
or even just call sched_tool(8).
|
|
Subclass off the core File class so we don't have to
worry about #size being defined. This will mainly
be useful to Rainbows! but allows us to simplify
our TeeInput implementation a little, too.
|
|
Found in Rainbows! testing. Reusing the buffer when finalizing
input for headers could be problematic because it would lead
to the @buf2 instance variable being clobbered; allowing the
trailers to "leak" into the body.
|
|
Constant scoping appears to be a bit different under 1.9
|
|
This must be called in the after_fork hook because there may be
Ruby modules that'll allow things such as CPU affinity and
scheduling class/priority to be set on a per-worker basis. So
we give the user the ability to change users at any time during
the after_fork hook.
|
|
?/ avoids allocating a String in 1.8 and in 1.9 short String
objects are cheap.
|
|
We follow the principle of least surprise now, so less
documentation is better documentation.
|
|
It makes more sense this way since users usually expect config
file directives to be order-independent.
|
|
Just in case anything depends on it, we'll have it set
correctly because it's usually set by the $SHELL
|
|
Allow people to use "~" and relative paths, like all
of our other paths.
|
|
This basically a prettier way of saying:
Dir.chdir(Unicorn::HttpServer::START_CTX[:cwd] = path)
In the config file. Unfortunately, this is configuration
directive where order matters and you should specify it
before any other path[1] directives if you're using relative
paths (relative paths are not recommended anyways)
[1] pid, stderr_path, stdout_path
|
|
Even if START_CTX[:cwd] is pointing to another directory,
avoid overriding the user's decision to Dir.chdir if they
do it in either the Unicorn config file or the config.ru.
|
|
split out uncommon code from the common path
|
|
`sh -c pwd` doesn't reliably read ENV["PWD"] on all platforms,
this means that directories that are symlinks may be ignored
and the real path is resolved. This can be problematic when
doing upgrades for common deployment systems such as Capistrano
which rely on the working directory being a symlink.
|
|
If fstat() fails on an open file descriptor in the master,
something is seriously wrong (like your kernel is broken/buggy)
and trying to restart the worker that owned that file descriptor
is likely masking the symptoms. Instead let the error propagate
up to the main loop to avoid wasting cycles to restart broken
workers.
|
|
This release fixes a regression introduced in 0.93.3 where
timed-out worker processes run a chance of not being killed off
at all if they're hung. While it's not ever advisable to have
requests take a long time, we realize it's easy to fix
everything :)
Eric Wong (3):
TODO: remove --kill
fix reliability of timeout kills
TODO: update for next version (possibly 1.0-pre)
|
|
The method introduced in commit
6c8a3d3c55997978bacaecc5dbbb7d03c2fee345 to avoid killing
workers after suspend/hibernate interacted badly with the change
for OpenBSD fchmod(2) compatibility introduced with the 0.93.3
release. This interaction lead to workers with files stuck in
the zero state to never be murdered off for timeout violations.
Additionally, the method to avoid killing processes off was
never completely reliable and has been reworked even if we
entered suspend/hibernate/STOP during client processing.
This regression was discovered during continued development of the
Rainbows! test suite (which we will bring over as it becomes ready).
|
|
This release mainly works around BSD stdio compatibility issues
that affect at least FreeBSD and OS X. While this issues was
documented and fixed in [ruby-core:26300][1], no production
release of MRI 1.8 has it, and users typically upgrade MRI more
slowly than gems. This issue does NOT affect 1.9 users. Thanks
to Vadim Spivak for reporting and testing this issue and Andrey
Stikheev for the fix.
Additionally there are small documentation bits, one error
handling improvement, and one minor change that should improve
reliability of signal delivery.
Andrey Stikheev (1):
workaround FreeBSD/OSX IO bug for large uploads
Eric Wong (7):
DESIGN: address concerns about on-demand and thundering herd
README: alter reply conventions for the mailing list
configurator: stop testing for non-portable listens
KNOWN_ISSUES: document Rack gem issue w/Rails 2.3.2
stop continually resends signals during shutdowns
add news bodies to site NEWS.atom.xml
configurator: fix broken example in RDoc
Suraj N. Kurapati (1):
show configuration file path in errors instead of '(eval)'
[1] http://redmine.ruby-lang.org/issues/show/2267
|
|
Under FreeBSD writing to the file in sync mode does not change current
position, so change position to the end of the file. Without this patch
multipart post requests with large data (image uploading) does not work
correctly:
Status: 500 Internal Server Error
bad content body
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.0/lib/rack/utils.rb:347:in `parse_multipart'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.0/lib/rack/utils.rb:319:in `loop'
/usr/local/lib/ruby/gems/1.8/gems/rack-1.0.0/lib/rack/utils.rb:319:in `parse_multipart'
File position behavior under FreeBSD :
ruby -v
ruby 1.8.7 (2009-04-08 patchlevel 160) [i386-freebsd7]
irb(main):001:0> b = File.new("abc", "w+")
=> #<File:abc>
irb(main):002:0> b.sync = true
=> true
irb(main):004:0> b.write("abc")
=> 3
irb(main):005:0> b.pos
=> 0
Acked-by: Eric Wong <normalperson@yhbt.net>
|
|
Thanks to Greg Melton for reporting.
|
|
Since our :QUIT and :TERM signal handlers are idempotent, we can
safely retry sending signals in case workers don't/can't handle
them them the first time around. This appears to be a problem
with the Thread-based concurrency models in Rainbows! not
behaving well (no surprise, though, since pthreads and signals
are difficult to manage/mix properly).
|
|
also __FILE__ did not reflect configuration file path
|
|
This release fixes compatibility with OpenBSD (and possibly
other Unices with stricter fchmod(2) implementations) thanks to
Jeremy Evans. Additionally there are small documentation
changes all around.
Eric Wong (11):
doc: expand on the SELF_PIPE description
fchmod heartbeat flips between 0/1 for compatibility
examples/init.sh: remove "set -u"
configurator: update with nginx fail_timeout=0 example
PHILOSOPHY: clarify experience other deployments
PHILOSOPHY: plug the Rainbows! spin-off project
README: remove unnecessary and extraneous dash
DESIGN: clarification and possibly improve HTML validity
README: remove the "non-existent" part
README: emphasize the "fast clients"-only part
drop the whitespace cleaner for Ragel->C
|
|
It has come to our attention that this setting is not very
well-known to the rest of the world...
|
|
This removes the Time.now.to_i comparison that was used to avoid
multiple, no-op fchmod() syscalls[1] within the same second.
This should allow us to run on OpenBSD where it can raise EINVAL
when Time.now.to_i is passed to it.
Reported-by: Jeremy Evans <jeremyevans0@gmail.com>
[1] - gettimeofday() from Time.now is not a real syscall on
VDSO-enabled x86_64 GNU/Linux systems where Unicorn is primarily
developed.
|
|
There seems to be a small amount of confusion regarding how it's
used (and some of the code is not very obvious). So explain our
usage of it and distinguish its use in the master vs worker(s).
|
|
Avoid truncated POST bodies from with URL-encoded forms in Rails
by switching TeeInput to use read-in-full semantics (only) when
a Content-Length: header exists. Chunked request bodies
continue to exhibit readpartial semantics to support
simultaneous bidirectional chunking.
The lack of return value checking in Rails to protect against a
short ios.read(length) is entirely reasonable even if not
pedantically correct. Most ios.read(length) implementations
return the full amount requested except right before EOF.
Also there are some minor documentation improvements.
Eric Wong (7):
Fix NEWS generation on single-paragraph tag messages
Include GPLv2 in docs
doc: make it clear contributors retain copyrights
TODO: removed Rainbows! (see rainbows.rubyforge.org)
Document the START_CTX hash contents
more-compatible TeeInput#read for POSTs with Content-Length
tests for read-in-full vs readpartial semantics
|
|
There are existing applications and libraries that don't check
the return value of env['rack.input'].read(length) (like Rails
:x). Those applications became broken under the IO#readpartial
semantics of TeeInput#read when handling larger request bodies.
We'll preserve the IO#readpartial semantics _only_ when handling
chunked requests (as long as Rack allows it, it's useful for
real-time processing of audio/video streaming uploads,
especially with Rainbows! and mobile clients) but use
read-in-full semantics for TeeInput#read on requests with a
known Content-Length.
|
|
Modifying this can be useful for esoteric cases like switching
entire Ruby installations or if the app was originally started
in a no-longer-existent directory and we can't upgrade because
we can't chdir to it.
|
|
Fix permissions for release tarballs/gems, no other changes.
Thanks to Jay Reitz for reporting this.
|
|
The one minor bugfix is only for Rails 2.3.x+ users who set the
RAILS_RELATIVE_URL_ROOT environment variable in a config file.
Users of the "--path" switch or those who set the environment
variable in the shell were unaffected by this bug. Note that we
still don't have relative URL root support for Rails < 2.3, and
are unlikely to bother with it unless there is visible demand
for it.
New features includes support for :tries and :delay when
specifying a "listen" in an after_fork hook. This was inspired
by Chris Wanstrath's example of binding per-worker listen
sockets in a loop while migrating (or upgrading) Unicorn.
Setting a negative value for :tries means we'll retry the listen
indefinitely until the socket becomes available.
So you can do something like this in an after_fork hook:
after_fork do |server, worker|
addr = "127.0.0.1:#{9293 + worker.nr}"
server.listen(addr, :tries => -1, :delay => 5)
end
There's also the usual round of added documentation, packaging
fixes, code cleanups, small fixes and minor performance
improvements that are viewable in the "git log" output.
Eric Wong (54):
build: hardcode the canonical git URL
build: manifest dropped manpages
build: smaller ChangeLog
doc/LATEST: remove trailing newline
http: don't force -fPIC if it can't be used
.gitignore on *.rbc files Rubinius generates
README/gemspec: a better description, hopefully
GNUmakefile: add missing .manifest dep on test installs
Add HACKING document
configurator: fix user switch example in RDoc
local.mk.sample: time and perms enforcement
unicorn_rails: show "RAILS_ENV" in help message
gemspec: compatibility with older Rubygems
Split out KNOWN_ISSUES document
KNOWN_ISSUES: add notes about the "isolate" gem
gemspec: fix test_files regexp match
gemspec: remove tests that fork from test_files
test_signals: ensure we can parse pids in response
GNUmakefile: cleanup test/manifest generation
util: remove APPEND_FLAGS constant
http_request: simplify and remove handle_body method
http_response: simplify and remove const dependencies
local.mk.sample: fix .js times
TUNING: notes about benchmarking a high :backlog
HttpServer#listen accepts :tries and :delay parameters
"make install" avoids installing multiple .so objects
Use Configurator#expand_addr in HttpServer#listen
configurator: move initialization stuff to #initialize
Remove "Z" constant for binary strings
cgi_wrapper: don't warn about stdoutput usage
cgi_wrapper: simplify status handling in response
cgi_wrapper: use Array#concat instead of +=
server: correctly unset reexec_pid on child death
configurator: update and modernize examples
configurator: add colons in front of listen() options
configurator: remove DEFAULT_LOGGER constant
gemspec: clarify commented-out licenses section
Add makefile targets for non-release installs
cleanup: use question mark op for 1-byte comparisons
RDoc for Unicorn::HttpServer::Worker
small cleanup to pid file handling + documentation
rails: RAILS_RELATIVE_URL_ROOT may be set in Unicorn config
unicorn_rails: undeprecate --path switch
manpages: document environment variables
README: remove reference to different versions
Avoid a small window when a pid file can be empty
configurator: update some migration examples
configurator: listen :delay must be Numeric
test: don't rely on .manifest for test install
SIGNALS: state that we stole semantics from nginx
const: DEFAULT_PORT as a string doesn't make sense
test_helper: unused_port rejects 8080 unconditionally
GNUmakefile: SINCE variable may be unset
tests: GIT-VERSION-GEN is a test install dependency
|
|
TCP ports are always integers, and it was always allowing a
randomly-generated value of 8080 through in the unused_port
method of test_helper.
|
|
:delay may be a Float to represent fractional seconds.
|
|
We now give an example of how a before_fork hook can be used
to incrementally migrate off the old code base without hitting
a thundering herd (especially in the "preload_app false") case.
Also comment on the per-worker listen usage in the RDoc, not
just a hidden comment.
|
|
There's always been a small window of opportunity for a script
to do File.read(pid).to_i would cause File.read() to read an
empty file and return "". This closes that window while
hopefully retaining backwards compatibility...
We've always checked for dirname(pid) writability in
Configurator, so we can safely write to a temporary file in the
intended directory and then atomically rename() it to the
destination path.
|
|
It's pointless to try and stat a file before trying to read it.
Instead just try opening it and rescue ENOENT because it
would've been racy anyways.
Additionally add some comments to keep us from forgetting
why we did the things we did with the pid file management.
|
|
I'd rather document and maintain a stable interface for the
Worker class than to have to deal with potential (portability
and security) issues with with supporting user privilege
management right now.
There's already an example of user/group-switching support in
the after_fork() hook and the error handling involved may be
different depending on the application and environment so I
remain hesitant to add official support for it...
|
|
It's compatible with both Ruby 1.8 and 1.9 without
needing a Range object.
|
|
We no longer have external lookups for it so just stick it in
the DEFAULTS hash for now. Since the Configurator::DEFAULTS
hash can be considered a stable interface for other modules to
interact with, they can eventually just use it instead of
relying on another constant.
|
|
Hopefuly make it more obvious that they're Ruby symbols and not
strings. While we're at it, fix ordering of :{rcv,snd}buf
descriptions to (logically) match the order of mention.
|
|
* Use the new :tries and :default parameters for listen()
instead of the ugly and less-effective "rescue nil"
* ActiveRecord connection management examples for hooks when
using for "preload_app true"
* combine "preload_app true" example with REE COW-friendly
optimization for memory savings
Some of these are based on Chris Wanstrath's configuration
posted here: http://gist.github.com/189623
|
|
Sometimes the upgraded version won't survive and we can fail to
unset that pid and instead accidentally create a local variable.
This is unlikely to be a problem in practice because this
variable is immediately reclobbered when we fork.
|
|
Array#concat avoids an intermediate Array object from being
allocated (yes, still supporting Rails <= 1.2.x apps...)
|
|
Our HttpResponse class interprets non-Integer string status
now as well as falling back if it can't be looked up.
|
|
It really shouldn't be a problem for existing CGI apps
to write to the StringIO object..
|
|
We've started using magic comments to ensure any strings we
create are binary instead. Additionally, ensure we create any
StringIO objects with an explicit string (which default to
binary) to ensure the StringIO object is binary. This is
because StringIO.new (with no arguments) will always use the
process-wide default encoding since it does not know about
magic comments (and couldn't, really...)
|
|
Avoids making the #listen method any noisier than it should be.
|
|
This may be redundant for the "normal" configuration file
directive, but allows the same syntax to be used in after_fork
hooks where HttpServer#listen() may be called.
|
|
This allows per-worker listeners to be configured to retry and
and not continue until the equivalent worker belonging to a
previous master (or even another server) has released the
socket.
In the Configurator RDoc, include better examples for
per-worker server.listen calls using these :tries == -1.
Inspired by an example by Chris Wanstrath.
|
|
We don't need the Z constant anymore and inlining the header
writing gives a small overall performance improvement in
microbenchmarks. This also makes this method reentrant and
thread-safe for Rainbows as well.
|