| Date | Commit message (Collapse) |
|---|
|
Client shutdowns/errors when streaming "rack.input" into the
Rack application are quieter now. Rev and EventMachine workers
now shutdown correctly when the master dies. Worker processes
now fail gracefully if log reopening fails. ThreadSpawn and
ThreadPool models now load Unicorn classes in a thread-safe way.
There's also an experimental RevThreadSpawn concurrency
model which may be heavily reworked in the future...
Eric Wong (30):
Threaded models have trouble with late loading under 1.9
cleanup worker heartbeat and master deathwatch
tests: allow use of alternative sha1 implementations
rev/event_machine: simplify keepalive checking a bit
tests: sha1.ru now handles empty bodies
rev: split out further into separate files for reuse
rev: DeferredResponse is independent of parser state
remove unnecessary class variable
ev_core: cleanup handling of APP constant
rev: DeferredResponse: always attach to main loop
initial cut of the RevThreadSpawn model
rev_thread_spawn/revactor: fix TeeInput for short reads
rev_thread_spawn: make 1.9 TeeInput performance tolerable
tests: add executable permissions to t0102
tests: extra check to avoid race in reopen logs test
rev_thread_spawn: 16K chunked reads work better
tests: ensure proper accounting of worker_connections
tests: heartbeat-timeout: simplify and avoid possible race
tests: ensure we process "START" from FIFO when starting
http_response: don't "rescue nil" for body.close
cleanup error handling pieces
tests: more stringent tests for error handling
revactor/tee_input: unnecessary error handling
gracefully exit workers if reopening logs fails
revactor/tee_input: raise ClientDisconnect on EOFError
bump versions since we depend on Unicorn::ClientShutdown
revactor/tee_input: share error handling with superclass
RevThreadSpawn is still experimental
Revert "Threaded models have trouble with late loading under 1.9"
Rakefile: add raa_update task
|
|
|
|
This reverts commit e1dcadef6ca242e36e99aab19e3e040bf01070f9.
This is fixed separately in Unicorn 0.95.0 (commit
560216c2fecfc5cf3489f749dc7a0221fd78eb26)
|
|
|
|
Less stuff to maintain is good.
|
|
|
|
|
|
Based on unicorn.git commit e4256da292f9626d7dfca60e08f65651a0a9139a
raise Unicorn::ClientShutdown if client aborts in TeeInput
Leaving the EOFError exception as-is bad because most
applications/frameworks run an application-wide exception
handler to pretty-print and/or log the exception with a huge
backtrace.
Since there's absolutely nothing we can do in the server-side
app to deal with clients prematurely shutting down, having a
backtrace does not make sense. Having a backtrace can even be
harmful since it creates unnecessary noise for application
engineers monitoring or tracking down real bugs.
|
|
Permissions for the logs could've been badly set by the master.
So we we'll let the master reopen them and refork children to
get around this problem. We have to be more careful when
reopening logs because we can reopen them in the middle of
client requests (we have to) whereas Unicorn has the luxury
of _knowing_ it has no active clients when it does the reopen.
|
|
We're doomed if the client socket EOFs on us while we're reading
it. So don't hide it and let the exception bubble all the way
up the stack.
|
|
Make sure any aborted/broken clients don't screw up
our connection accounting.
|
|
Unicorn 0.94.0 got a more generic handle_error function
that's useful in the Thread* models. The Revactor one
is a little different but similar to be worth refactoring
to match our standard pieces.
|
|
This can hide bugs in Rack applications/middleware. Most other
Rack handlers/servers seem to follow this route as well, so
this helps ensure broken things will break loudly and more
consistently across all Rack-enabled servers.
|
|
Avoid the chances of misfiring when waiting on the master
process to start in case something bad happens or we're
sharing the FIFO for other purposes.
|
|
It seems possible to have a race condition here with
the FIFO being overloaded for both start detection
and blocking. Since SIGSTOP is unavoidable, just use
that instead and sleep immediately afterwards in case
SIGSTOP is not processed in time.
|
|
Counting worker connections is easy-to-forget when implementing
new concurrency models and forgetting to do it means new clients
cannot be accepted. Fortunately some concurrency models tend
to do it for us.
|
|
When reading 4K chunks, performance is dismal under 1.8
|
|
In Unicorn, the master reopens logs before the workers do in
case the workers die while reopening logs. But for our test
cases (and real-world usage) we need to ensure the workers have
reopened logs as well.
|
|
|
|
Somehow 1.8 performance blows with shorter reads in the Rack
application. This may be because the Rev framework uses
a default 16K IO size and our test applications may request
less.
|
|
Explicitly requested short reads may cause too much data to be
returned, which would be bad and potentially break the
application. We need to ensure proper IO#readpartial-like
semantics in both of these models.
|
|
Seems to pass all tests, but that may only mean our
test cases are lacking...
|
|
It's too complicated to deal with multiple Rev loops
so only use the main one for now under 1.9.
|
|
It'll make development of future ev_core-derived things
easier, hopefully.
|
|
It's already global...
|
|
In the upcoming RevThread* models, the parser may be parsing
other requests already by the time DeferredResponse is called.
|
|
This will make things easier to manage with more
Rev-based concurrency models.
|
|
env['rack.input']read(length) may return nil zero-sized inputs
|
|
Since the HTTP parser is frozen during app dispatch, there's
no point in checking for HTTP keepalive sooner. Of course we
check G.alive as late as possible since we could've received a
:QUIT signal while app.call was running.
|
|
sha1sum(1) is only common GNU systems, and it may be installed
as gsha1sum on *BSDs. We'll also try using the openssl sha1
implementation, too. And finally, we'll provide our own Ruby
sha1sum.rb implementation as a last resort.
We go to great lengths to avoid our own Ruby version because we
want to avoid putting too much trust in ourselves, our Ruby
skills, and even the Ruby implementations. This is especially
with regard to our knowledge and correct usage of Ruby 1.9
encoding support. It would actually be *easier* to only use
sha1sum.rb and call it a day. We just choose to support
SHA1 implementations provided by third parties if possible.
Performance is not a factor since sha1sum.rb performance is very
close to the C implementations.
|
|
It turns out neither the EventMachine and Rev classes
checked for master death in its heartbeat mechanism.
Since we managed to forget the same thing twice, we
now have a test case for it and also centralized the
code to remove duplication.
|
|
Loading TeeInput or HttpResponse late does not always work well
in multithreaded situations and have been causing random test
failures on heavily loaded multicore boxes.
|
|
We depend on the just-released Unicorn 0.94.0 for the fixed
trailer handling. As with `unicorn', the `rainbows' executable
now sets and respects ENV["RACK_ENV"]. Also small fixes and
cleanups including better FreeBSD 7.2 compatibility and
less likely to over-aggressively kill slow/idle workers
when a very low timeout is set.
Eric Wong (20):
rev: split out heartbeat class
bump Unicorn dependency to (consistently) pass tests
tests: avoid single backquote in echo
event_machine: avoid slurping when proxying
tests: make timeout tests reliable under 1.9
thread_pool: comment for potential SMP issue under 1.9
Allow 'use "model"' as a string as well as symbol
Rev model is the only user of deferred_bodies
ev_core: use Tempfile instead of Unicorn::Util::tmpio
ev_core: ensure quit is triggered on all errors
rainbows: set and use process-wide ENV["RACK_ENV"]
http_server: add one second to any requested timeout
thread_pool: update fchmod heartbeat every second
t0004: tighten up timeout test
ev_core: remove Tempfile usage once again
cleanup: remove unused t????.ru test files
tests: staggered trailer upload test
ensure RACK_ENV is inherited from the parent env
t0100: more precise `expr` usage
|
|
It's also more portable since "+" isn't portable on
FreeBSD.
|
|
Add tests to ensure we set it correctly and it gets
passed down to the app.
|
|
This test lead to two separate bugfixes in Unicorn, one in the
HttpParser and the other in TeeInput. Ironically, this test was
spawned from what I initially thought was a bug in the EvCore
module used by Rev and EventMachine, but there was no bug in
EvCore...
|
|
|
|
We're simply too uncomfortable with the weird GC issues
associated with Tempfile and having linked temporary files at
all. Instead just depend on the #size-aware TmpIO class that
Unicorn 0.94.0 provides for us.
|
|
We've worked around trigger happy timeouts in the
master since we track the timeout at a lower resolution
here.
|
|
Like the rest of the concurrency models. This gives us
more flexibility in case a process-wide blocking operation
started during an "unlucky" period when the join timeout
was about to expire.
|
|
This is because our timeout implementations are less precise
than Unicorn. Since we handle multiple clients with the same
process, we sacrifice precision for performance and instead
implement our fchmod heartbeats at a fixed rate, as doing
fchmod() repeated for short-lived connections would hurt
performance and we have to call fchmod even when connected
clients are idle.
|
|
Merb, Sinatra, Thin, Passenger and now even Unicorn will it, so
we set and use it too because that's what all the cool kids are
doing.
|
|
Just in case something goes wrong with the write
or the logger, make sure we've triggered a quit.
|
|
Since we're geared towards slower clients, we may be able to
make gains from using userspace IO buffering. This allows us to
avoid metadef-ing a #size method for every File we allocate
and save memory.
|
|
We don't use it in EventMachine since EM has its own
built-in ways to handle deferred bodies.
|
|
Since const_get works with a string as well as a symbol,
allow that to be used. It's easier and simpler to just
allow strings as use arguments than to error check and
raise exceptions.
So both of the following should now work:
Rainbows! do
use :Revactor
end
Rainbows! do
use "Revactor"
end
Rainbows! will always use the symbol variant internally,
however, so applications can alway expect env['rainbows.model']
to be a symbol.
|
|
The problem is unconfirmed at the moment, but I've long
anticipated it. I just need to remember the next time I
log into a monster machine.
|
|
We need to resort to SIGSTOP to block off processes entirely
since 1.9 uses native threads.
|
|
Avoid slurping in case we're a fast backend writing to a slow
client. This should prevent our memory usage from exploding
when clients are reading slowly.
|
|
It's not portable to FreeBSD 7.2 /bin/sh
|