Date | Commit message (Collapse) |
|
In our race to have more concurrency options than real sites
using this server, we've added two new and fully supported
concurrency models: WriterThreadSpawn and WriterThreadPool
They're both designed to for serving large static files and work
best with IO.copy_stream (sendfile!) under Ruby 1.9. They may
also be used to dynamically generate long running, streaming
responses after headers are sent (use "proxy_buffering off" with
nginx).
Unlike most concurrency options in Rainbows!, these are designed
to run behind nginx (or haproxy if you don't support POST/PUT
requests) and are vulnerable to slow client denial of service
attacks.
I floated the idea of doing something along these lines back in
the early days of Unicorn, but deemed it too dangerous for some
applications. But nothing is too dangerous for Rainbows! So
here they are now for your experimentation.
|
|
|
|
Mostly internal cleanups and small improvements.
The only backwards incompatible change was the addition of the
"client_max_body_size" parameter to limit upload sizes to
prevent DoS. This defaults to one megabyte (same as nginx), so
any apps relying on the limit-less behavior of previous will
have to configure this in the Unicorn/Rainbows! config file:
Rainbows! do
# nil for unlimited, or any number in bytes
client_max_body_size nil
end
The ThreadSpawn and ThreadPool models are now optimized for serving
large static files under Ruby 1.9 using IO.copy_stream[1].
The EventMachine model has always had optimized static file
serving (using EM::Connection#stream_file_data[2]).
The EventMachine model (finally) gets conditionally deferred app
dispatch in a separate thread, as described by Ezra Zygmuntowicz
for Merb, Ebb and Thin[3].
[1] - http://euruko2008.csrug.cz/system/assets/documents/0000/0007/tanaka-IOcopy_stream-euruko2008.pdf
[2] - http://eventmachine.rubyforge.org/EventMachine/Connection.html#M000312
[3] - http://brainspl.at/articles/2008/04/18/deferred-requests-with-merb-ebb-and-thin
|
|
* avoid needless links to /Rainbows.html
* keepalive_timeout has been 5 seconds by default for a while
* update "Gemcutter" references to "RubyGems.org"
|
|
|
|
It should make life easier when dealing with and testing
complex dependencies, especially ones like Cramp which
depend on prerelease versions of gems.
|
|
Unicorn 0.97.0 has a bunch of internal cleanups and small fixes
and this is mainly to resync with those changes.
keepalive_timeout now defaults to 5 seconds (from 2 seconds
previous). This should help out clients on slower connections.
Some small fixes and cleanups:
* Rainbows::Fiber::IO objects may leak if a rare app uses them
explicitly with FiberSpawn/FiberPool-only (not RevFiberSpawn)
* quiet down ENOTCONN handling, there's nothing we can do about
this error so we won't fill our logs with it.
|
|
This release should fix ThreadSpawn green thread blocking issues
under MRI 1.8. Excessive socket closing is avoided when using
Thread* models with Sunshowers (or clients disconnecting
during uploads).
There is a new RevFiberSpawn concurrency model which combines
Rev with the traditional FiberSpawn model.
|
|
Oops.
|
|
This release introduces compatibility with Sunshowers, a library
for Web Sockets, see http://rainbows.rubyforge.org/sunshowers
for more information. Several small cleanups and fixes.
Eric Wong (20):
add RevThreadPool to README
rev: do not initialize a Rev::Loop in master process
rainbows.1: update headers
do not log IOError raised during app processing
move "async.callback" constant to EvCore
larger thread pool default sizes ({Rev,}ThreadPool)
ev_core: no need to explicitly close TmpIOs
EventMachine: allow usage as a base class
NeverBlock: resync with recent our EM-related expansion
RevThread*: move warning message to a saner place
EventMachineDefer: preliminary (and) broken version
TODO: add EM Deferrables
RevThread*: remove needless nil assignment
README: HTML5 Web Sockets may not be supported, yet...
env["hack.io"] for Fiber*, Revactor, Thread* models
EventMachineDefer is experimental
README: add Sunshowers reference
Rakefile: resync with Unicorn
doc/comparison: add Web Sockets to comparison
README updates
|
|
keepalive_timeout (default: 2 seconds) is now supported to
disconnect idle connections. Several new concurrency models
added include: NeverBlock, FiberSpawn and FiberPool; all of
which have only been lightly tested. RevThreadSpawn loses
streaming input support to become simpler and faster for the
general cases. AppPool middleware is now compatible with all
Fiber-based models including Revactor and NeverBlock.
A new document gives a summary of all the options we give you:
http://rainbows.rubyforge.org/Summary.html
If you're using any of the Rev-based concurrency models, the
latest iobuffer (0.1.3) gem will improve performance. Also,
RevThreadSpawn should become usable under MRI 1.8 with the next
release of Rev (0.3.2).
|
|
RDoc likes them for some reason
|
|
|
|
Don't show ChangeLog for unreleased things
|
|
This will hopefully make many things clearer about the project.
|
|
|
|
Client shutdowns/errors when streaming "rack.input" into the
Rack application are quieter now. Rev and EventMachine workers
now shutdown correctly when the master dies. Worker processes
now fail gracefully if log reopening fails. ThreadSpawn and
ThreadPool models now load Unicorn classes in a thread-safe way.
There's also an experimental RevThreadSpawn concurrency
model which may be heavily reworked in the future...
Eric Wong (30):
Threaded models have trouble with late loading under 1.9
cleanup worker heartbeat and master deathwatch
tests: allow use of alternative sha1 implementations
rev/event_machine: simplify keepalive checking a bit
tests: sha1.ru now handles empty bodies
rev: split out further into separate files for reuse
rev: DeferredResponse is independent of parser state
remove unnecessary class variable
ev_core: cleanup handling of APP constant
rev: DeferredResponse: always attach to main loop
initial cut of the RevThreadSpawn model
rev_thread_spawn/revactor: fix TeeInput for short reads
rev_thread_spawn: make 1.9 TeeInput performance tolerable
tests: add executable permissions to t0102
tests: extra check to avoid race in reopen logs test
rev_thread_spawn: 16K chunked reads work better
tests: ensure proper accounting of worker_connections
tests: heartbeat-timeout: simplify and avoid possible race
tests: ensure we process "START" from FIFO when starting
http_response: don't "rescue nil" for body.close
cleanup error handling pieces
tests: more stringent tests for error handling
revactor/tee_input: unnecessary error handling
gracefully exit workers if reopening logs fails
revactor/tee_input: raise ClientDisconnect on EOFError
bump versions since we depend on Unicorn::ClientShutdown
revactor/tee_input: share error handling with superclass
RevThreadSpawn is still experimental
Revert "Threaded models have trouble with late loading under 1.9"
Rakefile: add raa_update task
|
|
We depend on the just-released Unicorn 0.94.0 for the fixed
trailer handling. As with `unicorn', the `rainbows' executable
now sets and respects ENV["RACK_ENV"]. Also small fixes and
cleanups including better FreeBSD 7.2 compatibility and
less likely to over-aggressively kill slow/idle workers
when a very low timeout is set.
Eric Wong (20):
rev: split out heartbeat class
bump Unicorn dependency to (consistently) pass tests
tests: avoid single backquote in echo
event_machine: avoid slurping when proxying
tests: make timeout tests reliable under 1.9
thread_pool: comment for potential SMP issue under 1.9
Allow 'use "model"' as a string as well as symbol
Rev model is the only user of deferred_bodies
ev_core: use Tempfile instead of Unicorn::Util::tmpio
ev_core: ensure quit is triggered on all errors
rainbows: set and use process-wide ENV["RACK_ENV"]
http_server: add one second to any requested timeout
thread_pool: update fchmod heartbeat every second
t0004: tighten up timeout test
ev_core: remove Tempfile usage once again
cleanup: remove unused t????.ru test files
tests: staggered trailer upload test
ensure RACK_ENV is inherited from the parent env
t0100: more precise `expr` usage
|
|
It's more common form for externally-visible/modifiable
variables in Makefiles and shell scripts.
|
|
This release adds preliminary Rev support for network
concurrency under Ruby 1.8 and Ruby 1.9. There are caveats to
this model and reading the RDoc for Rainbows::Rev is
recommended.
Rainbows::AppPool Rack middleware is now available to limit
application concurrency on a per-process basis independently of
network concurrency. See the RDoc for this class for further
details.
Per-client timeouts have been removed, see
http://mid.gmane.org/20091013062602.GA13128@dcvr.yhbt.net
for the reasoning.
Rack environment changes:
* "rack.multithread" is now only true for models with "Thread"
in their name. Enabling thread-safe (but not reentrant) code
may actually be harmful for Revactor.
* "rainbows.model" is now exposed so the application can easily
figure out which network concurrency model is in use.
Bugfixes include better shutdown and error handling for all
existing models, OpenBSD compatibility for the per-process
heartbeat (same as found in unicorn v0.93.3).
Eric Wong (53):
add SIGNALS doc to RDoc
SIGNALS: add Rainbows!-specific notes
doc: better "Rainbows!" RDoc examples and linkage
tests: generate random_blob once for all tests
tests: move trash files to their own trash/ directory
t0000: basic test includes keepalive + pipelining
tests: simplify temporary file management
tests: add dbgcat() utility method
fchmod heartbeat flips between 0/1
tests: add revactor pipelining/keepalive test
thread_spawn: trap EAGAIN on accept_nonblock
thread_spawn: more robust loop
thread_spawn: non-blocking accept() shouldn't EINTR
tests: enable pipefail shell option if possible
README for test suite
tests: TEST_OPTS => SH_TEST_OPTS
tests: update TRACER examples in makefile
tests: create a bad exit code by default
thread_spawn: clean up nuking of timed-out threads
factor out common listen loop error handling
graceful exit on trap TypeError from IO.select
expand and share init_worker_process
revactor: break on EBADF in the accepting actors
revactor: cleanups and remove redundancy
No need to be halving timeout, already done for us
revactor: graceful death of keepalive clients
revactor: continue fchmod beat in graceful exit
cleanup thread models, threads no longer time out
revactor: fix graceful shutdown timeouts
Fix graceful shutdowns for threaded models
SIGINT/SIGTERM shuts down instantly in workers
tests: check for common exceptions with "Error"
DEPLOY: update with notes on DoS potential
tests: add reopen logs test for revactor
vs Unicorn: use diagrams for concurrency models
vs Unicorn: fix wording to be consistent with diagrams
vs Unicorn: fix copy+paste errors and grammar fail
README: alter reply conventions for the mailing list
preliminary Rev support
local.mk.sample: use ksh93 as default $(SHELL)
rack.multithread is only true for Thread* models
Rev: general module documentation + caveats
Rev: fix error handling for parser errors
t3003: set executable bit
documentation updates (mostly on network models)
rack: expose "rainbows.model" in Rack environment
tests: enforce rack.multithread and rainbows.model
README: update URLs
README: update with Rev model caveats
Add Rainbows::AppPool Rack middleware
t4003: chmod +x
local.mk.sample: use rev 0.3.1 instead
README: link to AppPool and extra note about Rev model
|
|
|
|
|
|
I'd rather write shell scripts in shell than shell scripts in
Ruby like was done with Unicorn. We're a *nix-only project so
we'll embrace *nix tools to their fullest extent and as a
pleasant side-effect these test cases are immune to internal API
changes.
|
|
No tests yet, but the old "gossamer" and "rainbows" branches
seem to be basically working.
|