| Date | Commit message (Collapse) |
|---|
|
* Use a frozen empty array and a class variable for TCP_Info to avoid
garbage. As far as I can tell, this shouldn't result in any garbage on
any requests (other than on the first request).
* Pass listener socket to #read to only check the client connection on
a TCP server.
* Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's
the most common state after ESTABLISHED, it makes the numbers
un-ordered, though. But comment should make it OK.
* Definition of of `check_client_connection` based on whether
Raindrops::TCP_Info is defined, instead of the class variable
approach.
* Changed the unit tests to pass a `nil` listener.
Tested on our staging environment, and still works like a dream.
I should note that I got the idea between this patch into Puma as well!
https://github.com/puma/puma/pull/1227
[ew: squashed in temporary change for oob_gc.rb, but we'll come
up with a different change to avoid breaking gctools
<https://github.com/tmm1/gctools>]
Acked-by: Eric Wong <e@80x24.org>
|
|
Fixes: 2af91a1fef70d654 ("Add after_worker_exit configuration option")
|
|
This option is executed in the master process following all
worker process exits. It is most useful in the case where
the worker process crashes the ruby interpreter, as the worker
process may not be able to send error notifications
appropriately.
For example, let's say you have a specific request that crashes a
worker process, which you expect to be due to a improperly
programmed C extension. By modifying your worker to save request
related data in a temporary file and using this option, you can get
a record of what request is crashing the application, which will
make debugging easier.
Example:
after_worker_exit do |server, worker, status|
server.logger.info "worker #{status.success? ? 'exit' : 'crash'}: #{status}"
file = "request.#{status.pid}.txt"
if File.exist?(file)
do_something_with(File.read(file)) unless status.success?
File.delete(file)
end
end
|
|
This ensures we won't have duplicate objects in Ruby 2.0-2.4.
For Ruby 2.5.0dev+, this avoids any duplicate cleanup
introduced as of r57471: https://bugs.ruby-lang.org/issues/13085
|
|
IO#write already elides the write(2) syscall for empty buffers,
so there's no need to complicate our instruction sequence
footprint for the rare case of an empty buffer.
The only cases a Rack app will have an empty buffer are:
1) `env['rack.input'].read` without args
2) `env['rack.input'].gets`
Neither of these calls are safe for server-independent Rack apps
as the client can OOM the app.
unicorn itself provides no facility for limiting maximum
rack.input size. Instead, unicorn relies on nginx to limit
input size using the client_max_body_size directive.
|
|
HTTPS helps some with reader privacy and Let's Encrypt seems to
be working well enough the past few months.
This change will allow us to reduce subjectAltName bloat in our
TLS certificate over time. It will also promote domain name
agility to support mirrors or migrations to other domains
(including a Tor hidden service mirror).
http://bogomips.org/unicorn/ will remain available for people on
legacy systems without usable TLS. There is no plan for automatic
redirecting from HTTP to HTTPS at this time.
|
|
Gmane's NNTP server remains up, but the HTTP site is down:
https://lars.ingebrigtsen.no/2016/07/28/the-end-of-gmane/
Anyways, our own archives are designed to be mirror-able via git:
git clone --mirror https://bogomips.org/unicorn-public
And the code is self-hostable: git clone https://public-inbox.org
|
|
We do not want to pull in a newer or older version of rack depending
on an the application running under it requires. Furthermore, it
has always been possible to use unicorn without any middleware at
all.
Without rack, we'll be missing descriptive status text in the first
response line, but any valid HTTP/1.x parser should be able to
handle it properly.
ref:
http://bogomips.org/unicorn-public/20160121201255.GA6186@dcvr.yhbt.net/t/#u
Thanks-to: Adam Duke <adam.v.duke@gmail.com>
Thanks-to: Aaron Patterson <tenderlove@ruby-lang.org>
|
|
* add nntp_url to the olddoc website footer
* update legacy support status for 4.x (not 4.8.x)
* update copyright range to 2016
* note all of our development tools are Free Software, too
* remove cgit mention; it may not always be cgit
(but URLs should remain compatible).
* discourage downloading snapshot tarballs;
"git clone" + periodic "git fetch" is more efficient
* remove most mentions of unicorn_rails as that
was meant for ancient Rails 1.x/2.x users
* update path reference to Ruby 2.3.0
* fix nginx upstream module link to avoid redirect
* shorten Message-ID example to avoid redirects
and inadvertant linkage
|
|
This blatantly violates Rack SPEC, but we've had this bug since
March 2009[1]. Thus, we cannot expect all existing applications
and middlewares to fix this bug and will probably have to
support it forever.
Unfortunately, supporting this bug contributes to application
server lock-in, but at least we'll document it as such.
[1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84
("HttpResponse: speed up non-multivalue headers")
Reported-by: Owen Ou <o@heroku.com>
Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
|
|
The PID of a process can never be zero as kill(2) interprets a '0'
PID arg as "every process in caller's process group", so there's no
risk of the 'nil.to_i => 0' conversion resulting in a truth value
when compared to $$.
|
|
Re-enable and expand on the test case while we're at it for new
Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We
shall assume anybody running a pre-release 2.3.0 at this point is
running a fairly recent snapshot, so we won't bother doing a
finer-grained check in the test for an exact revision number.
|
|
We no longer need Ruby 1.8 compatibility, so use String#clear
to reduce argument passing and code size.
|
|
They'll continue to be maintained, but we're no longer advertising
them. Also, favor lowercase "unicorn" while we're at it since that
matches the executable and gem name to avoid unnecessary escaping
for RDoc.
|
|
Linux users are effectively capped to 128 on stock installations
and may wonder why connections get rejected with overloaded apps
sooner rather than later.
|
|
systemd socket emulation shares FDs across execve, just like
the built-in SIGUSR2 upgrade process in unicorn. Thus it is
easy to support inheriting sockets from systemd.
Tested-by: Christos Trochalakis <yatiohi@ideopolis.gr>
|
|
This should save over 100 bytes of bytecode overhead due to
reduced method dispatch points. The performance difference
when this is actually hit could go either way depending on
how String#<< and realloc(3) interact, but it's uncommon
enough that nobody is expected to notice either way.
|
|
Ruby (MRI) 2.1 optimizes allocations away on String#freeze with
literal strings.
Furthermore, Ruby 2.2 optimizes away literal string allocations
when they are used as arguments to Hash#[] and Hash#[]=
Thus we can avoid expensive constant lookups and cache overhead
by taking advantage of advancements in Ruby.
Since Ruby 2.2 has been out for 7 months, now; it ought to be safe
to introduce minor performance regressions for folks using older
Rubies (1.9.3+ remains supported) to benefit folks on the latest
Ruby.
This should recover the performance lost in the
"reflect changes in Rack::Utils::HTTP_STATUS_CODES" change
in synthetic benchmarks.
|
|
Applications may want to alter the message associated with HTTP
status codes in Rack::Utils::HTTP_STATUS_CODES. Avoid memoizing
status lines ahead-of-time
Note: this introduces a minor performance regression, but ought to
be unnoticeable unless you're running "Hello world"-type apps.
|
|
TCP socket options are now set when inheriting existing sockets from
a parent process. I'm fairly certain all the TCP setsockopt knobs
we use are idempotent and harmless to change.
If anything, the only directive I'd be uncomfortable changing is
shortening the listen(2) (aka :backlog) size, but we've always
changed that anyways since it also applies to UNIX sockets.
Note: removing a configuration knob in a unicorn config file can not
reset the value to the OS-provided default setting. Inherited
sockets must use a new setting to override existing ones.
(or the socket needs to be closed and re-created in the process
launcher before unicorn inherits it).
Noticed-by: Christos Trochalakis <yatiohi@ideopolis.gr>
<20150626114129.GA25883@luke.ws.skroutz.gr>
|
|
Most of these were found by the `linkchecker' package
in Debian.
|
|
Middlewares such as Rack::Lock (used by Rails) break badly unless
the response body is closed on hijack, so we will close it to follow
the lead of other popular Rack servers.
While it's unclear if there's anybody using rack.hijack with unicorn,
we'll try to emulate the behavior of other servers as much as
possible.
ref: https://github.com/ngauthier/tubesock/issues/10
|
|
Combined with the previous commit to eliminate the `@socket'
instance variable, this eliminates the last instance variable
in the Unicorn::HttpRequest class.
Eliminating the last instance variable avoids the creation of a
internal hash table used for implementing the "generic" instance
variables found in non-pure-Ruby classes. Method entry overhead
remains the same.
While this change doesn't do a whole lot for unicorn memory usage
where the HttpRequest is a singleton, it helps other HTTP servers
which rely on this code where thousands of clients may be connected.
|
|
This avoids the expensive generic instance variable for @socket
and exposes the socket as `env["unicorn.socket"]' to the Rack
application.
As as nice side-effect, applications may access
`env["unicorn.socket"]' as part of the API may be useful for
3rd-party bits such as Raindrops::TCP_Info for reading the tcp_info
struct on Linux-based systems.
Yes, `env["unicorn.socket"]' is a proprietary API in unicorn!
News at 11! But then again, unicorn is not the first Rack server
to implement `env["#{servername}.socket"]', either...
|
|
It's ugly and less-readable to have redundant \z statements, and
according to ObjectSpace.memsize_of, this saves 4 bytes on x86-64.
|
|
Unnecessarily exposed accessors and constants take up unnecessary
memory in constant/method tables as well as using extra space in
instruction sequences.
Preforking servers like unicorn are a bloated pigs anyways,
but saving a few hundred bytes here and there can add up and
make them marginally less bad.
|
|
Rack 1.4 and earlier will soon die out, so avoid having extra,
overengineered code and method dispatch to silently drop support
for mis-hijacking with old Rack versions.
This will cause improperly hijacked responses in all versions of
Rack to fail, but allows properly hijacked responses to work
regardless of Rack version.
Followup-to: commit fdf09e562733f9509d275cb13c1c1a04e579a68a
("http_request: support rack.hijack by default")
|
|
Some process managers such as foreman and daemontools rely on
unicorn not daemonizing, but we still want to be able to process
SIGWINCH in that case.
stdout and stderr may be redirected to a pipe (for cronolog or
similar process), so those are less likely to be attached to a TTY
than stdin. This also allows users to process SIGWINCH when running
inside a regular terminal if they redirect stdin to /dev/null.
Reported-by: Dan Moore <dan@vaporwa.re>
References: <etPan.555b4293.5b47a5b7.e617@danbookpro>
<20150519232858.GA23515@dcvr.yhbt.net>
|
|
proc creation is expensive, so merely use a 48-byte generic ivar
hash slot for @socket instead.
|
|
Rack 1.4 and earlier will soon die out, so avoid having extra code
The only minor overhead is assigning two hash slots and
the extra hash checks when running ancient versions of Rack,
so it is unlikely anybody cares about that overhead with Rack 1.5
and later.
|
|
kgio_wait_readable is superior for single FDs in that it may use the
ppoll syscall on Linux via Ruby, making it immune to the slowdown
high FDs with select() and the array allocations enforced by the
Ruby wrapper interface.
Note: IO#wait in the io/wait stdlib has the same effect, but as of
2.2 still needlessly checks the FIONREAD ioctl. So avoid needing to
force a new require on users which also incur shared object loading
costs. The longer term plan is to rely entirely on Ruby IO
primitives entirely and drop kgio, but that won't happen until we
can depend on Ruby 2.3 for exception-free accept_nonblock
(which will be released December 2015).
|
|
Rack::TempfileReaper was added in rack 1.6 to cleanup temporary
files. Make Unicorn::TmpIO ducktype-compatible so
Rack::TempfileReaper may be used to free up space used by temporary
buffer files.
Ref: <CY1PR0301MB078011EB5A22B733EB222A45A4EE0@CY1PR0301MB0780.namprd03.prod.outlook.com>
Reported-by: Mike Mulvaney <MMulvaney@bna.com>
|
|
Literal regexps cost over 450 bytes of memory per-site and
unnecessary use of them costs memory in places where raw execution
speed does not matter. Nowadays, we can rely on String#end_with?
(introduced in 1.8.7) for improved readability, too.
|
|
We had HTTPS support but dropped it(*) and some wacky servers out
there do work better with TCP_DEFER_ACCEPT disabled.
(*) No, we will not support HTTP/2, that is for nginx
|
|
Literal String#freeze avoids allocations since Ruby 2.1 via the
opt_str_freeze instruction, so we can start relying on it in
some places as Ruby 2.1 adoption increases. The 100-continue
handling is a good place to start since it is an uncommonly-used
code path which benefits from size reduction and the negative
performance impact is restricted to a handful of users.
HTTP_RESPONSE_START can safely live in http_request.rb as its
usage does not cross namespace boundaries
The goal is to eventually eliminate Unicorn::Const entirely.
|
|
Rainbows! (in maintenance mode) will need to define it's own
constants in the future. We'll trim down our constant usage in
subsequent commits as we take advantage of Ruby VM improvements.
|
|
Oops, this should've been explained long ago but apparently not.
In response to a comment on
http://www.sitepoint.com/the-self-pipe-trick-explained/
> Does anybody know why both unicorn and foreman read 11 bytes from
> self-pipe?
Unfortunately I couldn't find a way to comment on the site on a
JavaScript-free browser nor does it seem possible without
registering.
Again, anybody can send plain-text mail to:
unicorn-public@bogomips.org
No registration, no real name policy, no terms-of-service, just
plain-text. Feel free to use Tor, mixmaster or any anonymity
service, too.
|
|
In 1.9+ at least, instance variables use less space than constants
in class tables and bytecode, leading to ~700 byte reduction in
bytecode overhead on 64-bit and a reduction in constant table/entries
of the Unicorn::HttpServer class.
|
|
The former is shorter Ruby code and also generates smaller
bytecode.
|
|
unicorn 5 will not support Ruby 1.8 anymore.
Drop mentions of Rubinius, too, it's too difficult to support due to
the proprietary and registration-required nature of its bug tracker.
The smaller memory footprint and CoW-friendly memory allocator in
mainline Ruby is a better fit for unicorn, anyways.
Since Ruby 1.9+ bundles RubyGems and gem startup is faster nowadays,
we'll just depend on that instead of not loading RubyGems.
Drop the local.mk.sample file, too, since it's way out-of-date
and probably isn't useful (I have not used it in a while).
|
|
require_relative appeared in Ruby 1.9.2 to speed up load times by
avoiding needless open() syscalls. This has no effect if you're using
RUBYLIB or the '-I' option when running ruby(1), but avoids searching
paths in other gems.
This does not affect unicorn greatly as unicorn does not activate many
gems, but still leads to reducing ~45 syscalls during startup.
|
|
IO#close_on_exec* methods are available since Ruby 1.9.1. It
allows us to use less bytecode as it requires fewer operands and
avoids constant lookups.
|
|
We're requiring Ruby 1.9.3+, so we can safely depend
on IO#autoclose= being available in 1.9+ and shave off
some bloat.
|
|
In Ruby 1.9.2+, socket options may be specified using symbols
instead of constants to avoid the need to import Socket::Constants
into the namespace. This also has a nice side-effect of reducing
the size of the bytecode by trading 3 instructions (getinlinecache,
getconstant, setinlinecache) for one "putobject" instruction.
Nowadays, we may also avoid defining OS-specific constants ourselves
since 1.9+ versions of Ruby already provide them to further reduce
bytecode size.
getsockopt also returns Socket::Option objects in 1.9.2+,
allowing us to avoid the larger "unpack('i')" method dispatch
for an operand-free "int" method call.
Finally, favor Object#nil? calls rather than "== nil" comparisons
to reduce bytecode size even more.
Since this code is only called at startup time, it does not benefit
from inline caching of constant lookups in current mainline Ruby.
Combined, these changes reduce YARV bytecode size by around 2K on a
64-bit system.
|
|
While it was technically interesting and fun to tunnel arbitrary
protocols over a semi-compliant Rack interface, nobody actually does
it (and anybody who does can look in our git history). This was
from back in 2009 when this was one of the few servers that could
handle chunked uploads,were one of the few users of chunked uploads,
nowadays everyone does it! (or do they? :)
A newer version of exec_cgi.rb still lives on in the repository of
yet another horribly-named server, but there's no point in bloating
the installation footprint of somewhat popular server such as unicorn.
|
|
The monotonic clock is immune to time adjustments so it is not
thrown off by misconfigured clocks. Process.clock_gettime also
generates less garbage on 64-bit systems due to the use of Flonum.
|
|
It is redundant given the existence of File#size in Ruby 1.9+
This saves 1440 bytes of bytecode on x86-64 under 2.2.0,
and at least another 120 bytes for the method entry,
hash table entry, and method definition overhead.
|
|
We implemented barely-advertised support for SSL for two reasons:
1) to detect corruption on LANs beyond what TCP offers
2) to support other servers based on unicorn (never happened)
Since this feature is largely not useful for unicorn itself,
there's no reason to penalize unicorn 5.x users with bloat.
In our defense, SSL support appeared in version 4.2.0 :)
|
|
Replacing the Regexp argument to a rarely-called String#split with a
literal String can save a little memory. The removed Regexp memsize
is 469 bytes on Ruby 2.1:
ObjectSpace.memsize_of(/,/) => 469
Is slightly smaller at 453 bytes on 2.2.0dev (r48474).
These numbers do not include the 40-byte object overhead.
Nevertheless, this is a waste for non-performance-critical code
during the socket inheritance phase. A literal string has less
overhead at 88 bytes:
* 48 bytes for table entry in the frozen string table
* 40 bytes for the object itself
The downside of using a literal string for the String#split argument
is a 40-byte string object gets allocated on every call, but this
piece of code is only called once in a process lifetime.
|
|
This has not been used since unicorn 4.0.0 over three years ago.
This is an incompatible change, but hopefully nobody uses this in
before_fork/after_fork hooks anywhere.
|