Date | Commit message (Collapse) |
|
Ruby (MRI) 2.1 optimizes allocations away on String#freeze with
literal strings.
Furthermore, Ruby 2.2 optimizes away literal string allocations
when they are used as arguments to Hash#[] and Hash#[]=
Thus we can avoid expensive constant lookups and cache overhead
by taking advantage of advancements in Ruby.
Since Ruby 2.2 has been out for 7 months, now; it ought to be safe
to introduce minor performance regressions for folks using older
Rubies (1.9.3+ remains supported) to benefit folks on the latest
Ruby.
This should recover the performance lost in the
"reflect changes in Rack::Utils::HTTP_STATUS_CODES" change
in synthetic benchmarks.
|
|
Applications may want to alter the message associated with HTTP
status codes in Rack::Utils::HTTP_STATUS_CODES. Avoid memoizing
status lines ahead-of-time
Note: this introduces a minor performance regression, but ought to
be unnoticeable unless you're running "Hello world"-type apps.
|
|
TCP socket options are now set when inheriting existing sockets from
a parent process. I'm fairly certain all the TCP setsockopt knobs
we use are idempotent and harmless to change.
If anything, the only directive I'd be uncomfortable changing is
shortening the listen(2) (aka :backlog) size, but we've always
changed that anyways since it also applies to UNIX sockets.
Note: removing a configuration knob in a unicorn config file can not
reset the value to the OS-provided default setting. Inherited
sockets must use a new setting to override existing ones.
(or the socket needs to be closed and re-created in the process
launcher before unicorn inherits it).
Noticed-by: Christos Trochalakis <yatiohi@ideopolis.gr>
<20150626114129.GA25883@luke.ws.skroutz.gr>
|
|
Most of these were found by the `linkchecker' package
in Debian.
|
|
Middlewares such as Rack::Lock (used by Rails) break badly unless
the response body is closed on hijack, so we will close it to follow
the lead of other popular Rack servers.
While it's unclear if there's anybody using rack.hijack with unicorn,
we'll try to emulate the behavior of other servers as much as
possible.
ref: https://github.com/ngauthier/tubesock/issues/10
|
|
Combined with the previous commit to eliminate the `@socket'
instance variable, this eliminates the last instance variable
in the Unicorn::HttpRequest class.
Eliminating the last instance variable avoids the creation of a
internal hash table used for implementing the "generic" instance
variables found in non-pure-Ruby classes. Method entry overhead
remains the same.
While this change doesn't do a whole lot for unicorn memory usage
where the HttpRequest is a singleton, it helps other HTTP servers
which rely on this code where thousands of clients may be connected.
|
|
This avoids the expensive generic instance variable for @socket
and exposes the socket as `env["unicorn.socket"]' to the Rack
application.
As as nice side-effect, applications may access
`env["unicorn.socket"]' as part of the API may be useful for
3rd-party bits such as Raindrops::TCP_Info for reading the tcp_info
struct on Linux-based systems.
Yes, `env["unicorn.socket"]' is a proprietary API in unicorn!
News at 11! But then again, unicorn is not the first Rack server
to implement `env["#{servername}.socket"]', either...
|
|
It's ugly and less-readable to have redundant \z statements, and
according to ObjectSpace.memsize_of, this saves 4 bytes on x86-64.
|
|
Unnecessarily exposed accessors and constants take up unnecessary
memory in constant/method tables as well as using extra space in
instruction sequences.
Preforking servers like unicorn are a bloated pigs anyways,
but saving a few hundred bytes here and there can add up and
make them marginally less bad.
|
|
Rack 1.4 and earlier will soon die out, so avoid having extra,
overengineered code and method dispatch to silently drop support
for mis-hijacking with old Rack versions.
This will cause improperly hijacked responses in all versions of
Rack to fail, but allows properly hijacked responses to work
regardless of Rack version.
Followup-to: commit fdf09e562733f9509d275cb13c1c1a04e579a68a
("http_request: support rack.hijack by default")
|
|
Some process managers such as foreman and daemontools rely on
unicorn not daemonizing, but we still want to be able to process
SIGWINCH in that case.
stdout and stderr may be redirected to a pipe (for cronolog or
similar process), so those are less likely to be attached to a TTY
than stdin. This also allows users to process SIGWINCH when running
inside a regular terminal if they redirect stdin to /dev/null.
Reported-by: Dan Moore <dan@vaporwa.re>
References: <etPan.555b4293.5b47a5b7.e617@danbookpro>
<20150519232858.GA23515@dcvr.yhbt.net>
|
|
proc creation is expensive, so merely use a 48-byte generic ivar
hash slot for @socket instead.
|
|
Rack 1.4 and earlier will soon die out, so avoid having extra code
The only minor overhead is assigning two hash slots and
the extra hash checks when running ancient versions of Rack,
so it is unlikely anybody cares about that overhead with Rack 1.5
and later.
|
|
kgio_wait_readable is superior for single FDs in that it may use the
ppoll syscall on Linux via Ruby, making it immune to the slowdown
high FDs with select() and the array allocations enforced by the
Ruby wrapper interface.
Note: IO#wait in the io/wait stdlib has the same effect, but as of
2.2 still needlessly checks the FIONREAD ioctl. So avoid needing to
force a new require on users which also incur shared object loading
costs. The longer term plan is to rely entirely on Ruby IO
primitives entirely and drop kgio, but that won't happen until we
can depend on Ruby 2.3 for exception-free accept_nonblock
(which will be released December 2015).
|
|
rack 1.6 added a TempfileReaper middleware to cleanup temporary
files. Enable it by default for users running rack 1.6 or later
to avoid leaving temporary files around.
|
|
Rack::TempfileReaper was added in rack 1.6 to cleanup temporary
files. Make Unicorn::TmpIO ducktype-compatible so
Rack::TempfileReaper may be used to free up space used by temporary
buffer files.
Ref: <CY1PR0301MB078011EB5A22B733EB222A45A4EE0@CY1PR0301MB0780.namprd03.prod.outlook.com>
Reported-by: Mike Mulvaney <MMulvaney@bna.com>
|
|
Literal regexps cost over 450 bytes of memory per-site and
unnecessary use of them costs memory in places where raw execution
speed does not matter. Nowadays, we can rely on String#end_with?
(introduced in 1.8.7) for improved readability, too.
|
|
We had HTTPS support but dropped it(*) and some wacky servers out
there do work better with TCP_DEFER_ACCEPT disabled.
(*) No, we will not support HTTP/2, that is for nginx
|
|
Empty classes do not need a heavy class definition scope.
|
|
Literal String#freeze avoids allocations since Ruby 2.1 via the
opt_str_freeze instruction, so we can start relying on it in
some places as Ruby 2.1 adoption increases. The 100-continue
handling is a good place to start since it is an uncommonly-used
code path which benefits from size reduction and the negative
performance impact is restricted to a handful of users.
HTTP_RESPONSE_START can safely live in http_request.rb as its
usage does not cross namespace boundaries
The goal is to eventually eliminate Unicorn::Const entirely.
|
|
Rainbows! (in maintenance mode) will need to define it's own
constants in the future. We'll trim down our constant usage in
subsequent commits as we take advantage of Ruby VM improvements.
|
|
Oops, this should've been explained long ago but apparently not.
In response to a comment on
http://www.sitepoint.com/the-self-pipe-trick-explained/
> Does anybody know why both unicorn and foreman read 11 bytes from
> self-pipe?
Unfortunately I couldn't find a way to comment on the site on a
JavaScript-free browser nor does it seem possible without
registering.
Again, anybody can send plain-text mail to:
unicorn-public@bogomips.org
No registration, no real name policy, no terms-of-service, just
plain-text. Feel free to use Tor, mixmaster or any anonymity
service, too.
|
|
In 1.9+ at least, instance variables use less space than constants
in class tables and bytecode, leading to ~700 byte reduction in
bytecode overhead on 64-bit and a reduction in constant table/entries
of the Unicorn::HttpServer class.
|
|
The former is shorter Ruby code and also generates smaller
bytecode.
|
|
unicorn 5 will not support Ruby 1.8 anymore.
Drop mentions of Rubinius, too, it's too difficult to support due to
the proprietary and registration-required nature of its bug tracker.
The smaller memory footprint and CoW-friendly memory allocator in
mainline Ruby is a better fit for unicorn, anyways.
Since Ruby 1.9+ bundles RubyGems and gem startup is faster nowadays,
we'll just depend on that instead of not loading RubyGems.
Drop the local.mk.sample file, too, since it's way out-of-date
and probably isn't useful (I have not used it in a while).
|
|
require_relative appeared in Ruby 1.9.2 to speed up load times by
avoiding needless open() syscalls. This has no effect if you're using
RUBYLIB or the '-I' option when running ruby(1), but avoids searching
paths in other gems.
This does not affect unicorn greatly as unicorn does not activate many
gems, but still leads to reducing ~45 syscalls during startup.
|
|
IO#close_on_exec* methods are available since Ruby 1.9.1. It
allows us to use less bytecode as it requires fewer operands and
avoids constant lookups.
|
|
We're requiring Ruby 1.9.3+, so we can safely depend
on IO#autoclose= being available in 1.9+ and shave off
some bloat.
|
|
In Ruby 1.9.2+, socket options may be specified using symbols
instead of constants to avoid the need to import Socket::Constants
into the namespace. This also has a nice side-effect of reducing
the size of the bytecode by trading 3 instructions (getinlinecache,
getconstant, setinlinecache) for one "putobject" instruction.
Nowadays, we may also avoid defining OS-specific constants ourselves
since 1.9+ versions of Ruby already provide them to further reduce
bytecode size.
getsockopt also returns Socket::Option objects in 1.9.2+,
allowing us to avoid the larger "unpack('i')" method dispatch
for an operand-free "int" method call.
Finally, favor Object#nil? calls rather than "== nil" comparisons
to reduce bytecode size even more.
Since this code is only called at startup time, it does not benefit
from inline caching of constant lookups in current mainline Ruby.
Combined, these changes reduce YARV bytecode size by around 2K on a
64-bit system.
|
|
While it was technically interesting and fun to tunnel arbitrary
protocols over a semi-compliant Rack interface, nobody actually does
it (and anybody who does can look in our git history). This was
from back in 2009 when this was one of the few servers that could
handle chunked uploads,were one of the few users of chunked uploads,
nowadays everyone does it! (or do they? :)
A newer version of exec_cgi.rb still lives on in the repository of
yet another horribly-named server, but there's no point in bloating
the installation footprint of somewhat popular server such as unicorn.
|
|
The monotonic clock is immune to time adjustments so it is not
thrown off by misconfigured clocks. Process.clock_gettime also
generates less garbage on 64-bit systems due to the use of Flonum.
|
|
It is redundant given the existence of File#size in Ruby 1.9+
This saves 1440 bytes of bytecode on x86-64 under 2.2.0,
and at least another 120 bytes for the method entry,
hash table entry, and method definition overhead.
|
|
We implemented barely-advertised support for SSL for two reasons:
1) to detect corruption on LANs beyond what TCP offers
2) to support other servers based on unicorn (never happened)
Since this feature is largely not useful for unicorn itself,
there's no reason to penalize unicorn 5.x users with bloat.
In our defense, SSL support appeared in version 4.2.0 :)
|
|
Replacing the Regexp argument to a rarely-called String#split with a
literal String can save a little memory. The removed Regexp memsize
is 469 bytes on Ruby 2.1:
ObjectSpace.memsize_of(/,/) => 469
Is slightly smaller at 453 bytes on 2.2.0dev (r48474).
These numbers do not include the 40-byte object overhead.
Nevertheless, this is a waste for non-performance-critical code
during the socket inheritance phase. A literal string has less
overhead at 88 bytes:
* 48 bytes for table entry in the frozen string table
* 40 bytes for the object itself
The downside of using a literal string for the String#split argument
is a 40-byte string object gets allocated on every call, but this
piece of code is only called once in a process lifetime.
|
|
This has not been used since unicorn 4.0.0 over three years ago.
This is an incompatible change, but hopefully nobody uses this in
before_fork/after_fork hooks anywhere.
|
|
Whatever compatibility reasons which existed in 2009 likely do not exist
now. Other servers (e.g. thin, puma) seem to work alright without it,
so there's no reason to waste precious bytes.
|
|
This has long been considered a mistake and not documented for very
long.
I considered removing X-Forwarded-Proto and X-Forwarded-SSL
handling, too, so rack.url_scheme is always "http", but that might
lead to compatibility issues in rare apps if Rack::Request#scheme is
not used.
|
|
Update the old mailing list info with our new public-inbox info.
The old mongrel.rubyforge.org links have been dead for years,
oh well. There's only a few days left of RubyForge left...
|
|
When daemonizing, it is possible for the grandparent to be
terminated by another process before the master can notify
it. Do not abort the master in this case.
This may fix the following issue:
https://github.com/kostya/eye/issues/49
(which I was notified of privately via email)
|
|
Thanks to Sam Saffron for the heads up.
|
|
I've never liked OobGC, so "hot potato!" :)
|
|
We close SELF_PIPE in the worker immediately, but signal handlers
do not get setup immediately. So prevent workers from erroring out
due to invalid SELF_PIPE.
|
|
This protects us from two problems:
1) we (or our app) somehow called IO#close on one of the sockets
we listen on without removing it from the readers array.
We'll ignore IOErrors from IO#close and assume we wanted to
close it.
2) our SIGQUIT handler is interrupted by itself. This can happen as
a fake signal from the master could be handled and a real signal
from an outside user is sent to us (e.g. from unicorn-worker-killer)
or if a user uses the killall(1) command.
|
|
Signaling using normal kill(2) is preserved, but the master now
prefers to signal workers using a pipe rather than kill(2).
Non-graceful signals (:TERM/:KILL) are still sent using kill(2),
as they ask for immediate shutdown.
This change is necessary to avoid triggering the ubf (unblocking
function) for rb_thread_call_without_gvl (and similar) functions
extensions. Most notably, this fixes compatibility with newer
versions of the 'pg' gem which will cancel a running DB query if
signaled[1].
This also has the nice side-effect of allowing a premature
master death (assuming preload_app didn't cause the master to
spawn off rogue child daemons).
Note: users should also refrain from using "killall" if using the
'pg' gem or something like it.
Unfortunately, this increases FD usage in the master as the writable
end of the pipe is preserved in the master. This limit the number
of worker processes the master may run to the open file limit of the
master process. Increasing the open file limit of the master
process may be needed. However, the FD use on the workers is
reduced by one as the internal self-pipe is no longer used. Thus,
overall pipe allocation for the kernel remains unchanged.
[1] - pg is correct to cancel a query, as it cannot know if
the signal was for a) graceful unicorn shutdown or
b) oh-noes-I-started-a-bad-query-ABORT-ABORT-ABORT!!
|
|
This reduces the window for a non-existent PID for folks who monitor
PIDs (not a great idea anyways). Unfortunately, this change also brings
us back to the case where having a PID later (for other process monitors)
is beneficial but more unicorn releases exist where we write the PID
early.
Thanks to Jimmy Soho for reporting this issue.
ref: <CAHStS5gFYcPBDxkVizAHrOeDKAkjT69kruFdgaY0CbB+vLbK8Q@mail.gmail.com>
This partially reverts 7d6ac0c17eb29a00a5b74099dbb3d4d015999f27
Folks: please monitor your app with HTTP requests rather than checking
processes, a stuck/wedged Ruby VM is still a running one.
|
|
This can avoid IOError from being seen by the application, and also
reduces points where IO#close may be called. This is a good thing
if we eventually port this code into a low-level server like
cmogstored where per-client memory space is defined by FD number of
a client.
Reported-by: Andrew Hobson <ahobson@gmail.com>
|
|
This renables the ability for Ruby 1.8.6 environments to perform reexecs
[ew: clarified this is for 1.8.6,
favor literal {} over Hash.new,
tweaked LISTENERS.map => LISTENERS.each, thanks to Hleb Valoshka
]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
In case we (and Linux) supports other values in the future,
we can update it then. Until now, ensure users only set
true or false for this option.
|
|
There is currently no GPLv4, so this change has no effect at the
moment.
In case the GPLv4 arrives and I am not alive to approve/review it,
the lesser of evils is have give blanket approval of all future GPL
versions (as published by the FSF). The worse evil is to be stuck
with a license which cannot guarantee the Free-ness of this project
in the future.
This unfortunately means the FSF can theoretically come out with
license terms I do not agree with, but the GPLv2 and GPLv3 will
always be an option to all users.
|
|
Thanks to Hongli Lai for noticing my typo. While we're at it, finish up
a halfway-written comment for the EXDEV case
|