| Date | Commit message (Collapse) |
|---|
|
It's fairly easy given unicorn was designed with synchronous I/O
in mind. The overhead of backtraces from EOFError on
readpartial should be rare given our requirement to only accept
requests from fast, reliable clients on LAN (e.g. nginx or
yet-another-horribly-named-server).
|
|
Latency from redirects is painful, and HTTPS can protect privacy
in some cases.
|
|
This allows for the equivalent of the
-N/--no-default_middleware command line option to be
specified in the configuration file so it doesn't
need to be specified on the command line every time
unicorn is executed.
It explicitly excludes the use of -N/--no-default_middleware
as an embedded configuration option in the rackup file, by
ignoring the options after ARGV is parsed.
In order to allow the configuration method to work, have
the lambda that Unicorn.builder returns accept two arguments.
Technically, only one argument is needed for the HttpServer
instance, but I'm guessing if the lambda accepts a single
argument, we expect that to be a rack application instead
of a lambda that returns a rack application.
The command line option option to disable default middleware
will take precedence over the unicorn configuration file option
if both are present.
For backwards compatibility, if the lambda passed to
HttpServer accepts 0 arguments, then call it without
arguments.
[ew: fix precedence for arity checking in build_app!
configurator: ensure -N is respected when set in command-line]
|
|
Slowly reducing dependencies on kgio. 'io/wait' is required by
'socket' these days, so it's no extra relocations for .so
loading, either.
|
|
It's not unicorn 6, yet, but we dropped Ruby 1.8 support at
unicorn 5. Stable Ruby 1.9+ releases have always reseeded the
PRNG at fork.
|
|
Ruby trunk started warning about more mismatched indentations
starting around r62836.
|
|
Hijackers may capture and reuse `env' indefinitely, so we must
not use it in those cases for future requests. For non-hijack
requests, we continue to reuse the `env' object to reduce
memory recycling.
Reported-and-tested-by: Sam Saffron <sam.saffron@gmail.com>
|
|
These three cold call sites instruction sequence size by a few
hundred bytes combined since we no longer support Ruby 1.8.6.
The "?/" shorthand is esoteric and no longer avoids allocation
in Ruby 1.9+ (not that this is hot code).
|
|
* origin/worker_exec:
Don't pass a block for fork when forking workers
Add worker_exec configuration option
|
|
This quiets down warnings when run with '-w'
|
|
This reduces the stack depth, making GC more efficient.
|
|
The worker_exec configuration option makes all worker processes
exec after forking. This initializes the worker processes with
separate memory layouts, defeating address space discovery
attacks on operating systems supporting address space layout
randomization, such as Linux, MacOS X, NetBSD, OpenBSD, and
Solaris.
Support for execing workers is very similar to support for reexecing
the master process. The main difference is the worker's to_i and
master pipes also need to be inherited after worker exec just as the
listening sockets need to be inherited after reexec.
Because execing working is similar to reexecing the master, this
extracts a couple of methods from reexec (listener_sockets and
close_sockets_on_exec), so they can be reused in worker_spawn.
|
|
This adds a hook that is called after the application has
been loaded by the worker process, directly before it starts
accepting requests. This hook is necessary if your application
needs to gain access to resources during initialization,
and then drop privileges before serving requests.
This is especially useful in conjunction with chroot support
so the app can load all the normal ruby libraries it needs
to function, and then chroot before accepting requests.
If you are preloading the app, it's possible to drop privileges
or chroot in after_fork, but if you are not preloading the app,
the only way to currently do this is to override the private
HttpServer#init_worker_process method, and overriding private
methods is a recipe for future breakage if the internals are
modified. This hook allows for such functionality to be
supported and not break in future versions of Unicorn.
|
|
This option is executed in the master process following all
worker process exits. It is most useful in the case where
the worker process crashes the ruby interpreter, as the worker
process may not be able to send error notifications
appropriately.
For example, let's say you have a specific request that crashes a
worker process, which you expect to be due to a improperly
programmed C extension. By modifying your worker to save request
related data in a temporary file and using this option, you can get
a record of what request is crashing the application, which will
make debugging easier.
Example:
after_worker_exit do |server, worker, status|
server.logger.info "worker #{status.success? ? 'exit' : 'crash'}: #{status}"
file = "request.#{status.pid}.txt"
if File.exist?(file)
do_something_with(File.read(file)) unless status.success?
File.delete(file)
end
end
|
|
HTTPS helps some with reader privacy and Let's Encrypt seems to
be working well enough the past few months.
This change will allow us to reduce subjectAltName bloat in our
TLS certificate over time. It will also promote domain name
agility to support mirrors or migrations to other domains
(including a Tor hidden service mirror).
http://bogomips.org/unicorn/ will remain available for people on
legacy systems without usable TLS. There is no plan for automatic
redirecting from HTTP to HTTPS at this time.
|
|
* add nntp_url to the olddoc website footer
* update legacy support status for 4.x (not 4.8.x)
* update copyright range to 2016
* note all of our development tools are Free Software, too
* remove cgit mention; it may not always be cgit
(but URLs should remain compatible).
* discourage downloading snapshot tarballs;
"git clone" + periodic "git fetch" is more efficient
* remove most mentions of unicorn_rails as that
was meant for ancient Rails 1.x/2.x users
* update path reference to Ruby 2.3.0
* fix nginx upstream module link to avoid redirect
* shorten Message-ID example to avoid redirects
and inadvertant linkage
|
|
The PID of a process can never be zero as kill(2) interprets a '0'
PID arg as "every process in caller's process group", so there's no
risk of the 'nil.to_i => 0' conversion resulting in a truth value
when compared to $$.
|
|
Re-enable and expand on the test case while we're at it for new
Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We
shall assume anybody running a pre-release 2.3.0 at this point is
running a fairly recent snapshot, so we won't bother doing a
finer-grained check in the test for an exact revision number.
|
|
They'll continue to be maintained, but we're no longer advertising
them. Also, favor lowercase "unicorn" while we're at it since that
matches the executable and gem name to avoid unnecessary escaping
for RDoc.
|
|
systemd socket emulation shares FDs across execve, just like
the built-in SIGUSR2 upgrade process in unicorn. Thus it is
easy to support inheriting sockets from systemd.
Tested-by: Christos Trochalakis <yatiohi@ideopolis.gr>
|
|
TCP socket options are now set when inheriting existing sockets from
a parent process. I'm fairly certain all the TCP setsockopt knobs
we use are idempotent and harmless to change.
If anything, the only directive I'd be uncomfortable changing is
shortening the listen(2) (aka :backlog) size, but we've always
changed that anyways since it also applies to UNIX sockets.
Note: removing a configuration knob in a unicorn config file can not
reset the value to the OS-provided default setting. Inherited
sockets must use a new setting to override existing ones.
(or the socket needs to be closed and re-created in the process
launcher before unicorn inherits it).
Noticed-by: Christos Trochalakis <yatiohi@ideopolis.gr>
<20150626114129.GA25883@luke.ws.skroutz.gr>
|
|
Most of these were found by the `linkchecker' package
in Debian.
|
|
Middlewares such as Rack::Lock (used by Rails) break badly unless
the response body is closed on hijack, so we will close it to follow
the lead of other popular Rack servers.
While it's unclear if there's anybody using rack.hijack with unicorn,
we'll try to emulate the behavior of other servers as much as
possible.
ref: https://github.com/ngauthier/tubesock/issues/10
|
|
Unnecessarily exposed accessors and constants take up unnecessary
memory in constant/method tables as well as using extra space in
instruction sequences.
Preforking servers like unicorn are a bloated pigs anyways,
but saving a few hundred bytes here and there can add up and
make them marginally less bad.
|
|
Some process managers such as foreman and daemontools rely on
unicorn not daemonizing, but we still want to be able to process
SIGWINCH in that case.
stdout and stderr may be redirected to a pipe (for cronolog or
similar process), so those are less likely to be attached to a TTY
than stdin. This also allows users to process SIGWINCH when running
inside a regular terminal if they redirect stdin to /dev/null.
Reported-by: Dan Moore <dan@vaporwa.re>
References: <etPan.555b4293.5b47a5b7.e617@danbookpro>
<20150519232858.GA23515@dcvr.yhbt.net>
|
|
kgio_wait_readable is superior for single FDs in that it may use the
ppoll syscall on Linux via Ruby, making it immune to the slowdown
high FDs with select() and the array allocations enforced by the
Ruby wrapper interface.
Note: IO#wait in the io/wait stdlib has the same effect, but as of
2.2 still needlessly checks the FIONREAD ioctl. So avoid needing to
force a new require on users which also incur shared object loading
costs. The longer term plan is to rely entirely on Ruby IO
primitives entirely and drop kgio, but that won't happen until we
can depend on Ruby 2.3 for exception-free accept_nonblock
(which will be released December 2015).
|
|
Literal regexps cost over 450 bytes of memory per-site and
unnecessary use of them costs memory in places where raw execution
speed does not matter. Nowadays, we can rely on String#end_with?
(introduced in 1.8.7) for improved readability, too.
|
|
Literal String#freeze avoids allocations since Ruby 2.1 via the
opt_str_freeze instruction, so we can start relying on it in
some places as Ruby 2.1 adoption increases. The 100-continue
handling is a good place to start since it is an uncommonly-used
code path which benefits from size reduction and the negative
performance impact is restricted to a handful of users.
HTTP_RESPONSE_START can safely live in http_request.rb as its
usage does not cross namespace boundaries
The goal is to eventually eliminate Unicorn::Const entirely.
|
|
Oops, this should've been explained long ago but apparently not.
In response to a comment on
http://www.sitepoint.com/the-self-pipe-trick-explained/
> Does anybody know why both unicorn and foreman read 11 bytes from
> self-pipe?
Unfortunately I couldn't find a way to comment on the site on a
JavaScript-free browser nor does it seem possible without
registering.
Again, anybody can send plain-text mail to:
unicorn-public@bogomips.org
No registration, no real name policy, no terms-of-service, just
plain-text. Feel free to use Tor, mixmaster or any anonymity
service, too.
|
|
In 1.9+ at least, instance variables use less space than constants
in class tables and bytecode, leading to ~700 byte reduction in
bytecode overhead on 64-bit and a reduction in constant table/entries
of the Unicorn::HttpServer class.
|
|
The former is shorter Ruby code and also generates smaller
bytecode.
|
|
unicorn 5 will not support Ruby 1.8 anymore.
Drop mentions of Rubinius, too, it's too difficult to support due to
the proprietary and registration-required nature of its bug tracker.
The smaller memory footprint and CoW-friendly memory allocator in
mainline Ruby is a better fit for unicorn, anyways.
Since Ruby 1.9+ bundles RubyGems and gem startup is faster nowadays,
we'll just depend on that instead of not loading RubyGems.
Drop the local.mk.sample file, too, since it's way out-of-date
and probably isn't useful (I have not used it in a while).
|
|
IO#close_on_exec* methods are available since Ruby 1.9.1. It
allows us to use less bytecode as it requires fewer operands and
avoids constant lookups.
|
|
We're requiring Ruby 1.9.3+, so we can safely depend
on IO#autoclose= being available in 1.9+ and shave off
some bloat.
|
|
The monotonic clock is immune to time adjustments so it is not
thrown off by misconfigured clocks. Process.clock_gettime also
generates less garbage on 64-bit systems due to the use of Flonum.
|
|
We implemented barely-advertised support for SSL for two reasons:
1) to detect corruption on LANs beyond what TCP offers
2) to support other servers based on unicorn (never happened)
Since this feature is largely not useful for unicorn itself,
there's no reason to penalize unicorn 5.x users with bloat.
In our defense, SSL support appeared in version 4.2.0 :)
|
|
Replacing the Regexp argument to a rarely-called String#split with a
literal String can save a little memory. The removed Regexp memsize
is 469 bytes on Ruby 2.1:
ObjectSpace.memsize_of(/,/) => 469
Is slightly smaller at 453 bytes on 2.2.0dev (r48474).
These numbers do not include the 40-byte object overhead.
Nevertheless, this is a waste for non-performance-critical code
during the socket inheritance phase. A literal string has less
overhead at 88 bytes:
* 48 bytes for table entry in the frozen string table
* 40 bytes for the object itself
The downside of using a literal string for the String#split argument
is a 40-byte string object gets allocated on every call, but this
piece of code is only called once in a process lifetime.
|
|
This has long been considered a mistake and not documented for very
long.
I considered removing X-Forwarded-Proto and X-Forwarded-SSL
handling, too, so rack.url_scheme is always "http", but that might
lead to compatibility issues in rare apps if Rack::Request#scheme is
not used.
|
|
When daemonizing, it is possible for the grandparent to be
terminated by another process before the master can notify
it. Do not abort the master in this case.
This may fix the following issue:
https://github.com/kostya/eye/issues/49
(which I was notified of privately via email)
|
|
We close SELF_PIPE in the worker immediately, but signal handlers
do not get setup immediately. So prevent workers from erroring out
due to invalid SELF_PIPE.
|
|
This protects us from two problems:
1) we (or our app) somehow called IO#close on one of the sockets
we listen on without removing it from the readers array.
We'll ignore IOErrors from IO#close and assume we wanted to
close it.
2) our SIGQUIT handler is interrupted by itself. This can happen as
a fake signal from the master could be handled and a real signal
from an outside user is sent to us (e.g. from unicorn-worker-killer)
or if a user uses the killall(1) command.
|
|
Signaling using normal kill(2) is preserved, but the master now
prefers to signal workers using a pipe rather than kill(2).
Non-graceful signals (:TERM/:KILL) are still sent using kill(2),
as they ask for immediate shutdown.
This change is necessary to avoid triggering the ubf (unblocking
function) for rb_thread_call_without_gvl (and similar) functions
extensions. Most notably, this fixes compatibility with newer
versions of the 'pg' gem which will cancel a running DB query if
signaled[1].
This also has the nice side-effect of allowing a premature
master death (assuming preload_app didn't cause the master to
spawn off rogue child daemons).
Note: users should also refrain from using "killall" if using the
'pg' gem or something like it.
Unfortunately, this increases FD usage in the master as the writable
end of the pipe is preserved in the master. This limit the number
of worker processes the master may run to the open file limit of the
master process. Increasing the open file limit of the master
process may be needed. However, the FD use on the workers is
reduced by one as the internal self-pipe is no longer used. Thus,
overall pipe allocation for the kernel remains unchanged.
[1] - pg is correct to cancel a query, as it cannot know if
the signal was for a) graceful unicorn shutdown or
b) oh-noes-I-started-a-bad-query-ABORT-ABORT-ABORT!!
|
|
This reduces the window for a non-existent PID for folks who monitor
PIDs (not a great idea anyways). Unfortunately, this change also brings
us back to the case where having a PID later (for other process monitors)
is beneficial but more unicorn releases exist where we write the PID
early.
Thanks to Jimmy Soho for reporting this issue.
ref: <CAHStS5gFYcPBDxkVizAHrOeDKAkjT69kruFdgaY0CbB+vLbK8Q@mail.gmail.com>
This partially reverts 7d6ac0c17eb29a00a5b74099dbb3d4d015999f27
Folks: please monitor your app with HTTP requests rather than checking
processes, a stuck/wedged Ruby VM is still a running one.
|
|
This renables the ability for Ruby 1.8.6 environments to perform reexecs
[ew: clarified this is for 1.8.6,
favor literal {} over Hash.new,
tweaked LISTENERS.map => LISTENERS.each, thanks to Hleb Valoshka
]
Signed-off-by: Eric Wong <normalperson@yhbt.net>
|
|
Thanks to Hongli Lai for noticing my typo. While we're at it, finish up
a halfway-written comment for the EXDEV case
|
|
Ruby 1.9 and later includes IO#autoclose=, so we can use it
and prevent some dead IO objects from hanging around.
|
|
This will preserve mtime on successful renames for comparisions.
While we're at it, avoid writing the new PID until the listeners are
inherited successfully. This can be useful to avoid accidentally
clobbering a good PID if binding the listener or building the app
(preload_app==true) fails
|
|
We do not attempt to write HTTP responses for socket errors if
clients disconnect from us unexpectedly.
Additionally, we do not hide backtraces EINVAL/EBADF errors, since
they are indicative of real bugs which must be fixed.
We do continue to hide hide EOF, ECONNRESET, ENOTCONN, and EPIPE
because clients (even "friendly") ones will break connections due to
client crashes or network failure (which is common for me :P), and
the backtraces from those will cause excessive logging and even
become a DoS vector.
|
|
Once a connection is hijacked, we ignore it completely and leave
the connection at the mercy of the application.
|
|
Rack 1.5.0 (protocol version [1,2]) adds support for
hijacking the client socket (removing it from the control
of unicorn (or any other Rack webserver)).
Tested with rack 1.5.0.
|