about summary refs log tree commit homepage
path: root/lib
AgeCommit message (Collapse)AuthorFilesLines
2019-05-03Rescue failed pipe resizes due to permissionsStephen Demjanenko1-0/+3
When running: ``` require 'kgio' require 'raindrops' F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/ Kgio::Pipe.new.each do |io| io.close_on_exec = true if defined?(F_SETPIPE_SZ) begin puts "setting" io.fcntl(F_SETPIPE_SZ, Raindrops::PAGE_SIZE) rescue Errno::EINVAL puts "rescued" rescue => e puts ["FAILED HARD", e].inspect end end end ``` on a few servers to test some Unicorn boot failures I saw: ``` ["FAILED HARD", #<Errno::EPERM: Operation not permitted>] ``` The `EPERM` error gets raised by the Linux kernel if: ``` (too_many_pipe_buffers_hard(pipe->user) || too_many_pipe_buffers_soft(pipe->user)) && !capable(CAP_SYS_RESOURCE) && !capable(CAP_SYS_ADMIN) ``` Given that the resize is not strictly necessary Unicorn should rescue the error and continue booting.
2018-10-18doc: update more URLs to use HTTPS and avoid redirectsEric Wong4-4/+4
Latency from redirects is painful, and HTTPS can protect privacy in some cases.
2018-09-21Support default_middleware configuration optionJeremy Evans3-9/+18
This allows for the equivalent of the -N/--no-default_middleware command line option to be specified in the configuration file so it doesn't need to be specified on the command line every time unicorn is executed. It explicitly excludes the use of -N/--no-default_middleware as an embedded configuration option in the rackup file, by ignoring the options after ARGV is parsed. In order to allow the configuration method to work, have the lambda that Unicorn.builder returns accept two arguments. Technically, only one argument is needed for the HttpServer instance, but I'm guessing if the lambda accepts a single argument, we expect that to be a rack application instead of a lambda that returns a rack application. The command line option option to disable default middleware will take precedence over the unicorn configuration file option if both are present. For backwards compatibility, if the lambda passed to HttpServer accepts 0 arguments, then call it without arguments. [ew: fix precedence for arity checking in build_app! configurator: ensure -N is respected when set in command-line]
2018-09-13Make Worker#user support different process primary group and log file groupJeremy Evans1-2/+16
Previously, Unicorn always used the process's primary group as the the group of the log file. However, there are reasons to use a separate group for the log files, such as when you have many applications where each application uses it's own user and primary group, but you want to be able to have a user read the log files for all applications. Some operating systems have a fairly small limit on the number of groups per user, and it may not be feasible to have a user be in the primary group for all applications. a primary group
2018-08-20socket_helper: add hint for FreeBSD users for accf_http(9)Eric Wong1-0/+1
Because I forget to load accf_http on new FreeBSD installs, too :x
2018-08-20shrink pipes under LinuxEric Wong3-4/+17
We have never had any need for pipes with the default 64K capacity on Linux. Our pipes are only used for tiny writes in signal handlers and to perform parent shutdown detection. With the current /proc/sys/fs/pipe-user-pages-soft default, only 1024 pipes can be created by an unprivileged user before Linux clamps down the pipe size to 4K (a single page) for newly-created pipes[1]. So avoid penalizing OTHER pipe users who could benefit from the increased capacity and use only a single page for ourselves. [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/pipe.c?h=v4.18#n642
2018-07-23use IO#wait instead of kgio_wait_readableEric Wong2-1/+2
Slowly reducing dependencies on kgio. 'io/wait' is required by 'socket' these days, so it's no extra relocations for .so loading, either.
2018-07-23remove random seed reset atforkEric Wong1-3/+0
It's not unicorn 6, yet, but we dropped Ruby 1.8 support at unicorn 5. Stable Ruby 1.9+ releases have always reseeded the PRNG at fork.
2018-05-01quiet some mismatched indentation warningsEric Wong3-16/+16
Ruby trunk started warning about more mismatched indentations starting around r62836.
2017-12-16avoid reusing env on hijackEric Wong3-4/+5
Hijackers may capture and reuse `env' indefinitely, so we must not use it in those cases for future requests. For non-hijack requests, we continue to reuse the `env' object to reduce memory recycling. Reported-and-tested-by: Sam Saffron <sam.saffron@gmail.com>
2017-11-16require 'pp' if $DEBUG is set by Rack appEric Wong1-1/+4
While "unicorn -d" requires 'pp' when setting $DEBUG, we did not account for (rare) Rack applications setting $DEBUG at load time. Thanks-to: James P (Jim) Robinson Jr <James.Robinson3@Cigna.com>
2017-04-08reduce method calls with String#start_with?Eric Wong3-3/+3
These three cold call sites instruction sequence size by a few hundred bytes combined since we no longer support Ruby 1.8.6. The "?/" shorthand is esoteric and no longer avoids allocation in Ruby 1.9+ (not that this is hot code).
2017-03-26Check for Socket::TCP_INFO constant before trying to get TCP_INFODylan Thacker-Smith1-14/+3
The ruby constant Socket::TCP_INFO is only defined if TCP_INFO is defined in C, so we can just check for the presence of that ruby constant instead of rescuing SocketError from the call to getsockopt.
2017-03-24Check for SocketError on first ccc attemptJeremy Evans1-4/+14
On OpenBSD, getsockopt(2) does not support TCP_INFO. With the current code, this results in a 500 for all clients if check_client_connection is enabled on OpenBSD. This patch rescues SocketError on the first getsockopt call, and if SocketError is raised, it doesn't check in the future. This should be the same behavior as if TCP_INFO was supported but inspect did not return a string in the expected format.
2017-03-24doc: note after_worker_exit is also 5.3.0+Eric Wong1-0/+2
Followup-to: 650e01ab0b118803486b56f3ee59521d59042dae ("doc: add version annotations for new features")
2017-03-23doc: add version annotations for new featuresEric Wong2-1/+8
We will inevitably have people running old unicorn versions for many years to come; but they may be reading the latest documentation online. Annotate when the new features (will) appear to avoid misleading users on old versions.
2017-03-23Merge remote-tracking branch 'origin/worker_exec'Eric Wong3-19/+73
* origin/worker_exec: Don't pass a block for fork when forking workers Add worker_exec configuration option
2017-03-23Merge branch 'ccc-tcp-v3'Eric Wong1-9/+29
* ccc-tcp-v3: test_ccc: use a pipe to synchronize test http_request: support proposed Raindrops::TCP states on non-Linux
2017-03-23http_server: initialize @pid ivarEric Wong1-0/+1
This quiets down warnings when run with '-w'
2017-03-23input: update documentation and hide internals.Eric Wong3-18/+18
rack 2.x exists nowadays still allows rewindable input as an option, and we will still enable it by default to avoid breaking any existing code. Hide the internal documentation since we do not want people depending on unicorn internals; so there's no reason to confuse or overwhelm people with documentation about it. Arguably, TeeInput and StreamInput should not be documented publically at all, but I guess that ship has long sailed...
2017-03-21http_request: support proposed Raindrops::TCP states on non-LinuxEric Wong1-9/+29
raindrops 0.18+ will have Raindrops::TCP state hash for portable mapping of TCP states to their respective numeric values. This was necessary because TCP state numbers (and even macro names) differ between FreeBSD and Linux (and possibly other OSes). Favor using the Raindrops::TCP state hash if available, but fall back to the hard-coded values since older versions of raindrops did not support TCP_INFO on non-Linux systems. While we're in the area, favor "const_defined?" over "defined?" to reduce the inline constant cache footprint for branches which are only evaluated once. Patches to implement Raindrops::TCP for FreeBSD are available at: https://bogomips.org/raindrops-public/20170316031652.17433-1-e@80x24.org/T/
2017-03-15Merge remote-tracking branch 'origin/ccc-tcp-v3'Eric Wong2-7/+83
* origin/ccc-tcp-v3: http_request: reduce insn size for check_client_connection support "struct tcp_info" on non-Linux and Ruby 2.2+ revert signature change to HttpServer#process_client new test for check_client_connection check_client_connection: use tcp state on linux
2017-03-14doc: fix links to raindrops projectEric Wong1-1/+1
bogomips.org is dropping prefixes to reduce subjectAltName bloat in TLS certificates.
2017-03-14freebsd: avoid EINVAL when setting accept filterEric Wong1-2/+5
Accept filters can only be set on listen sockets, and it also fails with EINVAL if it's already set. Untested, but I suppose changing the accept filter on a listening socket is not supported, either; since that could affect in-flight sockets.
2017-03-14http_request: reduce insn size for check_client_connectionEric Wong1-3/+4
Unlike constants and instance variables, class variable access is not optimized in the mainline Ruby VM. Use a constant instead, to take advantage of inline constant caching. This further reduces runtime instruction size by avoiding a branch by allocating the Raindrops::TCP_Info object up front. This reduces the method size by roughly 300 bytes on 64-bit.
2017-03-13Don't pass a block for fork when forking workers worker_execJeremy Evans1-8/+6
This reduces the stack depth, making GC more efficient.
2017-03-10Add worker_exec configuration optionJeremy Evans3-21/+77
The worker_exec configuration option makes all worker processes exec after forking. This initializes the worker processes with separate memory layouts, defeating address space discovery attacks on operating systems supporting address space layout randomization, such as Linux, MacOS X, NetBSD, OpenBSD, and Solaris. Support for execing workers is very similar to support for reexecing the master process. The main difference is the worker's to_i and master pipes also need to be inherited after worker exec just as the listening sockets need to be inherited after reexec. Because execing working is similar to reexecing the master, this extracts a couple of methods from reexec (listener_sockets and close_sockets_on_exec), so they can be reused in worker_spawn.
2017-03-08oob_gc: rely on opt_aref_with optimization on Ruby 2.2+Eric Wong1-2/+1
Maybe oob_gc probably isn't heavily used anymore, maybe some Ruby 2.2+ users will benefit from this constant reduction. Followup-to: fb2f10e1d7a72e67 ("reduce constants and optimize for Ruby 2.2")
2017-03-08support "struct tcp_info" on non-Linux and Ruby 2.2+Eric Wong1-1/+31
Ruby 2.2+ can show "struct tcp_info" as a string via Socket::Option#inspect, and we can attempt to parse it out to extract the information we need. Parsing this string is inefficient, but does not depend on the ordering of the tcp_info struct.
2017-03-08revert signature change to HttpServer#process_clientEric Wong4-12/+24
We can force kgio_tryaccept to return an internal class for TCP objects by subclassing Kgio::TCPServer. This avoids breakage in any unfortunate projects which depend on our undocumented internal APIs, such as gctools <https://github.com/tmm1/gctools>
2017-03-08check_client_connection: use tcp state on linux ccc-tcp-v2Simon Eskildsen3-11/+44
* Use a frozen empty array and a class variable for TCP_Info to avoid garbage. As far as I can tell, this shouldn't result in any garbage on any requests (other than on the first request). * Pass listener socket to #read to only check the client connection on a TCP server. * Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's the most common state after ESTABLISHED, it makes the numbers un-ordered, though. But comment should make it OK. * Definition of of `check_client_connection` based on whether Raindrops::TCP_Info is defined, instead of the class variable approach. * Changed the unit tests to pass a `nil` listener. Tested on our staging environment, and still works like a dream. I should note that I got the idea between this patch into Puma as well! https://github.com/puma/puma/pull/1227 [ew: squashed in temporary change for oob_gc.rb, but we'll come up with a different change to avoid breaking gctools <https://github.com/tmm1/gctools>] Acked-by: Eric Wong <e@80x24.org>
2017-02-23Add after_worker_ready configuration option chrootJeremy Evans2-2/+24
This adds a hook that is called after the application has been loaded by the worker process, directly before it starts accepting requests. This hook is necessary if your application needs to gain access to resources during initialization, and then drop privileges before serving requests. This is especially useful in conjunction with chroot support so the app can load all the normal ruby libraries it needs to function, and then chroot before accepting requests. If you are preloading the app, it's possible to drop privileges or chroot in after_fork, but if you are not preloading the app, the only way to currently do this is to override the private HttpServer#init_worker_process method, and overriding private methods is a recipe for future breakage if the internals are modified. This hook allows for such functionality to be supported and not break in future versions of Unicorn.
2017-02-23Add support for chroot to Worker#userJeremy Evans1-3/+10
Any chrooting would need to happen inside Worker#user, because you can't chroot until after you have parsed the list of groups, and you must chroot before dropping root privileges. chroot adds an extra layer of security, so that if the unicorn process is exploited, file system access is limited to the chroot directory instead of the entire file system.
2017-02-23Fix code example in after_worker_exit documentationJeremy Evans1-1/+1
Fixes: 2af91a1fef70d654 ("Add after_worker_exit configuration option")
2017-02-21Add after_worker_exit configuration optionJeremy Evans2-2/+23
This option is executed in the master process following all worker process exits. It is most useful in the case where the worker process crashes the ruby interpreter, as the worker process may not be able to send error notifications appropriately. For example, let's say you have a specific request that crashes a worker process, which you expect to be due to a improperly programmed C extension. By modifying your worker to save request related data in a temporary file and using this option, you can get a record of what request is crashing the application, which will make debugging easier. Example: after_worker_exit do |server, worker, status| server.logger.info "worker #{status.success? ? 'exit' : 'crash'}: #{status}" file = "request.#{status.pid}.txt" if File.exist?(file) do_something_with(File.read(file)) unless status.success? File.delete(file) end end
2017-02-13http_request: freeze constant strings passed IO#writeEric Wong1-4/+1
This ensures we won't have duplicate objects in Ruby 2.0-2.4. For Ruby 2.5.0dev+, this avoids any duplicate cleanup introduced as of r57471: https://bugs.ruby-lang.org/issues/13085
2017-02-10tee_input: simplify condition for IO#writeEric Wong1-3/+1
IO#write already elides the write(2) syscall for empty buffers, so there's no need to complicate our instruction sequence footprint for the rare case of an empty buffer. The only cases a Rack app will have an empty buffer are: 1) `env['rack.input'].read` without args 2) `env['rack.input'].gets` Neither of these calls are safe for server-independent Rack apps as the client can OOM the app. unicorn itself provides no facility for limiting maximum rack.input size. Instead, unicorn relies on nginx to limit input size using the client_max_body_size directive.
2016-10-25relocate website to https://bogomips.org/unicorn/ website-moveEric Wong2-4/+4
HTTPS helps some with reader privacy and Let's Encrypt seems to be working well enough the past few months. This change will allow us to reduce subjectAltName bloat in our TLS certificate over time. It will also promote domain name agility to support mirrors or migrations to other domains (including a Tor hidden service mirror). http://bogomips.org/unicorn/ will remain available for people on legacy systems without usable TLS. There is no plan for automatic redirecting from HTTP to HTTPS at this time.
2016-07-28doc: update gmane URLs to point to our own archivesEric Wong1-2/+3
Gmane's NNTP server remains up, but the HTTP site is down: https://lars.ingebrigtsen.no/2016/07/28/the-end-of-gmane/ Anyways, our own archives are designed to be mirror-able via git: git clone --mirror https://bogomips.org/unicorn-public And the code is self-hostable: git clone https://public-inbox.org
2016-01-27doc update for ClientShutdown exceptions classEric Wong1-1/+3
State explicitly applications should not rely on it, and instead rescue the generic EOFError exception. This class will stick around because there may inevitably be things which rely on it, but we should not encourage it, either.
2016-01-27rack is optional at runtime, required for devEric Wong2-20/+32
We do not want to pull in a newer or older version of rack depending on an the application running under it requires. Furthermore, it has always been possible to use unicorn without any middleware at all. Without rack, we'll be missing descriptive status text in the first response line, but any valid HTTP/1.x parser should be able to handle it properly. ref: http://bogomips.org/unicorn-public/20160121201255.GA6186@dcvr.yhbt.net/t/#u Thanks-to: Adam Duke <adam.v.duke@gmail.com> Thanks-to: Aaron Patterson <tenderlove@ruby-lang.org>
2016-01-07various documentation updatesEric Wong2-4/+5
* add nntp_url to the olddoc website footer * update legacy support status for 4.x (not 4.8.x) * update copyright range to 2016 * note all of our development tools are Free Software, too * remove cgit mention; it may not always be cgit (but URLs should remain compatible). * discourage downloading snapshot tarballs; "git clone" + periodic "git fetch" is more efficient * remove most mentions of unicorn_rails as that was meant for ancient Rails 1.x/2.x users * update path reference to Ruby 2.3.0 * fix nginx upstream module link to avoid redirect * shorten Message-ID example to avoid redirects and inadvertant linkage
2015-11-17http_response: allow nil values in response headersEric Wong1-1/+1
This blatantly violates Rack SPEC, but we've had this bug since March 2009[1]. Thus, we cannot expect all existing applications and middlewares to fix this bug and will probably have to support it forever. Unfortunately, supporting this bug contributes to application server lock-in, but at least we'll document it as such. [1] commit 1835c9e2e12e6674b52dd80e4598cad9c4ea1e84 ("HttpResponse: speed up non-multivalue headers") Reported-by: Owen Ou <o@heroku.com> Ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
2015-11-01golf down conditional for socket activationEric Wong1-1/+1
The PID of a process can never be zero as kill(2) interprets a '0' PID arg as "every process in caller's process group", so there's no risk of the 'nil.to_i => 0' conversion resulting in a truth value when compared to $$.
2015-10-27sd_listen_fds emulation cleanupEric Wong1-2/+2
Re-enable and expand on the test case while we're at it for new Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We shall assume anybody running a pre-release 2.3.0 at this point is running a fairly recent snapshot, so we won't bother doing a finer-grained check in the test for an exact revision number.
2015-08-22stream_input: favor String#clear over String#replaceEric Wong1-3/+3
We no longer need Ruby 1.8 compatibility, so use String#clear to reduce argument passing and code size.
2015-07-15doc: remove references to old serversEric Wong6-27/+21
They'll continue to be maintained, but we're no longer advertising them. Also, favor lowercase "unicorn" while we're at it since that matches the executable and gem name to avoid unnecessary escaping for RDoc.
2015-07-15configurator: document net.core.somaxconn sysctl dependencyEric Wong1-0/+5
Linux users are effectively capped to 128 on stock installations and may wonder why connections get rejected with overloaded apps sooner rather than later.
2015-07-05emulate sd_listen_fds for systemd supportEric Wong1-3/+13
systemd socket emulation shares FDs across execve, just like the built-in SIGUSR2 upgrade process in unicorn. Thus it is easy to support inheriting sockets from systemd. Tested-by: Christos Trochalakis <yatiohi@ideopolis.gr>
2015-06-30http_response: reduce size of multi-line header pathEric Wong1-1/+1
This should save over 100 bytes of bytecode overhead due to reduced method dispatch points. The performance difference when this is actually hit could go either way depending on how String#<< and realloc(3) interact, but it's uncommon enough that nobody is expected to notice either way.