about summary refs log tree commit homepage
DateCommit message (Collapse)
2024-01-15update dependency to Ruby 2.5+ HEAD master
We actually need Ruby 2.3+ for `accept_nonblock(exception: false)'; and (AFAIK) we can't easily use a subclass of `Socket' while using Socket#accept_nonblock to inject WriteSplat support for `IO#write(*multi_args)' So just depend on Ruby 2.5+ since all my Ruby is already on the already-ancient Ruby 2.7+ anyways.
2024-01-15kill off remaining kgio uses
kgio is an extra download and shared object which costs users bandwidth, disk space, startup time and memory. Ruby 2.3+ provides `Socket#accept_nonblock(exception: false)' support in addition to `exception: false' support in IO#*_nonblock methods from Ruby 2.1. We no longer distinguish between TCPServer and UNIXServer as separate classes internally; instead favoring the `Socket' class of Ruby for both. This allows us to use `Socket#accept_nonblock' and get a populated `Addrinfo' object off accept4(2)/accept(2) without resorting to a getpeername(2) syscall (kgio avoided getpeername(2) in the same way). The downside is there's more Ruby-level argument passing and stack usage on our end with HttpRequest#read_headers (formerly HttpRequest#read). I chose this tradeoff since advancements in Ruby itself can theoretically mitigate the cost of argument passing, while syscalls are a high fixed cost given modern CPU vulnerability mitigations. Note: no benchmarks have been run since I don't have a suitable system.
2024-01-15remove kgio from all read(2) and write(2) wrappers
It's fairly easy given unicorn was designed with synchronous I/O in mind. The overhead of backtraces from EOFError on readpartial should be rare given our requirement to only accept requests from fast, reliable clients on LAN (e.g. nginx or yet-another-horribly-named-server).
2023-10-03README: fix wording
Reported-by: <ideal.water4095@fastmail.com>
2023-09-11doc: various updates ahead of the release
The damage unicorn has done to the entire Ruby, Rack and Rails ecosystems with its ability to tolerate buggy code is unforgivable. Update the documentation to further discourage its use and clarify a few wordings noticed along the way.
2023-09-10tests: fold early shutdown() tests into t/integration.t
This means fewer redundant tests and more chances to notice Ruby incompatibilities.
2023-09-10tests: move broken app test to Perl 5 integration test
Less Ruby means fewer incompatibilities to worry about with every new version.
2023-09-10tests: fold SO_KEEPALIVE check to Perl 5 integration
No need to startup more processes than necessary.
2023-09-10tests: use Time::HiRes `sleep' and `time' everywhere
The time(2) syscall use by CORE::time is inaccurate[1]. It's also easier to read `sleep 0.01' rather than the longer `select' equivalent. [1] a6463151bd1db5b9 (httpdate: favor gettimeofday(2) over time(2) for correctness, 2023-06-01)
2023-09-10tests: use more common variable names between tests
Stuff like $u_conf, $daemon_pid, $pid_file, etc. will reduce cognitive overhead.
2023-09-10tests: introduce `do_req' helper sub
While early tests required fine-grained control in trickling requests, many of our later tests can use a short one-liner w/o having to spawn curl.
2023-09-10tests: rewrite SIGWINCH && SIGTTIN test in Perl 5
No need to deal with full second sleeps, here.
2023-09-10tests: port reopen logs test over to Perl 5
Being able to do subsecond sleeps is one welcome advantage over POSIX (not GNU) sleep(1) in portable Bourne sh.
2023-09-10tests: port t/heartbeat-timeout to Perl 5
I absolutely detest and regret adding this feature, but I'm hell bent on supporting it until the end of days because we don't break compatibility.
2023-09-10tests: port working_directory tests to Perl 5
We can fold a bunch of them into one test to save startup time, inodes, and FS activity.
2023-09-10tests: port some bad config tests to Perl 5
We can fold some tests into one test to save on Perl startup time (but Ruby startup time is a lost cause).
2023-06-20unicorn_http_common.rl: use only ASCII spaces for compatibility
Ragel 6.10 on FreeBSD 12.4 amd64 complains and fails on this, yet the same Ragel version on Debian 11.x i386 and amd64 never has. I suspect this can fix compatibility on s390x, arm64, armel, and armhf Debian builds: https://buildd.debian.org/status/fetch.php?pkg=unicorn&arch=s390x&ver=6.1.0-1&stamp=1687156375&file=log https://buildd.debian.org/status/fetch.php?pkg=unicorn&arch=arm64&ver=6.1.0-1&stamp=1687156478&file=log https://buildd.debian.org/status/fetch.php?pkg=unicorn&arch=armel&ver=6.1.0-1&stamp=1687156619&file=log https://buildd.debian.org/status/fetch.php?pkg=unicorn&arch=armhf&ver=6.1.0-1&stamp=1687156807&file=log Fixes: d5fbbf547203061b (Add some tolerance (RFC2616 sec. 19.3), 2016-10-20)
2023-06-20tests: ensure t/random_blob exists before Perl tests
Allow overriding `PROVE=' while we're at it, too; since development installations of Perl5 may name it `prove5.$MINOR' or similar.
2023-06-20tests: handle $/ assignment deprecation
...by testing less. `env["rack.input"].gets' users are out-of-luck if they want anything other than "\n" or `nil', I suppose... `$/' is non-thread-local and thus non-thread-safe, which doesn't affect unicorn itself, but Ruby deprecates it for single-threaded code, too, unfortunately. Rack::Lint doesn't allow separator arguments for #gets, either, so we can't support that, either...
2023-06-20t/active-unix-socket: sleep for init(8) to reap worker
Unfortunately, we need a sleep loop here since kill(2) succeeds on zombies and init(8) doesn't reap the worker soon enough on a FreeBSD VM.
2023-06-20t/lib.perl: FreeBSD: ignore accf_* messages
Testers may not have accf_http loaded nor the permissions to run `kldload accf_http', thus we must ignore these messages.
2023-06-20epollexclusive: handle future rb_io_t deprecation
It looks like Ruby 3.3+ will hide rb_io_t internals and get rid of the venerable `GetOpenFile' macro in favor of `rb_io_descriptor'. `rb_io_descriptor' has been public API since Ruby 3.1 and should be safe to use, and is necessary for `raindrops' (a dependency of ours): https://yhbt.net/raindrops-public/20230609104805.39022-1-samuel.williams@oriontransfer.co.nz/ https://bugs.ruby-lang.org/issues/19057#note-17 We'll also avoid an unnecessary call to `rb_io_get_io' in `get_readers' since `epio' (aka `self') can only be of the Unicorn::Waiter IO subclass. However, we must still use `rb_io_get_io' when handling non-self args in `prep_readers'.
2023-06-05LISTEN_FDS-inherited sockets are immortal across SIGHUP
When using systemd-style socket activation, consider the inherited socket immortal and do not drop it on SIGHUP. This means configs w/o any `listen' directives at all can continue to work after SIGHUP. I only noticed this while writing some tests in Perl 5 and the test suite is two lines shorter to test this feature :>
2023-06-05drop redundant IO#close_on_exec=false calls
Passing the `{ FD => IO }' mapping to #spawn or #exec already ensures Ruby will clear FD_CLOEXEC on these FDs before execve(2).
2023-06-05tests: move test_upload.rb tests to t/integration.t
The overread tests are ported over, and checksumming alone is enough to guard against data corruption. Randomizing the size of `read' calls on the client side will shake out any boundary bugs on the server side.
2023-06-05t/integration.t: switch PUT tests to MD5, reuse buffers
MD5 is faster, and these tests aren't meant to be secure, they're just for checking for data corruption. Furthermore, Content-MD5 is a supported HTTP trailer and we can verify that here to obsolete other tests. Furthermore, we can reuse buffers on env['rack.input'].read calls to avoid malloc(3) and GC overhead. Combined, these give roughly a 3% speedup for t/integration.t on my system.
2023-06-05test_server: drop early_hints test
t/integration.t already is more complete in that it tests both Rack 2 and 3 along with both possible values of check_client_connection.
2023-06-05early_hints supports Rack 3 array headers
We can hoist out append_headers into a new method and use it in both e103_response_write and http_response_write. t/integration.t now tests early_hints with both possible values of check_client_connection.
2023-06-05tests: get rid of sha1sum.rb and rsha1() sh function
These are no longer needed since Perl has long included Digest::SHA
2023-06-05port t/t0116-client_body_buffer_size.sh to Perl 5
While I'm fine with depending on curl for certain things, there's no need for it here since unicorn has had lazy rack.input for over a decade, at this point.
2023-06-05port t9000-preread-input.sh to Perl 5
Stuffing it into t/integration.t for now so we can save on startup costs.
2023-06-05tests: consistent tcp_start and unix_start across Perl 5 tests
I'll be using Unix sockets more in tests since there's no risk of system-wide conflicts with TCP port allocation. Furthermore, curl supports `--unix-socket' nowadays; so there's little reason to rely on TCP sockets and the conflicts they bring in tests.
2023-06-05tests: check_stderr consistently in Perl 5 tests
The Bourne shell tests did, so lets not let stuff sneak past us.
2023-06-05test_exec: drop test_basic and test_config_ru_alt_path
We already have coverage for these basic things elsewhere.
2023-06-05test_exec: drop sd_listen_fds emulation test
The Perl 5 tests already rely on this implicitly, and there was never a point when Perl 5 couldn't emulate systemd behavior.
2023-06-05port t0019-max_header_len.sh to Perl 5
This was the final socat requirement for integration tests. I think curl will remain an optional dependency for tests since it's probably the most widely-installed HTTP client.
2023-06-05tests: use autodie to simplify error checking
autodie is bundled with Perl 5.10+ and simplifies error checking in most cases. Some subroutines aren't perfectly translatable and their call sites had to be tweaked, but most of them are.
2023-06-05port t0100-rack-input-tests.sh to Perl 5
Yet another socat dependency gone \o/
2023-06-05port t0011-active-unix-socket.sh to Perl 5
Another socat dependency down... I've also started turning FD_CLOEXEC off on a pipe as a mechanism to detect daemonized process death in tests.
2023-06-05t/integration.t: use start_req to simplify test slighly
Less code is usually better.
2023-06-05port t0002-parser-error.sh to Perl 5
Another socat dependency down...
2023-06-05port t0000-http-basic.sh to Perl 5
One more socat dependency down...
2023-06-05port t0018-write-on-close.sh to Perl 5
This doesn't require restarting, so it's a perfect candidate.
2023-06-05support rack 3 multi-value headers
The first step in adding Rack 3 support. Rack supports multi-value headers via array rather than newlines. Tested-by: Martin Posthumus <martin.posthumus@gmail.com> Link: https://yhbt.net/unicorn-public/7c851d8a-bc57-7df8-3240-2f5ab831c47c@gmail.com/
2023-06-05switch unit/test_response.rb to Perl 5 integration test
http_response_write may benefit from API changes for Rack 3 support. Since there's no benefit I can see from using a unit test, switch to an integration test to avoid having to maintain the unit test if our internal http_response_write method changes. Of course, I can't trust tests written in Ruby since I've had to put up with a constant stream of incompatibilities over the past two decades :< Perl is more widely installed than socat[1], and nearly all the Perl I wrote 20 years ago still works unmodified today. [1] the rarest dependency of the Bourne shell integration tests
2023-06-05httpdate: fix build with Ruby 2.7 (at least)
<time.h> is still required for gmtime_r(3), and not all versions of <ruby.h> include <time.h>, already. Fixes: a6463151bd1db5b9 (httpdate: favor gettimeofday(2) over time(2) for correctness, 2023-06-01)
2023-06-05httpdate: favor gettimeofday(2) over time(2) for correctness
While scanning the git@vger.kernel.org mailing list, I've learned time(2) may return the wrong value in the first 1 to 2.5 ms of every second. While I'm not sure if the Date: response header matters to anyone, returning the correct time seems prudent. Link: https://lore.kernel.org/git/20230320230507.3932018-1-gitster@pobox.com/ Link: https://inbox.sourceware.org/libc-alpha/20230306160321.2942372-1-adhemerval.zanella@linaro.org/T/ Link: https://sourceware.org/bugzilla/show_bug.cgi?id=30200
2023-06-05ext: remove a vestige of ruby <2.0 support
The actual `id_clear' declaration was removed last year, but I missed it's (unused) initialization :x Fixes: c56eb04d683e ("drop Ruby 1.9.3 support, require 2.0+ for now")
2023-06-05chunk unterminated HTTP/1.1 responses
Rack::Chunked will be gone in Rack 3.1, so provide a non-middleware fallback which takes advantage of IO#write supporting multiple arguments in Ruby 2.5+. We still need to support Ruby 2.4, at least, since Rack 3.0 does. So a new (GC-unfriendly) Unicorn::WriteSplat module now exists for Ruby <= 2.4 users.
2023-06-05Support Rack 3 and fix tests on Rack 3
Most changes are to the tests to avoid uppercase characters in header keys, which are no longer allowed in rack 3 (to allow for O(1) access). This also changes a few places where an array of headers was used to switch to a hash, as a hash is requierd in Rack 3. Newer versions of curl use a 000 http_code for invalid http codes, so switch from "42 -eq" to "500 -ne" in the test, as Rack::Lint will always raise a 500 error. There is one test that fails on OpenBSD when opening a fifo. This is unrelated to unicorn as far as I can see, so skip the remaining part of the test in that case on OpenBSD. Tests still pass on Rack 2, and presumably Rack 1 as well, though I didn't test Rack 1. Co-authored-by: Eric Wong <bofh@yhbt.net>