| Date | Commit message (Collapse) |
|---|
|
It's fairly easy given unicorn was designed with synchronous I/O
in mind. The overhead of backtraces from EOFError on
readpartial should be rare given our requirement to only accept
requests from fast, reliable clients on LAN (e.g. nginx or
yet-another-horribly-named-server).
|
|
In preparation for kgio removal, I want to ensure we can
maintain existing performance when swapping kgio_tryaccept
for accept_nonblock on Ruby 2.3+
There's plenty of TCP benchmarking tools, but TCP port reuse
delays hurt predictability since unicorn doesn't do persistent
connections.
So this is exclusively for Unix sockets and uses Perl instead
of Ruby since I don't want to be bothered with GC
unpredictability on the client side.
|
|
This is intended to demonstrate how badly we suck at dealing
with slow clients making uploads. It can help users evaluate
alternative fully-buffering reverse proxies, because nginx
should not be the only option.
|
|
This is intended to demonstrate how badly we suck at dealing
with slow clients. It can help users evaluate alternative
fully-buffering reverse proxies, because nginx should not
be the only option.
Update the benchmark README while we're at it
|
|
This release fixes and works around issues from v5.5.0 (2019-03-04)
Stephen Demjanenko worked around a pipe resource accounting bug
present in old Linux kernels. Linux 3.x users from 3.16.57 and
on are unaffected. Linux 4.x users from 4.9 and on are
unaffected.
https://bogomips.org/unicorn-public/1556922018-24096-1-git-send-email-sdemjanenko@gmail.com/
Stan Pitucha reported a bug with the old `unicorn_rails' wrapper
(intended for Rails 2.x users) which was promptly fixed by
Jeremy Evans:
https://bogomips.org/unicorn-public/CAJ2_uEPifcv_ux4sX=t0C4zHTHGhqVfcLcSB2kTU3Rb_6pQ3nw@mail.gmail.com/
There's also some doc updates to warn users off `unicorn_rails';
the homepage is now energy-efficient for OLEDs and CRTs;
and I'm no longer advertising mailing list subscriptions
(because I hate centralization and mail archives are the priority)
Eric Wong (3):
doc: unicorn_rails: clarify that it is intended for rails <= 2.x
doc: stop advertising mailing list subscription
doc: switch homepage to dark216
Jeremy Evans (1):
unicorn_rails: fix regression with Rails >= 3.x in app build
Stephen Demjanenko (1):
Rescue failed pipe resizes due to permissions
|
|
When running: ```
require 'kgio'
require 'raindrops'
F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/
Kgio::Pipe.new.each do |io|
io.close_on_exec = true
if defined?(F_SETPIPE_SZ)
begin
puts "setting"
io.fcntl(F_SETPIPE_SZ, Raindrops::PAGE_SIZE)
rescue Errno::EINVAL
puts "rescued"
rescue => e
puts ["FAILED HARD", e].inspect
end
end
end
```
on a few servers to test some Unicorn boot failures I saw:
```
["FAILED HARD", #<Errno::EPERM: Operation not permitted>]
```
The `EPERM` error gets raised by the Linux kernel if:
```
(too_many_pipe_buffers_hard(pipe->user) ||
too_many_pipe_buffers_soft(pipe->user)) &&
!capable(CAP_SYS_RESOURCE) && !capable(CAP_SYS_ADMIN)
```
Given that the resize is not strictly necessary Unicorn should
rescue the error and continue booting.
|
|
Because Earth Day!:
https://80x24.org/olddoc-public/20190422014906.4253-1-e@80x24.org/
|
|
It's still available, but no point in advertising something
which increases the dependency on a centralized subscriber list.
Subscription will never be required to write to our
centralization-resistant public inbox (which anybody can
read/mirror using HTTP(S) or NNTP).
The mailing list has only attracted a tiny handful of
subscribers since Rubyforge died, and more than half of the
subscribers imported from Rubyforge have bounced off
(or unsubscribed).
|
|
Hopefully the wording is a little more explicit and clearer
by stating its purpose in the first line of the description.
|
|
Note: `unicorn_rails' was only intended for Rails <= 2.x projects
in the old days.
Fixes: 5985dd50a9bd7238 ("Support default_middleware configuration option")
From: Jeremy Evans <code@jeremyevans.net>
cf. https://bogomips.org/unicorn-public/20190306055734.GC61406@jeremyevans.local/
Signed-off-by: Eric Wong <e@80x24.org>
[ew: commit message]
|
|
Mostly identical to 5.5.0.pre1, which I didn't hear any feedback
from:
https://bogomips.org/unicorn-public/20181220222842.GA27382@dcvr/
> Jeremy Evans contributed the "default_middleware" configuration option:
>
> https://bogomips.org/unicorn-public/20180913192055.GD48926@jeremyevans.local/
>
> Jeremy also contributed the ability to use separate groups for the process
> and log files:
>
> https://bogomips.org/unicorn-public/20180913192449.GE48926@jeremyevans.local/
>
> There's also a couple of uninteresting minor optimizations and
> documentation additions.
Otherwise, there's one extra change to use
rb_gc_register_mark_object which is finally a documented part of
the Ruby C-API, but has existed since the 1.9 days.
|
|
unicorn has no future, but at least some of our past is archived.
This was taken from nntp://news.gmane.org/gmane.comp.lang.ruby.mongrel.devel
as the HTTP parts of gmane are dead. Unfortunately, Gmane only
started archiving mongrel-development@rubyforge.org in 2008 and
discussions prior to that may be lost. If anybody has mbox, Maildir
or MH archives from those days, I'd be glad to import them.
|
|
Since Ruby 2.6, it's a documented part of the API and we may depend
on it: https://bugs.ruby-lang.org/issues/9894
It's been around since the early Ruby 1.9 days, and reduces
overhead compared to relying on rb_global_variable:
https://bogomips.org/unicorn-public/20170301002854.29198-1-e@80x24.org/
|
|
Jeremy Evans contributed the "default_middleware" configuration option:
https://bogomips.org/unicorn-public/20180913192055.GD48926@jeremyevans.local/
Jeremy also contributed the ability to use separate groups for the process
and log files:
https://bogomips.org/unicorn-public/20180913192449.GE48926@jeremyevans.local/
There's also a couple of uninteresting minor optimizations and
documentation additions.
Eric Wong (10):
remove random seed reset atfork
use IO#wait instead of kgio_wait_readable
Merge branch '5.4-stable'
shrink pipes under Linux
socket_helper: add hint for FreeBSD users for accf_http(9)
tests: ensure -N/--no-default-middleware not supported in config.ru
doc: update more URLs to use HTTPS and avoid redirects
deduplicate strings VM-wide in Ruby 2.5+
doc/ISSUES: add links to git clone-able mail archives of our dependencies
README: minor updates and additional disclaimer
Jeremy Evans (2):
Make Worker#user support different process primary group and log file group
Support default_middleware configuration option
|
|
Nowadays, I mainly rely on systemd (and not USR2) for
zero-downtime upgrades. Also, CoW-friendliness is standard
in mainline Ruby since 2.0.
There also needs to be a disclaimer to point out the unfortunate
side-effect of robustness for hosting buggy apps.
|
|
Archives are crucial to preserving history and knowledge in Free
Software projects, so promote them for projects we depend on.
Naq lrf, gur nepuviny fbsgjner qrirybcrq sbe nepuvivat gur
havpbea znvyvat yvfg unf ybat fhecnffrq gur hfrshyarff bs
havpbea vgfrys :C
|
|
String#-@ deduplicates strings starting with Ruby 2.5.0
Hash#[]= deduplicates strings starting in Ruby 2.6.0-rc1
This allows us to save a small amount of memory by sharing
objects with other parts of the stack (e.g. Rack).
|
|
Latency from redirects is painful, and HTTPS can protect privacy
in some cases.
|
|
This allows for the equivalent of the
-N/--no-default_middleware command line option to be
specified in the configuration file so it doesn't
need to be specified on the command line every time
unicorn is executed.
It explicitly excludes the use of -N/--no-default_middleware
as an embedded configuration option in the rackup file, by
ignoring the options after ARGV is parsed.
In order to allow the configuration method to work, have
the lambda that Unicorn.builder returns accept two arguments.
Technically, only one argument is needed for the HttpServer
instance, but I'm guessing if the lambda accepts a single
argument, we expect that to be a rack application instead
of a lambda that returns a rack application.
The command line option option to disable default middleware
will take precedence over the unicorn configuration file option
if both are present.
For backwards compatibility, if the lambda passed to
HttpServer accepts 0 arguments, then call it without
arguments.
[ew: fix precedence for arity checking in build_app!
configurator: ensure -N is respected when set in command-line]
|
|
Continue to make it easy to migrate AWAY from unicorn because
vendor lock-in is the worst thing, especially if it's on us.
|
|
Previously, Unicorn always used the process's primary group as the
the group of the log file. However, there are reasons to use a
separate group for the log files, such as when you have many
applications where each application uses it's own user and primary
group, but you want to be able to have a user read the log files
for all applications. Some operating systems have a fairly small
limit on the number of groups per user, and it may not be feasible
to have a user be in the primary group for all applications.
a primary group
|
|
Because I forget to load accf_http on new FreeBSD installs,
too :x
|
|
We have never had any need for pipes with the default 64K
capacity on Linux. Our pipes are only used for tiny writes
in signal handlers and to perform parent shutdown detection.
With the current /proc/sys/fs/pipe-user-pages-soft
default, only 1024 pipes can be created by an unprivileged
user before Linux clamps down the pipe size to 4K (a single page)
for newly-created pipes[1].
So avoid penalizing OTHER pipe users who could benefit from the
increased capacity and use only a single page for ourselves.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/pipe.c?h=v4.18#n642
|
|
* 5.4-stable:
unicorn 5.4.1
|
|
Slowly reducing dependencies on kgio. 'io/wait' is required by
'socket' these days, so it's no extra relocations for .so
loading, either.
|
|
It's not unicorn 6, yet, but we dropped Ruby 1.8 support at
unicorn 5. Stable Ruby 1.9+ releases have always reseeded the
PRNG at fork.
|
|
This release quiets some warnings for Ruby 2.6 preview releases
and enables tests to pass under Ruby 1.9.3. Otherwise, nothing
interesting for Ruby 2.0..2.5 users. *YAWN*
Eric Wong (1):
quiet some mismatched indentation warnings
Fumiaki MATSUSHIMA (1):
Use IO#wait instead to fix test for Ruby 1.9
|
|
Ruby trunk started warning about more mismatched indentations
starting around r62836.
|
|
IO#wait_readable is introduced since 2.0
I confirmed we can pass tests for all versions of Ruby with this patch.
https://github.com/mtsmfm/unicorn/pull/2
|
|
Rack hijack support improves as the app code can capture and use
the Rack `env' privately without copying it (to avoid clobbering
by another client). Thanks to Sam Saffron for reporting and
testing this new feature:
https://bogomips.org/unicorn-public/CAAtdryPG3nLuyo0jxfYW1YHu1Q+ZpkLkd4KdWC8vA46B5haZxw@mail.gmail.com/T/
We also now support $DEBUG being set by the Rack app (instead of
relying on the "-d" CLI switch). Thanks to James P Robinson Jr
for reporting this bug:
https://bogomips.org/unicorn-public/D6324CB4.7BC3E%25james.robinson3@cigna.com/T/
(Coincidentally, this fix will be irrelevant for Ruby 2.5
which requires 'pp' by default)
There's a few minor test cleanups and documentation updates, too.
All commits since v5.3.1 (2017-10-03):
reduce method calls with String#start_with?
require 'pp' if $DEBUG is set by Rack app
avoid reusing env on hijack
tests: cleanup some unused variable warnings
ISSUES: add a note about Debian BTS interopability
Roughly all mailing discussions since the last release:
https://bogomips.org/unicorn-public/?q=d:20171004..20171223
|
|
This isn't anything new, just explicitly stating what's always
been the case.
In other news, I hate formal things.
|
|
Add a new "check-warnings" target to the GNUmakefile to make
checking for this easier. Warnings aren't fatal, and newer
versions of Ruby tend to increase warnings.
|
|
Hijackers may capture and reuse `env' indefinitely, so we must
not use it in those cases for future requests. For non-hijack
requests, we continue to reuse the `env' object to reduce
memory recycling.
Reported-and-tested-by: Sam Saffron <sam.saffron@gmail.com>
|
|
While "unicorn -d" requires 'pp' when setting $DEBUG, we did not
account for (rare) Rack applications setting $DEBUG at load time.
Thanks-to: James P (Jim) Robinson Jr <James.Robinson3@Cigna.com>
|
|
* origin/5.3-stable:
unicorn 5.3.1
fix GC issue on rb_global_variable array
|
|
This release fixes an occasional GC problem introduced in v5.3.0
to reduce global variable overhead (commit 979ebcf91705709b)
Thanks to Xuanzhong Wei for the patch which lead to this release:
https://bogomips.org/unicorn-public/20171003182054.76392-1-azrlew@gmail.com/T/#u
https://bogomips.org/unicorn-public/20171003145718.30404-1-azrlew@gmail.com/T/#u
Xuanzhong Wei (1):
fix GC issue on rb_global_variable array
|
|
We need to add the array to ruby's global_list right after created it;
otherwise it probably gets GCed.
(cherry picked from commit e85de00a3065461f6fac466276d915b809f40c7c)
|
|
We need to add the array to ruby's global_list right after created it;
otherwise it probably gets GCed.
|
|
These three cold call sites instruction sequence size by a few
hundred bytes combined since we no longer support Ruby 1.8.6.
The "?/" shorthand is esoteric and no longer avoids allocation
in Ruby 1.9+ (not that this is hot code).
|
|
A couple of portability fixes from Dylan Thacker-Smith and
Jeremy Evans since 5.3.0.pre1 over a week ago, but this looks
ready for a stable release, today.
When I started this over 8 years ago, I wondered if this would
just end up being an April Fools' joke. Guess not. I guess I
somehow tricked people into using a terribly marketed web server
that cannot talk directly to untrusted clients :x Anyways,
unicorn won't be able to handle slow clients 8 years from now,
either, or 80 years from now. And I vow never to learn to use
new-fangled things like epoll, kqueue, or threads :P
Anyways, this is a largish release with several new features,
and no backwards incompatibilities.
Simon Eskildsen contributed heavily using TCP_INFO under Linux
to implement the (now 5 year old) check_client_connection feature:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-check_client_connection
https://bogomips.org/unicorn-public/?q=s:check_client_connection&d:..20170401&x=t
This also led to FreeBSD and OpenBSD portability improvements in
one of our dependencies, raindrops:
https://bogomips.org/raindrops-public/20170323024829.GA5190@dcvr/T/#u
Jeremy Evans contributed several new features. First he
implemented after_worker_exit to aid debugging:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-after_worker_exit
https://bogomips.org/unicorn-public/?q=s:after_worker_exit&d:..20170401&x=t#t
And then security-related features to isolate workers. Workers
may now chroot to drop access to the master filesystem, and the
new after_worker_ready configuration hook now exists to aid with
chroot support in workers:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-after_worker_ready
https://bogomips.org/unicorn/Unicorn/Worker.html#method-i-user
https://bogomips.org/unicorn-public/?q=s:after_worker_ready&d:..20170401&x=t#t
https://bogomips.org/unicorn-public/?q=s:chroot&d:..20170401&x=t#t
Additionally, workers may run in a completely different VM space
(nullifying preload_app and any CoW savings) with the new
worker_exec option:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-worker_exec
https://bogomips.org/unicorn-public/?q=s:worker_exec&d:..20170401&x=t#t
There are also several improvements to FreeBSD and OpenBSD
support with the addition of these features.
shortlog of changes since v5.2.0 (2016-10-31):
Dylan Thacker-Smith (1):
Check for Socket::TCP_INFO constant before trying to get TCP_INFO
Eric Wong (30):
drop rb_str_set_len compatibility replacement
TUNING: document THP caveat for Linux users
tee_input: simplify condition for IO#write
remove response_start_sent
http_request: freeze constant strings passed IO#write
Revert "remove response_start_sent"
t/t0012-reload-empty-config.sh: access ivars directly if needed
t0011-active-unix-socket.sh: fix race condition in test
new test for check_client_connection
revert signature change to HttpServer#process_client
support "struct tcp_info" on non-Linux and Ruby 2.2+
unicorn_http: reduce rb_global_variable calls
oob_gc: rely on opt_aref_with optimization on Ruby 2.2+
http_request: reduce insn size for check_client_connection
freebsd: avoid EINVAL when setting accept filter
test-lib: expr(1) portability fix
tests: keep disabled tests defined
test_exec: SO_KEEPALIVE value only needs to be true
doc: fix links to raindrops project
http_request: support proposed Raindrops::TCP states on non-Linux
ISSUES: expand on mail archive info + subscription disclaimer
test_ccc: use a pipe to synchronize test
doc: remove private email support address
input: update documentation and hide internals.
http_server: initialize @pid ivar
gemspec: remove olddoc from build dependency
doc: add version annotations for new features
unicorn 5.3.0.pre1
doc: note after_worker_exit is also 5.3.0+
test_exec: SO_KEEPALIVE value only needs to be true (take #2)
Jeremy Evans (7):
Add after_worker_exit configuration option
Fix code example in after_worker_exit documentation
Add support for chroot to Worker#user
Add after_worker_ready configuration option
Add worker_exec configuration option
Don't pass a block for fork when forking workers
Check for SocketError on first ccc attempt
Simon Eskildsen (1):
check_client_connection: use tcp state on linux
|
|
We need to ensure the portability of the sd_listen_fds emulation
test, too, which didn't get tested on my FreeBSD 10.3 install
due to it being on Ruby 2.2
Followup-to: 4ce6b00f75f1 ("test_exec: SO_KEEPALIVE value only needs to be true")
|
|
The ruby constant Socket::TCP_INFO is only defined if TCP_INFO is defined
in C, so we can just check for the presence of that ruby constant instead
of rescuing SocketError from the call to getsockopt.
|
|
On OpenBSD, getsockopt(2) does not support TCP_INFO. With the current code,
this results in a 500 for all clients if check_client_connection is enabled
on OpenBSD.
This patch rescues SocketError on the first getsockopt call, and
if SocketError is raised, it doesn't check in the future. This
should be the same behavior as if TCP_INFO was supported but
inspect did not return a string in the expected format.
|
|
Followup-to: 650e01ab0b118803486b56f3ee59521d59042dae
("doc: add version annotations for new features")
|
|
A largish release with several new features.
Simon Eskildsen contributed heavily using TCP_INFO under Linux
to implement the (now 5 year old) check_client_connection feature:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-check_client_connection
https://bogomips.org/unicorn-public/?q=s:check_client_connection&d:..20170324&x=t
This also led to FreeBSD and OpenBSD portability improvements in
one of our dependencies, raindrops:
https://bogomips.org/raindrops-public/20170323024829.GA5190@dcvr/T/#u
Jeremy Evans contributed several new features. First he
implemented after_worker_exit to aid debugging:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-after_worker_exit
https://bogomips.org/unicorn-public/?q=s:after_worker_exit&d:..20170324&x=t#t
And then security-related features to isolate workers. Workers
may now chroot to drop access to the master filesystem, and the
new after_worker_ready configuration hook now exists to aid with
chroot support in workers:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-after_worker_ready
https://bogomips.org/unicorn/Unicorn/Worker.html#method-i-user
https://bogomips.org/unicorn-public/?q=s:after_worker_ready&d:..20170324&x=t#t
https://bogomips.org/unicorn-public/?q=s:chroot&d:..20170324&x=t#t
Additionally, workers may run in a completely different VM space
(nullifying preload_app and any CoW savings) with the new
worker_exec option:
https://bogomips.org/unicorn/Unicorn/Configurator.html#method-i-worker_exec
https://bogomips.org/unicorn-public/?q=s:worker_exec&d:..20170324&x=t#t
There are also several improvements to FreeBSD and OpenBSD
support with the addition of these features.
34 changes since 5.2.0 (2016-10-31):
Eric Wong (27):
drop rb_str_set_len compatibility replacement
TUNING: document THP caveat for Linux users
tee_input: simplify condition for IO#write
remove response_start_sent
http_request: freeze constant strings passed IO#write
Revert "remove response_start_sent"
t/t0012-reload-empty-config.sh: access ivars directly if needed
t0011-active-unix-socket.sh: fix race condition in test
new test for check_client_connection
revert signature change to HttpServer#process_client
support "struct tcp_info" on non-Linux and Ruby 2.2+
unicorn_http: reduce rb_global_variable calls
oob_gc: rely on opt_aref_with optimization on Ruby 2.2+
http_request: reduce insn size for check_client_connection
freebsd: avoid EINVAL when setting accept filter
test-lib: expr(1) portability fix
tests: keep disabled tests defined
test_exec: SO_KEEPALIVE value only needs to be true
doc: fix links to raindrops project
http_request: support proposed Raindrops::TCP states on non-Linux
ISSUES: expand on mail archive info + subscription disclaimer
test_ccc: use a pipe to synchronize test
doc: remove private email support address
input: update documentation and hide internals.
http_server: initialize @pid ivar
gemspec: remove olddoc from build dependency
doc: add version annotations for new features
Jeremy Evans (6):
Add after_worker_exit configuration option
Fix code example in after_worker_exit documentation
Add support for chroot to Worker#user
Add after_worker_ready configuration option
Add worker_exec configuration option
Don't pass a block for fork when forking workers
Simon Eskildsen (1):
check_client_connection: use tcp state on linux
|
|
We will inevitably have people running old unicorn versions
for many years to come; but they may be reading the latest
documentation online.
Annotate when the new features (will) appear to avoid misleading
users on old versions.
|
|
It's a little less DRY, and there'll be no NEWS file generated,
but it's one less thing to install, so perhaps that's worth it.
The website at https://bogomips.org/unicorn/ will continue
to use olddoc, of course,
|
|
* origin/worker_exec:
Don't pass a block for fork when forking workers
Add worker_exec configuration option
|
|
* ccc-tcp-v3:
test_ccc: use a pipe to synchronize test
http_request: support proposed Raindrops::TCP states on non-Linux
|
|
This quiets down warnings when run with '-w'
|