From: Eric Wong <normalperson@yhbt.net>
To: unicorn list <mongrel-unicorn@rubyforge.org>
Subject: Re: Exception `Errno::EAGAIN'
Date: Sun, 15 Nov 2009 15:47:39 -0800 [thread overview]
Message-ID: <20091115234739.GA26275@dcvr.yhbt.net> (raw)
In-Reply-To: <4B008CFE.6020206@lovedthanlost.net>
James Turnbull <james@lovedthanlost.net> wrote:
> I get this repeated error - about once a second:
>
> Exception `Errno::EAGAIN' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
> Resource temporarily unavailable - accept(2)
> Exception `Errno::EAGAIN' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
> Resource temporarily unavailable - accept(2)
> Exception `Errno::EAGAIN' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:643 -
> Resource temporarily unavailable - accept(2)
>
> Which seems to be related to:
>
> ready.each do |sock|
> begin
> process_client(sock.accept_nonblock)
> nr += 1
> alive.chmod(m = 0 == m ? 1 : 0)
> rescue Errno::EAGAIN, Errno::ECONNABORTED
> end
> break if nr < 0
> end
Hi James,
This is expected. The kernel wakes up all the workers when there's
_one_ connection and they all race to accept() one client connection.
One wins and accepts the connection, 3 lose and go back to sleep. With
lots of worker processes, this can be a thundering herd problem but
that's why we encourage Unicorn as a backend server, not as a frontend
server.
If we only have one listener, we could do a blocking accept() call to
avoid thundering herds under Linux, but testing on a 16-core box, I
don't remember being able to measure a performance improvement. But
otherwise we have to use select() with multiple listeners. nginx uses a
variety of non-portable locking mechanisms to implement its accept
mutex, but that's too much work under Ruby for little/no benefit. In
the nginx case, sometimes even disabling the accept mutex entirely gives
better performance.
> My unicorn.conf is:
>
> worker_processes 4
> working_directory "/etc/puppet"
> listen '/tmp/puppet.sock', :backlog => 1
> listen 8140, :tcp_nopush => true
> timeout 10
> pid "/tmp/puppet.pid"
>
> preload_app true
> GC.respond_to?(:copy_on_write_friendly=) and
> GC.copy_on_write_friendly = true
>
> before_fork do |server, worker|
> # the following is recomended for Rails + "preload_app true"
> # as there's no need for the master process to hold a connection
> # defined?(ActiveRecord::Base) and
> # ActiveRecord::Base.connection.disconnect!
>
> # the following allows a new master process to incrementally
> # phase out the old master process with SIGTTOU to avoid a
> # thundering herd (especially in the "preload_app false" case)
> # when doing a transparent upgrade. The last worker spawned
> # will then kill off the old master process with a SIGQUIT.
> old_pid = "#{server.config[:pid]}.oldbin"
> if old_pid != server.pid
> begin
> sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
> Process.kill(sig, File.read(old_pid).to_i)
> rescue Errno::ENOENT, Errno::ESRCH
> end
> end
>
> # optionally throttle the master from forking too quickly by
> sleeping
> sleep 1
> end
>
> The platform is Fedora 10, Ruby 1.8.6.
>
> Oddly, if I comment out the preload I get:
>
> Exception `Errno::EPERM' at
> /usr/lib/ruby/gems/1.8/gems/unicorn-0.95.0/lib/unicorn.rb:639 -
> Operation not permitted - /tmp/0.520680132392046
Do you by any chance do user switching in your application
or config.ru?
Instead, since 0.94.0, I would do this:
after_fork do |server, worker|
worker.user('user', 'group') if Process.euid == 0
end
That way only the worker process drops permissions. The master really
doesn't do anything interesting. Of course *I* would just start
Unicorn as a regular user and forget the complexity of user switching
entirely...
--
Eric Wong
next prev parent reply other threads:[~2009-11-15 23:57 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-11-15 23:21 Exception `Errno::EAGAIN' James Turnbull
2009-11-15 23:47 ` Eric Wong [this message]
2009-11-16 0:08 ` James Turnbull
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/unicorn/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20091115234739.GA26275@dcvr.yhbt.net \
--to=normalperson@yhbt.net \
--cc=mongrel-unicorn@rubyforge.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).