unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: John-Paul Bader <hukl@h3q.com>
To: unicorn list <mongrel-unicorn@rubyforge.org>
Subject: Re: "upstream timed out" after upgrades
Date: Thu, 4 Feb 2010 09:48:50 +0100	[thread overview]
Message-ID: <5D2B92C5-8D0A-4810-B86B-06F670C6EFDB@h3q.com> (raw)
In-Reply-To: <F6F8107F-B3A8-44DF-BF70-D9DAD5927D0A@berlin.ccc.de>

Hrm its getting weirder and weirder.

I started unicorn_rails now manually in the app root which is now running on rails 2.3.3 again: 

sudo unicorn_rails -E production -d -c /usr/local/etc/unicorn.rb

Now first of all it throws lots of Exceptions it couldn't find several (important) gems and then it produces this every second or so:

Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)

When I request the javascript in question I get:

Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)

This is the same no matter if I run it with -c or not. 

I am puzzled.

To give you the entire output of unicorn starting:

[hukl@cccms /usr/local/www/cccms]$ sudo unicorn_rails -E production -d -p 9090
{:daemonize=>false,
 :unicorn_options=>{:listeners=>["0.0.0.0:9090"]},
 :app=>
  #<Proc:0x0000000801782388@/usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/bin/unicorn_rails:124>}
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/cache
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/pids
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sessions
Exception `Errno::EEXIST' at /usr/local/lib/ruby/1.8/fileutils.rb:243 - File exists - tmp/sockets
I, [2010-02-04T09:42:08.141034 #38499]  INFO -- : listening on addr=213.73.89.122:9090 fd=3
Exception `LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- Win32API
Exception `LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- Win32API
I, [2010-02-04T09:42:08.158275 #38499]  INFO -- : worker=0 spawning...
I, [2010-02-04T09:42:08.159352 #38499]  INFO -- : master process ready
I, [2010-02-04T09:42:08.159471 #38501]  INFO -- : worker=0 spawned pid=38501
I, [2010-02-04T09:42:08.197082 #38501]  INFO -- : Refreshing Gem list
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem builder (~> 2.1.2)

Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem memcache-client (>= 1.7.4)

Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem tzinfo (~> 0.3.12)

Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem i18n (~> 0.1.3)

Exception `TypeError' at (eval):4 - can't modify frozen object
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- fast_xs
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- json
Using c extension for JSON.
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem tmail (~> 1.2.3)

Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- tmail/tmailscanner
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- tmail/tmailscanner
Exception `MissingSourceFile' at /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:162 - no such file to load -- tmail/tmailscanner
Exception `Gem::LoadError' at /usr/local/lib/ruby/site_ruby/1.8/rubygems.rb:827 - Could not find RubyGem activerecord-postgresql-adapter (>= 0)

Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- pg
WARNING: Nokogiri was built against LibXML version 2.7.3, but has dynamically loaded 2.7.6
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- ruby-debug
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- classtree
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- classtree
Exception `MissingSourceFile' at /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:162 - no such file to load -- classtree
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31 - no such file to load -- methodsig
Exception `MissingSourceFile' at /usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:38 - no such file to load -- methodsig
Exception `MissingSourceFile' at /usr/local/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/dependencies.rb:162 - no such file to load -- methodsig
=> Debugger enabled
worker=0 ready
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)
Exception `Errno::EINVAL' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn/http_response.rb:67 - Invalid argument
Exception `Errno::EAGAIN' at /usr/local/lib/ruby/gems/1.8/gems/unicorn-0.95.3/lib/unicorn.rb:640 - Resource temporarily unavailable - accept(2)


I seriously have no idea went wrong - It ran all so smooth couple of days ago. What am I missing - there must be something plain wrong. I left the js files out of the public site for now - it isn't needed that much and its more important to serve the content for now. In case you are trying to get it.

Kind regards, John


On 04.02.2010, at 09:16, John-Paul Bader wrote:

> Hey guys,
> 
> 
> i recently updated several components of my website setup: 
> 
> varnish   2.0.5 -> 2.0.6
> nginx      7.4.2 -> 7.4.5
> unicorn  0.95.3 -> 0.96
> ruby18  to the latest patch level 
> and rails from 2.3.2
> 
> It all seemed to work great but recently I noticed that the website hangs loading. The weird thing is that it hangs because 2 or 3 javascript files take about a minute to finish and when they do they are incomplete as in cut of in the middle of the content. The nginx error log gives the following output every few seconds:
> 
> 2010/02/04 08:58:41 [error] 29389#0: *471 upstream timed out (60: Operation timed out) while reading upstream, client: 213.73.89.122, server: www.ccc.de, request: "GET /javascripts/jquery-1.3.2.min.js?1246657683 HTTP/1.1", upstream: "http://unix:/var/run/unicorn.sock:/javascripts/jquery-1.3.2.min.js?1246657683", host: "www.ccc.de"
> 
> I don't think its varnishs fault because i get Internal Server Errors (500) if I run curl on the machine against nginx, not at first though - several requests for that jquery file run just fine and then suddenly i get a 500 on a request with the content being cut of in the middle.
> 
> Next thing I did was configuring unicorn to listen on 213.73.89.122:9090 rather than on a socket so i could run curl directly against it and even after a fresh start of unicorn gives me:
> 
> 
> curl http://213.73.89.122:9090/javascripts/jquery-1.3.2.min.js
> (lots of javascript)…,rowspan:"rowSpan",tabindex:"tabIndHTTP/1.1 500 Internal Server Error
> 
> curl: (18) transfer closed with 16256 bytes remaining to read
> 
> I also downgraded back to 0.95.3 but that didn't change anything. So now after 8h of try and error I'm out of ideas and would be happy to hear some suggestions.
> 
> Kind regards, John
> _______________________________________________
> Unicorn mailing list - mongrel-unicorn@rubyforge.org
> http://rubyforge.org/mailman/listinfo/mongrel-unicorn
> Do not quote signatures (like this one) or top post when replying
> 

_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying


  reply	other threads:[~2010-02-04  8:59 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2010-02-04  8:16 "upstream timed out" after upgrades John-Paul Bader
2010-02-04  8:48 ` John-Paul Bader [this message]
2010-02-04  9:28   ` John-Paul Bader
2010-02-04 10:11     ` Eric Wong
2010-02-04 12:26       ` John-Paul Bader
2010-02-04 14:22         ` John-Paul Bader
2010-02-04 15:11           ` John-Paul Bader
2010-02-04 19:42             ` Eric Wong
2010-02-04 20:01               ` John-Paul Bader
2010-02-04 20:14                 ` Eric Wong
2010-02-04 20:23                   ` John-Paul Bader
2010-02-04 20:48                     ` John-Paul Bader
2010-02-04 21:05                       ` John-Paul Bader
2010-02-04 21:25                       ` Eric Wong
2010-02-04 21:56                         ` John-Paul Bader
2010-02-04 21:27                     ` Eric Wong
2010-02-04 20:27                   ` John-Paul Bader
2010-02-04 21:52                     ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://yhbt.net/unicorn/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5D2B92C5-8D0A-4810-B86B-06F670C6EFDB@h3q.com \
    --to=hukl@h3q.com \
    --cc=mongrel-unicorn@rubyforge.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).