* Unicorn returns blank page after no use
@ 2015-07-01 16:08 Farjad Adamjee
2015-07-01 17:30 ` Eric Wong
0 siblings, 1 reply; 2+ messages in thread
From: Farjad Adamjee @ 2015-07-01 16:08 UTC (permalink / raw)
To: unicorn-public
Hello,
I am not sure if this is the correct place for this, I have googled
around to see if anyone else has encountered such an issue, but I did
not find anything.
I am having this issue where after a period of time of no use (on beta
[production] environment), maybe 12 hours or so, the unicorn process
sleeps. When I go and re-request the page, it returns blank.
I am using Apache proxy to Unicorn.
My unicorn.stderr.log states:
I, [2015-07-01T03:07:05.168511 #13258] INFO -- : master done reopening logs
I, [2015-07-01T03:07:05.232546 #16222] INFO -- : worker=2 done
reopening logs
I, [2015-07-01T03:07:05.239934 #13308] INFO -- : worker=0 done
reopening logs
I, [2015-07-01T03:07:05.307616 #13311] INFO -- : worker=1 done
reopening logs
When I request the page again, it loads fine. It is only during the
process of "reopening logs" when it returns a blank page.
My unicorn.rb file:
worker_processes 3
# Load app into the master before forking workers
# for super-fast worker spawn times
preload_app true
# Listen on
listen "127.0.0.1:8550", :backlog => 1024
# Restart any workers that haven't responded in 180 seconds
timeout 600
# PID for the Unicorn master
pid "/var/www/r/shared/pids/unicorn.pid"
# Unicorn Log paths
stderr_path "/var/www/r/shared/log/unicorn.stderr.log"
stdout_path "/var/www/r/shared/log/unicorn.stdout.log"
before_fork do |server, worker|
##
# When sent a USR2, Unicorn will suffix its pidfile with .oldbin and
# immediately start loading up a new version of itself (loaded with a new
# version of our app). When this new Unicorn is completely loaded
# it will begin spawning workers. The first worker spawned will check to
# see if an .oldbin pidfile exists. If so, this means we've just
booted up
# a new Unicorn and need to tell the old one that it can now die. To
do so
# we send it a QUIT.
#
# Using this method we get 0 downtime deploys.
defined?(ActiveRecord::Base) and
ActiveRecord::Base.connection.disconnect!
defined?(OathKeeper) and OathKeeper.disconnect!
old_pid = '/var/www/r/shared/pids/unicorn.pid.oldbin'
if File.exists?(old_pid) && server.pid != old_pid
begin
Process.kill("QUIT", File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
# someone else did our job for us
end
end
end
after_fork do |server, worker|
##
# # Unicorn master loads the app then forks off workers - because of
the way
# # Unix forking works, we need to make sure we aren't using any
of the parent's
# # sockets, e.g. db connection
#
defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection
defined?(OathKeeper) and OathKeeper.reconnect
end
Any help would be appreciated!
Thank you!
Farjad
--
Farjad Adamjee
844.246.4018
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Unicorn returns blank page after no use
2015-07-01 16:08 Unicorn returns blank page after no use Farjad Adamjee
@ 2015-07-01 17:30 ` Eric Wong
0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2015-07-01 17:30 UTC (permalink / raw)
To: Farjad Adamjee; +Cc: unicorn-public
Farjad Adamjee <fadamjee@vailsys.com> wrote:
> Hello,
>
> I am not sure if this is the correct place for this, I have googled
> around to see if anyone else has encountered such an issue, but I
> did not find anything.
This is the correct place :) Plain-text email is the only way I
handle support for ease-of-archival and distribution.
> I am having this issue where after a period of time of no use (on
> beta [production] environment), maybe 12 hours or so, the unicorn
> process sleeps. When I go and re-request the page, it returns blank.
This sounds like a database (or any other persistent connection
backend) connection expiring due to an idle timeout.
This could also be a firewall/router timeout problem between the
server running unicorn and the database host.
Tony experienced this last year:
http://bogomips.org/unicorn-public/m/CAKM1sPNRsES6H6ByK6bO9Djwa8WvYV6HJ-rEaHopRUYBVFfuhg@mail.gmail.com
You can confirm by running lsof on a worker processes to look for
open sockets. In case the connection loss isn't noticed by the
OS, use strace (or any other similar syscall tracer for your OS)
instead and watch connection activity.
With strace (or similar) knock yourself down to one worker
process via SIGTTOU to the master to ensure you're always tracing
the correct one.
> I am using Apache proxy to Unicorn.
By the way, this is not recommended for slow clients.
nginx (Open Source edition) is currently the only supported proxy.
> # Using this method we get 0 downtime deploys.
> defined?(ActiveRecord::Base) and
> ActiveRecord::Base.connection.disconnect!
> defined?(OathKeeper) and OathKeeper.disconnect!
I suggest disabling any idle timeout you might have, enabling TCP
keepalive and enabling reconnects (see AR and your database driver
documentation). Worst case (firewall/router problem you can't
fix via configuration), is to send a request to unicorn every
few minutes/hours to keep things going.
Perhaps OathKeeper has the same problem, too...
I've never heard of it until now.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2015-07-01 17:30 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-07-01 16:08 Unicorn returns blank page after no use Farjad Adamjee
2015-07-01 17:30 ` Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).