* leakage of sockets or activerecord connections @ 2013-08-22 22:13 Corin Langosch [not found] ` <52168D10.8080407-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> 0 siblings, 1 reply; 4+ messages in thread From: Corin Langosch @ 2013-08-22 22:13 UTC (permalink / raw) To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw Hi, I'm using rainbows to power my own small middleware. I doen't use rails (or any other framework), only activerecord for database access. I chose XEpollThreadSpawn, set worker_processes 1 and worker_connections 25. All classes are eager loaded, no reloading of anything while the server is running. AR connection pool size is set to 100. Now it seems that every request opens a new connection and never frees/ closes it. So after 100 requests I get an AR connection pool exception. I also see exactly 100 postgresql clients connected. When I kill the server all clients get disconnected. I wonder if I have to setup and hooks (like in unicorn before_fork etc.)? In fact I'd expect this happens automatically as the thread exits after the request is completed? Thanks, Corin _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <52168D10.8080407-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org>]
* Re: leakage of sockets or activerecord connections [not found] ` <52168D10.8080407-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> @ 2013-08-22 22:45 ` Eric Wong 2013-08-22 22:46 ` Lin Jen-Shin (godfat) 1 sibling, 0 replies; 4+ messages in thread From: Eric Wong @ 2013-08-22 22:45 UTC (permalink / raw) To: Rainbows! list Corin Langosch <info-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> wrote: > I'm using rainbows to power my own small middleware. I doen't use > rails (or any other framework), only activerecord for database > access. I chose XEpollThreadSpawn, set worker_processes 1 and > worker_connections 25. All classes are eager loaded, no reloading of > anything while the server is running. AR connection pool size is set > to 100. I'm not sure how AR connection pool works (if it uses thread-local variables for storing the connections). If it's using thread-locals, it could be reliant on GC, and that would require: a) GC to run frequently enough to reap connections b) your Postgres bindings being GC-aware (and really being unreachable in your VM) > Now it seems that every request opens a new connection and never > frees/ closes it. So after 100 requests I get an AR connection pool > exception. I also see exactly 100 postgresql clients connected. When > I kill the server all clients get disconnected. > > I wonder if I have to setup and hooks (like in unicorn before_fork > etc.)? In fact I'd expect this happens automatically as the thread > exits after the request is completed? Can you reproduce the issue with XEpollThreadPool? If it's using thread-local storage, you probably need to have a Rack middleware push the connection back into the pool when it's done working on a request. Something like this in your config.ru: class ReleaseConnections def initialize(app) @app = app end def call(env) begin @app.call(env) ensure Thread.current[:whatever_sockets].release_to_pool end end end use ReleaseConnections ... run YourApp.new In any case, I'd dig through the Rails/AR APIs to see how it works and how to release resources back to the pool. _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: leakage of sockets or activerecord connections [not found] ` <52168D10.8080407-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> 2013-08-22 22:45 ` Eric Wong @ 2013-08-22 22:46 ` Lin Jen-Shin (godfat) [not found] ` <CAA2_N1soVYiNkRgZqa+mLO26KUzvTAXuk0gA9Qo7MPrZ0+bNXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 1 sibling, 1 reply; 4+ messages in thread From: Lin Jen-Shin (godfat) @ 2013-08-22 22:46 UTC (permalink / raw) To: Rainbows! list On Fri, Aug 23, 2013 at 6:13 AM, Corin Langosch <info-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> wrote: > Hi, > > I wonder if I have to setup and hooks (like in unicorn before_fork etc.)? In > fact I'd expect this happens automatically as the thread exits after the > request is completed? > > Thanks, > Corin No, I think you should probably insert this middleware on top of your app: ActiveRecord::ConnectionAdapters::ConnectionManagement This is actually what Rails did for itself. Here's what it is doing: class ConnectionManagement def initialize(app) @app = app end def call(env) testing = env.key?('rack.test') response = @app.call(env) response[2] = ::Rack::BodyProxy.new(response[2]) do ActiveRecord::Base.clear_active_connections! unless testing end response rescue ActiveRecord::Base.clear_active_connections! unless testing raise end end p.s. By pasting you this, now I understood why Rails works a bit differently with threads in tests. Oh well. Maybe I should patch it. _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <CAA2_N1soVYiNkRgZqa+mLO26KUzvTAXuk0gA9Qo7MPrZ0+bNXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: leakage of sockets or activerecord connections [not found] ` <CAA2_N1soVYiNkRgZqa+mLO26KUzvTAXuk0gA9Qo7MPrZ0+bNXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2013-08-27 12:31 ` Corin Langosch 0 siblings, 0 replies; 4+ messages in thread From: Corin Langosch @ 2013-08-27 12:31 UTC (permalink / raw) To: Rainbows! list Am 23.08.2013 00:46, schrieb Lin Jen-Shin (godfat): > No, I think you should probably insert this middleware on top of your app: > > ActiveRecord::ConnectionAdapters::ConnectionManagement > > Thank you, works fine now! :) Corin _______________________________________________ Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org http://rubyforge.org/mailman/listinfo/rainbows-talk Do not quote signatures (like this one) or top post when replying ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2013-08-27 12:38 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-08-22 22:13 leakage of sockets or activerecord connections Corin Langosch [not found] ` <52168D10.8080407-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> 2013-08-22 22:45 ` Eric Wong 2013-08-22 22:46 ` Lin Jen-Shin (godfat) [not found] ` <CAA2_N1soVYiNkRgZqa+mLO26KUzvTAXuk0gA9Qo7MPrZ0+bNXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 2013-08-27 12:31 ` Corin Langosch
Code repositories for project(s) associated with this public inbox https://yhbt.net/rainbows.git/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).