leakage of sockets or activerecord connections

leakage of sockets or activerecord connections

[Corin Langosch]

Hi,

I'm using rainbows to power my own small middleware. I doen't use rails (or any 
other framework), only activerecord for database access. I chose 
XEpollThreadSpawn, set worker_processes 1 and worker_connections 25. All classes 
are eager loaded, no reloading of anything while the server is running. AR 
connection pool size is set to 100.

Now it seems that every request opens a new connection and never frees/ closes 
it. So after 100 requests I get an AR connection pool exception. I also see 
exactly 100 postgresql clients connected. When I kill the server all clients get 
disconnected.

I wonder if I have to setup and hooks (like in unicorn before_fork etc.)? In 
fact I'd expect this happens automatically as the thread exits after the request 
is completed?

Thanks,
Corin

_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: leakage of sockets or activerecord connections

[Eric Wong]

Corin Langosch <info-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> wrote:
> I'm using rainbows to power my own small middleware. I doen't use
> rails (or any other framework), only activerecord for database
> access. I chose XEpollThreadSpawn, set worker_processes 1 and
> worker_connections 25. All classes are eager loaded, no reloading of
> anything while the server is running. AR connection pool size is set
> to 100.

I'm not sure how AR connection pool works (if it uses thread-local
variables for storing the connections).  If it's using thread-locals,
it could be reliant on GC, and that would require:

a) GC to run frequently enough to reap connections
b) your Postgres bindings being GC-aware (and really being unreachable
   in your VM)

> Now it seems that every request opens a new connection and never
> frees/ closes it. So after 100 requests I get an AR connection pool
> exception. I also see exactly 100 postgresql clients connected. When
> I kill the server all clients get disconnected.
> 
> I wonder if I have to setup and hooks (like in unicorn before_fork
> etc.)? In fact I'd expect this happens automatically as the thread
> exits after the request is completed?

Can you reproduce the issue with XEpollThreadPool?

If it's using thread-local storage, you probably need to have a Rack
middleware push the connection back into the pool when it's done working
on a request.

Something like this in your config.ru:

class ReleaseConnections
  def initialize(app)
    @app = app
  end

  def call(env)
    begin
      @app.call(env)
    ensure
      Thread.current[:whatever_sockets].release_to_pool
    end
  end
end

use ReleaseConnections
...
run YourApp.new

In any case, I'd dig through the Rails/AR APIs to see how it works
and how to release resources back to the pool.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: leakage of sockets or activerecord connections

[Lin Jen-Shin (godfat)]

On Fri, Aug 23, 2013 at 6:13 AM, Corin Langosch <info-FIgL9nsKG9THeUWFKdsAYQC/G2K4zDHf@public.gmane.org> wrote:
> Hi,
>
> I wonder if I have to setup and hooks (like in unicorn before_fork etc.)? In
> fact I'd expect this happens automatically as the thread exits after the
> request is completed?
>
> Thanks,
> Corin

No, I think you should probably insert this middleware on top of your app:

    ActiveRecord::ConnectionAdapters::ConnectionManagement

This is actually what Rails did for itself. Here's what it is doing:

    class ConnectionManagement
      def initialize(app)
        @app = app
      end

      def call(env)
        testing = env.key?('rack.test')

        response = @app.call(env)
        response[2] = ::Rack::BodyProxy.new(response[2]) do
          ActiveRecord::Base.clear_active_connections! unless testing
        end

        response
      rescue
        ActiveRecord::Base.clear_active_connections! unless testing
        raise
      end
    end

p.s. By pasting you this, now I understood why Rails works a bit
differently with threads in tests. Oh well. Maybe I should patch it.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying

Re: leakage of sockets or activerecord connections

[Corin Langosch]

Am 23.08.2013 00:46, schrieb Lin Jen-Shin (godfat):
> No, I think you should probably insert this middleware on top of your app:
>
>      ActiveRecord::ConnectionAdapters::ConnectionManagement
>
>

Thank you, works fine now! :)

Corin
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying