From: Cedric Maion <cedric@maion.com>
To: Eric Wong <e@80x24.org>
Cc: unicorn-public@bogomips.org
Subject: Re: Log reopening broken on Rails 4 with config.autoflush_log = false?
Date: Fri, 4 Jul 2014 11:13:17 +0200 [thread overview]
Message-ID: <20140704091317.GA19662@cedric-maion.com> (raw)
In-Reply-To: <20140703172410.GA17685@dcvr.yhbt.net>
Hi Eric,
Thanks for your answer (and update to the FAQ ;) ).
I'm playing with autoflush_log = true and see if I find a performance
impact (I'm logging approx 20KB per sec).
sys.vm.dirty* sysctls are already changed, but with low ratios as those
systems are also writing large files (and I don't want those pages in
memory for too long).
If a performance impact can be measured, I believe that I will turn
autoflush_log to false again and modify Unicorn::Util#is_log? to accept
!fp.sync ;) (I'm willing to accept occasional log lines corruption esp
during rotation)... or just restart unicorn after rotation instead.
Or, thinking out loud... keep autoflush_log to false and have unicorn
flush logs itself regularly, outside the HTTP request path (and when
reopening logs), to avoid blocking on IO during a log flush in the
middle of a request? (I need to refresh myself on the sizes of the ruby
& system IO buffers though, not sure if it's worth the pain).
Cedric
On Thu, Jul 03, 2014 at 05:24:10PM +0000, Eric Wong wrote:
>
> Cedric Maion <cedric@maion.com> wrote:
> > On a Rails 4.0 app with 'config.autoflush_log = false', it seems to me
> > that telling unicorn to reopen log files fails (e.g., after a log
> > rotation and USR1 signal, Rails/unicorn still writes to the old rotated
> > file instead of reopening a new one).
> >
> > Without this config option, logs does properly get reopened.
> >
> > Is this something known?
>
> Yes, log buffering via File#sync=true is incompatible with reopening in
> multiprocess servers. File#sync=true does not guarantee writes are
> atomic on line boundaries, so it's dangerous to assume they're logs.
>
> Even without reopening, enabling this option on a multiprocess server
> might corrupt logs (depending on the buffering implementation)
> because the synchronization is within each process rather for the
> entire OS.
>
> Looking at the rails source:
> ==> railties/lib/rails/application/bootstrap.rb <==
> f = File.open path, 'a'
> f.binmode
> f.sync = config.autoflush_log # if true make sure every write flushes
>
> You're not likely to notice any performance difference unless you're
> logging excessively. Keep in mind the Linux kernel also does buffering
> and you can tune that via the sys.vm.dirty_* sysctls.
prev parent reply other threads:[~2014-07-04 9:13 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-07-03 14:40 Log reopening broken on Rails 4 with config.autoflush_log = false? Cedric Maion
2014-07-03 17:24 ` Eric Wong
2014-07-03 19:28 ` [PATCH] FAQ: add entry for Rails autoflush_log Eric Wong
2014-07-04 9:13 ` Cedric Maion [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/unicorn/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140704091317.GA19662@cedric-maion.com \
--to=cedric@maion.com \
--cc=e@80x24.org \
--cc=unicorn-public@bogomips.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).