Rainbows! Rack HTTP server user/dev discussion
 help / Atom feed
From: Eric Wong <normalperson@yhbt.net>
To: mongrel-unicorn@rubyforge.org
Cc: rainbows-talk@rubyforge.org
Subject: [RFC] workaround reopen atomicity issues for stdio vs non-stdio
Date: Sun, 20 Oct 2013 04:44:20 +0000
Message-ID: <20131020044420.GA2734@dcvr.yhbt.net> (raw)

In multithreaded apps, we must use dup2/dup3 with a temporary
descriptor to reopen log files atomically.  This is the only way
to protect all concurrent userspace access to a file when reopening.

ref: http://bugs.ruby-lang.org/issues/9036
ref: yahns commit bcb10abe53cfb1d6a8ef7daef59eb10ced397c8a
---
 Review of this patch is greatly appreciated.  This doesn't affect most
 unicorn users unless they spawn threads themselves and write to log
 files in their app.  This does affect Rainbows! users who configure
 Rainbows! to use threads, though.


 Also, I guess I should announce yahns on these lists for those not on
 ruby-talk:  http://yahns.yhbt.net/README
 git clone git://yhbt.net/yahns - not for production, yet, but soon
 I can write HTTP servers all day long, really, I just can't stand
 web browsers :P

 lib/unicorn/util.rb | 24 ++++++++++++++++++++++--
 1 file changed, 22 insertions(+), 2 deletions(-)

diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb
index f84241c..94c4e37 100644
--- a/lib/unicorn/util.rb
+++ b/lib/unicorn/util.rb
@@ -39,7 +39,7 @@ module Unicorn::Util
     to_reopen.each do |fp|
       orig_st = begin
         fp.stat
-      rescue IOError, Errno::EBADF
+      rescue IOError, Errno::EBADF # race
         next
       end
 
@@ -50,8 +50,28 @@ module Unicorn::Util
       end
 
       begin
-        File.open(fp.path, 'a') { |tmpfp| fp.reopen(tmpfp) }
+        # stdin, stdout, stderr are special.  The following dance should
+        # guarantee there is no window where `fp' is unwritable in MRI
+        # (or any correct Ruby implementation).
+        #
+        # Fwiw, GVL has zero bearing here.  This is tricky because of
+        # the unavoidable existence of stdio FILE * pointers for
+        # std{in,out,err} in all programs which may use the standard C library
+        if fp.fileno <= 2
+          # We do not want to hit fclose(3)->dup(2) window for std{in,out,err}
+          # MRI will use freopen(3) here internally on std{in,out,err}
+          fp.reopen(fp.path, "a")
+        else
+          # We should not need this workaround, Ruby can be fixed:
+          #    http://bugs.ruby-lang.org/issues/9036
+          # MRI will not call call fclose(3) or freopen(3) here
+          # since there's no associated std{in,out,err} FILE * pointer
+          # This should atomically use dup3(2) (or dup2(2)) syscall
+          File.open(fp.path, "a") { |tmpfp| fp.reopen(tmpfp) }
+        end
+
         fp.sync = true
+        fp.flush # IO#sync=true may not implicitly flush
         new_st = fp.stat
 
         # this should only happen in the master:
-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying


                 reply index

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://bogomips.org/rainbows/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20131020044420.GA2734@dcvr.yhbt.net \
    --to=normalperson@yhbt.net \
    --cc=mongrel-unicorn@rubyforge.org \
    --cc=rainbows-talk@rubyforge.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Rainbows! Rack HTTP server user/dev discussion

Archives are clonable:
	git clone --mirror https://bogomips.org/rainbows-public
	git clone --mirror http://ou63pmih66umazou.onion/rainbows-public

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.lang.ruby.rainbows
	nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.rainbows

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox