[PATCH 0/3] final round of changes for unicorn 5 compatibility

[PATCH 0/3] final round of changes for unicorn 5 compatibility

[Eric Wong]

* http_server: add master_pid attribute
* stream_response_epoll: remove hijack_prepare call
* bump to unicorn 5.0.1, use monotonic clock

There'll probably be another round of small changes since we
can drop Ruby 1.8 support and debloat our code a little.

 lib/rainbows.rb                            | 19 ++++++++++++++++---
 lib/rainbows/coolio/client.rb              |  2 +-
 lib/rainbows/coolio/heartbeat.rb           |  2 +-
 lib/rainbows/epoll/client.rb               |  4 ++--
 lib/rainbows/fiber.rb                      |  2 +-
 lib/rainbows/fiber/base.rb                 |  4 ++--
 lib/rainbows/fiber/coolio/heartbeat.rb     |  2 +-
 lib/rainbows/fiber/io.rb                   |  2 +-
 lib/rainbows/http_server.rb                |  1 +
 lib/rainbows/join_threads.rb               |  4 ++--
 lib/rainbows/process_client.rb             |  4 ++--
 lib/rainbows/stream_response_epoll.rb      |  2 +-
 lib/rainbows/thread_timeout.rb             |  8 ++++----
 lib/rainbows/xepoll_thread_pool/client.rb  |  4 ++--
 lib/rainbows/xepoll_thread_spawn/client.rb |  4 ++--
 rainbows.gemspec                           |  2 +-
 t/test_isolate.rb                          |  2 +-
 17 files changed, 41 insertions(+), 27 deletions(-)

[PATCH 1/3] http_server: add master_pid attribute

[Eric Wong]

We'll need this for unicorn 5, as they dropped this publically
accessible attribute while retaining the ivar.  Eventually we
may not have to check this attribute at all, instead detecting
parent death via worker pipe.
---
 lib/rainbows/http_server.rb | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/rainbows/http_server.rb b/lib/rainbows/http_server.rb
index 09f2589..429539c 100644
--- a/lib/rainbows/http_server.rb
+++ b/lib/rainbows/http_server.rb
@@ -8,6 +8,7 @@ class Rainbows::HttpServer < Unicorn::HttpServer
   attr_accessor :client_header_buffer_size
   attr_accessor :client_max_body_size
   attr_reader :use
+  attr_reader :master_pid
 
   def self.setup(block)
     Rainbows.server.instance_eval(&block)
-- 
EW

[PATCH 2/3] stream_response_epoll: remove hijack_prepare call

[Eric Wong]

unicorn 5.0.0 assumes Rack hijack is always available if
the application tries to use it, so the wrapper method
is removed.
---
 lib/rainbows/stream_response_epoll.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/rainbows/stream_response_epoll.rb b/lib/rainbows/stream_response_epoll.rb
index c36acaf..7c2f4f6 100644
--- a/lib/rainbows/stream_response_epoll.rb
+++ b/lib/rainbows/stream_response_epoll.rb
@@ -34,7 +34,7 @@ module Rainbows::StreamResponseEpoll
       headers.each do |key, value|
         case key
         when "rack.hijack"
-          hijack = hijack_prepare(value)
+          hijack = value
           body = nil # ensure we do not close body
         else
           if /\n/ =~ value
-- 
EW

[PATCH 3/3] bump to unicorn 5.0.1, use monotonic clock

[Eric Wong]

The timeout (mis)feature in unicorn uses the monotonic clock
if available.  We must follow suit to avoid having our timeout
functionality completely broken.
---
 lib/rainbows.rb                            | 19 ++++++++++++++++---
 lib/rainbows/coolio/client.rb              |  2 +-
 lib/rainbows/coolio/heartbeat.rb           |  2 +-
 lib/rainbows/epoll/client.rb               |  4 ++--
 lib/rainbows/fiber.rb                      |  2 +-
 lib/rainbows/fiber/base.rb                 |  4 ++--
 lib/rainbows/fiber/coolio/heartbeat.rb     |  2 +-
 lib/rainbows/fiber/io.rb                   |  2 +-
 lib/rainbows/join_threads.rb               |  4 ++--
 lib/rainbows/process_client.rb             |  4 ++--
 lib/rainbows/thread_timeout.rb             |  8 ++++----
 lib/rainbows/xepoll_thread_pool/client.rb  |  4 ++--
 lib/rainbows/xepoll_thread_spawn/client.rb |  4 ++--
 rainbows.gemspec                           |  2 +-
 t/test_isolate.rb                          |  2 +-
 15 files changed, 39 insertions(+), 26 deletions(-)

diff --git a/lib/rainbows.rb b/lib/rainbows.rb
index 0af5620..f23b387 100644
--- a/lib/rainbows.rb
+++ b/lib/rainbows.rb
@@ -75,8 +75,8 @@ module Rainbows
   end
 
   def self.tick
-    @worker.tick = Time.now.to_i
-    exit!(2) if @expire && Time.now >= @expire
+    @worker.tick = now.to_i
+    exit!(2) if @expire && now >= @expire
     @alive && @server.master_pid == Process.ppid or quit!
   end
 
@@ -88,7 +88,7 @@ module Rainbows
     unless @expire
       @alive = false
       Rainbows::HttpParser.quit
-      @expire = Time.now + (@server.timeout * 2.0)
+      @expire = now + (@server.timeout * 2.0)
       tmp = @readers.dup
       @readers.clear
       tmp.each { |s| s.close rescue nil }.clear
@@ -100,6 +100,19 @@ module Rainbows
     false
   end
 
+  # try to use the monotonic clock in Ruby >= 2.1, it is immune to clock
+  # offset adjustments and generates less garbage (Float vs Time object)
+  begin
+    Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    def self.now
+      Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    end
+  rescue NameError, NoMethodError
+    def self.now # Ruby <= 2.0
+      Rainbows.now
+    end
+  end
+
   autoload :Base, "rainbows/base"
   autoload :WriterThreadPool, "rainbows/writer_thread_pool"
   autoload :WriterThreadSpawn, "rainbows/writer_thread_spawn"
diff --git a/lib/rainbows/coolio/client.rb b/lib/rainbows/coolio/client.rb
index ad827f6..cd22c22 100644
--- a/lib/rainbows/coolio/client.rb
+++ b/lib/rainbows/coolio/client.rb
@@ -154,7 +154,7 @@ class Rainbows::Coolio::Client < Coolio::IO
         # buf == :wait_readable
         unless enabled?
           enable
-          KATO[self] = Time.now
+          KATO[self] = Rainbows.now
         end
       else
         on_read(''.freeze)
diff --git a/lib/rainbows/coolio/heartbeat.rb b/lib/rainbows/coolio/heartbeat.rb
index 3ae9e16..fcfbb0f 100644
--- a/lib/rainbows/coolio/heartbeat.rb
+++ b/lib/rainbows/coolio/heartbeat.rb
@@ -13,7 +13,7 @@ class Rainbows::Coolio::Heartbeat < Coolio::TimerWatcher
 
   def on_timer
     if (ot = KEEPALIVE_TIMEOUT) >= 0
-      ot = Time.now - ot
+      ot = Rainbows.now - ot
       KATO.delete_if { |client, time| time < ot and client.timeout? }
     end
     exit if (! Rainbows.tick && CONN.size <= 0)
diff --git a/lib/rainbows/epoll/client.rb b/lib/rainbows/epoll/client.rb
index 6dcbb81..fe04258 100644
--- a/lib/rainbows/epoll/client.rb
+++ b/lib/rainbows/epoll/client.rb
@@ -14,10 +14,10 @@ module Rainbows::Epoll::Client
   Rainbows.at_quit { KATO.each_key { |k| k.timeout! }.clear }
   Rainbows.config!(self, :keepalive_timeout)
   EP = Rainbows::EP
-  @@last_expire = Time.now
+  @@last_expire = Rainbows.now
 
   def self.expire
-    return if ((now = Time.now) - @@last_expire) < 1.0
+    return if ((now = Rainbows.now) - @@last_expire) < 1.0
     if (ot = KEEPALIVE_TIMEOUT) >= 0
       ot = now - ot
       KATO.delete_if { |client, time| time < ot and client.timeout! }
diff --git a/lib/rainbows/fiber.rb b/lib/rainbows/fiber.rb
index a0f3f99..a37512f 100644
--- a/lib/rainbows/fiber.rb
+++ b/lib/rainbows/fiber.rb
@@ -29,7 +29,7 @@ module Rainbows::Fiber
   # right?).  Calling this directly is deprecated, use
   # Rainbows.sleep(seconds) instead.
   def self.sleep(seconds)
-    ZZ[Fiber.current] = Time.now + seconds
+    ZZ[Fiber.current] = Rainbows.now + seconds
     Fiber.yield
   end
 
diff --git a/lib/rainbows/fiber/base.rb b/lib/rainbows/fiber/base.rb
index e8f5b16..7c4fb59 100644
--- a/lib/rainbows/fiber/base.rb
+++ b/lib/rainbows/fiber/base.rb
@@ -40,7 +40,7 @@ module Rainbows::Fiber::Base
   # woken and returns an interval to IO.select on
   def schedule_sleepers
     max = nil
-    now = Time.now
+    now = Rainbows.now
     fibs = []
     ZZ.delete_if { |fib, time|
       if now >= time
@@ -54,7 +54,7 @@ module Rainbows::Fiber::Base
 
     max_sleep = 1.0 # wake up semi-frequently to prevent SIGKILL from master
     if max
-      max -= Time.now
+      max -= Rainbows.now
       return 0 if max < 0.0
       return max_sleep if max > max_sleep
       max
diff --git a/lib/rainbows/fiber/coolio/heartbeat.rb b/lib/rainbows/fiber/coolio/heartbeat.rb
index 6b1e4f9..7014dbe 100644
--- a/lib/rainbows/fiber/coolio/heartbeat.rb
+++ b/lib/rainbows/fiber/coolio/heartbeat.rb
@@ -5,7 +5,7 @@ class Rainbows::Fiber::Coolio::Heartbeat < Coolio::TimerWatcher
   ZZ = Rainbows::Fiber::ZZ
   def on_timer
     exit if (! Rainbows.tick && Rainbows.cur <= 0)
-    now = Time.now
+    now = Rainbows.now
     fibs = []
     ZZ.delete_if { |fib, time| now >= time ? fibs << fib : ! fib.alive? }
     fibs.each { |fib| fib.resume if fib.alive? }
diff --git a/lib/rainbows/fiber/io.rb b/lib/rainbows/fiber/io.rb
index b121854..111132f 100644
--- a/lib/rainbows/fiber/io.rb
+++ b/lib/rainbows/fiber/io.rb
@@ -63,7 +63,7 @@ class Rainbows::Fiber::IO
     expire = nil
     case rv = Kgio.tryread(@to_io, 16384, buf)
     when :wait_readable
-      return if expire && expire < Time.now
+      return if expire && expire < Rainbows.now
       expire ||= read_expire
       kgio_wait_readable
     else
diff --git a/lib/rainbows/join_threads.rb b/lib/rainbows/join_threads.rb
index 7ab2004..2ae6aa4 100644
--- a/lib/rainbows/join_threads.rb
+++ b/lib/rainbows/join_threads.rb
@@ -5,12 +5,12 @@ module Rainbows::JoinThreads
 
   # blocking acceptor threads must be forced to run
   def self.acceptors(threads)
-    expire = Time.now + Rainbows.server.timeout
+    expire = Rainbows.now + Rainbows.server.timeout
     threads.delete_if do |thr|
       Rainbows.tick
       begin
         # blocking accept() may not wake up properly
-        thr.raise(Errno::EINTR) if Time.now > expire && thr.stop?
+        thr.raise(Errno::EINTR) if Rainbows.now > expire && thr.stop?
 
         thr.run
         thr.join(0.01)
diff --git a/lib/rainbows/process_client.rb b/lib/rainbows/process_client.rb
index a39d6cd..4bf9878 100644
--- a/lib/rainbows/process_client.rb
+++ b/lib/rainbows/process_client.rb
@@ -9,7 +9,7 @@ module Rainbows::ProcessClient
   Rainbows.config!(self, :client_header_buffer_size, :keepalive_timeout)
 
   def read_expire
-    Time.now + KEEPALIVE_TIMEOUT
+    Rainbows.now + KEEPALIVE_TIMEOUT
   end
 
   # used for reading headers (respecting keepalive_timeout)
@@ -18,7 +18,7 @@ module Rainbows::ProcessClient
     begin
       case rv = kgio_tryread(CLIENT_HEADER_BUFFER_SIZE, buf)
       when :wait_readable
-        return if expire && expire < Time.now
+        return if expire && expire < Rainbows.now
         expire ||= read_expire
         kgio_wait_readable(KEEPALIVE_TIMEOUT)
       else
diff --git a/lib/rainbows/thread_timeout.rb b/lib/rainbows/thread_timeout.rb
index 8348272..8739d29 100644
--- a/lib/rainbows/thread_timeout.rb
+++ b/lib/rainbows/thread_timeout.rb
@@ -63,7 +63,7 @@ class Rainbows::ThreadTimeout
 
   # The MRI 1.8 won't be usable in January 2038, we'll raise this
   # when we eventually drop support for 1.8 (before 2038, hopefully)
-  NEVER = Time.at(0x7fffffff)
+  NEVER = 0x7fffffff
 
   def initialize(app, opts)
     # @timeout must be Numeric since we add this to Time
@@ -114,7 +114,7 @@ class Rainbows::ThreadTimeout
     # is hopeless and we might as well just die anyways.
     # initialize guarantees @timeout will be Numeric
     start_watchdog(env) unless @watchdog
-    @active[Thread.current] = Time.now + @timeout
+    @active[Thread.current] = Rainbows.now + @timeout
 
     begin
       # It is important to unlock inside this begin block
@@ -162,7 +162,7 @@ class Rainbows::ThreadTimeout
         # that are about to release themselves from the eye of the
         # watchdog thread.
         @lock.synchronize do
-          now = Time.now
+          now = Rainbows.now
           @active.delete_if do |thread, expire_at|
             # We also use this loop to get the maximum possible time to
             # sleep for if we're not killing the thread.
@@ -184,7 +184,7 @@ class Rainbows::ThreadTimeout
           sleep(@timeout)
         else
           # sleep until the next known thread is about to expire.
-          sec = next_expiry - Time.now
+          sec = next_expiry - Rainbows.now
           sec > 0.0 ? sleep(sec) : Thread.pass # give other threads a chance
         end
       rescue => e
diff --git a/lib/rainbows/xepoll_thread_pool/client.rb b/lib/rainbows/xepoll_thread_pool/client.rb
index 001e69d..760bbde 100644
--- a/lib/rainbows/xepoll_thread_pool/client.rb
+++ b/lib/rainbows/xepoll_thread_pool/client.rb
@@ -46,7 +46,7 @@ module Rainbows::XEpollThreadPool::Client
     LOCK.synchronize { clients = KATO.keys; KATO.clear }
     clients.each { |io| io.closed? or io.close }
   end
-  @@last_expire = Time.now
+  @@last_expire = Rainbows.now
 
   def kato_set
     LOCK.synchronize { KATO[self] = @@last_expire }
@@ -70,7 +70,7 @@ module Rainbows::XEpollThreadPool::Client
   end
 
   def self.expire
-    return if ((now = Time.now) - @@last_expire) < 1.0
+    return if ((now = Rainbows.now) - @@last_expire) < 1.0
     if (ot = KEEPALIVE_TIMEOUT) >= 0
       ot = now - ot
       defer = []
diff --git a/lib/rainbows/xepoll_thread_spawn/client.rb b/lib/rainbows/xepoll_thread_spawn/client.rb
index a3277cf..67c5976 100644
--- a/lib/rainbows/xepoll_thread_spawn/client.rb
+++ b/lib/rainbows/xepoll_thread_spawn/client.rb
@@ -35,7 +35,7 @@ module Rainbows::XEpollThreadSpawn::Client
     LOCK.synchronize { clients = KATO.keys; KATO.clear }
     clients.each { |io| io.closed? or io.shutdown }
   end
-  @@last_expire = Time.now
+  @@last_expire = Rainbows.now
 
   def kato_set
     LOCK.synchronize { KATO[self] = @@last_expire }
@@ -59,7 +59,7 @@ module Rainbows::XEpollThreadSpawn::Client
   end
 
   def self.expire
-    return if ((now = Time.now) - @@last_expire) < 1.0
+    return if ((now = Rainbows.now) - @@last_expire) < 1.0
     if (ot = KEEPALIVE_TIMEOUT) >= 0
       ot = now - ot
       defer = []
diff --git a/rainbows.gemspec b/rainbows.gemspec
index afc8b5a..6ddf630 100644
--- a/rainbows.gemspec
+++ b/rainbows.gemspec
@@ -27,7 +27,7 @@ Gem::Specification.new do |s|
   # we need unicorn for the HTTP parser and process management
   # we need unicorn 4.8.0+ since we depend on undocumented/unsupported
   # unicorn internals.
-  s.add_dependency(%q<unicorn>, ["~> 4.8"])
+  s.add_dependency(%q<unicorn>, ["~> 5.0"])
 
   s.add_development_dependency(%q<isolate>, "~> 3.1")
   s.add_development_dependency(%q<olddoc>, "~> 1.0")
diff --git a/t/test_isolate.rb b/t/test_isolate.rb
index 98918f6..58bc4dc 100644
--- a/t/test_isolate.rb
+++ b/t/test_isolate.rb
@@ -20,7 +20,7 @@ Isolate.now!(opts) do
   gem 'rack', '1.6.4'
   gem 'kcar', '0.5.0'
   gem 'raindrops', '0.13.0'
-  gem 'unicorn', '4.9.0'
+  gem 'unicorn', '5.0.1'
 
   if engine == "ruby"
     gem 'sendfile', '1.2.2'
-- 
EW