[PATCH 0/6] misc updates for Ruby-trunk and gem changes

[PATCH 0/6] misc updates for Ruby-trunk and gem changes

[Eric Wong]

Ruby keeps changing, and so do gems (and I'm to blame for the
kgio change) which requires removing t0044.

Oh well.  Maybe I'll add MSG_MORE support to Rainbows! someday,
but yahns already supports it :)

Eric Wong (6):
      build: fix quoting issue with double parens
      response: avoid unnecessary args to IO.copy_stream
      t/close-has-env.ru: ensure close is idempotent
      sync_close: This fix breakage from Ruby-trunk r50118
      t/t0044-autopush.sh: remove test
      t/test_isolate.rb: updates for various gem versions

 lib/rainbows/response.rb   |   2 +-
 lib/rainbows/sync_close.rb |  12 ++++-
 t/GNUmakefile              |   2 +-
 t/close-has-env.ru         |   1 +
 t/t0044-autopush.sh        | 121 ---------------------------------------------
 t/test_isolate.rb          |  16 +++---
 6 files changed, 21 insertions(+), 133 deletions(-)

[PATCH 1/6] build: fix quoting issue with double parens

[Eric Wong]

Apparently GNU make parses that strangely and mangles it for the
shell...
---
 t/GNUmakefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/t/GNUmakefile b/t/GNUmakefile
index c456877..97b1985 100644
--- a/t/GNUmakefile
+++ b/t/GNUmakefile
@@ -42,7 +42,7 @@ ifeq ($(RUBY_ENGINE),ruby)
     models += FiberSpawn
     models += FiberPool
 
-    RUBY_LE_2_1 := $(shell $(RUBY) -e 'puts((RUBY_VERSION.to_f <= 2.1).to_s))')
+    RUBY_LE_2_1 := $(shell $(RUBY) -e 'puts(RUBY_VERSION.to_f <= 2.1)')
     ifeq ($(RUBY_LE_2_1), true)
       models += Coolio
       models += CoolioThreadPool
-- 
EW

[PATCH 2/6] response: avoid unnecessary args to IO.copy_stream

[Eric Wong]

Not all responses are seekable, so do not attempt to pass seek
arguments to them since Ruby may attempt to seek (and fail!).
---
 lib/rainbows/response.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/rainbows/response.rb b/lib/rainbows/response.rb
index 8a0daf8..8d0de1d 100644
--- a/lib/rainbows/response.rb
+++ b/lib/rainbows/response.rb
@@ -129,7 +129,7 @@ module Rainbows::Response
       module CopyStream
         def write_body_file(body, range)
           range ? COPY_STREAM.copy_stream(body, self, range[1], range[0]) :
-                  COPY_STREAM.copy_stream(body, self, nil, 0)
+                  COPY_STREAM.copy_stream(body, self)
         end
       end
       include CopyStream
-- 
EW

[PATCH 3/6] t/close-has-env.ru: ensure close is idempotent

[Eric Wong]

Ruby 2.3.0 will have idempotent IO#close, so closing an IO
twice will be idempotent and not raise an IOError exception.
Ensure we do not rely on the IOError exception to catch
our own errors.
---
 t/close-has-env.ru | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/close-has-env.ru b/t/close-has-env.ru
index 471f605..3b6ec88 100644
--- a/t/close-has-env.ru
+++ b/t/close-has-env.ru
@@ -10,6 +10,7 @@ class ClosablePipe < ::IO
   end
 
   def close
+    return if closed? # idempotent for Ruby 2.3.0 compatibility
     super
     $stdout.syswrite "path_info=#{@env['PATH_INFO']}\n"
   end
-- 
EW

[PATCH 4/6] sync_close: This fix breakage from Ruby-trunk r50118

[Eric Wong]

By providing read+readpartial methods

IO.copy_stream behaves slightly differently when operating on
non-IO-subclassed objects nowadays.

Ref:
> * io.c (copy_stream_body): use the arguments without conversion if
>   having read, readpartial, and write methods, than conversion by
>   to_path method.  [ruby-core:68676] [Bug #11015]
---
 lib/rainbows/sync_close.rb | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/lib/rainbows/sync_close.rb b/lib/rainbows/sync_close.rb
index 75f119e..8738cae 100644
--- a/lib/rainbows/sync_close.rb
+++ b/lib/rainbows/sync_close.rb
@@ -16,6 +16,14 @@ class Rainbows::SyncClose
     @body.respond_to?(m)
   end
 
+  def readpartial(*args)
+    @body.readpartial(*args)
+  end
+
+  def read(*args)
+    @body.read(*args)
+  end
+
   def to_path
     @body.to_path
   end
@@ -31,7 +39,7 @@ class Rainbows::SyncClose
   # called by the writer thread to wake up the original thread (in #initialize)
   def close
     @body.close
-    ensure
-      @mutex.synchronize { @cv.signal }
+  ensure
+    @mutex.synchronize { @cv.signal }
   end
 end
-- 
EW

[PATCH 5/6] t/t0044-autopush.sh: remove test

[Eric Wong]

kgio no longer does anything with autopush, so this test is
broken.  Autopush was overkill and badly done (MSG_MORE is
better on Linux, FreeBSD should copy it :P).
---
 t/t0044-autopush.sh | 121 ----------------------------------------------------
 1 file changed, 121 deletions(-)
 delete mode 100644 t/t0044-autopush.sh

diff --git a/t/t0044-autopush.sh b/t/t0044-autopush.sh
deleted file mode 100644
index 103f9fc..0000000
--- a/t/t0044-autopush.sh
+++ /dev/null
@@ -1,121 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-STRACE=$(which strace 2>/dev/null || :)
-if ! test -x "$STRACE"
-then
-	t_info "strace not found, skipping $T"
-	exit 0
-fi
-if test x"$(uname -s)" != xLinux
-then
-	t_info "Linux is the only supported OS for $T"
-	exit 0
-fi
-
-# these buffer internally in external libraries, so we can't detect when
-# to use TCP_CORK
-skip_models EventMachine NeverBlock
-skip_models StreamResponseEpoll
-skip_models Coolio CoolioThreadPool CoolioThreadSpawn
-skip_models Revactor Rev RevThreadPool RevThreadSpawn
-
-# not sure why, but we don't have time to care about Ruby 1.8 too much
-case $RUBY_VERSION in
-1.8.*) skip_models WriterThreadSpawn WriterThreadPool ;;
-esac
-
-t_plan 13 "Kgio autopush tests"
-
-start_strace () {
-	# dbgcat strace_out
-	> $strace_out
-	sleep 1
-	strace -p $worker_pid -e '!futex' -f -o $strace_out &
-	strace_pid=$!
-	while ! test -s $strace_out; do sleep 1; done
-}
-
-check_TCP_CORK () {
-	nr=0
-	while test 2 -gt $(grep TCP_CORK $strace_out | count_lines)
-	do
-		nr=$(( $nr + 1 ))
-		if test $nr -gt 30
-		then
-			dbgcat strace_out
-			die "waited too long ($nr seconds) for TCP_CORK"
-		fi
-		sleep 1
-	done
-
-	test 2 -eq $(grep TCP_CORK $strace_out | count_lines)
-	fgrep 'SOL_TCP, TCP_CORK, [0],' $strace_out
-	fgrep 'SOL_TCP, TCP_CORK, [1],' $strace_out
-}
-
-t_begin "setup and start" && {
-	rainbows_setup $model 1 1
-	rtmpfiles strace_out
-	ed -s $unicorn_config <<EOF
-,s/^listen.*/listen "$listen", :tcp_nodelay => true, :tcp_nopush => true/
-w
-EOF
-	rainbows -D large-file-response.ru -c $unicorn_config -E none
-	rainbows_wait_start
-}
-
-t_begin "read worker pid" && {
-	worker_pid=$(curl -sSf http://$listen/pid)
-	kill -0 $worker_pid
-}
-
-t_begin "start strace on worker" && start_strace
-
-t_begin "reading RSS uncorks" && {
-	curl -sSf http://$listen/rss >/dev/null
-}
-
-t_begin "restart strace on worker" && {
-	kill $strace_pid
-	wait
-	start_strace
-}
-
-t_begin "reading static file uncorks" && {
-	curl -sSf http://$listen/random_blob >/dev/null
-	check_TCP_CORK
-}
-
-t_begin "stop strace on worker" && {
-	kill $strace_pid
-	wait
-}
-
-t_begin "enable sendfile" && {
-	echo >> $unicorn_config 'require "sendfile"'
-	kill -HUP $rainbows_pid
-	test xSTART = x"$(cat $fifo)"
-}
-
-t_begin "reread worker pid" && {
-	worker_pid=$(curl -sSf http://$listen/pid)
-	kill -0 $worker_pid
-}
-
-t_begin "restart strace on the worker" && start_strace
-
-t_begin "HTTP/1.x GET on static file with sendfile uncorks" && {
-	curl -sSf http://$listen/random_blob >/dev/null
-	check_TCP_CORK
-}
-
-t_begin "killing succeeds" && {
-	kill $strace_pid
-	wait
-	# dbgcat strace_out
-	kill $rainbows_pid
-}
-
-t_begin "check stderr" && check_stderr
-
-t_done
-- 
EW

[PATCH 6/6] t/test_isolate.rb: updates for various gem versions

[Eric Wong]

---
 t/test_isolate.rb | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/t/test_isolate.rb b/t/test_isolate.rb
index 4ae4894..98918f6 100644
--- a/t/test_isolate.rb
+++ b/t/test_isolate.rb
@@ -16,18 +16,18 @@ $stdout.reopen($stderr)
 lock = File.open(__FILE__, "rb")
 lock.flock(File::LOCK_EX)
 Isolate.now!(opts) do
-  gem 'kgio', '2.9.2'
-  gem 'rack', '1.5.2'
-  gem 'kcar', '0.4.0'
+  gem 'kgio', '2.10.0'
+  gem 'rack', '1.6.4'
+  gem 'kcar', '0.5.0'
   gem 'raindrops', '0.13.0'
-  gem 'unicorn', '4.8.3'
+  gem 'unicorn', '4.9.0'
 
   if engine == "ruby"
     gem 'sendfile', '1.2.2'
+    gem 'eventmachine', '1.0.8'
+    gem 'async_sinatra', '1.2.0'
     if RUBY_VERSION.to_f < 2.2
       gem 'cool.io', '1.1.0'
-      gem 'eventmachine', '1.0.3'
-      gem 'async_sinatra', '1.0.0'
       gem 'neverblock', '0.1.6.2'
     end
   end
@@ -40,10 +40,10 @@ Isolate.now!(opts) do
   end
 
   if RUBY_PLATFORM =~ /linux/
-    gem 'sleepy_penguin', '3.3.0'
+    gem 'sleepy_penguin', '3.4.1'
 
     # is 2.6.32 new enough?
-    gem 'io_splice', '4.3.0' if `uname -r`.strip > '2.6.32'
+    gem 'io_splice', '4.4.0' if `uname -r`.strip > '2.6.32'
   end
 end
 
-- 
EW

Re: [PATCH 5/6] t/t0044-autopush.sh: remove test

[Eric Wong]

Btw, I've reverted this removal with the release of kgio 2.11.0
and commit bd7596e50bd094edf5e5842afb8239c158fe2491

   Revert "t/t0044-autopush.sh: remove test"

   Stuck between a rock and a hard place..
   This reverts commit 5c911ab2470a2ca9c140b3a2e20ae810bf758590
   and restores autopush support when using kgio 2.11.0+

   Explanations on the kgio mailing list below:

   https://bogomips.org/kgio-public/20161216-kgio-2.11.0-released@fail/
   https://bogomips.org/kgio-public/20161216000110.GA7366@untitled/T/#u

   We can drop kgio when we drop Ruby <= 2.2 support in a few years.

https://bogomips.org/rainbows.git/patch?id=bd7596e50bd0

Oh well...