raindrops RubyGem user+dev discussion/patches/pulls/bugs/help
 help / Atom feed
* [PATCH 0/3] aggregate/pmq: various fixes and cleanups
@ 2017-03-16  3:14 Eric Wong
  2017-03-16  3:14 ` [PATCH 1/3] aggregate/pmq: avoid false sharing of lock buffers Eric Wong
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Eric Wong @ 2017-03-16  3:14 UTC (permalink / raw)
  To: raindrops-public

Now that we rely on Ruby 1.9+, we can rely on IO.copy_stream
and remove the io-extra requirement.  Future Rubies may even
have pread/pwrite support natively.

Eric Wong (3):
      aggregate/pmq: avoid false sharing of lock buffers
      aggregate/pmq: remove io-extra requirement
      aggregate/pmq: avoid File#stat allocation

 lib/raindrops/aggregate/pmq.rb | 22 ++++++++++++++--------
 raindrops.gemspec              |  1 -
 2 files changed, 14 insertions(+), 9 deletions(-)


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 1/3] aggregate/pmq: avoid false sharing of lock buffers
  2017-03-16  3:14 [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong
@ 2017-03-16  3:14 ` Eric Wong
  2017-03-16  3:14 ` [PATCH 2/3] aggregate/pmq: remove io-extra requirement Eric Wong
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2017-03-16  3:14 UTC (permalink / raw)
  To: raindrops-public

And rely on frozen string optimizations in Ruby while we're at it.
---
 lib/raindrops/aggregate/pmq.rb | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/lib/raindrops/aggregate/pmq.rb b/lib/raindrops/aggregate/pmq.rb
index a2dd45e..f543302 100644
--- a/lib/raindrops/aggregate/pmq.rb
+++ b/lib/raindrops/aggregate/pmq.rb
@@ -39,9 +39,9 @@ class Raindrops::Aggregate::PMQ
   # :stopdoc:
   # These constants are for Linux.  This is designed for aggregating
   # TCP_INFO.
-  RDLOCK = [ Fcntl::F_RDLCK ].pack("s @256")
-  WRLOCK = [ Fcntl::F_WRLCK ].pack("s @256")
-  UNLOCK = [ Fcntl::F_UNLCK ].pack("s @256")
+  RDLOCK = [ Fcntl::F_RDLCK ].pack("s @256".freeze).freeze
+  WRLOCK = [ Fcntl::F_WRLCK ].pack("s @256".freeze).freeze
+  UNLOCK = [ Fcntl::F_UNLCK ].pack("s @256".freeze).freeze
   # :startdoc:
 
   # returns the number of dropped messages sent to a POSIX message
@@ -185,10 +185,12 @@ def lock! io, type # :nodoc:
   def synchronize io, type # :nodoc:
     @mutex.synchronize do
       begin
+        type = type.dup
         lock! io, type
         yield io
       ensure
-        lock! io, UNLOCK
+        lock! io, type.replace(UNLOCK)
+        type.clear
       end
     end
   end
-- 
EW


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 2/3] aggregate/pmq: remove io-extra requirement
  2017-03-16  3:14 [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong
  2017-03-16  3:14 ` [PATCH 1/3] aggregate/pmq: avoid false sharing of lock buffers Eric Wong
@ 2017-03-16  3:14 ` Eric Wong
  2017-03-16  3:14 ` [PATCH 3/3] aggregate/pmq: avoid File#stat allocation Eric Wong
  2017-03-21 23:48 ` [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2017-03-16  3:14 UTC (permalink / raw)
  To: raindrops-public

IO.copy_stream is standard in 1.9+ and can use pread when
given an offset.  We do not need to use pwrite with fcntl
locking, actually.
---
 lib/raindrops/aggregate/pmq.rb | 12 ++++++++----
 raindrops.gemspec              |  1 -
 2 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/lib/raindrops/aggregate/pmq.rb b/lib/raindrops/aggregate/pmq.rb
index f543302..5a6a1f6 100644
--- a/lib/raindrops/aggregate/pmq.rb
+++ b/lib/raindrops/aggregate/pmq.rb
@@ -3,8 +3,8 @@
 require "aggregate"
 require "posix_mq"
 require "fcntl"
-require "io/extra"
 require "thread"
+require "stringio"
 
 # \Aggregate + POSIX message queues support for Ruby 1.9 and \Linux
 #
@@ -19,7 +19,6 @@
 # or libraries:
 #
 # * aggregate (tested with 0.2.2)
-# * io-extra  (tested with 1.2.3)
 # * posix_mq  (tested with 1.0.0)
 #
 # == Design
@@ -84,6 +83,7 @@ def initialize(params = {})
       @wr = File.open(t.path, "wb")
       @rd = File.open(t.path, "rb")
     end
+    @wr.sync = true
     @cached_aggregate = @aggregate
     flush_master
     @mq_send = if opts[:lossy]
@@ -151,7 +151,10 @@ def aggregate
     @cached_aggregate ||= begin
       flush
       Marshal.load(synchronize(@rd, RDLOCK) do |rd|
-        IO.pread rd.fileno, rd.stat.size, 0
+        dst = StringIO.new
+        dst.binmode
+        IO.copy_stream(rd, dst, rd.stat.size, 0)
+        dst.string
       end)
     end
   end
@@ -163,7 +166,8 @@ def flush_master
     dump = Marshal.dump @aggregate
     synchronize(@wr, WRLOCK) do |wr|
       wr.truncate 0
-      IO.pwrite wr.fileno, dump, 0
+      wr.rewind
+      wr.write(dump)
     end
   end
 
diff --git a/raindrops.gemspec b/raindrops.gemspec
index c00a6b5..4651fa9 100644
--- a/raindrops.gemspec
+++ b/raindrops.gemspec
@@ -22,7 +22,6 @@
   s.test_files = test_files
   s.add_development_dependency('aggregate', '~> 0.2')
   s.add_development_dependency('test-unit', '~> 3.0')
-  s.add_development_dependency('io-extra', [ '~> 1.2', '>= 1.2.3'])
   s.add_development_dependency('posix_mq', '~> 2.0')
   s.add_development_dependency('rack', [ '>= 1.2', '< 3.0' ])
   s.add_development_dependency('olddoc', '~> 1.0')
-- 
EW


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH 3/3] aggregate/pmq: avoid File#stat allocation
  2017-03-16  3:14 [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong
  2017-03-16  3:14 ` [PATCH 1/3] aggregate/pmq: avoid false sharing of lock buffers Eric Wong
  2017-03-16  3:14 ` [PATCH 2/3] aggregate/pmq: remove io-extra requirement Eric Wong
@ 2017-03-16  3:14 ` Eric Wong
  2017-03-21 23:48 ` [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2017-03-16  3:14 UTC (permalink / raw)
  To: raindrops-public

File#size is available in modern Rubies so the extra syscall
is avoided.
---
 lib/raindrops/aggregate/pmq.rb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lib/raindrops/aggregate/pmq.rb b/lib/raindrops/aggregate/pmq.rb
index 5a6a1f6..98d4169 100644
--- a/lib/raindrops/aggregate/pmq.rb
+++ b/lib/raindrops/aggregate/pmq.rb
@@ -153,7 +153,7 @@ def aggregate
       Marshal.load(synchronize(@rd, RDLOCK) do |rd|
         dst = StringIO.new
         dst.binmode
-        IO.copy_stream(rd, dst, rd.stat.size, 0)
+        IO.copy_stream(rd, dst, rd.size, 0)
         dst.string
       end)
     end
-- 
EW


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH 0/3] aggregate/pmq: various fixes and cleanups
  2017-03-16  3:14 [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong
                   ` (2 preceding siblings ...)
  2017-03-16  3:14 ` [PATCH 3/3] aggregate/pmq: avoid File#stat allocation Eric Wong
@ 2017-03-21 23:48 ` Eric Wong
  3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2017-03-21 23:48 UTC (permalink / raw)
  To: raindrops-public

On a related note, posix_mq got some fixes found when testing
aggregate/pmq under FreeBSD:

https://bogomips.org/ruby-posix-mq/20170320-posix-mq-2.4.0-rele@sed-for-ruby/

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-16  3:14 [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong
2017-03-16  3:14 ` [PATCH 1/3] aggregate/pmq: avoid false sharing of lock buffers Eric Wong
2017-03-16  3:14 ` [PATCH 2/3] aggregate/pmq: remove io-extra requirement Eric Wong
2017-03-16  3:14 ` [PATCH 3/3] aggregate/pmq: avoid File#stat allocation Eric Wong
2017-03-21 23:48 ` [PATCH 0/3] aggregate/pmq: various fixes and cleanups Eric Wong

raindrops RubyGem user+dev discussion/patches/pulls/bugs/help

Archives are clonable:
	git clone --mirror https://bogomips.org/raindrops-public
	git clone --mirror http://ou63pmih66umazou.onion/raindrops-public

Newsgroups are available over NNTP:
	nntp://news.public-inbox.org/inbox.comp.lang.ruby.raindrops
	nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.raindrops

 note: .onion URLs require Tor: https://www.torproject.org/
       or Tor2web: https://www.tor2web.org/

AGPL code for this site: git clone https://public-inbox.org/ public-inbox