ChangeLog from bogomips.org/ruby_io_splice.git (v1.0.0..v4.1.1)
commit 45436d6251b3310e3ac02f1590e473a2602762ac
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue May 17 17:15:49 2011 -0700
io_splice 4.1.1 - workaround socket -> pipe issues
IO::Splice.copy_stream now uses SPLICE_F_NONBLOCK whenever the
destination is a pipe. Our documentation is also updated to
reflect the issue of splicing from sockets into pipes.
ref: http://lkml.org/lkml/2009/1/13/478
commit 1e4b4af9d8fa01b59b933fe94b405288f668a7fc
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue May 17 17:12:48 2011 -0700
expand doc for non-blocking splice into a pipe
We should warn our users to avoid a blocking splice() from a
socket into a pipe if the socket buffers are full unless there's
a blocking read/splice on the other end of the pipe.
commit 86508cdc09ef36d8c5ae708ef9e3e19a82844e98
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 16 20:18:12 2011 -0700
copy_stream: always use SPLICE_F_NONBLOCK for partial copy
The pipe may be full from small buffers due to how TCP
works, so we need to use non-blocking I/O on the pipe.
ref: http://lkml.org/lkml/2009/1/13/478
commit b9cf769a639d22e63d2c611541000b46e5d07d45
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 16 09:28:48 2011 -0700
io_splice 4.1.0 - copy_stream enhancement for 1.9
IO::Splice.copy_stream coerces based on the availability of
the #to_path method in source and destination arguments.
This is to be compatible with IO.copy_stream and also
Rack::File since this may be used in web servers.
This is NOT visible with Ruby 1.8 since File.open
doesn't coerce with #to_path on the given object.
commit e662a7f7262cef0820c014ae533bdf199f0104d6
Author: Eric Wong <normalperson@yhbt.net>
Date: Fri May 13 12:59:13 2011 -0700
copy_stream coerces based on #to_path
This is to be compatible with IO.copy_stream and also
Rack::File since this may be used in web servers.
This is NOT compatible with Ruby 1.8 since File.open
doesn't coerce with #to_path on the given object.
commit fdec276346fb0aa764499f7ad0483b6ba51685a5
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu May 12 19:39:12 2011 -0700
io_splice 4.0.0 - easier to use!
Methods now automatically retry on EINTR after checking
IO#closed? and executing interrupts.
There is a new IO::Splice::WAITALL flag (implemented in
userspace) which may be used to cause IO.splice and IO.tee
to retry (while the GVL is released) until the operation
is complete.
IO::Splice.copy_stream now works correctly for large copies
if source offset is specified.
MRI 1.8 is only weakly supported now, MRI 1.9 is the
preferred platform though Rubinius appears to work, too.
commit 48bfb5ce0706334fc8ba42e2f517623fbe90d8b4
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu May 12 19:28:41 2011 -0700
fix up documentation for fd vs IO objects
We support both, but IO objects are easier to use so
refer to those in the documentation.
commit 1ec67bc61d534232b5eec5c3f32da362898c6508
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu May 12 19:13:57 2011 -0700
copy_stream: enough to get this working under MRI 1.8
This way at least 1.8 users can have something akin to
IO.copy_stream.
commit 83bdc390d98e9af4439c6db92679a9a84eb9f52a
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu May 12 18:59:44 2011 -0700
test_copy_stream: fix userspace buffering assumption
And enable UNIX socket tests for this
commit ec812c7f2d83681b7beecdf21334e3a43c2b7090
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu May 12 17:25:08 2011 -0700
add copy_stream test from MRI
This helps flesh out some inconsistencies between our
implementation and IO.copy_stream in MRI. Some things
we won't care about though, like userspace buffering.
commit e887427daad73f460c1e7abbbefecc22cddbadc3
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu May 12 17:22:57 2011 -0700
copy_stream: update src_offset on retries
Oops :x
commit 7dd97ae08a4c6755427a4e2162d4232cfd1d8cac
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue May 10 12:42:38 2011 -0700
copy_stream attempts to use IO::Splice::WAITALL
This reduces Ruby method dispatches
commit b8b2d02b56e29466a0437b4bb8e8e0608a933f89
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue May 10 12:36:59 2011 -0700
test_io_splice: disable timeout-related tests in non-MRI
timeout doesn't seem to interrupt C extensions reliably
in some cases...
commit 5e73ae9c477d0ba64f497f0517be20e65ba0c253
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 9 17:57:10 2011 -0700
add IO::Splice::WAITALL flag support
This allows splice-in-full and tee-in-full behavior
to simplify user code.
commit 40b985ed0a6d103f2bf123ff6c288e5617b76999
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 9 13:54:54 2011 -0700
test_io_splice_eintr: beef up test
Also works under Rubinius
commit 7a7ea76be30e212c35132829959c0b57e5acb621
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 9 13:45:28 2011 -0700
use ssize_t/size_t instead of long/unsigned long
Easier to read this way since it matches upstream
functions.
commit c797cea2cdddbbc5534be54ac37d95d0c8af6e61
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 9 13:43:39 2011 -0700
EINTR test only works under 1.9
Not that we feel like supporting 1.8 any longer..
commit 4cbd3c3ea9e251eadd74e0971b0510fc05f95aa4
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 9 13:28:19 2011 -0700
detect closed descriptors on EINTR
We don't want to retry with the same FD if it somehow
got closed and reopened in a different thread. We
could possibly have Errno::EBADF, too, but
rb_thread_io_blocking_region doesn't allow multiple
file descriptors...
commit 184f664d0773ea41c56b9db8dcf69878b9713855
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 9 13:18:42 2011 -0700
my_fileno: use FIX2INT instead of NUM2INT
No chance in hell a file descriptor can be a Bignum...
commit af9da960a4b2683d27c946c308ae105958abdd26
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon May 9 13:12:25 2011 -0700
retry on EINTR
This makes things consistent with Ruby core IO methods
commit 0b07f9c8ac10219590beebb9038f92efd6c3ebaf
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun May 1 14:11:11 2011 -0700
io_splice 3.1.0 - IO::Splice.copy_stream improvement
IO::Splice.copy_stream no longer modifies the actual file offset
of the given source file (if it's a regular file). This follows
IO.copy_stream and IO#sendfile semantics, allowing multiple
threads/processes to simultaneously stream a single regular file
descriptor to multiple sockets/pipes.
commit 5e544bc745e5216f2455fa0ad398e39097debe60
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun May 1 13:58:23 2011 -0700
minor documentation updates
IO::Splice.copy_stream is more useful, now
commit b5ec116e331c22c064bee46f63145e325785e33f
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun May 1 13:29:51 2011 -0700
IO::Splice.copy_stream doesn't change offset of source file
This is for compatibility with IO.copy_stream and IO#sendfile,
both of which use pread()-semantics when given a source file
offset. This allows multiple threads/processes to copy from
the same file handle.
commit b39448e121f8f060c274ee1ee3c473f7b20d92e1
Author: Eric Wong <normalperson@yhbt.net>
Date: Wed Apr 27 11:36:33 2011 -0700
extconf: remove unnecessary dir_config statement
I didn't know mkmf at the time :<
commit 73b61c10c0085ee4a26a321375938bce16fb8a6d
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue Mar 1 09:29:57 2011 +0000
io_splice 3.0.0 - kinder, gentler Linux zero-copy
This release adds the IO.trysplice and IO.trytee interfaces
to avoid expensive EAGAIN exceptions for non-blocking I/O.
There is no IO.tryvmsplice method as we still haven't figured
out a good use for IO.vmsplice in Ruby, and non-blocking I/O
with iovecs is just painful! If you want more zero-copy fun
without needing mmap(2), check out the "sendfile" RubyGem and
IO.copy_stream (1.9). As of Linux 2.6.33+, sendfile(2) can copy
mmap-able files to +any+ descriptor, not just sockets.
Please email us at ruby.io.splice@librelist.com if you can think
of a good use for IO.vmsplice or IO.trysplice in Ruby.
commit 9230fba2a4231965687d2e590f8d6ad01a2fa3a8
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Feb 28 01:24:01 2011 +0000
doc: more notes for IO.vmsplice
HALP!
commit 7652c110b2d635300a7e817b69ee4bef6a28d71d
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Feb 28 01:08:44 2011 +0000
remove VERSION constant
Needless cruft and maintenance overhead.
commit 30250538e6c48b97a77114705e83fb619f362592
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Feb 28 01:05:15 2011 +0000
doc: RDoc for new interfaces and changes
And yes, encourage them for non-blocking I/O users.
commit 38273a2a81db814eacd2257665eac2fe3ecd3b94
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Feb 28 00:21:28 2011 +0000
IO.vmsplice flags argument defaults to zero
For consistency with all of our other methods.
commit 813c5300294046f9ae0f4bac8449a4734ddd7fd9
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Feb 27 23:40:44 2011 +0000
add IO.trytee interface
This allows more-efficient use of non-blocking I/O with
tee(2) and will return :EAGAIN instead of generating an
expensive Errno::EAGAIN exception.
commit c32ddca888259c391e126154ff62fdcdb2e3759c
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 16:47:40 2011 -0800
trysplice implies SPLICE_F_NONBLOCK for flags
It makes more sense this way if non-blocking I/O is expected.
commit 4ba0bb32bb717d1f77c2f4f3cb41e646eb243a7f
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 16:44:17 2011 -0800
flags are optional for splice and trysplice
No point in requiring the flags argument if it's going to be
zero, so default them to zero if they're unset.
commit 24a2f6db438537ff81081a41b5ebf02cda2dec57
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 16:35:56 2011 -0800
shorten calls to rb_thread_blocking_region
Long lines bother me
commit 844ed3d36faf14813a2e6f2476920f63bcb3bc89
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 14:45:04 2011 -0800
avoid signed vs unsigned comparison
Caught when compiling with the CFLAGS of Ruby 1.9.3dev
commit b122f037c3af81b8e20deb36b9e0706fe8535668
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 14:43:08 2011 -0800
copy_stream: File.open encoding doesn't matter
No need to open in binary mode since no file data
goes through luserspace!
commit 5c9dc339b355546e68013be26aa54cc0e9978409
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 14:41:19 2011 -0800
test: fix broken assertions on Ruby 1.9.3dev
Oops :x
commit fd131dcbf7a9c098099c00790f504908bc2c94a4
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 14:40:48 2011 -0800
copy_stream uses IO.trysplice instead of IO.splice
Things should be slightly faster this way.
commit b9119424bdd41e417afae354329e9b6d298baff4
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 14:20:37 2011 -0800
test: remove unused_port function
It's wholly unnecessary when we don't do exec() for other
processes.
commit 62f706a496eba0bc581fb2c277fed2dc29f60eb0
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 26 14:17:25 2011 -0800
add IO.trysplice interface
This reduces the cost of EAGAIN and EOF since they
are not considered exceptional conditions.
commit 1dd7ade30687dfbe220315ab412c33e2edbfa14f
Author: Eric Wong <normalperson@yhbt.net>
Date: Wed Dec 22 10:57:58 2010 -0800
io/splice: unindent
This seems to fix the issue we had with older RDoc, too
commit 31f77b2a01f69956b034d35a7f12038947b6bdbc
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 5 03:27:54 2011 -0800
test: cleanup unused_port function
Duh!
commit cba967458df157f3e4aa2dc885eb87013bece175
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Feb 5 03:25:47 2011 -0800
clean up packaging and use pkg.mk
commit f62107bf4b5eb01dbc22b68e56b61db78c73336c
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Dec 27 16:55:28 2010 +0000
doc: switch to wrongdoc
No more JavaScript!
commit 09cd762eeebc07dd7fd0eef6c3bea40a0f74b030
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Dec 27 16:19:30 2010 +0000
GNUmakefile: sync examples for documentation publishing
The examples on our site were out-of-date.
commit 4df9ceb637abe416ac36608038e440e30bbb6a35
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Aug 2 08:14:08 2010 +0000
GNUmakefile: update RAA on new releases
commit fd4ee32882f8c59b085f097dbb2d143c9086a255
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Aug 2 07:55:57 2010 +0000
Ruby io_splice 2.2.0 - updates for Linux 2.6.35
This release adds IO#pipe_size and IO#pipe_size= accessors for
resizing the kernel pipe buffer under Linux 2.6.35+. This
functionality may be used to either reduce memory footprint or
make large copies with fewer method dispatches and system calls.
These methods are only available when run under Linux 2.6.35 or
later (but always built, so there's no need to recompile this
library after upgrading your kernel).
http://bogomips.org/ruby_io_splice/doc/IO.html#method-i-pipe_size
http://bogomips.org/ruby_io_splice/doc/IO.html#method-i-pipe_size%3D
commit a6a3ba543c3b9069437ade7741c694ce3da2e884
Author: Eric Wong <normalperson@yhbt.net>
Date: Mon Aug 2 07:40:48 2010 +0000
add IO#pipe_size and IO#pipe_size= accessors
These are friendly wrappers around F_GETPIPE_SZ
and F_SETPIPE_SZ fcntl()s in Linux 2.6.35.
commit dcd710c622530052be50b226d245caebb6d2a970
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu Jun 24 18:46:14 2010 +0000
test_io_splice: fix broken test
Apparently timing variations were allowing it to succeed under
Ruby 1.8/1.9, but Rubinius tripped and exposed the bug.
commit 52ec5be1ba9768499b948186589c1404ccc64724
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 20:27:36 2010 +0000
Ruby io_splice 2.1.0 - IO::Splice.copy_stream fixes
This fixes the issue with our copy_stream where non-pipe
descriptors with the O_NONBLOCK flag set were not handled
correctly. Like IO.copy_stream in Ruby 1.9, our
IO::Splice.copy_stream will _always_ block until IO is available
(or raise if there are errors).
IO::Splice.copy_stream now closes all file descriptors that were
opened within the method, reducing pressure on the garbage
collector.
There are also new singleton convenience methods:
* IO::Splice.full - blocks and splices the full amount
* IO::Splice.partial - blocks until something is spliceable
They are used internally to implement IO::Splice.copy_stream,
but may also be useful to end users.
Rcov shows 100% test coverage \o/
commit 1db6afb8369cafdfe05680798ee68fe3081251d2
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 20:26:37 2010 +0000
doc: help RDoc read IO::Splice singleton methods
commit a2026eae297ee6b911b77d07e1d96a0136e1a114
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 19:49:12 2010 +0000
document +src+ and +dst+ for IO::Splice.{full,partial}
Unlike IO::Splice.copy_stream, these require at least
one of the source or destination parameters to be a
pipe.
commit be048fd9d58604feb7aed434a53b5abd694c06f2
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 19:46:05 2010 +0000
GNUmakefile: add rcov target
commit 87332294a9cb7e3ce426050a7c949569200a4f01
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 19:21:42 2010 +0000
copy_stream: handle and block on EAGAIN from the non-pipe IO
splice(2) (called without SPLICE_F_NONBLOCK) only blocks on the
pipe descriptor, but may trigger EAGAIN from the non-pipe
descriptor. So use IO.select to deal with the case where the
non-pipe descriptor is non-blocking and wait on it.
While we're at it, add the IO::Splice.partial and
IO::Splice.full singleton methods to make our life easier
to implement copy_stream with. These methods may also
be used directly by users who may find them useful.
commit e86706ab91e3d44297f03d1d9428968e237a1d2a
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 19:21:17 2010 +0000
GNUmakefile: allow passing options to test/unit
commit a301f99563deeb03c93d275330e20ff672519680
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 17:59:10 2010 +0000
copy_stream: cleanup after ourselves
It's a good idea to clean up any file descriptors
opened within our method to avoid unnecessary GC
invocations.
commit 8aab8c93421519f0ad33ab7d7c5049e12a1362b3
Author: Eric Wong <normalperson@yhbt.net>
Date: Sun Jun 6 06:50:10 2010 +0000
COPYING: bundle the LGPLv2.1 instead of LGPLv3
The choice of version is up to the user, but bundle the one
more likely to be invoked.
commit 2b95ba46ee144eb3a22054ae98a6416996136469
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Jun 5 08:36:16 2010 +0000
Ruby io_splice 2.0.0
There is a new IO::Splice.copy_stream method which is similar to
the IO.copy_stream method in 1.9 core, but uses splice()
internally to avoid copies to userspace in situations where
sendfile() is not possible.
With Linux 2.6.35 (and header files to match), we will also
export the F_GETPIPE_SZ and and F_SETPIPE_SZ constants so
they can be used with fcntl() to resize pipe buffers.
The documentation is also updated to reflect the returned
usefulness of the IO::Splice::F_MOVE constant for FUSE
filesystems under Linux 2.6.35.
I've also relicensed to LGPLv2.1+ (from LGPLv3-only) to allow
bundling with GPLv2-only applications (I'm currently the sole
copyright holder).
Signed-off-by: Eric Wong <normalperson@yhbt.net>
commit 901e6061348490b3ea5d1e08be5b08c9601ad310
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Jun 5 08:34:17 2010 +0000
update PIPE_CAPA documentation for Linux 2.6.35
commit 70efa04bf48a3187e60d0f459b6a4202fb0885b2
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Jun 5 08:22:19 2010 +0000
relicense to LGPLv2.1+ (from LGPLv3 only)
This allows GPLv2-only programs to bundle us.
Signed-off-by: Eric Wong <normalperson@yhbt.net>
commit b755f909f88c3954d421437e54eb465be74df652
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Jun 5 06:27:24 2010 +0000
GNUmakefile: make our unit tests PHONY
commit 70c9bdf4ef12212a99a50ffb1284287920953a06
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Jun 5 06:19:17 2010 +0000
examples/splice-cp: simplify with IO::Splice.copy_stream
commit fc64ffe36d831c1a29a603a1a1ad50b7b68d55e8
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Jun 5 06:02:30 2010 +0000
io_splice_ext: require errno.h
It seems needed for Rubinius
commit e8b06376c21a74e3fc2e6e9ace4eda8b2359ab62
Author: Eric Wong <normalperson@yhbt.net>
Date: Sat Jun 5 05:42:54 2010 +0000
copy_stream takes source offset like the non-splice version
This should make it a drop-in replacement for IO.copy_stream,
except it won't deal with things like StringIO.
commit 5e222a0cedd7146bdc2b0f07d5c36ce7f38b740e
Author: Eric Wong <normalperson@yhbt.net>
Date: Thu Jun 3 07:00:43 2010 +0000
allow IO.vmsplice to take a single string
This could be useful if a Ruby implementation eventually gives
us page-aligned strings that we can IO::Splice::F_GIFT
commit 01ced6c651db7694ecb73fbaa145d319f0b1e15d
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue Jun 1 16:59:57 2010 -0700
copy_stream: always try to move pages instead of copying
For most case in the forseeable future it's a no-op, but
since FUSE will support it in 2.6.35, some folks will be
able to use it again.
commit bb6791da7cf37e326846edc44a7c32b79021f9da
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue Jun 1 16:54:24 2010 -0700
preliminary support for F_GETPIPE_SZ and F_SETPIPE_SZ
One day soon (2.6.35), we'll be able to resize pipes. We may
support this independently of the C library since that can take
a few months/years to catch up, taking distro release cycles
into account, too.
commit e4bdec2a41932a6ebd444d003f21cc025223cfdd
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue Jun 1 16:46:18 2010 -0700
update IO::Splice::F_MOVE documentation for 2.6.35
FUSE devices will finally gain splice() support and also
support SPLICE_F_MOVE in 2.6.35. SPLICE_F_MOVE is still
ignored for non-FUSE devices.
commit 5b34f7048cae6161c5180f57074272d558373133
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue Jun 1 16:08:46 2010 -0700
simplify examples for 1.0.0 API
commit ecb5f27598a6f53b6e02817bb0c1644fc4c75f09
Author: Eric Wong <normalperson@yhbt.net>
Date: Tue Jun 1 16:01:58 2010 -0700
implement IO::Splice.copy_stream
This is like IO.copy_stream in Ruby 1.9, but uses
the splice() system call internally.
commit 09ae46671a8829dbc2fba3ec8f2de9127e3c3315
Author: Eric Wong <normalperson@yhbt.net>
Date: Fri Mar 12 16:21:01 2010 -0800
fix indentation bug
Originally generated with the Darkfish Rdoc Generator 2, modified by wrongdoc.