The splice family of Linux system calls can transfer data between file descriptors without the need to copy data into userspace. Instead of a userspace buffer, they rely on an ordinary Unix pipe as a kernel-level buffer.
Efficient zero-copy I/O avoids data copies into userspace, reducing garbage for the Ruby garbage collector as a side effect.
More flexible than sendfile, may be used to enable copies between arbitrary file descriptors (assuming kernel support), not just file-to-socket (or file-to-anything in newer Linux).
Thread-safe blocking operations under Ruby 1.9+, releases GVL if blocking operations are used.
Safely usable with non-blocking I/O frameworks (unlike IO.copy_stream) when combined with the IO::Splice::F_NONBLOCK flag.
Fully-documented library API and examples
Operating system support for the splice(2), tee(2) and vmsplice(2) system calls is required. Currently, only Linux 2.6.17 or later with a modern C library (glibc 2.5 or later) support these system calls. Using the latest stable Linux kernel is HIGHLY recommended as there have been numerous bugs in the early releases of these system calls.
If you're using a packaged Ruby distribution, make sure you have a C compiler and the matching Ruby development libraries and headers.
If you use RubyGems:
gem install io_splice
Otherwise grab the latest tarball from:
Unpack it, and run "ruby setup.rb"
Our API matches the C API closely, see the RDoc for full API documentation and the Linux manpages for more details.
All of these system calls are fairly new have seen limited usage anywhere, even outside of the Ruby world.
If you encounter problems (privilege escalation, memory leaks (in the kernel), poor performance, corrupt data, etc..), try upgrading the Linux kernel to the latest stable version.
These system calls have a lot of potential, and will hopefully be standardized and available in non-Linux kernels some day.
You can get the latest source via git from the following locations:
git://bogomips.org/ruby_io_splice.git git://repo.or.cz/ruby_io_splice.git (mirror)
You may browse the code from the web and download the latest snapshot tarballs here:
Inline patches (from "git format-patch") to the mailing list are preferred because they allow code review and comments in the reply to the patch.
We will adhere to mostly the same conventions for patch submissions as git itself. See the Documentation/SubmittingPatches document distributed with git on on patch submission guidelines to follow. Just don't email the git mailing list or maintainer with io_splice patches.
All feedback (bug reports, user/development discussion, patches, pull requests) go to the mailing list: firstname.lastname@example.org
Send patches (from "git format-patch") with "git send-email" and do not send HTML email or attachments. We are very responsive to email and you will usually get a response within 24-72 hours.
We operate a public-inbox which feeds the mailing list. You may subscribe either using ssoma or by sending a mail to email@example.com
ssoma is a mail archiver/fetcher using git. It operates in a similar fashion to tools such as slrnpull, fetchmail, or getmail. ssoma subscription instructions:
URL=git://bogomips.org/ruby-io-splice LISTNAME=ruby-io-splice # to initialize a maildir (this may be a new or existing maildir, # ssoma will not touch existing messages) # If you prefer mbox, use mbox:/path/to/mbox as the last argument # You may also use imap://$MAILSERVER/INBOX for an IMAP account # or imaps:// for an IMAPS account, as well. ssoma add $LISTNAME $URL maildir:/path/to/maildir # read with your favorite MUA (only using mutt as an example) mutt -f /path/to/maildir # (or /path/to/mbox) # to keep your mbox or maildir up-to-date, periodically run: ssoma sync $LISTNAME # your MUA may modify and delete messages from the maildir or mbox, # this does not affect ssoma functionality at all # to sync all your ssoma subscriptions ssoma sync # You may wish to sync in your cronjob ssoma sync --cron
We love to hear from you!
Email patches (using git send-email), pull requests (formatted using git request-pull), questions, bug reports, suggestions, etc. to us publically at:
Mail archives are available at: http://bogomips.org/ruby-io-splice/
Please send plain-text email only and do not waste bandwidth on HTML mail, HTML mail will not be read.
Quote as little as reasonable and do not top post.
For sensitive topics, email us privately at: firstname.lastname@example.org