We no longer send an empty zone= parameter in create_open calls. Additionally, if we detect send() failure on a socket, assume a tracker has been restarted and restart the request. There are also some minor code cleanups.
client changes: * new_file gains :create_open_args and :create_close_args which allows custom arguments to be passed to plugins. (this matches the Perl client behavior) * new_file also gains :info hash which can be populated with information normally retrieved with file_info * users with net-http-persistent installed will see a small speed boost and reduction of TIME_WAIT sockets when dealing with small data (useful for testing DB-intensive parts of MogileFS). No changes for admin.
client changes: * new_file gains :create_open_args and :create_close_args which allows custom arguments to be passed to plugins. * new_file also gains :info hash which can be populated with information normally retrieved with file_info * users with net-http-persistent installed will see a small speed boost and reduction of TIME_WAIT sockets when dealing with small data (useful for testing DB-intensive parts of MogileFS). No changes for admin.
Admin speedups for get_domains, get_hosts, and get_devices. Previous versions were completely unusable for parsing a list of 3000+ domains. Installations with thousands of hosts or devices should see noticeable performance improvements. The client interface gains the each_file_info iterator method. This behaves like each_key, but gives access to the entire response the file_info returns, including: checksum, devcount, file size, class, and domain.
No changes since 3.3.0-rc1. We now correctly timeout requests to slow trackers and avoid reusing the socket after timeouts (as old responses can arrive late). A big thanks to David Rasch for helping with timeout/idempotency issues for this release. Minor documentation/packaging updates, it should be easier to build a pre-release gem now (see HACKING doc).
We now correctly timeout requests to slow trackers and avoid reusing the socket after timeouts (as old responses can arrive late). A big thanks to David Rasch for helping with timeout/idempotency issues for this release. Minor documentation/packaging updates, it should be easier to build a pre-release gem now (see HACKING doc).
Changes since 3.2.0-rc1 (MogileFS::Admin-only): * admin supports "create_device" and "change_device_weight" commands * admin casts "reject_bad_md5" field in "get_device" return value to boolean (true/false). * minor code simplifications to admin Changes since 3.1.1 * "list_keys" and "exist?" client commands raise errors properly on failure. * backend connections no longer terminate on ERR responses, only on socket/connection errors. * support the "updateclass" client command. This is for updating the class of a given key and not to be confused with the "update_class" admin command. * "new_file" checksum usage is now documented since MogileFS 2.60 includes official support for checksums
* "list_keys" and "exist?" client commands raise errors properly on failure. * backend connections no longer terminate on ERR responses, only on socket/connection errors. * support the "updateclass" client command. This is for updating the class of a given key and not to be confused with the "update_class" admin command. * "new_file" checksum usage is now documented since MogileFS 2.60 includes official support for checksums
This releases fixes problems short reads when slurping files into memory. Thanks to Matthew Draper for this fix. There are also minor documentation updates.
* improved API support for uploading large files While we've always supported uploading large files, the (still-supported) existing APIs were somewhat awkward or required the file to exist on the file system. See MogileFS::NewFile for details and examples. * more informative exception messages for timed-out requests * :fail_timeout parameter, the timeout for retrying a failed tracker connection. This defaults to 5 seconds (same as previous versions where this was hard-coded. * :new_file_max_time parameter Controls the maximum of time spent creating and uploading a new file in MogileFS. This defaults to 1 hour (which matching the expiry time of a row in the MogileFS internal tempfile table). * store_file works on unlinked File/Tempfile objects * each_fid method in MogileFS::Admin fixed * stale_fid_checker example script added * mogstored_rack example split into a standalone RubyGem: http://bogomips.org/mogstored_rack/ * backend error constants are generated on const_missing, instead of when raised, making it easier to rescue exceptions we didn't explicitly enable * some internal cleanups and documentation improvements
Changes since 3.0.0-rc1:
* 1.8 copy_stream emulator respects creation umask
This matches the latest IO.copy_stream behavior.
[ruby-core:41308], r33851 in ruby/trunk
* higher timeouts for uploads to compensate for slow
(but not dead) servers.
Changes since v2.2.0:
=== client changes
* "store_file" now accepts any IO object capable of streaming
data (e.g. pipes and sockets). This uses chunked
Transfer-Encoding for PUTs, so backend storage nodes
will need to support this (latest mogstored does).
* "store_file" no longer uses an infinite timeout when
awaiting a response after a PUT, the new timeout for the
response is now calculated based on the time and size of
the PUT request.
* new commands: "file_debug" and "file_info" (new commands
in mogilefsd, be sure you have the latest version)
* "get_paths" takes optional ":pathcount" parameter
to control the number of returned paths.
* "get_file_data" supports offset and count for partial
transfer (requires support from storage node for
Range: requests, most HTTP servers are capable of this)
* IO.copy_stream is enabled by default under Ruby 1.9.3.
Expect performance improvements.
* "list_keys" with a passed block (for additional info) is
faster due to internal pipelining implementation and the
addition of "file_info" support.
* fixed handling of "+" in key/domain names (old bug)
* rare, truncated partial responses due to network/server
failure now raise MogileFS::InvalidResponseError
=== admin changes
Admin support is still a work-in-progress,
I usually just find myself using "mogadm" anyways.
* new admin commands: "replicate_now"
* "get_stats" no longer works on new mogilefsd versions
* get_domains handles "repl_policy" field correctly for classes
in MogileFS 2.x
* admin commands should convert all numeric fields to either
Integer or Float objects and not String representations
of numerics. Affected methods include:
get_hosts, get_devices, list_fids, each_fids, get_domains
=== miscellany
* we no longer add methods to standard Ruby classes
(at least we never /changed/ existing methods :P)
* {kgio}[http://bogomips.org/kgio] automatically used
if available, but not required to avoid exceptions
with non-blocking I/O
* dropped Ruby 1.8.6 support, 1.8.7 or later is required.
* Tested with MRI 1.8.7, 1.9.3 and Rubinius 1.2.4
* MogileFS::Mysql - deprecated, to be removed in 2012
* improved test suite
* some optional experimental features/changes,
see "git log" for details
* Added internal pipelining implementation, this is
not easy-to-use since the server can respond-out-of-order,
but still useful for things like a list_keys+file_info
loop.
=== client changes
* "store_file" now accepts any IO object capable of streaming
data (e.g. pipes and sockets). This uses chunked
Transfer-Encoding for PUTs, so backend storage nodes
will need to support this (latest mogstored does).
* "store_file" no longer uses an infinite timeout when
awaiting a response after a PUT, the new timeout for the
response is now calculated based on the time and size of
the PUT request.
* new commands: "file_debug" and "file_info" (new commands
in mogilefsd, be sure you have the latest version)
* "get_paths" takes optional ":pathcount" parameter
to control the number of returned paths.
* "get_file_data" supports offset and count for partial
transfer (requires support from storage node for
Range: requests, most HTTP servers are capable of this)
* IO.copy_stream is enabled by default under Ruby 1.9.3.
Expect performance improvements.
* "list_keys" with a passed block (for additional info) is
faster due to internal pipelining implementation and the
addition of "file_info" support.
* fixed handling of "+" in key/domain names (old bug)
* rare, truncated partial responses due to network/server
failure now raise MogileFS::InvalidResponseError
=== admin changes
Admin support is still a work-in-progress,
I usually just find myself using "mogadm" anyways.
* new admin commands: "replicate_now"
* "get_stats" no longer works on new mogilefsd versions
* get_domains handles "repl_policy" field correctly for classes
in MogileFS 2.x
* admin commands should convert all numeric fields to either
Integer or Float objects and not String representations
of numerics. Affected methods include:
get_hosts, get_devices, list_fids, each_fids, get_domains
=== miscellany
* we no longer add methods to standard Ruby classes
(at least we never /changed/ existing methods :P)
* {kgio}[http://bogomips.org/kgio] automatically used
if available, but not required to avoid exceptions
with non-blocking I/O
* dropped Ruby 1.8.6 support, 1.8.7 or later is required.
* Tested with MRI 1.8.7, 1.9.3 and Rubinius 1.2.4
* MogileFS::Mysql - deprecated, to be removed in 2012
* improved test suite
* some optional experimental features/changes,
see "git log" for details
* Added internal pipelining implementation, this is
not easy-to-use since the server can respond-out-of-order,
but still useful for things like a list_keys+file_info
loop.
= 2.2.0 * internal cleanups (no public API breakage) * refactor backend socket/connection handling for reliability There'll probably be more aggressive cleanups/refactoring in future releases if I have time.
= 2.1.0
* MySQL interface returns integer length and devcount (API change)
* Ensure store_{content,file} always returns size (API fix)
* Add get_uris API method
* Respect timeout when doing get_file_data
* MySQL interface filters out URLs for down/dead hosts/devices
* Really remove all NFS support
* get_file in slurp mode slurps all output correctly
Eric Wong (18):
tests: retry random ports correctly
test_mogilefs: fix race conditions
MySQL interface returns integer length and devcount
Respect timeout when doing get_file_data
Unify internal HTTP GET/HEAD methods
Close socket we create before raising exceptions
Compact get_paths output so we don't have nils
Really remove all NFS support
README: add a note about emailing me
Extra checking for full_timeout in sysread_full
Add get_uris API method
verify_uris: use write_nonblock instead of syswrite
README: add links to the repo.or.cz mirror
mysql: filter out URLs for down/dead hosts/devices
mog: small cleanup
Ensure store_{content,file} always returns size
GNUmakefile: better logging/output control
mogilefs-client 2.1.0
godfat (1):
call IO.select([sock]) if sock is not ready.
Retroactively tagging this old release since it was in SVN or Perforce...
initial
* verify_uris method made more robust * preliminary Ruby 1.9 compatibility, tests still need some work * allow store_content to be used with a streamable object of known length * add setup.rb for non-RubyGems users
initial
* Fix missing MogileFS::Util include for sysrwloop in MogileFS::MogileFS
* Fixed MogileFS#rename. Bug #14465 submitted by Justin Dossey. * Removed infinite loop in MogileFS::HTTPFile#store_file. Patch #13789 submitted by Andy Lo-A-Foe. * Made MogileFS#get_file_data timeout configurable. Bug #13490 submitted by Andy Lo-A-Foe. * Add MogileFS#size. Feature Request #14484 submitted by Justin Dossey. * Fix MogileFS#get_file_data to return the data for HTTP mode. Bug #7133 submitted by John Wanko. * New maintainer: Eric Wong * Add `mog' command-line tool as a demo/example * Lower memory consumption with large files * Allow get_file_data to accept a block for large files * Fix each_keys loop termination condition * Apply error handling patch from Matthew Willson. Bug #15987 * Merge large file patch from Andy Lo-A-Foe. Bug #13764
Originally generated with the Darkfish Rdoc Generator 2, modified by wrongdoc.