HACKING History LICENSE NEWS README TODO
MogileFS Admin Backend Client MogileFS NewFile NewFile::Writer ReadOnlyError

Ruby mogilefs-client 3.12.2 / 2020-02-06 09:02 UTC

Only one documentation change to update URLs.

bogomips.org is expiring and with the price of the .org TLD
inevitably going up, I won't be able to afford to pay
extortionists at ICANN/Ethos/PIR.

Ruby mogilefs-client 3.12.1 / 2019-11-11 22:09 UTC

Minor cleanups, mainly to fix annoying warnings in Ruby 2.7.0dev.

4 changes since v3.12.0 (2019-01-02):

      pkg.mk: use dark216 olddoc theme for Earth Day 2019
      mogilefs: actually use kwarg for Net::HTTP::Persistent.new
      fix more indentation warnings
      test_client: quiet warnings about redefining accessors

Ruby mogilefs-client 3.12.0 / 2019-01-02 22:49 UTC

Minor cleanups, mainly to fix indentation warnings in Ruby 2.6+

7 changes since v3.11.1 (2017-03-23):

      pool: simplify empty class definition
      new_file/stream: remove unused variable
      mog: add updateclass command
      socket/pure_ruby: favor String#clear if available
      fix indentation warnings with ruby trunk
      doc: https:// URLs instead of git://
      update URLs to point to wiki

mogilefs-client 3.11.1 / 2017-03-23 02:04 UTC

This release updates the IO#write garbage workaround to be
enabled for Ruby 2.0 and 2.1 users; but avoids it on the
just-released Ruby 2.4.1; as that includes the backported bugfix
for https://bugs.ruby-lang.org/issues/13085

There's also some trivial doc and code aesthetics fixes.

6 changes since 3.11.0:

      .olddoc.yml: add trailing slash on URL
      socket_common: expand IO#write garbage workaround to 2.0 and 2.1
      socket_common: limit garbage workaround to Ruby <= 2.4.1
      socket_common: remove needless 'o' modifier for Regexp
      socket_common: improve readability of case statement
      GNUmakefile: fix incorrect comment

Ruby mogilefs-client 3.11.0 / 2017-01-31 18:32 UTC

This release provides compatibility with net-http-persistent 3.x
while retaining support for the (API-incompatible) 2.x series.

Users of mainline Ruby 2.2+ get a workaround for excessive
garbage for uploading files.  This excessive garbage on IO#write
should be fixed when Ruby 2.5 gets released on Dec 25, 2017:
https://bugs.ruby-lang.org/issues/13085

There's also a few minor garbage reductions of small strings,
relying on the maintained cmogstored for Content-Range PUT
testing, and some minor build system and documentation updates.

12 changes since 3.10.0 (Aug 2016):

      avoid pointless capture in regexp
      use opt_str_freeze to avoid allocations in a few places
      new_file/common: simplify empty class declarations
      support net-http-persistent 3.x
      test: switch to cmogstored for testing Content-Range PUTs
      avoid excessive garbage on uploads with Ruby 2.2+
      socket_common: limit IO#write garbage workaround to <= 2.4
      freeze string literals in a few more places
      update URLS to HTTPS
      README: explain removal of NFS support
      doc: remove private email address linkage
      gemspec: remove olddoc development dependency

Ruby mogilefs-client 3.10.0 / 2016-08-31 07:32 UTC

A bunch of minor tweaks to reduce garbage and exceptions.

In addition to the existing :timeout and :fail_timeout options,
there is a new :connect_timeout directive for all connection
classes which only controls the time spent waiting for a TCP
connection.  This defaults to the same value as the existing
:timeout option (3 seconds); but users are advised to lower
it to match ideal network conditions.

For users of Ruby 2.3+, this release takes advantage of
exception-less Socket#connect_nonblock using "exception: false".
Users of Ruby 2.1+ will also benefit from "exception: false"
usage for read_nonblock and write_nonblock calls.
kgio is no longer be useful with this release with Ruby 2.3+

This release also fixes a Ruby 1.8.7 compatibility bug for
non-kgio users.  However, this may be the last 1.8.7-compatible
release.  Fwiw, I wanted to remove Ruby 1.8 support around 5
years ago but several users were against it.   Maybe nobody will
complain, this time...

18 changes since 3.9.0:

      doc: avoid inadvertantly documenting the Process class
      admin: simplify utilization conversion
      more idiomatic comparisons with constants
      bigfile/filter: only update MD5 if non-nil
      bigfile: lazily require bigfile/filter
      backend: simplify regexp
      .olddoc.yml: add NNTP and mailing list archive URL
      minor garbage reductions for newer Rubies
      socket/pure_ruby: fix Ruby 1.8 compatibility
      socket/pure_ruby: use `:exception=>false' on Ruby 2.1+
      test_fresh: do not delete non-existent domain
      admin: map unset reject_bad_md5 field to nil
      socket/pure_ruby: connect with "exception:false" on Ruby 2.3+
      implement :connect_timeout option
      add .gitattributes for Ruby method detection
      README: stop mentioning cgit
      connect_timeout: match :timeout if unset
      pkg.mk: use --local option for gem installation

Ruby mogilefs-client 3.9.0 / 2015-06-25 11:12 UTC

shortlog of changes since v3.8.0:

* test/test_fresh.rb: fix breakage from Ruby-trunk r50118
* use monotonic clock if possible on Ruby 2.1+
* avoid defineclass instructions for empty classes
* HACKING: minor documentation updates
* mogilefs/socket/pure_ruby: use IO#wait_*able

Ruby mogilefs-client 3.8.0 / 2015-02-10 00:36 UTC

Mostly documentation updates and test cleanups.  Nothing interesting
for the regular client, but the admin client now shows the
checksum-related fields in get_devices.

Most notably, there's a public-inbox for those who don't want
to subscribe to the MogileFS list:

      mogilefs-client-public@bogomips.org

Archives are at: http://bogomips.org/mogilefs-client-public/

There is no new mailing list subscription, it is only a
public-inbox.  Cc-ing the MogileFS list at mogile@googlegroups.com
is encouraged if you do have a subscription.

shortlog since 3.7.1:

      add mog-sync example script
      admin (doc, get_hosts): display integers in results doc
      update documentation and packaging
      GNUmakefile: publish examples on the site, too
      examples: add usage_fetcher example
      TODO: remove Cool.io and EventMachine references
      admin: flesh out get_devices
      admin: reduce bytecode overhead of get_stats
      tests: create fresh intances for all integration tests
      include mogilefs/version.rb in the distro
      test: fixup object lifetimes and teardown
      new public-inbox: mogilefs-client-public@bogomips.org

Ruby mogilefs-client 3.7.1 / 2013-09-10 23:35 UTC

Only one bugfix:
  new_file/stream: correct declare errors array correctly

This only triggered on rare syscall errors
(Errno::EMFILE/Errno::ENFILE)

Ruby mogilefs-client 3.7.0 / 2013-07-18 06:40 UTC

MogileFS::MogileFS#each_key now accepts optional :after and :limit args

This allows easy iteration while respecting :after and :limit,
giving each_key an interface consistent with each_file_info.

Ruby mogilefs-client 3.6.0 / 2013-02-21 20:33 UTC

We no longer send an empty zone= parameter in create_open
calls.  Additionally, if we detect send() failure on a socket,
assume a tracker has been restarted and restart the request.
There are also some minor code cleanups.

mogilefs-client 3.5.0 / 2012-12-04 11:09 UTC

client changes:

* new_file gains :create_open_args and :create_close_args which
  allows custom arguments to be passed to plugins.
  (this matches the Perl client behavior)

* new_file also gains :info hash which can be populated with
  information normally retrieved with file_info

* users with net-http-persistent installed will see a small speed
  boost and reduction of TIME_WAIT sockets when dealing with
  small data (useful for testing DB-intensive parts of MogileFS).

No changes for admin.

mogilefs-client 3.5.0-rc1 / 2012-10-31 19:53 UTC

client changes:

* new_file gains :create_open_args and :create_close_args which
allows custom arguments to be passed to plugins.

* new_file also gains :info hash which can be populated with
information normally retrieved with file_info

* users with net-http-persistent installed will see a small speed
boost and reduction of TIME_WAIT sockets when dealing with
small data (useful for testing DB-intensive parts of MogileFS).

No changes for admin.

Ruby mogilefs-client 3.4.0 / 2012-10-09 03:06 UTC

Admin speedups for get_domains, get_hosts, and get_devices.
Previous versions were completely unusable for parsing a list of
3000+ domains.  Installations with thousands of hosts or devices
should see noticeable performance improvements.

The client interface gains the each_file_info iterator method.
This behaves like each_key, but gives access to the entire
response the file_info returns, including: checksum, devcount,
file size, class, and domain.

Ruby mogilefs-client 3.3.0 / 2012-08-11 11:12 UTC

No changes since 3.3.0-rc1.

We now correctly timeout requests to slow trackers and avoid
reusing the socket after timeouts (as old responses can arrive
late).  A big thanks to David Rasch for helping with
timeout/idempotency issues for this release.

Minor documentation/packaging updates, it should be easier
to build a pre-release gem now (see HACKING doc).

Ruby mogilefs-client 3.3.0-rc1 / 2012-08-03 21:50 UTC

We now correctly timeout requests to slow trackers and avoid
reusing the socket after timeouts (as old responses can arrive
late).  A big thanks to David Rasch for helping with
timeout/idempotency issues for this release.

Minor documentation/packaging updates, it should be easier
to build a pre-release gem now (see HACKING doc).

Ruby mogilefs-client 3.2.0 / 2012-06-29 22:28 UTC

Changes since 3.2.0-rc1 (MogileFS::Admin-only):

* admin supports "create_device" and "change_device_weight" commands

* admin casts "reject_bad_md5" field in "get_device" return
  value to boolean (true/false).

* minor code simplifications to admin

Changes since 3.1.1

* "list_keys" and "exist?" client commands raise errors properly
  on failure.

* backend connections no longer terminate on ERR responses, only
  on socket/connection errors.

* support the "updateclass" client command.  This is for updating
  the class of a given key and not to be confused with the
  "update_class" admin command.

* "new_file" checksum usage is now documented since MogileFS 2.60
  includes official support for checksums

Ruby mogilefs-client 3.2.0-rc1 / 2012-06-15 01:05 UTC

* "list_keys" and "exist?" client commands raise errors properly
  on failure.

* backend connections no longer terminate on ERR responses, only
  on socket/connection errors.

* support the "updateclass" client command.  This is for updating
  the class of a given key and not to be confused with the
  "update_class" admin command.

* "new_file" checksum usage is now documented since MogileFS 2.60
  includes official support for checksums

Ruby mogilefs-client 3.1.1 / 2012-02-28 22:12 UTC

This releases fixes problems short reads when slurping files
into memory.  Thanks to Matthew Draper for this fix.
There are also minor documentation updates.

Ruby mogilefs-client 3.1.0 / 2011-12-15 04:10 UTC

* improved API support for uploading large files
  While we've always supported uploading large files, the
  (still-supported) existing APIs were somewhat awkward or
  required the file to exist on the file system.  See
  MogileFS::NewFile for details and examples.

* more informative exception messages for timed-out requests

* :fail_timeout parameter, the timeout for retrying a failed
  tracker connection.  This defaults to 5 seconds (same as
  previous versions where this was hard-coded.

* :new_file_max_time parameter
  Controls the maximum of time spent creating and uploading
  a new file in MogileFS.   This defaults to 1 hour (which
  matching the expiry time of a row in the MogileFS internal
  tempfile table).

* store_file works on unlinked File/Tempfile objects

* each_fid method in MogileFS::Admin fixed

* stale_fid_checker example script added

* mogstored_rack example split into a standalone RubyGem:
  http://bogomips.org/mogstored_rack/

* backend error constants are generated on const_missing,
  instead of when raised, making it easier to rescue exceptions
  we didn't explicitly enable

* some internal cleanups and documentation improvements

Ruby mogilefs-client 3.0.0 / 2011-11-28 20:53 UTC

Changes since 3.0.0-rc1:

* 1.8 copy_stream emulator respects creation umask
  This matches the latest IO.copy_stream behavior.
  [ruby-core:41308], r33851 in ruby/trunk

* higher timeouts for uploads to compensate for slow
  (but not dead) servers.

Changes since v2.2.0:

=== client changes

* "store_file" now accepts any IO object capable of streaming
  data (e.g. pipes and sockets).   This uses chunked
  Transfer-Encoding for PUTs, so backend storage nodes
  will need to support this (latest mogstored does).

* "store_file" no longer uses an infinite timeout when
  awaiting a response after a PUT, the new timeout for the
  response is now calculated based on the time and size of
  the PUT request.

* new commands: "file_debug" and "file_info" (new commands
  in mogilefsd, be sure you have the latest version)

* "get_paths" takes optional ":pathcount" parameter
  to control the number of returned paths.

* "get_file_data" supports offset and count for partial
  transfer (requires support from storage node for
  Range: requests, most HTTP servers are capable of this)

* IO.copy_stream is enabled by default under Ruby 1.9.3.
  Expect performance improvements.

* "list_keys" with a passed block (for additional info) is
  faster due to internal pipelining implementation and the
  addition of "file_info" support.

* fixed handling of "+" in key/domain names (old bug)

* rare, truncated partial responses due to network/server
  failure now raise MogileFS::InvalidResponseError

=== admin changes

Admin support is still a work-in-progress,
I usually just find myself using "mogadm" anyways.

* new admin commands: "replicate_now"

* "get_stats" no longer works on new mogilefsd versions

* get_domains handles "repl_policy" field correctly for classes
  in MogileFS 2.x

* admin commands should convert all numeric fields to either
  Integer or Float objects and not String representations
  of numerics.  Affected methods include:
   get_hosts, get_devices, list_fids, each_fids, get_domains

=== miscellany

* we no longer add methods to standard Ruby classes
  (at least we never /changed/ existing methods :P)

* {kgio}[http://bogomips.org/kgio] automatically used
  if available, but not required to avoid exceptions
  with non-blocking I/O

* dropped Ruby 1.8.6 support, 1.8.7 or later is required.

* Tested with MRI 1.8.7, 1.9.3 and Rubinius 1.2.4

* MogileFS::Mysql - deprecated, to be removed in 2012

* improved test suite

* some optional experimental features/changes,
  see "git log" for details

* Added internal pipelining implementation, this is
  not easy-to-use since the server can respond-out-of-order,
  but still useful for things like a list_keys+file_info
  loop.

mogilefs-client 3.0.0-rc1 / 2011-11-21 02:48 UTC

=== client changes

* "store_file" now accepts any IO object capable of streaming
  data (e.g. pipes and sockets).   This uses chunked
  Transfer-Encoding for PUTs, so backend storage nodes
  will need to support this (latest mogstored does).

* "store_file" no longer uses an infinite timeout when
  awaiting a response after a PUT, the new timeout for the
  response is now calculated based on the time and size of
  the PUT request.

* new commands: "file_debug" and "file_info" (new commands
  in mogilefsd, be sure you have the latest version)

* "get_paths" takes optional ":pathcount" parameter
  to control the number of returned paths.

* "get_file_data" supports offset and count for partial
  transfer (requires support from storage node for
  Range: requests, most HTTP servers are capable of this)

* IO.copy_stream is enabled by default under Ruby 1.9.3.
  Expect performance improvements.

* "list_keys" with a passed block (for additional info) is
  faster due to internal pipelining implementation and the
  addition of "file_info" support.

* fixed handling of "+" in key/domain names (old bug)

* rare, truncated partial responses due to network/server
  failure now raise MogileFS::InvalidResponseError

=== admin changes

Admin support is still a work-in-progress,
I usually just find myself using "mogadm" anyways.

* new admin commands: "replicate_now"

* "get_stats" no longer works on new mogilefsd versions

* get_domains handles "repl_policy" field correctly for classes
  in MogileFS 2.x

* admin commands should convert all numeric fields to either
  Integer or Float objects and not String representations
  of numerics.  Affected methods include:
   get_hosts, get_devices, list_fids, each_fids, get_domains

=== miscellany

* we no longer add methods to standard Ruby classes
  (at least we never /changed/ existing methods :P)

* {kgio}[http://bogomips.org/kgio] automatically used
  if available, but not required to avoid exceptions
  with non-blocking I/O

* dropped Ruby 1.8.6 support, 1.8.7 or later is required.

* Tested with MRI 1.8.7, 1.9.3 and Rubinius 1.2.4

* MogileFS::Mysql - deprecated, to be removed in 2012

* improved test suite

* some optional experimental features/changes,
  see "git log" for details

* Added internal pipelining implementation, this is
  not easy-to-use since the server can respond-out-of-order,
  but still useful for things like a list_keys+file_info
  loop.

mogilefs-client 2.2.0 / 2011-01-13 02:10 UTC

= 2.2.0
* internal cleanups (no public API breakage)
* refactor backend socket/connection handling for reliability

There'll probably be more aggressive cleanups/refactoring in
future releases if I have time.

mogilefs-client 2.1.0 / 2009-04-12 03:04 UTC

= 2.1.0
* MySQL interface returns integer length and devcount (API change)
* Ensure store_{content,file} always returns size (API fix)
* Add get_uris API method
* Respect timeout when doing get_file_data
* MySQL interface filters out URLs for down/dead hosts/devices
* Really remove all NFS support
* get_file in slurp mode slurps all output correctly

Eric Wong (18):
      tests: retry random ports correctly
      test_mogilefs: fix race conditions
      MySQL interface returns integer length and devcount
      Respect timeout when doing get_file_data
      Unify internal HTTP GET/HEAD methods
      Close socket we create before raising exceptions
      Compact get_paths output so we don't have nils
      Really remove all NFS support
      README: add a note about emailing me
      Extra checking for full_timeout in sysread_full
      Add get_uris API method
      verify_uris: use write_nonblock instead of syswrite
      README: add links to the repo.or.cz mirror
      mysql: filter out URLs for down/dead hosts/devices
      mog: small cleanup
      Ensure store_{content,file} always returns size
      GNUmakefile: better logging/output control
      mogilefs-client 2.1.0

godfat (1):
      call IO.select([sock]) if sock is not ready.

mogilefs-client 1.2.1 / 2009-03-25 07:29 UTC

Retroactively tagging this old release since
it was in SVN or Perforce...

mogilefs-client 2.0.2 / 2009-03-02 21:56 UTC

initial

mogilefs-client v2.0.1 / 2009-02-25 08:13 UTC

* verify_uris method made more robust
* preliminary Ruby 1.9 compatibility, tests still need some work
* allow store_content to be used with a streamable object of known length
* add setup.rb for non-RubyGems users

mogilefs-client 2.0.0 / 2009-02-03 04:27 UTC

initial

v1.3.1 / 2008-10-03 03:01 UTC

* Fix missing MogileFS::Util include for sysrwloop in MogileFS::MogileFS

v1.3.0 / 2008-09-18 23:46 UTC

* Fixed MogileFS#rename.  Bug #14465 submitted by Justin Dossey.
* Removed infinite loop in MogileFS::HTTPFile#store_file.  Patch #13789
  submitted by  Andy Lo-A-Foe.
* Made MogileFS#get_file_data timeout configurable.  Bug #13490 submitted by
  Andy Lo-A-Foe.
* Add MogileFS#size.  Feature Request #14484 submitted by Justin Dossey.
* Fix MogileFS#get_file_data to return the data for HTTP mode.  Bug #7133
  submitted by John Wanko.
* New maintainer: Eric Wong
* Add `mog' command-line tool as a demo/example
* Lower memory consumption with large files
* Allow get_file_data to accept a block for large files
* Fix each_keys loop termination condition
* Apply error handling patch from Matthew Willson.  Bug #15987
* Merge large file patch from Andy Lo-A-Foe.  Bug #13764

mail archives: https://yhbt.net/mogilefs-client-public/
	http://ou63pmih66umazou.onion/mogilefs-client-public/ 
	nntp://news.public-inbox.org/inbox.comp.file-systems.mogilefs.ruby 
	nntp://ou63pmih66umazou.onion/inbox.comp.file-systems.mogilefs.ruby 
public: mogilefs-client-public@yhbt.net
source code: git clone https://yhbt.net/mogilefs-client.git
	torsocks git clone http://ou63pmih66umazou.onion/mogilefs-client.git