about summary refs log tree commit homepage
DateCommit message (Collapse)
2011-11-11list_keys/each_key: better handling of verbose listings pipeline
This allows us to implement "mog ls -l" much more efficiently
2011-11-11pipeline: make errors easier to debug by including the request
Otherwise it's outright painful to figure out what failed in a pipeline...
2011-11-11initial pipeline API
This is useful for the "verbose" listing of keys since we make a lot of file_info calls here. This API feels very awkward, but I think it's unavoidable...
2011-11-11test/test_backend: fix broken escaping test
This is a followup to commit 55de4a3375793fa31993a1e9b4be777007bd31b8 (url_unescape: fix ordering of "+" => " " of swap)
2011-11-11Rakefile: packaging updates
2011-11-11url_unescape: fix ordering of "+" => " " of swap
Otherwise we'll be converting "%2B" into " " instead of "+" when it appears in a file name.
2011-11-10backend: factor out dispatch unlocked
We may use this for pipelining
2011-11-10doc: misc updates
2011-11-10admin: use Object#__send__ since Object#send can be overwritten
2011-11-10backend: avoid eval for defining error classes
2011-11-10Revert "Compact get_paths output so we don't have nils"
This reverts commit f0ed5cb6ec6851a367175a93398e813e2d62667f. Unnecessary paranoia, our Backend is more robust nowadays by requiring CRLF before parsing
2011-11-10mogilefs/mogilefs: make @domain an accessor
It makes sense for use with MogileFS::Pool (in case somebody is pooling connections).
2011-11-10get_paths defaults to noverify: true
This restores 2.x behavior and is faster and safer since we're smart enough to deal with failover. Also dropping default @zone support since it doesn't seem that useful.
2011-11-10test_http_reader: fix stupid/broken assertion
I shouldn't be allowed to code.
2011-11-10test/test_http_reader: fix unused var warning
2011-11-10documentation/packaging updates
Trying to find a happy medium within Hoe while keeping my preference for gmake and forcing wrongdoc on readers: JavaScript and GUIs all suck :P
2011-11-10backend: more descriptive error for backends being unreachable
Each backend could have a different story to tell
2011-11-10ensure get_file_data users notice truncated responses
We don't want to silently truncate data on our users, that would be bad.
2011-11-10socket/pure_ruby: swap IO#wait for IO#select
The return value of IO#wait is strange and confusing, relying on FIONREAD is a waste of time anyways.
2011-11-10test: set Thread.abort_on_exception= true
Should make tests easier to fix and debug.
2011-11-10backend: fix idempotent retry logic
We don't want to blindly retry on invalid keys and such, only on invalid (truncated) responses, timeouts, and syscall errors.
2011-11-09fix warnings, oops
Found with 1.9.3
2011-11-09backend: automatically retry on idempotent commands
Read-only commands to the MogileFS tracker may be safely retried if a request is sent but no response is received.
2011-11-09avoid NilMethodError if get_file_data is passed an invalid key
It should raise on an unknown key, of course.
2011-11-08Rakefile: remove ZenTest test dependency
I don't use any of its tools
2011-11-08get_file_data supports offset and count
Just like IO.copy_stream
2011-11-08update Manifest (ugh)
2011-11-08get_paths: expand and improve tests
get_paths may take a Hash for its optional arguments and now supports the optional :pathcount argument. There may now be a default @zone for MogileFS::MogileFS objects as well (specified via :zone).
2011-11-08get_uri: object allocation reduction
Using Array#map! instead of Array#map can save us at least one object allocation.
2011-11-08test/aggregate: not all Ruby implementations support "-n" well
At least my year-old Rubinius installation does not
2011-11-08test/exec: remove "uuid" gem dependency on 1.8
Just borrow the code from 1.9.3
2011-11-08copy_stream (1.8) fix close issue
Not sure exactly what was causing it...
2011-11-08bigfile: explicitly close HTTPReader socket
No need to unnecessarily trigger GC nor hit EMFILE/ENFILE on VMs that rarely GC IO objects...
2011-11-08http_reader: remove support for overriding http_method
GET is all we need and we can save some code this way. If we ever need to use HEAD much again, we can use net/http/persistent since our internal HTTP classes are only optimized for large responses.
2011-11-08split deprecated paths_size to its own file
This is only needed for users on old MogileFS servers
2011-11-08http_* shorter error message for failures
Since we'll have multiple tries, try to limit errors.
2011-11-08backend: revert 1.8 splat breakage
Oh well, not a big savings there anyways unlike the inner loop.
2011-11-08get_file_data: avoid exposing users to copy_stream invocation
We'll now accept an optional argument which can be passed to IO.copy_stream directly. This should make life easier on users so they won't be exposed to our internals to make efficient copies of large files.
2011-11-08backend: small garbage reduction for url_decode
Obviously I could go farther, but not at the expense of readability. There are C libraries I could use, but MogileFS may move to a JSON-based protocol in the future anyways...
2011-11-08size/list_keys improvements
We can use the file_info command to get things faster, now.
2011-11-08test/integration: skip dummy test with minitest
setup is expensive with integration test since we wait for the monitor
2011-11-08tests: migrate tests for modifying classes to test_fresh
classids can get recycled, it seems.
2011-11-08mog: "ls -l" shows full key name
2011-11-08Fix Ruby 1.8 compatibility
Our custom copy_stream needs to flush data like it does under 1.9. 1.8 also can't do a blocking open(2) on a FIFO in a native thread so we'll fork() instead.
2011-11-08add support for the file_debug command
This was added in MogileFS 2.45
2011-11-08add file_info command support and test
This is a command added in MogileFS 2.45
2011-11-08tests: include test for opening FIFO paths
It's conceivable we'd need it.
2011-11-08Disable retries when storing non-rewindable data
If a user tries to pipe something to us and we can't rewind on failure, propagate that error all the way up to avoid risking a corrupted upload.
2011-11-07http_reader: fix usage with pure sockets
We won't redefine the "new" singleton method since that conflicts with existing usage.
2011-11-07avoid circular requires
Ruby 1.9.3 considers them harmful