about summary refs log tree commit homepage
DateCommit message (Collapse)
2013-02-28mgmt: check for client death during fsck checksumming socket_alive
Clients may start an fsck checksum request and not be around to read the response. So detect client death and abort checksumming if we have a dead socket. This is not extensively tested and may be overkill.
2013-02-23mgmt: fix fsck digest assert failure in graceful shutdown
Items in the low-priority fsck queue could trigger a assertion failure during graceful shutdown due to improper handling of the MOG_NEXT_IGNORE state in mog_mgmt_quit_step(). However, using the fsck queue in graceful shutdown (which is single-threaded) is probably a bad idea anyways, as the fsck digest could monopolize other requests. So give no special handling to fsck digest queries during graceful shutdown. This only affects users running fsck with checksumming enabled during a graceful shutdown of cmogstored. For checksums users, it is recommended to stop fsck from the trackers and wait for all tracker queues to drain before upgrading cmogstored (and using graceful shutdown on the old cmogstored).
2013-02-23http_get: comment about snprintf() being a hot spot
cmogstored is pretty fast, but it could be faster.
2013-02-21queue_common: update comments to match code
While we're at it, explain the use of cloexec.
2013-02-18document/reserve SIGWINCH/SIGHUP for future use v1.2.0
Despite having an extensive test suite and minimal room for user error, giving users the options to back out of a hot upgrade may be worth supporting.
2013-02-18copyright comment updates for 2013 (part 2)
Many files were missed the first time around in commit 37026af96dec638aa850d604003bf7218d90037d
2013-02-18manpage: document SIGUSR2 upgrades
This is a new feature and needs to be documented.
2013-02-18move cmogstored_exit() prototype to cmogstored.h
This fixes a missing prototype warning for cmogstored_exit() when checking exit.c with sparse.
2013-02-18queue_epoll: fix bad cast for epoll.event
The events field of struct epoll_event is a uint32_t, not int.
2013-02-18tests: add valgrind supp for epoll_ctl on 32-bit arch
The epoll_event.data union is 64-bits on 32-bit systems while pointers are 32-bit. We only use 32-bits of that union, but valgrind mistakenly complains about it (the kernel does not care about the user-supplied data union at all).
2013-02-18ioutil: fix memory access error on from mog_iou_write
sizeof(buf) returns the size of the pointer if buf is a passed parameter, even if it the function prototype dictates a fixed size for buf as we do in mog_iou_write. While we're at it, make our mog_iou_write buf parameter const. This bug was introduced in: commit a960a351b2248a196c91cdbf6256f98e1bc2ef37 "split iostat util% tracking from mountlist" and never affected an official release of cmogstored. This bug was caught while testing on a 32-bit GNU/Linux machine. My normal 32-bit FreeBSD 9.0 environment did not catch this as iostat on that platform only reports integer percentages and does not need more than 4 bytes.
2013-02-16handle pthread_create returning ENOMEM on old glibc
Older glibc will return ENOMEM on mprotect() failures. This bug was only fixed in 2011, so the long-term distros and old installations may not have the necessary backports. ref: http://www.sourceware.org/bugzilla/show_bug.cgi?id=386
2013-02-16graceful handling of pthread_create EAGAIN failure
pthread_create may return EAGAIN as a temporary failure, do not abort a running process if this is the case. For the initial mountlist scan, we must retry indefinitely for cmogstored to be usable. However, with our thread pools, we can always run fewer threads (as long as there is at least one thread per-pool).
2013-02-16test/http_idle_expire: hopefully improve test reliability
This is a tricky test and doesn't always succeed, since it's hard to tell how many file descriptors glibc will use internally.
2013-02-15sig: avoid pselect if ppoll is present in mog_sleep
We want to favor ppoll over pselect, since ppoll is a better interface and we can have a slightly smaller binary with fewer dependencies. While we're at it, use mog_sleep(-1) as an alias for mog_selfwake_wait to further reduce binary size.
2013-02-15avoid racy sleep on fork failure in master process
We need to atomically enable interrupts and sleep with the same syscall. Fortunately, using pselect (through mog_sleep) allows that and is POSIX-compliant, so use that.
2013-02-15mnt: inform user of slow mountlist scan
This will inform the user of why cmogstored may be slow to start, since we need the mountlist to be populated at startup. We also throw a pthread_cancel() in there to load libgcc_s under glibc, so we can avoid loading libgcc_s once we're under FD pressure. This makes test/http_idle_expire.rb more reliable.
2013-02-14test/http_range: do not allow webrick to perform lookups
DNS lookups cause webrick tests to fail or timeout. Our tests should not have external network dependencies.
2013-02-14inherit: avoid DNS lookup on upgrade
A typo caused unnecessary DNS lookups when inheriting sockets. While we're at it, fix another typo in the error message, too.
2013-02-14selfwake: use epoll_pwait on Linux instead of eventfd
This saves us a file descriptor in Linux, which provides epoll_pwait in 2.6.19+ (and ppoll for 2.6.18, the oldest kernel we support).
2013-02-11mnt: revert to mutex for protecting by_dev hash
Since we now update future copies of by_dev offline and only need a lock to swap in the new one, contention for by_dev should be less of a problem than it was before. This should make reader-writer locks an unnecessary risk. Reader-writer locks are riskier since writer starvation can potentially be an issue with many readers.
2013-02-11test/mogilefs_integration: increase test reliability
Use SO_REUSEADDR, since Linux requires both the new program (cmogstored) and this test both use SO_REUSEADDR for SO_REUSEADDR to be effective. Also, minimize the window for port conflicts. While there are hard-to-avoid race windows for conflicts when binding random ports, we can minimize those windows by holding those ports open in the parent as long as possible.
2013-02-11kqueue: update NOTIFYRD -> SELFWAKE
This was missed in the earlier changes to allow eventfd usage under Linux instead of using an notification pipe.
2013-02-11fix signal races when master process is used
In the absence of a pselect/ppoll-like version of waitpid; we must use a selfwake descriptor (pipe or eventfd) to wake the master up whenever a signal is received. So wait on the selfwake descriptor and always run waitpid with WNOHANG in a loop to ensure all children are reaped. The: mog_intr_disable(); waitpid(); mog_intr_enable() sequence was completely stupid I can't believe I wrote it.
2013-02-11allow self-wakeup to use eventfd under modern Linux
eventfd uses fewer resources than a pipe, so create less overhead for our users by using eventfd instead of a pipe.
2013-02-11pidfile: delay unlink of old file on aborted upgrades
We don't want to be without any pidfile if writing the new pidfile fails.
2013-02-11upgrade: do not disable interrupts in forked child
The child disables interrupts right away, so there's no reason to enable interrupts temporarily.
2013-02-11test/upgrade: more thorough PID file checking
We need to ensure the PID file is non-empty, not just that it exists.
2013-02-11prioritize upgrade before exit in main loop
If we receive both SIGUSR2 and SIGQUIT in a short time period; we should trigger the upgrade before gsince raceful exit; as no user will (intentionally) send SIGQUIT before SIGUSR2.
2013-02-11test/upgrade: teardown more careful about killing
We don't want to accidentally kill ourselves by targeting PID=0 if the PID file is empty.
2013-02-09tests: fix several Ruby warnings
Unused variables and unset Content-Type for Net::HTTP requests
2013-02-09test/inherit: fix Ruby 2.0.0 close-on-exec compatibility
FD inheritance from exec() must be done explicitly in Ruby 2.0
2013-02-08mnt: move stat/lstat logic to mnt_usable
This centralizes the mountpoint suitability logic in one place. In the future, it may also allow us to parallelize the work of scanning filesystems.
2013-02-08upgrade: fix env placeholder for valgrind
Having a NULL at the beginning of the list caused iteration in the destructor to stop, allowing valgrind to detect a memory leak.
2013-02-08cfg: require PATH to be set for --daemonize
Maybe some weird users do not have PATH
2013-02-08upgrade: avoid non-async-safe functions in child
execvp may malloc internally in its path lookup, so use find_in_path to perform this lookup in the parent instead. Additionally, putenv() may not be async-signal-safe either, but execve is, so use execve.
2013-02-07cfg: disallow trailing ':' in PATH with daemonize
Trailing ':' in PATH means using the current path, which is now incompatible with daemonize.
2013-02-07upgrade: avoid potential deadlock from post-fork mutex use
Pthreads implementations do not require mutexes be in a consistent/usable state in a forked child Since we don't need the mutex in a single-threaded forked child, we can just skip it and avoid reinitializing it entirely.
2013-02-07rename fs_usable to mnt_usable
It should be clearer this code is only called from inside mnt.c and not fs.c (the latter is for general filesystem operations, not operations on a mount point).
2013-02-07release memory allocated for upgrade at exit
This is not strictly necessary as this memory is freed anyways, but stop valgrind from complaining and avoid unnecessary suppressions (since shutdown performance is not important).
2013-02-07forbid relative paths with daemonization
Relative paths are incompatible with daemonization, as it does not work for SIGUSR2 upgrades (since daemonize forces the server to run in "/"). Relative paths are confusing and error-prone anyways, so do not allow users to specify them along with --daemonize.
2013-02-07omit trailing newline from die() and warn() calls
The GNU error() function already emits a newline at the end of these messages.
2013-02-07favor error.h GNU system header over gnulib one
error.h is available on GNU/Linux (and presumably GNU/kFreeBSD and GNU/Hurd, so favor that system-wide header over the gnulib one.
2013-02-06remove warn module and alias it to error() in gnulib
There is no need to maintain our own code for this.
2013-02-06queue_epoll: change fprintf(stderr, ...) to use warn()
This makes it easier alter how/if we write to stderr.
2013-01-31ioutil: avoid assigned but unused variable
Noticed with gcc 4.7.2 in Debian testing (4.7.2-5)
2013-01-31cmogstored: initialize syslog before inheriting
This ensures the: inherited $ADDRESS:$PORT on fd=... messages are prefixed with the PID in logs.
2013-01-31cfg: daemonize is a boolean, not an integer
This project uses C99 features (and some GNU extensions), so bool is usable.
2013-01-31sockaddr*-related data structure size reductions
We do not need all the weight of sockaddr_storage or NI_MAXHOST. cmogstored currently only supports IPv4 and IPv6[1] and (like any respectable server) will not perform reverse DNS lookups. This allows us to reduce our stack usage in some places and keep caches hotter. [1] MogileFS does not support IPv6, yet, even
2013-01-31minimize interrupt windows for master process
Code is easier to follow when interrupts occur at well-defined points. The worker processes (and master-less standalone) already follows this.