diff options
author | Eric Wong <normalperson@yhbt.net> | 2012-11-14 21:27:11 +0000 |
---|---|---|
committer | Eric Wong <normalperson@yhbt.net> | 2012-11-14 21:33:47 +0000 |
commit | 07f7cfcddc1d1baacf76e0a84007901292a710eb (patch) | |
tree | e8e603c5d6b57df7ab98789debae813d64eafcc1 | |
parent | b615dd35ac805176cd1284145b0c0a75f2ce5554 (diff) | |
download | cmogstored-0.8.0.tar.gz |
Fixes manpage references to include sections.
-rw-r--r-- | README | 8 | ||||
-rw-r--r-- | cmogstored.x | 4 | ||||
-rw-r--r-- | doc/queues.txt | 38 |
3 files changed, 30 insertions, 20 deletions
@@ -14,15 +14,15 @@ Features ======== * low memory footprint, more memory for file system caches -* multithreaded design keeps queues in rotational disks busy +* multithreaded design[1] keeps queues in rotational disks busy * epoll/kqueue used to cheaply maintain persistent connections -* supports MD5 checksum extension for mogstored +* supports checksumming features in MogileFS::Server 2.60+ * easily installable on modern GNU/Linux and FreeBSD without Perl * supports HTTP/1.1 persistent connections and pipelining * supports chunked and partial (Content-Range) HTTP/1.1 PUT requests * supports partial GET requests * may reject PUTs based on Content-MD5 header/trailer verification -* graceful shutdown via SIGQUIT won't terminate active requests +* graceful shutdown via SIGQUIT will not terminate active requests * extensive test suite, code coverage and Valgrind-tested Getting Started @@ -82,3 +82,5 @@ http://bogomips.org/cmogstored/README The latest releases are announced via Atom feed: http://bogomips.org/cmogstored/NEWS.atom.xml + +[1] http://bogomips.org/cmogstored/queues.txt diff --git a/cmogstored.x b/cmogstored.x index 8ef9d0c..80552c3 100644 --- a/cmogstored.x +++ b/cmogstored.x @@ -1,3 +1,6 @@ +[SIGNALS] +SIGQUIT - gracefully shutdown the server + [REPORTING BUGS] Only bug reports in plain-text email will be read. You may also report bugs publically to the MogileFS mailing list @@ -16,4 +19,3 @@ You can learn more about MogileFS: http://mogilefsd.org/ cmogstored source code is available via git: git clone git://bogomips.org/cmogstored.git - diff --git a/doc/queues.txt b/doc/queues.txt index 432571e..a43613a 100644 --- a/doc/queues.txt +++ b/doc/queues.txt @@ -8,13 +8,13 @@ cmogstored. Multithreading is required for concurrent disk (and page cache) I/O. Most operations with cmogstored are not CPU-intensive, so pinning -cmogstored to a single CPU or core via schedtool /may/ even be +cmogstored to a single CPU or core via schedtool(1) /may/ even be beneficial. cmogstored is designed for disk-intensive operations where I/O latency is unpredictable and often high. This design is susceptible to the "ping-pong" effect where per-client data gets bounced between different -CPUs, so it may be suboptimal for CPU/L1/L2-intensive applications. +CPUs, so it may be suboptimal for CPU/L1/L2-intensive applications listen queue + accept() thread pool @@ -24,31 +24,34 @@ Like all TCP servers, there is a standard listen queue for every listen socket we have (mgmt/http). Each listen queue has a dedicated thread pool running _blocking_ -accept() syscall in a loop. We use dedicated threads and blocking -accept() to benefit from "wake-one" behavior in the Linux kernel. +accept(2) (or accept4(2)) syscall in a loop. We use dedicated threads +and blocking accept to benefit from "wake-one" behavior in the Linux +kernel. -There is one accept() thread pool for every listen socket. +There is one accept(2) thread pool for every listen socket. mog_queue thread pool --------------------- -epoll (or kqueue) descriptor is the heart of "struct mog_queue". +epoll(2) (or kqueue(2)) descriptor is the heart of "struct mog_queue". There is one thread pool in cmogstored dedicated to sharing a single mog_queue object. This design allows clients to migrate between different threads. This is beneficial if one thread is blocked on disk I/O (including seeks), a thread will only service one client at a time. -Currently the mog_queue is a singleton, giving it optimal fairness in -worst-case scenarios at the expense of concurrent throughput. +Currently the mog_queue is a singleton, giving it optimal fairness +across the machine in worst-case scenarios at the expense of concurrent +throughput. idle queue ========== -This is either an epoll or kqueue descriptor. Unlike traditional -poll()/select(), epoll/kqueue easily allows clients to migrate -between threads as they become ready. +This is either an epoll(2) or kqueue(2) descriptor. Unlike traditional +poll(2)/select(2), epoll/kqueue easily allows clients to migrate between +threads as client sockets become ready. -To implement queue-like behavior, we rely on one-shot notifications. +To implement queue-like behavior, we rely exclusively on one-shot +notifications (EPOLLONESHOT or EV_ONESHOT). active queue ============ @@ -58,16 +61,19 @@ both more complex and /theoretically/ open to starvation given a perfect storm of idle time. cmogstored currently places an object back in the epoll/kqueue object if -it's considered "active", and both systems queue internally to preserve -ordering of unretrieved readiness events. +it is considered "active", as both systems internally to preserve +ordering of unretrieved readiness events. The practice of queueing +"active" clients (clients which have not triggered EAGAIN) prevents +starvation when a client pipelines requests to us. It is analogous to +the use of sched_yield(2) in purely multi-threaded designs. Queue flow ---------- -Once a client is accept()-ed, it is immediately pushed into the +Once a client is accept(2)-ed, it is immediately pushed into the mog_queue. This mimics the effect of TCP_DEFER_ACCEPT[1] (in Linux) and the "dataready" accept filter (in FreeBSD) from the perspective -of the epoll_wait/kqueue caller. +of the epoll_wait(2)/kqueue(2) caller. TCP_DEFER_ACCEPT itself is not used as it has some documented and unresolved issues: |