about summary refs log tree commit
david - an HTTP load testing tool aimed at fork/threading servers

david (as in david vs. Goliath) is a slow HTTP client intended for
server administrators test behavior when many slow clients are connected
to it.  It is very small, but can open many active connections which can
cause the server to deny connections from other clients.

Highly recommended reading: http://kegel.com/c10k.html


======= The Problem =======

Misconfigured forking/threading servers can come under a lot of load
when lots of traffic from slow clients connect to it and processes.

Per-IP connection limits can prevent david from doing harm to other
clients attempting to access a server, but it cannot prevent real-world
scenarios where many users on slow Internet connections are all
attempting to to access an HTTP site.

Avoiding HTTP servers that use the per-client thread/process model is a
good start.  However if that's not possible (heavy reliance on mod_perl,
mod_php, etc), then using a reverse proxy (nginx[1] is a good candidate)
to buffer and spoon-feed slow clients works well, too.

Servers using one client per thread/fork require an entry in the process
table (or thread table), and additional stack memory usage.  Even with
shared memory and copy-on-write fork, this memory usage is much bigger
than an implementation that avoids these things entirely and relies on
I/O readiness notification for each.  The addition of kernel thread or
processes for each connection also places additional burden on the
kernel scheduler and giving other tasks less time to run.

[1] nginx just seems to drop connections that are going too slowly (1
byte/second).  I doubt many users actually use connections that slow.

======= Inspiration =======

Many application servers I've seen often use at least 20-30 megabytes of
memory (however not taking shared memory after fork into account).
With Apache, when many clients connect, more processes are spawned (up
to a limit defined in a configuration file).

These server processes exist to execute application code quickly,
however while they are often underutilized because they are waiting on
I/O from slow clients.  So while waiting on I/O from slow clients,
the application does little else other than using a sizable chunk
of memory on the system.


======= Usage =======

david sends HTTP requests from the first filename passed to it on the
command-line so it can easily be customized by the user.  Included is
are barebones HTTP 1.0 and HTTP 1.1 request I use for testing.

Configuration is done mostly via editing pre-processor #defines at the
top of the source file.  The HTTP_HOST environment variable must be
defined for david to connect to a server.  HTTP_PORT is optional and
defaults to 80.

Command-line:

  HTTP_HOST=www.example.com ./david http_1.1.txt


======= Implementation =======

david is a simple select()-based client.  However, it uses child
processes to get around file descriptor limits on select() and also
per-process limits.  I'm choosing select() for portability and
simplicity over more exotic, less-portable I/O readiness schemes.

david itself uses very little in the way of system resources other than
file descriptors.  Since it is used to simulate slow clients, it
reads and writes to the socket very slowly and is not performance
intensive.

This has only compiled this on a Linux 2.6-based system, but it is
believed to be portable to all systems supporting mmap, and non-blocking
TCP sockets.  mmap() is used so that a user could write arbitrarily
large requests without causing david to use large amounts of resident
memory.


======= Disclaimer =======

Like many tools, david can be misused with malicious intent.  The author
does not condone its usage on any machine without the express permission
of its administrator(s).


======= License =======

This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
General Public License for more details.  You should have received a
copy of the GNU General Public License along with this program; if not,
write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330,
Boston, MA  02111-1307  USA