Re: cmogstored response header

[Eric Wong <e@80x24.org>]

David Birdsong <david.birdsong@gmail.com> wrote:
> Hi, I'm using nginx in front of cmogstored and trying to use nginx to
> measure time-to-first-byte w/ the $upstream_header_time variable.
> 
> I'd like to use this measurement as a way to keep an eye on device I/O and
> request saturation.
> 
> Does cmogstored start writing response headers before successfully opening
> the file or does it write the header only after a successful stat?

Successful stat(), or more accurately: open/openat() + fstat().
cmogstored needs to know the size + mtime before writing the
response header.  So yes, relying on the header response time
should give you a good idea of seek latency...

Under Linux, we use MSG_MORE with send() in an attempt to bundle
the first chunk of the response body into the same packet of the
response header; so you may need to take that into account, too.

TCP_NOPUSH is used under FreeBSD for a similar effect.
We used to use TCP_CORK under Linux, but send(..., MSG_MORE) is
fewer syscalls...

<tangent>
Theoretically, we could use the iovecs with the *BSD sendfile
implementation to reduce syscalls under *BSD; but I'm hesitant
to complicate the code for OSes I rarely use.  Handling
partial writes with iovecs in *BSD sendfile can get tricky.
</tangent>