From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-2.8 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00 shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: cmogstored-public@bogomips.org Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 01D9920431; Mon, 4 Apr 2016 00:29:45 +0000 (UTC) Date: Mon, 4 Apr 2016 00:29:45 +0000 From: Eric Wong To: mogile@googlegroups.com, David Birdsong Cc: cmogstored-public@bogomips.org Subject: Re: cmogstored response header Message-ID: <20160404002945.GA29666@dcvr.yhbt.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: List-Id: David Birdsong wrote: > Hi, I'm using nginx in front of cmogstored and trying to use nginx to > measure time-to-first-byte w/ the $upstream_header_time variable. > > I'd like to use this measurement as a way to keep an eye on device I/O and > request saturation. > > Does cmogstored start writing response headers before successfully opening > the file or does it write the header only after a successful stat? Successful stat(), or more accurately: open/openat() + fstat(). cmogstored needs to know the size + mtime before writing the response header. So yes, relying on the header response time should give you a good idea of seek latency... Under Linux, we use MSG_MORE with send() in an attempt to bundle the first chunk of the response body into the same packet of the response header; so you may need to take that into account, too. TCP_NOPUSH is used under FreeBSD for a similar effect. We used to use TCP_CORK under Linux, but send(..., MSG_MORE) is fewer syscalls... Theoretically, we could use the iovecs with the *BSD sendfile implementation to reduce syscalls under *BSD; but I'm hesitant to complicate the code for OSes I rarely use. Handling partial writes with iovecs in *BSD sendfile can get tricky.