Peter Griess's Bloghttps://betablog.std.in/2020-11-23T16:32:00-06:00HTTP content negotiation with typemaps2020-11-23T16:32:00-06:002020-11-23T16:32:00-06:00Peter Griesstag:betablog.std.in,2020-11-23:/http-content-negotiation-with-typemaps/<p>In my earlier posts about HTTP content negotiation (see <a href="/http-content-negotiation-on-aws-cloudfront">part 1</a> and <a href="/http-content-negotiation-on-aws-cloudfront-part-2">part 2</a>), I built a negotiation engine that runs on the Lambda@Edge AWS proxy, with configuration and policy specified inline in the implementation.</p>
<p>While this does work for my specific needs, it has some drawbacks that prevent …</p><p>In my earlier posts about HTTP content negotiation (see <a href="/http-content-negotiation-on-aws-cloudfront">part 1</a> and <a href="/http-content-negotiation-on-aws-cloudfront-part-2">part 2</a>), I built a negotiation engine that runs on the Lambda@Edge AWS proxy, with configuration and policy specified inline in the implementation.</p>
<p>While this does work for my specific needs, it has some drawbacks that prevent it from being deployed un-modified in front of an arbitrary static site.</p>
<ul>
<li>Configuration can only be specified on a global basis rather than per-resource. For example, WebP is universally preferred over JPEG. While WebP is great, </li>
<li>We have to guess what the available representations are based on the request, for example by assuming that all images are served with a <code>.jpg</code> suffix.</li>
<li>Paths for different representations are computed at runtime based on simple transformations, mandating how data is laid out in the backing directory structure.</li>
<li>It is possible to share this setup between sites unless they have the same layout, support the same set of representations, etc.</li>
</ul>
<p>We can address these issues by moving policy and configuration out of the engine.</p>
<p>Apache's content negotiation implementation supports this by computing a list of possible representations for a given resource and then feeding this into a data-driven algorithm described <a href="https://httpd.apache.org/docs/2.4/content-negotiation.html#methods">here</a>. Apache provides two mechanisms of building this list:</p>
<p><strong>Multiviews</strong> At request time, Apache applies fixed transformations to the request path (e.g. mapping <code>foo.html</code> to <code>foo.html.gz</code>). This requires I/O to discover and parse all existing representations. The proxy must perform a remote directory scan, for which there is no standard HTTP mechanism. Without that, it must guess at some common transformations (e.g. try appending <code>.gz</code>) and issue a large number of requests upstream to determine which are viable.</p>
<p><strong>Typemaps</strong> At request time, Apache loads a typemap file corresponding to the request path. The typemap contains an operator-specified list of representations, URL from which to retrieve them, and associated metadata, including header attributes. This list can be generated at any time, but static sites likely want to do this as part of their build process. The ASCII format is simple enough to generate that it can be done with a trivial shell script. Since this is fetched from the origin over HTTP, these typemaps can be cached with other resources.</p>
<p>Typemaps solve all of the problems called out at the beginning of this post -- the metadata to allow the content engine to negotiate representations is computed outside of the proxy and in a format that is both easy to generate and parse. Because there is no specified mechainsm to generate this file, its contents can reflect whatever policy the origin wishes.</p>
<p>I've updated <a href="https://github.com/pgriess/http-negotiator">http-negotiator</a> to support typemaps, and now the Lambda@Edge request handler for this blog has no site-specific policy or configuration.</p>
<p>It now looks like the following</p>
<div class="highlight"><pre><span></span><code><span class="s1">'use strict'</span><span class="p">;</span>
<span class="kd">const</span> <span class="p">{</span>
<span class="nx">ValueTuple</span><span class="p">,</span>
<span class="nx">awsPerformTypemapNegotiation</span><span class="p">,</span>
<span class="nx">typemapParse</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'http-negotiator'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">http</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'http'</span><span class="p">);</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">URL</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'url'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">DEFAULT_DOCUMENT</span> <span class="o">=</span> <span class="s1">'index.html'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">ORIGIN_BASE_URL</span> <span class="o">=</span> <span class="s1">'http://s3.us-west-2.amazonaws.com/blog.std.in'</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">SERVER_ENCODING_WHITELIST</span> <span class="o">=</span> <span class="ow">new</span> <span class="nb">Set</span><span class="p">()</span>
<span class="kd">const</span> <span class="nx">SERVER_TYPE_WHITELIST</span> <span class="o">=</span> <span class="ow">new</span> <span class="nb">Set</span><span class="p">([</span><span class="s1">'image/gif'</span><span class="p">,</span> <span class="s1">'image/jpeg'</span><span class="p">,</span> <span class="s1">'text/html'</span><span class="p">,</span> <span class="s1">'text/plain'</span><span class="p">]);</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">context</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="p">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">request</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mf">0</span><span class="p">].</span><span class="nx">cf</span><span class="p">.</span><span class="nx">request</span><span class="p">;</span>
<span class="c1">// Pass through requests to fetch the typemap</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">.</span><span class="nx">endsWith</span><span class="p">(</span><span class="s1">'.var'</span><span class="p">))</span> <span class="p">{</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Generate the URL for the typemap; handle default documents</span>
<span class="kd">let</span> <span class="nx">varUri</span> <span class="o">=</span> <span class="nx">ORIGIN_BASE_URL</span> <span class="o">+</span> <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">varUri</span><span class="p">[</span><span class="nx">varUri</span><span class="p">.</span><span class="nx">length</span> <span class="o">-</span> <span class="mf">1</span><span class="p">]</span> <span class="o">==</span> <span class="s1">'/'</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">varUri</span> <span class="o">=</span> <span class="nx">varUri</span> <span class="o">+</span> <span class="nx">DEFAULT_DOCUMENT</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">varUri</span> <span class="o">+=</span> <span class="s1">'.var'</span><span class="p">;</span>
<span class="kd">let</span> <span class="nx">bodyBuffers</span> <span class="o">=</span> <span class="p">[];</span>
<span class="nx">http</span><span class="p">.</span><span class="nx">get</span><span class="p">(</span><span class="nx">varUri</span><span class="p">,</span> <span class="p">(</span><span class="nx">res</span><span class="p">)</span> <span class="p">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="p">{</span> <span class="nx">statusCode</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">res</span><span class="p">;</span>
<span class="c1">// No typemap found; pass though the request to the original object</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">statusCode</span> <span class="o">!=</span> <span class="mf">200</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">res</span>
<span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="s1">'error'</span><span class="p">,</span> <span class="p">(</span><span class="nx">err</span><span class="p">)</span> <span class="p">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="s1">'Error fetching typemap: '</span> <span class="o">+</span> <span class="nx">err</span><span class="p">.</span><span class="nx">message</span> <span class="o">+</span> <span class="s1">'; falling back to original'</span><span class="p">);</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="p">})</span>
<span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="p">(</span><span class="nx">buf</span><span class="p">)</span> <span class="p">=></span> <span class="p">{</span>
<span class="nx">bodyBuffers</span><span class="p">.</span><span class="nx">push</span><span class="p">(</span><span class="nx">buf</span><span class="p">);</span>
<span class="p">})</span>
<span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="s1">'end'</span><span class="p">,</span> <span class="p">()</span> <span class="p">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">res</span><span class="p">.</span><span class="nx">complete</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="s1">'Incomplete typemap; falling back to original'</span><span class="p">)</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Parse the typemap and perform negotiation</span>
<span class="kd">const</span> <span class="nx">selectedTm</span> <span class="o">=</span> <span class="nx">awsPerformTypemapNegotiation</span><span class="p">(</span>
<span class="nx">request</span><span class="p">.</span><span class="nx">headers</span><span class="p">,</span>
<span class="nx">typemapParse</span><span class="p">(</span><span class="nx">Buffer</span><span class="p">.</span><span class="nx">concat</span><span class="p">(</span><span class="nx">bodyBuffers</span><span class="p">).</span><span class="nx">toString</span><span class="p">()),</span>
<span class="ow">new</span> <span class="nb">Map</span><span class="p">([</span>
<span class="p">[</span><span class="s1">'accept'</span><span class="p">,</span> <span class="nx">SERVER_TYPE_WHITELIST</span><span class="p">],</span>
<span class="p">[</span><span class="s1">'accept-encoding'</span><span class="p">,</span> <span class="nx">SERVER_ENCODING_WHITELIST</span><span class="p">]]));</span>
<span class="c1">// XXX: This should return a 406; alas this requires using API gateway ;(</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">selectedTm</span> <span class="o">===</span> <span class="kc">null</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="s1">'Negotiation failed; falling back to original'</span><span class="p">)</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Swizzle the request URL, using the URL constructor to resolve</span>
<span class="c1">// relative URLs. The hostname is ignored so we fake it safely.</span>
<span class="kd">const</span> <span class="nx">u</span> <span class="o">=</span> <span class="ow">new</span> <span class="nx">URL</span><span class="p">(</span><span class="nx">selectedTm</span><span class="p">.</span><span class="nx">uri</span><span class="p">,</span> <span class="s1">'http://a/'</span> <span class="o">+</span> <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">);</span>
<span class="nx">request</span><span class="p">.</span><span class="nx">uri</span> <span class="o">=</span> <span class="nx">u</span><span class="p">.</span><span class="nx">pathname</span> <span class="o">+</span> <span class="nx">u</span><span class="p">.</span><span class="nx">search</span> <span class="o">+</span> <span class="nx">u</span><span class="p">.</span><span class="nx">hash</span><span class="p">;</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="p">});</span>
<span class="p">});</span>
<span class="p">};</span>
</code></pre></div>HTTP content negotiation on AWS CloudFront Part 22018-11-21T14:00:00-06:002018-11-21T14:00:00-06:00Peter Griesstag:betablog.std.in,2018-11-21:/http-content-negotiation-on-aws-cloudfront-part-2/<p><em>[Note -- this has been updated to fix a bug in handling Chrome]</em></p>
<p>My earlier <a href="https://blog.std.in/2018/09/14/http-content-negotiation-on-aws-cloudfront/">post on HTTP content negotiation in AWS CloudFront</a> covered support for negotiating the response encoding using the request's <code>Accept-Encoding</code> header. This post builds on that by adding support for negotiating <code>Content-Type</code> using the request's <code>Accept</code> header …</p><p><em>[Note -- this has been updated to fix a bug in handling Chrome]</em></p>
<p>My earlier <a href="https://blog.std.in/2018/09/14/http-content-negotiation-on-aws-cloudfront/">post on HTTP content negotiation in AWS CloudFront</a> covered support for negotiating the response encoding using the request's <code>Accept-Encoding</code> header. This post builds on that by adding support for negotiating <code>Content-Type</code> using the request's <code>Accept</code> header.</p>
<p>As <a href="https://www.igvita.com/2012/12/18/deploying-new-image-formats-on-the-web/">Ilya Grigorik laid out several years ago</a>, content negotiation tricky to get right, and yet particularly important when serving images. This is even more relevant today, as recently the browser ecosystem's support for WebP has taken a big step forward -- <a href="https://blogs.windows.com/msedgedev/2018/10/04/edgehtml-18-october-2018-update/">Edge just shipped support</a> and <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1294490">work on Firefox support has resumed after a long hiatus</a>. Furthermore, there continue to be interesting new image formats on the horizon such as AVIF and HEIF.</p>
<p>There are several reasons why <code>Content-Type</code> negotiation is more difficult than <code>Content-Encoding</code>. First, the media types being negotiated are hierarchical, supporting a type and subtype model, e.g. <code>image/png</code>. In addition, wildcards are supported, e.g. <code>image/*</code>. Finally, unlike <code>Accept-Encoding</code> when browsers explicitly send all of the encodings that they support, browsers tend not to do this with the <code>Accept</code> header. For example, Firefox sends <code>Accept: */*</code> when requesting images. This gives the HTTP server no indication of what the browser actually supports -- by the RFC the server would be allowed to return any content at all. As a result, servers are typically either conservative, returning only formats which are highly likely to be supported like <code>image/jpeg</code>, or fall back to heuristics like <code>User-Agent</code> sniffing to detect specific browser builds which support a server-preferred content type.</p>
<p>What is an HTTP server implementer to do? Apache's <code>mod_negotiation</code> has a <a href="https://httpd.apache.org/docs/current/content-negotiation.html">fairly sophisticated set of heuristics for supporting content negotiation</a> which covers some of this including working around usage of overly-permissive wildcards.</p>
<p>With AWS CloudFront, we can implement something similar to drive this process on Lambda@Edge. The code below is being used to serve this article, and will cause the image of a dog to be returned as WebP if your browser supports it.</p>
<div class="highlight"><pre><span></span><code><span class="s1">'use strict'</span><span class="p">;</span>
<span class="kd">const</span> <span class="p">{</span>
<span class="nx">ValueTuple</span><span class="p">,</span>
<span class="nx">awsPerformEncodingNegotiation</span><span class="p">,</span>
<span class="nx">awsPerformTypeNegotiation</span> <span class="p">}</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'http_content_negotiation'</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">SERVER_ENCODINGS</span> <span class="o">=</span> <span class="p">[</span>
<span class="ow">new</span> <span class="nx">ValueTuple</span><span class="p">(</span><span class="s1">'br'</span><span class="p">,</span> <span class="ow">new</span> <span class="nb">Map</span><span class="p">([[</span><span class="s1">'q'</span><span class="p">,</span> <span class="mf">1</span><span class="p">]])),</span>
<span class="ow">new</span> <span class="nx">ValueTuple</span><span class="p">(</span><span class="s1">'gzip'</span><span class="p">,</span> <span class="ow">new</span> <span class="nb">Map</span><span class="p">([[</span><span class="s1">'q'</span><span class="p">,</span> <span class="mf">0.9</span><span class="p">]])),</span>
<span class="ow">new</span> <span class="nx">ValueTuple</span><span class="p">(</span><span class="s1">'identity'</span><span class="p">,</span> <span class="ow">new</span> <span class="nb">Map</span><span class="p">([[</span><span class="s1">'q'</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">]]))];</span>
<span class="kd">const</span> <span class="nx">SERVER_IMAGE_TYPES</span> <span class="o">=</span> <span class="p">[</span>
<span class="ow">new</span> <span class="nx">ValueTuple</span><span class="p">(</span><span class="s1">'image/webp'</span><span class="p">,</span> <span class="ow">new</span> <span class="nb">Map</span><span class="p">([[</span><span class="s1">'q'</span><span class="p">,</span> <span class="mf">1</span><span class="p">]])),</span>
<span class="ow">new</span> <span class="nx">ValueTuple</span><span class="p">(</span><span class="s1">'image/jpeg'</span><span class="p">,</span> <span class="ow">new</span> <span class="nb">Map</span><span class="p">([[</span><span class="s1">'q'</span><span class="p">,</span> <span class="mf">0.5</span><span class="p">]]))];</span>
<span class="kd">const</span> <span class="nx">SERVER_IMAGE_WHITELIST</span> <span class="o">=</span> <span class="ow">new</span> <span class="nb">Set</span><span class="p">([</span>
<span class="s1">'image/jpeg'</span><span class="p">,</span>
<span class="p">]);</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">context</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="p">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">request</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mf">0</span><span class="p">].</span><span class="nx">cf</span><span class="p">.</span><span class="nx">request</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">.</span><span class="nx">endsWith</span><span class="p">(</span><span class="s1">'.jpg'</span><span class="p">))</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">type</span> <span class="o">=</span> <span class="nx">awsPerformTypeNegotiation</span><span class="p">(</span>
<span class="nx">request</span><span class="p">.</span><span class="nx">headers</span><span class="p">,</span> <span class="nx">SERVER_IMAGE_TYPES</span><span class="p">,</span> <span class="nx">SERVER_IMAGE_WHITELIST</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">type</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">uriWithoutExtension</span> <span class="o">=</span> <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">.</span><span class="nx">slice</span><span class="p">(</span><span class="mf">0</span><span class="p">,</span> <span class="o">-</span><span class="mf">3</span><span class="p">);</span>
<span class="k">switch</span> <span class="p">(</span><span class="nx">type</span><span class="p">.</span><span class="nx">value</span><span class="p">)</span> <span class="p">{</span>
<span class="k">case</span> <span class="s1">'image/webp'</span><span class="o">:</span>
<span class="nx">request</span><span class="p">.</span><span class="nx">uri</span> <span class="o">=</span> <span class="nx">uriWithoutExtension</span> <span class="o">+</span> <span class="s1">'webp'</span><span class="p">;</span>
<span class="k">break</span><span class="p">;</span>
<span class="k">case</span> <span class="s1">'image/jpeg'</span><span class="o">:</span>
<span class="c1">// Nothing to do</span>
<span class="k">break</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">else</span> <span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">.</span><span class="nx">startsWith</span><span class="p">(</span><span class="s1">'/gzip/'</span><span class="p">)</span> <span class="o">&&</span>
<span class="o">!</span><span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">.</span><span class="nx">startsWith</span><span class="p">(</span><span class="s1">'/br/'</span><span class="p">))</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">encoding</span> <span class="o">=</span> <span class="nx">awsPerformEncodingNegotiation</span><span class="p">(</span><span class="nx">request</span><span class="p">.</span><span class="nx">headers</span><span class="p">,</span> <span class="nx">SERVER_ENCODINGS</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">encoding</span> <span class="o">&&</span> <span class="nx">encoding</span><span class="p">.</span><span class="nx">value</span> <span class="o">!==</span> <span class="s1">'identity'</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">request</span><span class="p">.</span><span class="nx">uri</span> <span class="o">=</span> <span class="s1">'/'</span> <span class="o">+</span> <span class="nx">encoding</span><span class="p">.</span><span class="nx">value</span> <span class="o">+</span> <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div>
<p><img src="/images/39446898795_a8f4cb8a73_20p.jpg" width="100%" alt="Dog and purple flowers"/></p>
<h2>How does this work?</h2>
<p>Similar to my previous post, this uses the zero-dependency, MIT-licensed <a href="https://github.com/pgriess/http-content-negotiation-js">http-content-negotiation-js</a> library to run the content negotiation process. This library implements all of the requisite media range parsing and semantics, as well as some of the heuristics from <code>mod_negotiation</code>. For example, it treats matches against a subtype wildcard as having an implicit q-value of 0.02 if none of the media ranges in the request have an explicit q-value specified.</p>
<p>First, the <code>SERVER_IMAGE_TYPES</code> list of <code>ValueTuple</code> objects is created to represent our content type preferences for images. Note that we indicate that we have a preference for <code>image/webp</code> (implicit q-value 1) over <code>image/jpeg</code> (q-value 0.5), as the former compresses better. We also use <code>SERVER_IMAGE_WHITELIST</code> to track a whitelist of media types that we are willing to allow to match a wildcard. This handles the case where a browser sends <code>Accept: */*</code> but we're not sure if they really support WebP or not, in which case it's best to fall-back to something that we know is supported.</p>
<p>Next, the request handler looks for request URLs ending in <code>.jpg</code> and interprets this as a request for an image, performing type negotiation. We then rewrite the URL for the upstream request with a new file extension based on the negotiated content type.</p>
<p>Finally, it's worth noting that while type and encoding negotiation are not mutually exclusive, it is generally not worthwhile spending CPU cycles to encode and decode images. Because of this, we only bother performing encoding negotiation if we're not serving an image.</p>HTTP content negotiation on AWS CloudFront2018-09-14T14:10:00-05:002018-09-14T14:10:00-05:00Peter Griesstag:betablog.std.in,2018-09-14:/http-content-negotiation-on-aws-cloudfront/<p><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation">HTTP content negotiation</a> is a mechanism by which web servers consider request headers in addition to the the URL when determining which content to include in the response. A common use cases for this is response body compression, wherein a server may decide to gzip the content if the request …</p><p><a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Content_negotiation">HTTP content negotiation</a> is a mechanism by which web servers consider request headers in addition to the the URL when determining which content to include in the response. A common use cases for this is response body compression, wherein a server may decide to gzip the content if the request arrived with an <code>Accept-Encoding: gzip</code> header.</p>
<p>Support for content negotiation in HTTP servers is a mixed bag. Apache provides <a href="https://httpd.apache.org/docs/current/content-negotiation.html">good built-in support</a> for this. NGINX does not offer anything comparable although a <a href="https://www.igvita.com/2013/05/01/deploying-webp-via-accept-content-negotiation/">rough approximation</a> is possible via configuration directives. Unfortunately I can't find documentation on IIS. AWS S3 static website hosting, which is used to serve this blog, provides no facility for this whatsoever.</p>
<p>Over the past few years, CDNs have evolved to help address this problem in a few days.</p>
<p>Most CDNs can compress content on the fly, even if the origin only serves uncompressed. Support for gzip is de rigueur, with CloudFront supporting Brotli as well. In practice, however, this can be limited. For example, AWS CloudFront won't compress anything under 1KB or over 10MB. In addition, compression is typically more effective the more CPU you spend on it though this effect is non-linear. For example, <a href="https://www.rootusers.com/gzip-vs-bzip2-vs-xz-performance-comparison/">running gzip at level 9 can produce content that is 10s of percent smaller than level 1, but requires several times the processing power</a>. As a result, CDNs are typically configured to run at fairly low optimization levels.</p>
<p>Recently CDNs have also begun to allow applications to run business logic at the edge. CloudFlare workers, AWS Lambda@Edge and Fastly VCL are all examples of this.</p>
<p>Felice Geracitano had the clever idea to use Lambda@Edge on AWS CloudFront to <a href="https://medium.com/@felice.geracitano/brotli-compression-delivered-from-aws-7be5b467c2e1">implement a bare bones content negotiation scheme for the purpose of supporting Brotli</a>. While there are some issues with his implementation, the concept of performing content negotiation in JavaScript on the CDN and using the result to drive fetching a different resource from the origin is a powerful one.</p>
<h2>What does a good solution for this look like?</h2>
<ul>
<li>The origin server does not need to support content negotiation. This is both cheaper to operate and allows for offline processing of assets (e.g. to compress using gzip level 9 or Zopfli).</li>
<li>The content negotiation process should respect quality factors in the HTTP request headers, e.g. <code>Accept-Encoding: gzip, br;q=0.9</code> indicates that if content encodings for both <code>gzip</code> and <code>br</code> are available, it prefers <code>gzip</code>. </li>
<li>The CDN should serve content with a correct <code>Vary</code> header to ensure that downstream caches are not confused by the content negotiation process.</li>
<li>The results of JavaScript content negotiation logic should be cached by the CDN, and only re-executed on CDN cache misses.</li>
</ul>
<p>Below is an implementation of this for AWS CloudFront, and is being used to handle traffic to https://blog.std.in. The code is MIT licensed and is derived from <a href="https://github.com/pgriess/http-content-negotiation-js">pgriess/http-content-negotiation-js</a>.</p>
<p>First, the <code>http-content-negotiation</code> module:</p>
<div class="highlight"><pre><span></span><code><span class="cm">/*</span>
<span class="cm">MIT License</span>
<span class="cm">Copyright (c) 2018 Peter Griess</span>
<span class="cm">Permission is hereby granted, free of charge, to any person obtaining a copy</span>
<span class="cm">of this software and associated documentation files (the "Software"), to deal</span>
<span class="cm">in the Software without restriction, including without limitation the rights</span>
<span class="cm">to use, copy, modify, merge, publish, distribute, sublicense, and/or sell</span>
<span class="cm">copies of the Software, and to permit persons to whom the Software is</span>
<span class="cm">furnished to do so, subject to the following conditions:</span>
<span class="cm">The above copyright notice and this permission notice shall be included in all</span>
<span class="cm">copies or substantial portions of the Software.</span>
<span class="cm">THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR</span>
<span class="cm">IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,</span>
<span class="cm">FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE</span>
<span class="cm">AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER</span>
<span class="cm">LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,</span>
<span class="cm">OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE</span>
<span class="cm">SOFTWARE.</span>
<span class="cm">*/</span>
<span class="cm">/*</span>
<span class="cm">* Given an array of AWS Lambda header objects for headers that support</span>
<span class="cm">* ','-delimited list syntax, return a single array containing the values from</span>
<span class="cm">* all of these lists.</span>
<span class="cm">*</span>
<span class="cm">* Assumptions</span>
<span class="cm">*</span>
<span class="cm">* - HTTP headers arrive as an array of objects, each with a 'key' and 'value'</span>
<span class="cm">* property. We ignore the 'key' property as we assume the caller has supplied</span>
<span class="cm">* an array where these do not differ except by case.</span>
<span class="cm">*</span>
<span class="cm">* - The header objects specified have values which conform to section 7 of RFC</span>
<span class="cm">* 7230. For eample, Accept, Accept-Encoding support this. User-Agent does not.</span>
<span class="cm">*/</span>
<span class="kd">const</span> <span class="nx">splitHeaders</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">headers</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">headers</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">ho</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nx">ho</span><span class="p">[</span><span class="s1">'value'</span><span class="p">];</span> <span class="p">})</span>
<span class="p">.</span><span class="nx">reduce</span><span class="p">(</span>
<span class="kd">function</span><span class="p">(</span><span class="nx">acc</span><span class="p">,</span> <span class="nx">val</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">acc</span><span class="p">.</span><span class="nx">concat</span><span class="p">(</span><span class="nx">val</span><span class="p">.</span><span class="nx">replace</span><span class="p">(</span><span class="sr">/ +/g</span><span class="p">,</span> <span class="s1">''</span><span class="p">).</span><span class="nx">split</span><span class="p">(</span><span class="s1">','</span><span class="p">));</span>
<span class="p">},</span>
<span class="p">[]);</span>
<span class="p">};</span>
<span class="cm">/*</span>
<span class="cm">* Parse an HTTP header value with optional attributes, returning a tuple of</span>
<span class="cm">* (value name, attributes dictionary).</span>
<span class="cm">*</span>
<span class="cm">* For example 'foo;a=1;b=2' would return ['foo', {'a': 1, 'b': 2}].</span>
<span class="cm">*/</span>
<span class="kd">const</span> <span class="nx">parseHeaderValue</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">v</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">s</span> <span class="o">=</span> <span class="nx">v</span><span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="s1">';'</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">s</span><span class="p">.</span><span class="nx">length</span> <span class="o">==</span> <span class="mf">1</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="nx">v</span><span class="p">,</span> <span class="p">{}];</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">attrs</span> <span class="o">=</span> <span class="p">{};</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">forEach</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">av</span><span class="p">,</span> <span class="nx">idx</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">idx</span> <span class="o">===</span> <span class="mf">0</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="kd">const</span> <span class="nx">kvp</span> <span class="o">=</span> <span class="nx">av</span><span class="p">.</span><span class="nx">split</span><span class="p">(</span><span class="s1">'='</span><span class="p">,</span> <span class="mf">2</span><span class="p">)</span>
<span class="nx">attrs</span><span class="p">[</span><span class="nx">kvp</span><span class="p">[</span><span class="mf">0</span><span class="p">]]</span> <span class="o">=</span> <span class="nx">kvp</span><span class="p">[</span><span class="mf">1</span><span class="p">];</span>
<span class="p">});</span>
<span class="k">return</span> <span class="p">[</span><span class="nx">s</span><span class="p">[</span><span class="mf">0</span><span class="p">],</span> <span class="nx">attrs</span><span class="p">];</span>
<span class="p">};</span>
<span class="cm">/*</span>
<span class="cm">* Given an array of (value name, attribute dictionary) tuples, return a sorted</span>
<span class="cm">* array of (value name, q-value) tuples, ordered by the value of the 'q' attribute.</span>
<span class="cm">*</span>
<span class="cm">* If multiple instances of the same value are found, the last instance will</span>
<span class="cm">* override attributes of the earlier values. If no 'q' attribute is specified,</span>
<span class="cm">* a default value of 1 is assumed.</span>
<span class="cm">*</span>
<span class="cm">* For example given the below header values, the output of this function will</span>
<span class="cm">* be [['b', 3], ['a', 2]].</span>
<span class="cm">*</span>
<span class="cm">* [['a', {'q': '5'}], ['a', {'q': '2'}], ['b', {'q': '3'}]]</span>
<span class="cm">*/</span>
<span class="kd">const</span> <span class="nx">sortHeadersByQValue</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">headerValues</span><span class="p">)</span> <span class="p">{</span>
<span class="cm">/* Parse q attributes, ensuring that all to 1 */</span>
<span class="kd">var</span> <span class="nx">headerValuesWithQValues</span> <span class="o">=</span> <span class="nx">headerValues</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">vt</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">vn</span> <span class="o">=</span> <span class="nx">vt</span><span class="p">[</span><span class="mf">0</span><span class="p">];</span>
<span class="kd">var</span> <span class="nx">va</span> <span class="o">=</span> <span class="nx">vt</span><span class="p">[</span><span class="mf">1</span><span class="p">];</span>
<span class="k">if</span> <span class="p">(</span><span class="s1">'q'</span> <span class="ow">in</span> <span class="nx">va</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="nx">vn</span><span class="p">,</span> <span class="nb">parseFloat</span><span class="p">(</span><span class="nx">va</span><span class="p">[</span><span class="s1">'q'</span><span class="p">])];</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="nx">vn</span><span class="p">,</span> <span class="mf">1</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">});</span>
<span class="cm">/* Filter out duplicates by name, preserving the last seen */</span>
<span class="kd">var</span> <span class="nx">seen</span> <span class="o">=</span> <span class="p">{};</span>
<span class="kd">const</span> <span class="nx">filteredValues</span> <span class="o">=</span> <span class="nx">headerValuesWithQValues</span><span class="p">.</span><span class="nx">reverse</span><span class="p">().</span><span class="nx">filter</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">vt</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">vn</span> <span class="o">=</span> <span class="nx">vt</span><span class="p">[</span><span class="mf">0</span><span class="p">];</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">vn</span> <span class="ow">in</span> <span class="nx">seen</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="kc">false</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">seen</span><span class="p">[</span><span class="nx">vn</span><span class="p">]</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span>
<span class="k">return</span> <span class="kc">true</span><span class="p">;</span>
<span class="p">});</span>
<span class="cm">/* Sort by values with highest 'q' attribute */</span>
<span class="k">return</span> <span class="nx">filteredValues</span><span class="p">.</span><span class="nx">sort</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nx">b</span><span class="p">[</span><span class="mf">1</span><span class="p">]</span> <span class="o">-</span> <span class="nx">a</span><span class="p">[</span><span class="mf">1</span><span class="p">];</span> <span class="p">});</span>
<span class="p">};</span>
<span class="cm">/*</span>
<span class="cm">* Perform content negotiation.</span>
<span class="cm">*</span>
<span class="cm">* Given sorted arrays of supported (value name, q-value) tuples, select a</span>
<span class="cm">* value that is mutuaully acceptable. Returns null is nothing could be found.</span>
<span class="cm">*/</span>
<span class="kd">const</span> <span class="nx">performNegotiation</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">clientValues</span><span class="p">,</span> <span class="nx">serverValues</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">scores</span> <span class="o">=</span> <span class="p">[];</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span> <span class="o">=</span> <span class="mf">0</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">clientValues</span><span class="p">.</span><span class="nx">length</span><span class="p">;</span> <span class="o">++</span><span class="nx">i</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">cv</span> <span class="o">=</span> <span class="nx">clientValues</span><span class="p">[</span><span class="nx">i</span><span class="p">];</span>
<span class="kd">const</span> <span class="nx">sv</span> <span class="o">=</span> <span class="nx">serverValues</span><span class="p">.</span><span class="nx">find</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">sv</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nx">sv</span><span class="p">[</span><span class="mf">0</span><span class="p">]</span> <span class="o">===</span> <span class="nx">cv</span><span class="p">[</span><span class="mf">0</span><span class="p">];</span> <span class="p">});</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">sv</span> <span class="o">===</span> <span class="kc">undefined</span><span class="p">)</span> <span class="p">{</span>
<span class="k">continue</span><span class="p">;</span>
<span class="p">}</span>
<span class="nx">scores</span><span class="p">.</span><span class="nx">push</span><span class="p">([</span><span class="nx">cv</span><span class="p">[</span><span class="mf">0</span><span class="p">],</span> <span class="nx">cv</span><span class="p">[</span><span class="mf">1</span><span class="p">]</span> <span class="o">*</span> <span class="nx">sv</span><span class="p">[</span><span class="mf">1</span><span class="p">]]);</span>
<span class="p">}</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">scores</span><span class="p">.</span><span class="nx">length</span> <span class="o">===</span> <span class="mf">0</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="kc">null</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">scores</span><span class="p">.</span><span class="nx">sort</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nx">b</span><span class="p">[</span><span class="mf">1</span><span class="p">]</span> <span class="o">-</span> <span class="nx">a</span><span class="p">[</span><span class="mf">1</span><span class="p">];</span> <span class="p">})[</span><span class="mf">0</span><span class="p">][</span><span class="mf">0</span><span class="p">];</span>
<span class="p">};</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">context</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="p">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">request</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mf">0</span><span class="p">].</span><span class="nx">cf</span><span class="p">.</span><span class="nx">request</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">headers</span> <span class="o">=</span> <span class="nx">request</span><span class="p">.</span><span class="nx">headers</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="s1">'accept-encoding'</span> <span class="ow">in</span> <span class="nx">headers</span> <span class="o">&&</span>
<span class="o">!</span><span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">.</span><span class="nx">startsWith</span><span class="p">(</span><span class="s1">'/gzip/'</span><span class="p">)</span> <span class="o">&&</span>
<span class="o">!</span><span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">.</span><span class="nx">startsWith</span><span class="p">(</span><span class="s1">'/br/'</span><span class="p">))</span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">SERVER_WEIGHTS</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">[</span><span class="s1">'br'</span><span class="p">,</span> <span class="mf">1</span><span class="p">],</span>
<span class="p">[</span><span class="s1">'gzip'</span><span class="p">,</span> <span class="mf">0.9</span><span class="p">],</span>
<span class="p">[</span><span class="s1">'identity'</span><span class="p">,</span> <span class="mf">0.1</span><span class="p">],</span>
<span class="p">];</span>
<span class="kd">const</span> <span class="nx">sh</span> <span class="o">=</span> <span class="nx">splitHeaders</span><span class="p">(</span><span class="nx">headers</span><span class="p">[</span><span class="s1">'accept-encoding'</span><span class="p">]);</span>
<span class="kd">const</span> <span class="nx">ph</span> <span class="o">=</span> <span class="nx">sh</span><span class="p">.</span><span class="nx">map</span><span class="p">(</span><span class="nx">parseHeaderValue</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">qh</span> <span class="o">=</span> <span class="nx">sortHeadersByQValue</span><span class="p">(</span><span class="nx">ph</span><span class="p">);</span>
<span class="kd">const</span> <span class="nx">rep</span> <span class="o">=</span> <span class="nx">performNegotiation</span><span class="p">(</span><span class="nx">qh</span><span class="p">,</span> <span class="nx">SERVER_WEIGHTS</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">rep</span> <span class="o">&&</span> <span class="nx">rep</span> <span class="o">!==</span> <span class="s1">'identity'</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">request</span><span class="p">.</span><span class="nx">uri</span> <span class="o">=</span> <span class="s1">'/'</span> <span class="o">+</span> <span class="nx">rep</span> <span class="o">+</span> <span class="nx">request</span><span class="p">.</span><span class="nx">uri</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">request</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div>
<p>... and now the request handler:</p>
<div class="highlight"><pre><span></span><code><span class="cm">/*</span>
<span class="cm">MIT License</span>
<span class="cm">Copyright (c) 2018 Peter Griess</span>
<span class="cm">Permission is hereby granted, free of charge, to any person obtaining a copy</span>
<span class="cm">of this software and associated documentation files (the "Software"), to deal</span>
<span class="cm">in the Software without restriction, including without limitation the rights</span>
<span class="cm">to use, copy, modify, merge, publish, distribute, sublicense, and/or sell</span>
<span class="cm">copies of the Software, and to permit persons to whom the Software is</span>
<span class="cm">furnished to do so, subject to the following conditions:</span>
<span class="cm">The above copyright notice and this permission notice shall be included in all</span>
<span class="cm">copies or substantial portions of the Software.</span>
<span class="cm">THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR</span>
<span class="cm">IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,</span>
<span class="cm">FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE</span>
<span class="cm">AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER</span>
<span class="cm">LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,</span>
<span class="cm">OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE</span>
<span class="cm">SOFTWARE.</span>
<span class="cm">*/</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">handler</span> <span class="o">=</span> <span class="p">(</span><span class="nx">event</span><span class="p">,</span> <span class="nx">context</span><span class="p">,</span> <span class="nx">callback</span><span class="p">)</span> <span class="p">=></span> <span class="p">{</span>
<span class="kd">const</span> <span class="nx">response</span> <span class="o">=</span> <span class="nx">event</span><span class="p">.</span><span class="nx">Records</span><span class="p">[</span><span class="mf">0</span><span class="p">].</span><span class="nx">cf</span><span class="p">.</span><span class="nx">response</span><span class="p">;</span>
<span class="kd">const</span> <span class="nx">headers</span> <span class="o">=</span> <span class="nx">response</span><span class="p">.</span><span class="nx">headers</span><span class="p">;</span>
<span class="nx">headers</span><span class="p">[</span><span class="s1">'Vary'</span><span class="p">]</span> <span class="o">=</span> <span class="p">[{</span><span class="nx">key</span><span class="o">:</span> <span class="s1">'Vary'</span><span class="p">,</span> <span class="nx">value</span><span class="o">:</span> <span class="s1">'Accept-Encoding'</span><span class="p">}];</span>
<span class="nx">callback</span><span class="p">(</span><span class="kc">null</span><span class="p">,</span> <span class="nx">response</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div>
<h2>How does this work?</h2>
<p>The origin for https://blog.std.in/ is an S3 bucket configured for static website hosting. There are 3 different versions of each piece of content -- one un-processed, one compressed with gzip, and another compressed with Brotli. The compressed content lives in a shadow directory hierarchy under <code>/gzip</code> and <code>/br</code> respectively, allowing the path for compressed content to be computed by prepending the requisite directory.</p>
<p>There are two handlers -- an origin request handler and an origin response handler. There are no viewer handlers, allowing CloudFront to skip this logic entirely when serving a cache hit. The origin request handler performs the content negotiation, parsing the <code>Accept-Encoding</code> header and comparing its requirements with what's provided by the S3 bucket serving as the origin. It selects the best match and updates the URI to fetch from the origin. The origin response handler sets a <code>Vary: Accept-Encoding</code> header on the response indicating that content was negotiated based on the value of the <code>Accept-Encoding</code> header. The resulting response is then cached in CloudFront.</p>
<p>Finally, the CloudFront distribution is configured with the "Cache Based on Selected Request Headers" setting set to include <code>Accept-Encoding</code>. This has the effect of CloudFront incorporating the browser's <code>Accept-Encoding</code> header in its cache key when looking up a response. In addition, this prevents CloudFront from stripping the browser's <code>Accept-Encoding</code> header before the origin request handler has a chance to execute.</p>
<h2>A bug in CloudFront?</h2>
<p>It is surprising to me that the <code>Vary</code> header is not being added automatically by CloudFront as enabling "Cache Based on Selected Request Headers" and adding <code>Accept-Encoding</code> explicitly indicates that the content for the given URL may vary by the value of this header. This seems like a pretty clear indication that CloudFront should be adding a <code>Vary: Accept-Encoding</code> header to the response automatically.</p>Simple AWS Request Signing2018-09-04T08:11:00-05:002018-09-04T08:11:00-05:00Peter Griesstag:betablog.std.in,2018-09-04:/simple-aws-signing/<p>Amazon Web Services <a href="https://aws.amazon.com/">Amazon Web Services</a> has been
online for more than a decade, and now supports a dizzying array of services
backed by a relatively easy-to-use REST API. Unfortunately, while the ecosystem
of tools and libraries has expanded exponentially, working with these services
tends to require a deep scaffolding …</p><p>Amazon Web Services <a href="https://aws.amazon.com/">Amazon Web Services</a> has been
online for more than a decade, and now supports a dizzying array of services
backed by a relatively easy-to-use REST API. Unfortunately, while the ecosystem
of tools and libraries has expanded exponentially, working with these services
tends to require a deep scaffolding of dependencies. Lately I've been playing
with <a href="https://openwrt.org">OpenWRT</a> on my home network and wanted to make some
AWS REST API requests. The device has a fairly generous 128MB of storage, but
even so, pulling in the official <code>awscli</code> Python package with its dependencies
weighs in at over 30MB, not including Python itself.</p>
<p>Introducing <a href="https://github.com/pgriess/aws4sign">aws4sign</a>, a zero-dependency,
MIT-licensed, single-file Python2 library and CLI tool that computes the
<a href="https://docs.aws.amazon.com/general/latest/gr/sigv4_signing.html">AWS v4 signature</a>.</p>
<p>There are two ways to use this.</p>
<h2>Invoke as an executable</h2>
<p>The <code>aws4sign.py</code> file itself contains a simple <code>__main__</code> that makes it easy
to drive signature generation from anywhere that can invoke an executable.</p>
<p>The following bash snippet calls the AWS Route53 <code>hostedzone</code> API using <code>curl</code>:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># AWS keys</span>
<span class="nb">export</span> <span class="nv">AWS_ACCESS_KEY_ID</span><span class="o">=</span>...
<span class="nb">export</span> <span class="nv">AWS_SECRET_ACCESS_KEY</span><span class="o">=</span>...
<span class="c1"># Inputs to the signature algorithm; must be immutable</span>
<span class="c1">#</span>
<span class="c1"># The $now parameter in particular is interesting. The AWS signature algorithm</span>
<span class="c1"># includes the current time, which we pass to aws4sign.py using the -t option.</span>
<span class="nv">now</span><span class="o">=</span><span class="k">$(</span>date <span class="s1">'+%s'</span><span class="k">)</span>
<span class="nv">url</span><span class="o">=</span>https://route53.amazonaws.com/2013-04-01/hostedzone
<span class="c1"># Compute all headers</span>
<span class="nv">auth_header</span><span class="o">=</span><span class="k">$(</span>python2.7 ./aws4sign.py -t <span class="nv">$now</span> -p authorization <span class="nv">$url</span> <span class="p">|</span> cut -f2<span class="k">)</span>
<span class="nv">date_header</span><span class="o">=</span><span class="k">$(</span>python2.7 ./aws4sign.py -t <span class="nv">$now</span> -p x-amz-date <span class="nv">$url</span> <span class="p">|</span> cut -f2<span class="k">)</span>
<span class="nv">content_header</span><span class="o">=</span><span class="k">$(</span>python2.7 ./aws4sign.py -t <span class="nv">$now</span> -p x-amz-content-sha256 <span class="nv">$url</span> <span class="p">|</span> cut -f2<span class="k">)</span>
curl -s <span class="se">\</span>
-H <span class="s2">"authorization: </span><span class="nv">$auth_header</span><span class="s2">"</span> <span class="se">\</span>
-H <span class="s2">"x-amz-date: </span><span class="nv">$date_header</span><span class="s2">"</span> <span class="se">\</span>
-H <span class="s2">"x-amz-content-sha256: </span><span class="nv">$content_header</span><span class="s2">"</span> <span class="se">\</span>
<span class="nv">$url</span>
</code></pre></div>
<p>The only thing of note here is that we end up calling <code>aws4sign.py</code> once for
each header that we need to pass to <code>curl</code>, selecting the header to display
using the <code>-p</code> option. If we omitted this option, all headers would be emitted
(one per line), but parsing these is a bit more involved than desirable for
such a simple example. Instead, we just emit a single header each time and use
<code>cut</code> to grab its value. Note, however, that because the AWS signature
algorithm uses a timestamp, we need to ensure that <code>aws4sign.py</code> has a constant
notion of time across invocations. We do this by computing the current time
up-front and passing it using the <code>-t</code> option.</p>
<h2>Integrated with Python code</h2>
<p>Copy and paste the single 100-line
<a href="https://github.com/pgriess/aws4sign/blob/master/aws4sign.py"><code>aws4_signature_parts()</code></a>
function into your code. Or integrate it into a module of your own. Whatever.
No dependencies. No mucking with PIP. No incompatible licenses.</p>
<p>The following code invokes the Route53 <code>hostedzone</code> API using <code>urllib2</code>:</p>
<div class="highlight"><pre><span></span><code><span class="k">def</span> <span class="nf">aws4_signature_parts</span><span class="p">(</span><span class="o">...</span><span class="p">):</span>
<span class="o">...</span>
<span class="k">def</span> <span class="nf">aws_route53</span><span class="p">(</span><span class="n">aws_key</span><span class="p">,</span> <span class="n">aws_key_secret</span><span class="p">,</span> <span class="n">path</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
<span class="n">url</span> <span class="o">=</span> <span class="s1">'https://route53.amazonaws.com/2013-04-01/</span><span class="si">{}</span><span class="s1">'</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">path</span><span class="p">)</span>
<span class="n">_</span><span class="p">,</span> <span class="n">_</span><span class="p">,</span> <span class="n">headers</span> <span class="o">=</span> <span class="n">aws4_signature_parts</span><span class="p">(</span>
<span class="n">aws_key</span><span class="p">,</span>
<span class="n">aws_key_secret</span><span class="p">,</span>
<span class="s1">'GET'</span> <span class="k">if</span> <span class="n">data</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="s1">'POST'</span><span class="p">,</span>
<span class="n">url</span><span class="p">,</span>
<span class="n">data</span><span class="o">=</span><span class="s1">''</span> <span class="k">if</span> <span class="n">data</span> <span class="ow">is</span> <span class="kc">None</span> <span class="k">else</span> <span class="n">data</span><span class="p">)</span>
<span class="k">return</span> <span class="n">urllib2</span><span class="o">.</span><span class="n">urlopen</span><span class="p">(</span><span class="n">urllib2</span><span class="o">.</span><span class="n">Request</span><span class="p">(</span><span class="n">url</span><span class="p">,</span> <span class="n">headers</span><span class="o">=</span><span class="n">headers</span><span class="p">,</span> <span class="n">data</span><span class="o">=</span><span class="n">data</span><span class="p">))</span>
<span class="nb">print</span> <span class="n">aws_route53</span><span class="p">(</span><span class="n">aws_key</span><span class="p">,</span> <span class="n">aws_secret</span><span class="p">,</span> <span class="s1">'hostedzone'</span><span class="p">)</span><span class="o">.</span><span class="n">read</span><span class="p">()</span>
</code></pre></div>
<p>The first two return values from <code>aws4_signature_parts()</code> should probably be
ignored by most users -- they are mostly in place to provide visibility into
the signing process for validation and testing purposes.</p>
<p>That's it! Happy signing.</p>HTTP Response sizes and TCP2015-05-23T16:00:00-05:002015-05-23T16:00:00-05:00Peter Griesstag:betablog.std.in,2015-05-23:/http-response-sizes-and-tcp/<p>It's no secret that reducing the size of HTTP responses can lead to performance
improvements. Surprisingly, this is not a linear relationship; decreasing
response size only slightly can dramatically reduce the time required to
transfer the data.</p>
<p>This document explains the throughput characteristics of an established TCP
connection and how …</p><p>It's no secret that reducing the size of HTTP responses can lead to performance
improvements. Surprisingly, this is not a linear relationship; decreasing
response size only slightly can dramatically reduce the time required to
transfer the data.</p>
<p>This document explains the throughput characteristics of an established TCP
connection and how they can shape performance, often in surprising ways.</p>
<p><em><strong>Note</strong>: I making some simplifying assumptions here so that things are easier
to model: a pre-existing, idle, TCP connection, and no packet loss. This
effectively shows the best case scenario for how TCP can handle a response.</em></p>
<h2>A (brief) refresher on TCP</h2>
<p>TCP has several mechanisms that govern how fast the sender can send data.</p>
<p>While a comprehensive understanding of TCP is way, way beyond the scope of this
note (and not something the author would claim to possess anyway), the basic
flow control mechanisms are not horribly complicated.</p>
<p>First, a bit of vocabulary</p>
<ul>
<li><em>sender</em> -- the party sending data, e.g. an HTTP client when sending a
request, or an HTTP server when sending a response; both parties in a TCP
connection are senders and receivers</li>
<li><em>receiver</em> -- the party receiving data, e.g. an HTTP client when receiving a
response, or an HTTP server when receiving a request; both parties in a TCP
connection are senders and receivers</li>
<li><em>data segment</em> -- a single IP packet containing a TCP header and at least
one byte of application data</li>
<li><em>congestion window (cwnd)</em> -- the number of un-acknowledged data segments
issued by the sender that can be in-flight at once; changes over time as the
sender observes congestion on the connection</li>
<li><em>initial congestion window (IW or initcwnd)</em> -- the initial value of <code>cwnd</code>
for new connections; 10 is the standard value and what Facebook uses</li>
<li><em>maximum segment size (MSS)</em> -- the largest possible size of a single data
segment; negotiated during TCP handshaking</li>
<li><em>receive window (rwnd)</em> -- the number of bytes that the receiver is willing to
buffer for the application</li>
<li><em>round-trip time (RTT)</em> -- the amount of time it takes a packet to travel
from the sender to the receiver, and back again; colloquially known as "ping
time"</li>
</ul>
<p>The maximum amount of data in-flight from the sender to the receiver is defined
to be <code>min(MSS * cwnd, rwnd)</code>.</p>
<p>Each ACK for a data segment that arrives back at the sender frees up a slot in
the <code>cwnd</code>. If the sender is unable to send additional data segments because
there are already <code>cwnd</code> un-acknowledged segments in-flight, they can send out
new data each time an ACK arrives. In addition, the <code>cwnd</code> is incremented by 1
each time an ACK is received, effectively doubling the <code>cwnd</code> value each time a
flight of ACKs arrives for the outstanding data segments.</p>
<h2>Data flights and bandwidth</h2>
<p>The sender can have up to <code>cwnd</code> segments in-flight at a given time. Beyond
that, the sender is stuck waiting for ACKs before it can emit additional
segments. For large responses, this means that we typically see a pattern where
<code>cwnd</code> segments are emitted all at once, an RTT passes, and <code>cwnd</code> ACKs arrive
all at once. At this point, the sender can then send out another <code>cwnd</code> worth
of segments. As a result, output tends to be bursty, with periodicity equal to
the RTT.</p>
<p>Recall that each ACK received increments the <code>cwnd</code> by 1. For a large response
(i.e. the sender wants to send as much as possible at every opportunity), every
data flight is twice as large as the one before it.</p>
<h2>How long does it take to send a response?</h2>
<p>If we're able to fit our response into the first data flight, we will require
only a single round-trip to receive the response. The inverse is also true:
if our response is only a single byte too large, the full response will not be
available to the receiver for an additional RTT.</p>
<p>This illustrates an interesting property of TCP's congestion control algorithm:
<em>when investigating latency it's useful to think of transmission size in terms
of the number of data flights that are required to transmit it, rather than
the absolute byte counts</em>. That is, a single-byte response will take just as
much time to receive as an <code>cwnd * MSS</code> response.</p>
<p>Here is the amount of time required to transmit various data payloads on
typical cell networks around the world. Assumptions: MSS of 1300, <code>cwnd</code> of 10
(the IETF recommended <code>IW</code>), and RTTs as shown for various countries.</p>
<ul>
<li>1xRTT (USA 150ms; India 1200ms; Brazil 600ms): 1 byte - 13,000 bytes</li>
<li>2xRTT (USA 300ms; India 2400ms; Brazil 1200ms): 13,001 bytes - 39,000 bytes</li>
<li>3xRTT (USA 450ms; India 3600ms; Brazil 1800ms): 39,001 bytes - 91,000 bytes</li>
<li>4xRTT (USA 600ms; India 4800ms; Brazil 2400ms): 91,001 bytes - 195,000 bytes</li>
</ul>
<p>RTT values are hypothetical but realistic RTTs for cell network users in the
respective countries.</p>
<h2>Ok, how can we speed things up?</h2>
<p>Using the above table, we can see that if we have a response that tends to be
around 40k, the effort to reduce that below the 39k threshold will result in a
50% decrease in time to receive the data! Given that network time often
dominates performance, this can be a significant win.</p>
<p>If you are running your own server, you could also increase the <code>IW</code> value
directly, though you really want to be sure you know what you're doing; it's
easy to cause performance problems by introducing congestion into the network
that would have otherwise been avoided. For kicks, here's a link showing the
<a href="http://www.cdnplanet.com/blog/initcwnd-settings-major-cdn-providers/">IW values for major CDN providers</a>.</p>How to stream MP3 audio from Rdio2011-12-18T20:34:00-06:002011-12-18T20:34:00-06:00Peter Griesstag:betablog.std.in,2011-12-18:/how-to-stream-mp3-audio-from-rdio/<p>Recently, I was working on a personal project for which I wanted to
stream audio from my <a href="http://www.rdio.com/" title="Rdio">Rdio</a> (which,
incidentally, is a great service) account. Unfortunately, the
<a href="http://developer.rdio.com/" title="Rdio API documentation">documented Rdio
APIs</a> don't provide
a way to do this, instead providing streaming through a Flash player for
web apps or compiled libraries …</p><p>Recently, I was working on a personal project for which I wanted to
stream audio from my <a href="http://www.rdio.com/" title="Rdio">Rdio</a> (which,
incidentally, is a great service) account. Unfortunately, the
<a href="http://developer.rdio.com/" title="Rdio API documentation">documented Rdio
APIs</a> don't provide
a way to do this, instead providing streaming through a Flash player for
web apps or compiled libraries for iOS and Android. I spent a bit of
time reverse-engineering pieces of the Rdio ecosystem to figure out how
to do this and thought I'd post the resulting recipe for how to do this
in Python in case anyone is interested.</p>
<p>First of all, <a href="http://developer.rdio.com/apps/register" title="Register an Application">apply for an Rdio API
key</a>.</p>
<p>Next, you'll need to install a bit of software:</p>
<ul>
<li><a href="https://github.com/rdio/rdio-python" title="rdio-python">rdio-python</a>
package, which gives us access to the official Rdio REST API</li>
<li><a href="http://www.pyamf.org" title="PyAMF">PyAMF</a>, which we use to make calls to
the the un-documented Flash API</li>
<li><a href="http://rtmpdump.mplayerhq.hu" title="rtmpdump">rtmpdump</a>, which we use to
stream the FLV content from the RTMP server. Note that versions
after 2.1d don't work, as they refuse to talk to the server which
they deem to be not genuine Adobe</li>
<li><a href="http://www.ffmpeg.org/" title="ffmpeg">ffmpeg</a>, which we will use to
transcode the FLV audio into MP3</li>
</ul>
<p>Once you've got all that installed, the process is relatively
straight-forward: use the Rdio API to search for a track you want to
download and grab a playback token for it; use the (un-documented) Flash
API to retrieve parameters for constructing arguments to <code>rtmpdump</code>; run
<code>rtmpdump and pipe the output to ffmpeg</code> to trans-code the FLV to
MP3.</p>
<p>Here's a proof-of-concept script that implements that process:</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/env python</span>
<span class="c1">#</span>
<span class="c1"># Demo tool to generate the rtmpdump(1) command for streaming a song from Rdio.</span>
<span class="kn">import</span> <span class="nn">httplib</span>
<span class="kn">from</span> <span class="nn">optparse</span> <span class="kn">import</span> <span class="n">OptionParser</span>
<span class="kn">from</span> <span class="nn">os.path</span> <span class="kn">import</span> <span class="n">basename</span>
<span class="kn">from</span> <span class="nn">pprint</span> <span class="kn">import</span> <span class="n">pprint</span>
<span class="kn">from</span> <span class="nn">pyamf.remoting.client</span> <span class="kn">import</span> <span class="n">RemotingService</span>
<span class="kn">from</span> <span class="nn">rdioapi</span> <span class="kn">import</span> <span class="n">Rdio</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">from</span> <span class="nn">urlparse</span> <span class="kn">import</span> <span class="n">urlparse</span>
<span class="c1"># URL hosting the Flash player</span>
<span class="n">FLASH_PLAYER_URL</span> <span class="o">=</span> <span class="s1">'http://www.rdio.com/api/swf'</span>
<span class="c1"># API endpoint for AMF</span>
<span class="n">AMF_ENDPOINT</span> <span class="o">=</span> <span class="s1">'http://www.rdio.com/api/1/amf/'</span>
<span class="c1"># Exit with a message</span>
<span class="k">def</span> <span class="nf">fail</span><span class="p">(</span><span class="n">msg</span><span class="p">):</span>
<span class="nb">print</span> <span class="o">>></span> <span class="n">sys</span><span class="o">.</span><span class="n">stderr</span><span class="p">,</span> <span class="s1">'</span><span class="si">%s</span><span class="s1">: </span><span class="si">%s</span><span class="s1">'</span> <span class="o">%</span> <span class="p">(</span><span class="n">basename</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">0</span><span class="p">]),</span> <span class="n">msg</span><span class="p">)</span>
<span class="n">sys</span><span class="o">.</span><span class="n">exit</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span>
<span class="c1"># Recursively resolve the given URL, following 3xx redirects. Returns final URL</span>
<span class="c1"># that did not result in a redirect</span>
<span class="k">def</span> <span class="nf">resolve_url</span><span class="p">(</span><span class="n">url</span><span class="p">):</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">FLASH_PLAYER_URL</span>
<span class="k">while</span> <span class="kc">True</span><span class="p">:</span>
<span class="n">pr</span> <span class="o">=</span> <span class="n">urlparse</span><span class="p">(</span><span class="n">url</span><span class="p">)</span>
<span class="n">hc</span> <span class="o">=</span> <span class="n">httplib</span><span class="o">.</span><span class="n">HTTPConnection</span><span class="p">(</span><span class="n">pr</span><span class="o">.</span><span class="n">hostname</span><span class="p">)</span>
<span class="n">hc</span><span class="o">.</span><span class="n">request</span><span class="p">(</span><span class="s1">'GET'</span><span class="p">,</span> <span class="n">pr</span><span class="o">.</span><span class="n">path</span><span class="p">)</span>
<span class="n">hr</span> <span class="o">=</span> <span class="n">hc</span><span class="o">.</span><span class="n">getresponse</span><span class="p">()</span>
<span class="k">if</span> <span class="n">hr</span><span class="o">.</span><span class="n">status</span> <span class="o">/</span> <span class="mi">100</span> <span class="o">==</span> <span class="mi">3</span><span class="p">:</span>
<span class="n">url</span> <span class="o">=</span> <span class="n">hr</span><span class="o">.</span><span class="n">getheader</span><span class="p">(</span><span class="s1">'location'</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">return</span> <span class="n">url</span>
<span class="n">op</span> <span class="o">=</span> <span class="n">OptionParser</span><span class="p">(</span>
<span class="n">usage</span><span class="o">=</span><span class="s1">'%prog [options] <key> <secret> <query>'</span><span class="p">,</span>
<span class="n">description</span><span class="o">=</span><span class="s1">'''Emit an invocation of rtmpdump(1) that will fetch an FLV</span>
<span class="s1">file containing the audio to the first match of the given query to the Rdio</span>
<span class="s1">API.'''</span><span class="p">)</span>
<span class="n">opts</span><span class="p">,</span> <span class="n">args</span> <span class="o">=</span> <span class="n">op</span><span class="o">.</span><span class="n">parse_args</span><span class="p">()</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">args</span><span class="p">)</span> <span class="o"><</span> <span class="mi">3</span><span class="p">:</span>
<span class="n">op</span><span class="o">.</span><span class="n">error</span><span class="p">(</span><span class="s1">'missing required arguments'</span><span class="p">)</span>
<span class="n">key</span> <span class="o">=</span> <span class="n">args</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">secret</span> <span class="o">=</span> <span class="n">args</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">query</span> <span class="o">=</span> <span class="s1">' '</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">args</span><span class="p">[</span><span class="mi">2</span><span class="p">:])</span>
<span class="c1"># Figure out the URL for the Flash player that we're going to</span>
<span class="c1"># impersonate</span>
<span class="n">flash_url</span> <span class="o">=</span> <span class="n">resolve_url</span><span class="p">(</span><span class="n">FLASH_PLAYER_URL</span><span class="p">)</span>
<span class="c1"># Create a REST API client</span>
<span class="n">ra</span> <span class="o">=</span> <span class="n">Rdio</span><span class="p">(</span><span class="n">key</span><span class="p">,</span> <span class="n">secret</span><span class="p">,</span> <span class="p">{})</span>
<span class="c1"># Create an AMF API client</span>
<span class="n">svc</span> <span class="o">=</span> <span class="n">RemotingService</span><span class="p">(</span><span class="n">AMF_ENDPOINT</span><span class="p">,</span> <span class="n">referer</span><span class="o">=</span><span class="n">flash_url</span><span class="p">,</span> <span class="n">amf_version</span><span class="o">=</span><span class="mi">3</span><span class="p">)</span>
<span class="n">svc</span><span class="o">.</span><span class="n">addHeader</span><span class="p">(</span><span class="s1">'Auth'</span><span class="p">,</span> <span class="nb">chr</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span>
<span class="n">rdio_svc</span> <span class="o">=</span> <span class="n">svc</span><span class="o">.</span><span class="n">getService</span><span class="p">(</span><span class="s1">'rdio'</span><span class="p">)</span>
<span class="c1"># Search for the track to play</span>
<span class="n">results</span> <span class="o">=</span> <span class="n">ra</span><span class="o">.</span><span class="n">search</span><span class="p">(</span><span class="n">query</span><span class="o">=</span><span class="n">query</span><span class="p">,</span> <span class="n">types</span><span class="o">=</span><span class="s1">','</span><span class="o">.</span><span class="n">join</span><span class="p">([</span><span class="s1">'Track'</span><span class="p">]),</span>
<span class="n">count</span><span class="o">=</span><span class="mi">1</span><span class="p">)[</span><span class="s1">'results'</span><span class="p">]</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">results</span><span class="p">:</span>
<span class="n">fail</span><span class="p">(</span><span class="s1">'no results found'</span><span class="p">)</span>
<span class="c1"># Get a playback token</span>
<span class="n">token</span> <span class="o">=</span> <span class="n">ra</span><span class="o">.</span><span class="n">getPlaybackToken</span><span class="p">(</span><span class="n">domain</span><span class="o">=</span><span class="s1">'std.in'</span><span class="p">)</span>
<span class="c1"># Get playback information</span>
<span class="c1">#</span>
<span class="c1"># The end of the 'surl' value initially points to a 0:30 sample. Replace it to</span>
<span class="c1"># get the full track.</span>
<span class="n">pi</span> <span class="o">=</span> <span class="n">rdio_svc</span><span class="o">.</span><span class="n">getPlaybackInfo</span><span class="p">({</span>
<span class="s1">'domain'</span><span class="p">:</span> <span class="s1">'std.in'</span><span class="p">,</span>
<span class="s1">'playbackToken'</span><span class="p">:</span> <span class="n">token</span><span class="p">,</span>
<span class="s1">'manualPlay'</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
<span class="s1">'playerName'</span><span class="p">:</span> <span class="s1">'api_544189'</span><span class="p">,</span>
<span class="s1">'type'</span><span class="p">:</span> <span class="s1">'flash'</span><span class="p">,</span>
<span class="s1">'key'</span><span class="p">:</span> <span class="n">results</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">'key'</span><span class="p">]})</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">pi</span><span class="p">:</span>
<span class="n">fail</span><span class="p">(</span><span class="s1">'failed to get playback info'</span><span class="p">)</span>
<span class="n">auth</span> <span class="o">=</span> <span class="n">pi</span><span class="p">[</span><span class="s1">'auth'</span><span class="p">]</span>
<span class="n">surl</span> <span class="o">=</span> <span class="n">pi</span><span class="p">[</span><span class="s1">'surl'</span><span class="p">]</span><span class="o">.</span><span class="n">replace</span><span class="p">(</span><span class="s1">'30s-96'</span><span class="p">,</span> <span class="s1">'full-192'</span><span class="p">)</span>
<span class="c1"># Use rtmpdump(1) and ffmpeg(1) to grab the FLV file and then transcode it into</span>
<span class="c1"># an MP3</span>
<span class="nb">print</span> <span class="s1">'rtmpdump '</span> \
<span class="s1">'-p "http://blog.std.in/" '</span> \
<span class="s1">'--app "ondemand?</span><span class="si">%s</span><span class="s1">" '</span> \
<span class="s1">'-r "rtmpe://</span><span class="si">%s</span><span class="s1">/ondemand/mp3:</span><span class="si">%s</span><span class="s1">" '</span> \
<span class="s1">'-W "</span><span class="si">%s</span><span class="s1">" | '</span> \
<span class="s1">'ffmpeg '</span> \
<span class="s1">'-f flv '</span> \
<span class="s1">'-i - '</span> \
<span class="s1">'-f mp3 '</span> \
<span class="s1">'foo.mp3'</span> <span class="o">%</span> <span class="p">(</span>
<span class="n">auth</span><span class="p">,</span>
<span class="s1">'cp102543.edgefcs.net:1935'</span><span class="p">,</span>
<span class="n">surl</span><span class="p">,</span>
<span class="n">flash_url</span><span class="p">)</span>
</code></pre></div>
<p>To use use, invoke it with your API key, API secret and the query you
wish to use for searching for songs. It will perform the search and spit
out shell commands for fetching and trans-coding the song.</p>
<p>I've wrapped this behavior up in a simple Python library and dumped it
up on GitHub as <a href="https://github.com/pgriess/pyrdiostream">pyrdiostream</a>.</p>NodeJS and V82011-02-06T13:32:00-06:002011-02-06T13:32:00-06:00Peter Griesstag:betablog.std.in,2011-02-06:/nodejs-and-v8/<p>[This is a response to
<a href="http://www.olympum.com/future/nodejs-to-v8-or-not-to-v8/">this</a> blog
post by <a href="http://twitter.com/olympum">@olympum</a>, with the rest of the
thread being
<a href="http://joyeur.com/2011/02/05/on-brunos-concern-about-the-current-coupling-of-node-js-and-v8/">here</a>
and
<a href="http://www.olympum.com/future/answering-jason-on-v8-governance-and-impact-to-nodejs/">here</a>.]</p>
<p>Bruno makes three assertions about the relationship between NodeJS and
V8: that V8 was not designed as a server-side engine, that V8's lack of
threading inhibits adequate fault isolation, and …</p><p>[This is a response to
<a href="http://www.olympum.com/future/nodejs-to-v8-or-not-to-v8/">this</a> blog
post by <a href="http://twitter.com/olympum">@olympum</a>, with the rest of the
thread being
<a href="http://joyeur.com/2011/02/05/on-brunos-concern-about-the-current-coupling-of-node-js-and-v8/">here</a>
and
<a href="http://www.olympum.com/future/answering-jason-on-v8-governance-and-impact-to-nodejs/">here</a>.]</p>
<p>Bruno makes three assertions about the relationship between NodeJS and
V8: that V8 was not designed as a server-side engine, that V8's lack of
threading inhibits adequate fault isolation, and that lack of explicit
alignment between the V8 and NodeJS projects may lead to problems in the
future.</p>
<h2>V8 was not designed for server-side execution</h2>
<p>I'm not really sure what this means, as Bruno doesn't provide any
details on what he's concerned about.</p>
<p>When compared with the JVM, which offers distinct client and server
modes affecting primarily JIT compilation garbage collection strategies,
V8 is indeed less full featured. In fact, in probably virtually all
respects, the JVM is more mature and featureful than V8. However, that
does not imply that it's a more appropriate choice for a server-side
JavaScript runtime. After all, the JVM was designed from the ground up
to run something vastly different than JavaScript. Though the list of
alternative languages targeting the JVM is long and growing, it's
unclear to me that supporting these is a priority for the JVM team (the
<code>invokedynamic</code> instruction not withstanding).</p>
<p>I would be very interested to see some benchmarks of workloads which are
characteristic of server applications which compared Rhino, V8 and
SpiderMonkey, et all. The results on
<a href="http://arewefastyet.com">arewefastyet.com</a> are interesting, but don't
include the JVM. Unfortunately, I'm not enough of a JavaScript expert to
opine on the relevance of the benchmarks used there to a workload more
classically "server". Perhaps someone else from the community could
weigh in on this?</p>
<p>It would be wonderful if someone at Joyent or Yahoo! could contribute a
representative benchmark (it's <a href="http://hg.mozilla.org/users/danderson_mozilla.com/awfy">open
source</a>!) and/or
include a JVM-based engine to AWFY.</p>
<h2>V8's lack of threads inhibits fault isolation</h2>
<p>This is a specious argument, IMO.</p>
<p>In a system which runs each request in its own thread, fault isolation
is no better than a system which multiplexes requests over a single
thread.</p>
<p>Let's examine what happens when a fault occurs in a thread-per-request
model. In languages without direct memory access (JavaScript, Java,
etc), faults in threads are bubbled up to the top of the thread stack,
say by virtue of an exception. Other threads continue to soldier on in
their work, unaffected by this. I think this is what Bruno is referring
to. Of course, it is possible for a problem in one thread to cause
others to fail, particularly by exhausting available resources (e.g.
file descriptors, memory, database connections, etc). In addition, bugs
in application synchronization will cause other threads to deadlock, see
corrupted state, etc.</p>
<p>In a multiplexed model (e.g. NodeJS), things are largely the same: a
fault will bubble up to the top of the event loop as an exception where
it will be dealt with. Other requests are unaffected. Significantly, the
lack of parallelism within the same address space suggests that this
model may in fact be less likely to fail than a threaded model (no
locking bugs!).</p>
<p>This argument boils down to which VM is more likely to crash of its own
volition, taking all requests (be they in separate threads or not) with
it. I don't have any information one way or the other on whether the JVM
is more reliable than V8. Bruno, do you?</p>
<h2>The V8 team's commitment to NodeJS is uncertain</h2>
<p>While there doesn't seem to be a formal commitment here, anecdotal
evidence suggests that the V8 team is interested in seeing server-side
JavaScript (and NodeJS in particular) succeed.</p>
<p><a href="http://twitter.com/jasonh">@jasonh</a> makes several excellent points on
the nature of this relationship (and Joyent's) in <a href="http://joyeur.com/2011/02/05/on-brunos-concern-about-the-current-coupling-of-node-js-and-v8/">his blog
post</a>.</p>Benchmarking Web Socket servers with wsbench2010-09-24T12:35:00-05:002010-09-24T12:35:00-05:00Peter Griesstag:betablog.std.in,2010-09-24:/benchmarking-web-socket-servers/<p><a href="http://en.wikipedia.org/wiki/Web_Sockets">Web Sockets</a> are gaining
traction as a realtime full-duplex communication channel, with several
leading browsers (Chrome 5, Safari 5, Firefox 4, etc) having implemented
support for some flavor of the protocol. Server support exists, but is
not widespread, and is (entirely?) limited to specialized servers,
with <a href="http://socket.io/" title="Socket.IO">Socket.IO</a> (based on …</p><p><a href="http://en.wikipedia.org/wiki/Web_Sockets">Web Sockets</a> are gaining
traction as a realtime full-duplex communication channel, with several
leading browsers (Chrome 5, Safari 5, Firefox 4, etc) having implemented
support for some flavor of the protocol. Server support exists, but is
not widespread, and is (entirely?) limited to specialized servers,
with <a href="http://socket.io/" title="Socket.IO">Socket.IO</a> (based on
<a href="http://nodejs.org" title="NodeJS">NodeJS</a>) being perhaps the most well-known.
Finally, there appear to be no tools to do load or other testing on Web
Socket services.</p>
<p>Enter <a href="http://github.com/pgriess/wsbench">wsbench</a>, a benchmarking tool
for draft76 Web Socket servers featuring</p>
<ul>
<li>Ability to generate a high degree of load using a single client
process.</li>
<li>Easily change the number and rate of connections and number and size
of messages to send/receive with each connection.</li>
<li>Ability to target a specific port, path, and Web Socket protocol in
the target server.</li>
<li>The core request/session engine is easily scriptable using
JavaScript.</li>
</ul>
<p>You'll find that <code>wsbench</code> is quite easy to use.</p>
<p>Here, we open and close 1000 connections to a Web Socket server running
on localhost, port 8000. We generate 50 connection requests per second.</p>
<div class="highlight"><pre><span></span><code><span class="w"> </span><span class="c">% time ./wsbench -c 1000 -r 50 ws://localhost:8000</span><span class="w"></span>
<span class="w"> </span><span class="n">Success</span><span class="w"> </span><span class="n">rate</span><span class="p">:</span><span class="w"> </span><span class="mi">100</span><span class="c">% from 1000 connections</span><span class="w"></span>
<span class="w"> </span><span class="nb">real</span><span class="w"> </span>0<span class="n">m20</span><span class="p">.</span>379<span class="n">s</span><span class="w"></span>
<span class="w"> </span><span class="n">user</span><span class="w"> </span>0<span class="n">m1</span><span class="p">.</span>340<span class="n">s</span><span class="w"></span>
<span class="w"> </span><span class="n">sys</span><span class="w"> </span>0<span class="n">m0</span><span class="p">.</span>517<span class="n">s</span><span class="w"></span>
</code></pre></div>
<p>We can see a few interesting things in the output above.</p>
<ul>
<li>The error rate is tracked by <code>wsbench</code> and reported at the end.
Errors include failure to open a connection, failure to send a
message, failure to close the connection cleanly, etc.</li>
<li>The <code>wsbench</code> process running on a late-model Mac Book is able to
generate this load using less than 10% of the CPU.</li>
</ul>
<p>The above benchmarking run only tested establishing connections. We
didn't send (or receive) any messages. By passing the <code>-m 5</code> and
<code>-s 128</code> options to <code>wsbench</code>, we can send 5 128 byte messages per
connection. Invoke <code>wsbench</code> with the <code>-h</code> option to see full usage:</p>
<div class="highlight"><pre><span></span><code><span class="k">usage</span><span class="err">:</span><span class="w"> </span><span class="n">wsbench</span><span class="w"> </span><span class="o">[</span><span class="n">options</span><span class="o">]</span><span class="w"> </span><span class="o"><</span><span class="n">url</span><span class="o">></span><span class="w"></span>
<span class="n">Kick</span><span class="w"> </span><span class="k">off</span><span class="w"> </span><span class="n">a</span><span class="w"> </span><span class="n">benchmarking</span><span class="w"> </span><span class="n">run</span><span class="w"> </span><span class="n">against</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">given</span><span class="w"> </span><span class="nl">ws</span><span class="p">:</span><span class="o">//</span><span class="w"> </span><span class="n">URL</span><span class="p">.</span><span class="w"></span>
<span class="n">We</span><span class="w"> </span><span class="n">can</span><span class="w"> </span><span class="k">execute</span><span class="w"> </span><span class="n">our</span><span class="w"> </span><span class="n">workload</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">one</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">two</span><span class="w"> </span><span class="nl">ways</span><span class="p">:</span><span class="w"> </span><span class="n">serially</span><span class="p">,</span><span class="w"> </span><span class="n">wherein</span><span class="w"> </span><span class="k">each</span><span class="w"></span>
<span class="k">connection</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">closed</span><span class="w"> </span><span class="k">before</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="k">next</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">initiated</span><span class="p">;</span><span class="w"> </span><span class="ow">or</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">parallel</span><span class="p">,</span><span class="w"> </span><span class="n">wherein</span><span class="w"></span>
<span class="n">a</span><span class="w"> </span><span class="n">desired</span><span class="w"> </span><span class="n">rate</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">specified</span><span class="w"> </span><span class="ow">and</span><span class="w"> </span><span class="n">connections</span><span class="w"> </span><span class="n">initiated</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">meet</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="n">rate</span><span class="p">,</span><span class="w"></span>
<span class="n">independent</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="k">state</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">other</span><span class="w"> </span><span class="n">connections</span><span class="p">.</span><span class="w"> </span><span class="n">Serial</span><span class="w"> </span><span class="n">execution</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">the</span><span class="w"></span>
<span class="k">default</span><span class="p">,</span><span class="w"> </span><span class="ow">and</span><span class="w"> </span><span class="n">parallel</span><span class="w"> </span><span class="n">execution</span><span class="w"> </span><span class="n">can</span><span class="w"> </span><span class="n">be</span><span class="w"> </span><span class="n">specified</span><span class="w"> </span><span class="k">using</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="o">-</span><span class="n">r</span><span class="w"> </span><span class="o"><</span><span class="n">rate</span><span class="o">></span><span class="w"></span>
<span class="k">option</span><span class="p">.</span><span class="w"> </span><span class="n">Parallel</span><span class="w"> </span><span class="n">execution</span><span class="w"> </span><span class="k">is</span><span class="w"> </span><span class="n">bounded</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">total</span><span class="w"> </span><span class="n">number</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">connections</span><span class="w"></span>
<span class="k">to</span><span class="w"> </span><span class="n">be</span><span class="w"> </span><span class="n">made</span><span class="p">,</span><span class="w"> </span><span class="n">specified</span><span class="w"> </span><span class="k">by</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="o">-</span><span class="n">c</span><span class="w"> </span><span class="k">option</span><span class="p">.</span><span class="w"></span>
<span class="n">Available</span><span class="w"> </span><span class="nl">options</span><span class="p">:</span><span class="w"></span>
<span class="w"> </span><span class="o">-</span><span class="n">c</span><span class="p">,</span><span class="w"> </span><span class="o">--</span><span class="n">num</span><span class="o">-</span><span class="n">conns</span><span class="w"> </span><span class="n">NUMBER</span><span class="w"> </span><span class="n">number</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">connections</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="k">open</span><span class="w"> </span><span class="p">(</span><span class="k">default</span><span class="err">:</span><span class="w"> </span><span class="mi">100</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="o">-</span><span class="n">h</span><span class="p">,</span><span class="w"> </span><span class="o">--</span><span class="n">help</span><span class="w"> </span><span class="n">display</span><span class="w"> </span><span class="n">this</span><span class="w"> </span><span class="n">help</span><span class="w"></span>
<span class="w"> </span><span class="o">-</span><span class="n">m</span><span class="p">,</span><span class="w"> </span><span class="o">--</span><span class="n">num</span><span class="o">-</span><span class="n">msgs</span><span class="w"> </span><span class="n">NUMBER</span><span class="w"> </span><span class="n">number</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">messages</span><span class="w"> </span><span class="n">per</span><span class="w"> </span><span class="k">connection</span><span class="w"> </span><span class="p">(</span><span class="k">default</span><span class="err">:</span><span class="w"> </span><span class="mi">0</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="o">-</span><span class="n">p</span><span class="p">,</span><span class="w"> </span><span class="o">--</span><span class="n">protocol</span><span class="w"> </span><span class="n">PROTO</span><span class="w"> </span><span class="k">set</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">Web</span><span class="w"> </span><span class="n">Socket</span><span class="w"> </span><span class="n">protocol</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="k">use</span><span class="w"> </span><span class="p">(</span><span class="k">default</span><span class="err">:</span><span class="w"> </span><span class="n">empty</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="o">-</span><span class="n">r</span><span class="p">,</span><span class="w"> </span><span class="o">--</span><span class="n">rate</span><span class="w"> </span><span class="n">NUMBER</span><span class="w"> </span><span class="n">number</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">connections</span><span class="w"> </span><span class="n">per</span><span class="w"> </span><span class="k">second</span><span class="w"> </span><span class="p">(</span><span class="k">default</span><span class="err">:</span><span class="w"> </span><span class="mi">0</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="o">-</span><span class="n">s</span><span class="p">,</span><span class="w"> </span><span class="o">--</span><span class="n">msg</span><span class="o">-</span><span class="k">size</span><span class="w"> </span><span class="n">NUMBER</span><span class="w"> </span><span class="k">size</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">messages</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">send</span><span class="p">,</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="n">bytes</span><span class="w"> </span><span class="p">(</span><span class="k">default</span><span class="err">:</span><span class="w"> </span><span class="mi">32</span><span class="p">)</span><span class="w"></span>
<span class="w"> </span><span class="o">-</span><span class="n">S</span><span class="p">,</span><span class="w"> </span><span class="c1">--session FILE file to use for session logic (default: None)</span>
</code></pre></div>
<p>Beyond performance testing, it can be useful to run load against a
server continually to discover any resource leaks. This can be done by
passing the <code>-c 0</code> option -- 0 connections is interpreted as a special
"infinite" value. For example <code>-c 0 -r 100</code> will open/close 100
connections per second indefinitely (or until wsbench is terminated with
a <code>^C</code>).</p>
<p>For information on how to script the core of <code>wsbench</code>, take a look at
the Session Scripting section in the <a href="http://github.com/pgriess/wsbench">project
page</a> on GitHub. Because this tool is
written entirely in JavaScript (using NodeJS), you'll find that its
easily extensible using a familiar language.</p>Using sendfile(2) with NodeJS2010-09-09T14:07:00-05:002010-09-09T14:07:00-05:00Peter Griesstag:betablog.std.in,2010-09-09:/using-sendfile-with-nodejs/<p>NodeJS provides an interface to using the <code>sendfile(2)</code> system call.
Briefly, this system call allows the kernel to efficiently transport
data from a file on-disk to a socket without round-tripping the data
through user-space. This is one of the more important techniques that
HTTP servers use to get good …</p><p>NodeJS provides an interface to using the <code>sendfile(2)</code> system call.
Briefly, this system call allows the kernel to efficiently transport
data from a file on-disk to a socket without round-tripping the data
through user-space. This is one of the more important techniques that
HTTP servers use to get good performance when serving static files.</p>
<p>Using this is slightly tricky in NodeJS, as the <code>sendfile(2)</code> call is
not guaranteed to write all of a file's data to the given socket. Just
like the <code>write(2)</code> system call, it can declare success after only
writing a portion of the file contents to the given socket. This is
commonly the case with non-blocking sockets, as files larger than the TCP
send window cannot be buffered entirely in the TCP stack. At this point,
one must wait until some of this outstanding data has been flushed to
the other end of the TCP connection.</p>
<p>Without further ado, the following code implements a TCP server that
uses <code>sendfile(2)</code> to transfer the contents of a file to every client
that connects.</p>
<div class="highlight"><pre><span></span><code><span class="kd">var</span> <span class="nx">assert</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'assert'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">net</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'net'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">open</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">binding</span><span class="p">(</span><span class="s1">'fs'</span><span class="p">).</span><span class="nx">open</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">sendfile</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">binding</span><span class="p">(</span><span class="s1">'fs'</span><span class="p">).</span><span class="nx">sendfile</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">.</span><span class="nx">length</span> <span class="o"><</span> <span class="mf">4</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="s1">'usage: sendfile <port> <path>'</span><span class="p">);</span>
<span class="nx">process</span><span class="p">.</span><span class="nx">exit</span><span class="p">(</span><span class="mf">1</span><span class="p">);</span>
<span class="p">}</span>
<span class="kd">var</span> <span class="nx">port</span> <span class="o">=</span> <span class="nb">parseInt</span><span class="p">(</span><span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mf">2</span><span class="p">]);</span>
<span class="kd">var</span> <span class="nx">path</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">argv</span><span class="p">[</span><span class="mf">3</span><span class="p">];</span>
<span class="kd">var</span> <span class="nx">bufSz</span> <span class="o">=</span> <span class="mf">1</span> <span class="o"><<</span> <span class="mf">10</span><span class="p">;</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">log</span><span class="p">(</span><span class="s1">'Sending '</span> <span class="o">+</span> <span class="nx">path</span> <span class="o">+</span> <span class="s1">' to all connections on port '</span> <span class="o">+</span> <span class="nx">port</span><span class="p">);</span>
<span class="nx">net</span><span class="p">.</span><span class="nx">createServer</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">open</span><span class="p">(</span><span class="nx">path</span><span class="p">,</span> <span class="nx">process</span><span class="p">.</span><span class="nx">O_RDONLY</span><span class="p">,</span> <span class="mf">0</span><span class="p">,</span> <span class="kd">function</span><span class="p">(</span><span class="nx">err</span><span class="p">,</span> <span class="nx">fd</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Track our offset in the file outside of sendData() so that its value</span>
<span class="c1">// is stable across multiple invocations</span>
<span class="kd">var</span> <span class="nx">off</span> <span class="o">=</span> <span class="mf">0</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">sendData</span> <span class="o">=</span> <span class="kd">function</span><span class="p">()</span> <span class="p">{</span>
<span class="c1">// We only care about the 'drain' event if we're not done yet</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">removeListener</span><span class="p">(</span><span class="s1">'drain'</span><span class="p">,</span> <span class="nx">sendData</span><span class="p">);</span>
<span class="k">try</span> <span class="p">{</span>
<span class="c1">// Try to send file data until we either hit EOF, or fail the</span>
<span class="c1">// write due to EAGAIN</span>
<span class="k">do</span> <span class="p">{</span>
<span class="nx">nbytes</span> <span class="o">=</span> <span class="nx">sendfile</span><span class="p">(</span><span class="nx">s</span><span class="p">.</span><span class="nx">fd</span><span class="p">,</span> <span class="nx">fd</span><span class="p">,</span> <span class="nx">off</span><span class="p">,</span> <span class="nx">bufSz</span><span class="p">);</span>
<span class="nx">off</span> <span class="o">+=</span> <span class="nx">nbytes</span><span class="p">;</span>
<span class="p">}</span> <span class="k">while</span> <span class="p">(</span><span class="nx">nbytes</span> <span class="o">></span> <span class="mf">0</span><span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">end</span><span class="p">();</span>
<span class="p">}</span> <span class="k">catch</span> <span class="p">(</span><span class="nx">e</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Only EAGAIN is special; everything else is fatal</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">e</span><span class="p">.</span><span class="nx">errno</span> <span class="o">!==</span> <span class="nx">process</span><span class="p">.</span><span class="nx">EAGAIN</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="nx">e</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// When the socket has room for more data, start pumping again</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">on</span><span class="p">(</span><span class="s1">'drain'</span><span class="p">,</span> <span class="nx">sendData</span><span class="p">);</span>
<span class="c1">// Manually fire up the IOWatcher so that the 'drain' event</span>
<span class="c1">// fires. The net.Stream class usually manages this for you,</span>
<span class="c1">// but since we're going around it via sendfile(), we have to</span>
<span class="c1">// do this manually.</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">_writeWatcher</span><span class="p">.</span><span class="nx">start</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">err</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">err</span><span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">end</span><span class="p">();</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Kick off the transfer</span>
<span class="nx">sendData</span><span class="p">();</span>
<span class="p">});</span>
<span class="p">}).</span><span class="nx">listen</span><span class="p">(</span><span class="nx">port</span><span class="p">);</span>
</code></pre></div>
<p>This is doing a couple of interesting things</p>
<ul>
<li>We use the synchronous version of the <code>sendfile</code> API. We do this
because we don't want to round-trip through the libeio thread pool.</li>
<li>We need to handle the <code>EAGAIN</code> error status from the <code>sendfile(2)</code>
system call. NodeJS exposes this via a thrown exception. Rather than
issuing another <code>sendfile</code> call right away, we wait until the socket
is drained to try again (otherwise we're just busy-waiting). It's
possible that the performance cost of generating and handling this
exception is high enough that we'd be better off using the
asynchronous version of the <code>sendfile</code> API.</li>
<li>We have to kick the write <code>IOWatcher</code> on the <code>net.Stream</code> instance
ourselves to get the drain event to fire. This class only knows how
to start the watcher itself when it notices a <code>write(2)</code> system call
fail. Since we're using <code>sendfile(2)</code> behind its back, we have to
tell it to do this explicitly.</li>
<li>We notice that we've hit the end of our source file when
<code>sendfile(2)</code> returns 0 bytes written.</li>
</ul>More intelligent HTTP routing with NodeJS2010-07-22T09:40:00-05:002010-07-22T09:40:00-05:00Peter Griesstag:betablog.std.in,2010-07-22:/nodejs-more-intelligent-routing/<p>Earlier this week, I wrote <a href="http://developer.yahoo.net/blog/archives/2010/07/multicore_http_server_with_nodejs.html" title="Multi-Core HTTP Server with NodeJS">an article for
YDN</a>
covering some of the reasons why one might want to run a multi-core HTTP
server in NodeJS and some strategies for intelligently allocating
connections to different workers. While routing based on characteristics
of the TCP connection is useful, the approach outlined …</p><p>Earlier this week, I wrote <a href="http://developer.yahoo.net/blog/archives/2010/07/multicore_http_server_with_nodejs.html" title="Multi-Core HTTP Server with NodeJS">an article for
YDN</a>
covering some of the reasons why one might want to run a multi-core HTTP
server in NodeJS and some strategies for intelligently allocating
connections to different workers. While routing based on characteristics
of the TCP connection is useful, the approach outlined in that post has
a serious shortcoming - we cannot actually read any data off of the
socket when making these decisions. Doing so before passing off the file
descriptor would cause the worker process to miss critical request data,
choking the HTTP parser.</p>
<p>The above limitation precludes interrogating properties of the HTTP
request itself (e.g. headers, query parameters, etc) to make routing
decisions. In practice, there are a wide variety of use-cases where this
is important: routing by cookie, vhost, path, query parameters, etc. In
addition to cache affinity, this can provide some rudimentary forms of
access control (e.g. by running each vhost in a process with a different
UID or
<a href="http://linux.die.net/man/2/chroot" title="chroot(2) - Linux Man Page"><code>chroot(2)</code></a>
jail) or even QoS (e.g. by running each vhost in a process with its
<a href="http://linux.die.net/man/2/nice" title="nice(2) - Linux man page"><code>nice(2)</code></a>
value controlled).</p>
<p>Naively we could use NodeJS as a reverse HTTP proxy (and a pretty good
one, at that), but the overhead of proxying every byte of every request
is kind of a drag. As it turns out, we can use file descriptor passing
to efficiently hand off each TCP connection to the appropriate worker
once we've read enough of the request to make a routing decision. Thus,
once the routing process delegates a connection to a worker, that worker
owns it completely and the routing process has nothing more to do with
it. No juggling connections, no proxying traffic, nothing. The trick is
to do this in such a way that allows the routing process to parse as
much of the request as it needs to while ensuring that all socket data
remains available to the worker.</p>
<p>Step by step, we can do the following. Note that this does not work with
HTTP/1.1 keep-alive, which multiplexes multiple requests over a single
connection.</p>
<ol>
<li>Accept the TCP connection in the routing process</li>
<li>Set up a data handler for the TCP connection that both retains a
record of every byte received and uses a specially-constructed
instance of the interruptible HTTP parser (part of NodeJS core) to
parse as much of the request as we need</li>
<li>Once we've seen enough of the request, make a routing decision; here
we just use the vhost specified in the request</li>
<li>Hand off the file descriptor and all data seen thus far to the
worker</li>
<li>In the worker, construct a <code>net.Stream</code> connection around the
received FD and use it to emit a synthetic 'data' event to replay
data already read off of the socket by the routing process</li>
</ol>
<p>It's important to note that this does not rely on any modifications to
the HTTP stack in the worker - just plane vanilla NodeJS. In order to do
this, we have to recover from the fact that parsing the HTTP request in
the routing process is destructive - it's pulling bytes off of the
socket that are not available to the worker once it takes over the TCP
connection. To make sure that the worker doesn't miss a single byte seen
on the socket since its inception, we send over all data seen thus far
and replay it in the worker using the synthetic 'data' event.</p>
<p>First, <code>router.js</code>:</p>
<div class="highlight"><pre><span></span><code><span class="kd">var</span> <span class="nx">HTTPParser</span> <span class="o">=</span> <span class="nx">process</span><span class="p">.</span><span class="nx">binding</span><span class="p">(</span><span class="s1">'http_parser'</span><span class="p">).</span><span class="nx">HTTPParser</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">net</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'net'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">path</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'path'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">sys</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'sys'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">Worker</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'webworker/webworker'</span><span class="p">).</span><span class="nx">Worker</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">VHOSTS</span> <span class="o">=</span> <span class="p">[</span><span class="s1">'foo.bar.com'</span><span class="p">,</span> <span class="s1">'baz.bizzle.com'</span><span class="p">];</span>
<span class="kd">var</span> <span class="nx">WORKERS</span> <span class="o">=</span> <span class="p">{};</span>
<span class="nx">VHOSTS</span><span class="p">.</span><span class="nx">forEach</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">vh</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">WORKERS</span><span class="p">[</span><span class="nx">vh</span><span class="p">]</span> <span class="o">=</span> <span class="ow">new</span> <span class="nx">Worker</span><span class="p">(</span><span class="nx">path</span><span class="p">.</span><span class="nx">join</span><span class="p">(</span><span class="nx">__dirname</span><span class="p">,</span> <span class="s1">'worker.js'</span><span class="p">));</span>
<span class="p">});</span>
<span class="nx">net</span><span class="p">.</span><span class="nx">createServer</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">s</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">hp</span> <span class="o">=</span> <span class="ow">new</span> <span class="nx">HTTPParser</span><span class="p">(</span><span class="s1">'request'</span><span class="p">);</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'headers'</span> <span class="o">:</span> <span class="p">{</span>
<span class="p">},</span>
<span class="s1">'partial'</span> <span class="o">:</span> <span class="p">{</span>
<span class="s1">'field'</span> <span class="o">:</span> <span class="s1">''</span><span class="p">,</span>
<span class="s1">'value'</span> <span class="o">:</span> <span class="s1">''</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="kd">var</span> <span class="nx">seenData</span> <span class="o">=</span> <span class="s1">''</span><span class="p">;</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">onURL</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">buf</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">len</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">str</span> <span class="o">=</span> <span class="nx">buf</span><span class="p">.</span><span class="nx">toString</span><span class="p">(</span><span class="s1">'ascii'</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">start</span> <span class="o">+</span> <span class="nx">len</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">url</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">url</span> <span class="o">+=</span> <span class="nx">str</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">url</span> <span class="o">=</span> <span class="nx">str</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">onHeaderField</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">buf</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">len</span><span class="p">)</span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">value</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">headers</span><span class="p">[</span><span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">field</span><span class="p">]</span> <span class="o">=</span> <span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">value</span><span class="p">;</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'field'</span> <span class="o">:</span> <span class="s1">''</span><span class="p">,</span>
<span class="s1">'value'</span> <span class="o">:</span> <span class="s1">''</span>
<span class="p">};</span>
<span class="p">}</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">field</span> <span class="o">+=</span> <span class="nx">buf</span><span class="p">.</span><span class="nx">toString</span><span class="p">(</span>
<span class="s1">'ascii'</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">start</span> <span class="o">+</span> <span class="nx">len</span>
<span class="p">).</span><span class="nx">toLowerCase</span><span class="p">();</span>
<span class="p">};</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">onHeaderValue</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">buf</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">len</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">value</span> <span class="o">+=</span> <span class="nx">buf</span><span class="p">.</span><span class="nx">toString</span><span class="p">(</span>
<span class="s1">'ascii'</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">start</span> <span class="o">+</span> <span class="nx">len</span>
<span class="p">).</span><span class="nx">toLowerCase</span><span class="p">();</span>
<span class="p">};</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">onHeadersComplete</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">info</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Clean up partial state</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">field</span><span class="p">.</span><span class="nx">length</span> <span class="o">></span> <span class="mf">0</span> <span class="o">&&</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">value</span><span class="p">.</span><span class="nx">length</span> <span class="o">></span> <span class="mf">0</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">headers</span><span class="p">[</span><span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">field</span><span class="p">]</span> <span class="o">=</span> <span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">.</span><span class="nx">value</span><span class="p">;</span>
<span class="p">}</span>
<span class="ow">delete</span> <span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">partial</span><span class="p">;</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">version</span> <span class="o">=</span> <span class="p">{</span>
<span class="s1">'major'</span> <span class="o">:</span> <span class="nx">info</span><span class="p">.</span><span class="nx">versionMajor</span><span class="p">,</span>
<span class="s1">'minor'</span> <span class="o">:</span> <span class="nx">info</span><span class="p">.</span><span class="nx">versionMinor</span>
<span class="p">};</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">method</span> <span class="o">=</span> <span class="nx">info</span><span class="p">.</span><span class="nx">method</span><span class="p">;</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">upgrade</span> <span class="o">=</span> <span class="nx">info</span><span class="p">.</span><span class="nx">upgrade</span><span class="p">;</span>
<span class="k">if</span> <span class="p">(</span><span class="s1">'host'</span> <span class="ow">in</span> <span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">headers</span> <span class="o">&&</span>
<span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">headers</span><span class="p">.</span><span class="nx">host</span> <span class="ow">in</span> <span class="nx">WORKERS</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">pause</span><span class="p">();</span>
<span class="nx">WORKERS</span><span class="p">[</span><span class="nx">hp</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">headers</span><span class="p">.</span><span class="nx">host</span><span class="p">].</span><span class="nx">postMessage</span><span class="p">(</span>
<span class="nx">seenData</span><span class="p">,</span> <span class="nx">s</span><span class="p">.</span><span class="nx">fd</span>
<span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">write</span><span class="p">(</span>
<span class="s1">'HTTP/'</span> <span class="o">+</span> <span class="nx">info</span><span class="p">.</span><span class="nx">versionMajor</span> <span class="o">+</span> <span class="s1">'.'</span> <span class="o">+</span> <span class="nx">info</span><span class="p">.</span><span class="nx">versionMinor</span> <span class="o">+</span> <span class="s1">' '</span> <span class="o">+</span>
<span class="s1">'400 Host not found\r\n'</span>
<span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">write</span><span class="p">(</span><span class="s1">'\r\n'</span><span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">end</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">ondata</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">buf</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">end</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">seenData</span> <span class="o">+=</span> <span class="nx">buf</span><span class="p">.</span><span class="nx">toString</span><span class="p">(</span><span class="s1">'ascii'</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">end</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">ret</span> <span class="o">=</span> <span class="nx">hp</span><span class="p">.</span><span class="nx">execute</span><span class="p">(</span><span class="nx">buf</span><span class="p">,</span> <span class="nx">start</span><span class="p">,</span> <span class="nx">end</span> <span class="o">-</span> <span class="nx">start</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">ret</span> <span class="ow">instanceof</span> <span class="ne">Error</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">destroy</span><span class="p">(</span><span class="nx">ret</span><span class="p">);</span>
<span class="k">return</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">};</span>
<span class="p">}).</span><span class="nx">listen</span><span class="p">(</span><span class="mf">8080</span><span class="p">);</span>
</code></pre></div>
<p>... next, <code>worker.js</code>:</p>
<div class="highlight"><pre><span></span><code><span class="kd">var</span> <span class="nx">Buffer</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'buffer'</span><span class="p">).</span><span class="nx">Buffer</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">http</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'http'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">net</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'net'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">sys</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'sys'</span><span class="p">);</span>
<span class="kd">var</span> <span class="nx">srv</span> <span class="o">=</span> <span class="nx">http</span><span class="p">.</span><span class="nx">createServer</span><span class="p">(</span><span class="kd">function</span><span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">resp</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">resp</span><span class="p">.</span><span class="nx">writeHead</span><span class="p">(</span><span class="mf">200</span><span class="p">,</span> <span class="p">{</span><span class="s1">'Content-Type'</span> <span class="o">:</span> <span class="s1">'text/plain'</span><span class="p">});</span>
<span class="nx">resp</span><span class="p">.</span><span class="nx">write</span><span class="p">(</span><span class="s1">'Hello, vhost world!\n'</span><span class="p">);</span>
<span class="nx">resp</span><span class="p">.</span><span class="nx">end</span><span class="p">();</span>
<span class="p">});</span>
<span class="nx">onmessage</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">msg</span><span class="p">)</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">s</span> <span class="o">=</span> <span class="ow">new</span> <span class="nx">net</span><span class="p">.</span><span class="nx">Stream</span><span class="p">(</span><span class="nx">msg</span><span class="p">.</span><span class="nx">fd</span><span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">type</span> <span class="o">=</span> <span class="nx">srv</span><span class="p">.</span><span class="nx">type</span><span class="p">;</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">server</span> <span class="o">=</span> <span class="nx">srv</span><span class="p">;</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">resume</span><span class="p">();</span>
<span class="nx">srv</span><span class="p">.</span><span class="nx">emit</span><span class="p">(</span><span class="s1">'connection'</span><span class="p">,</span> <span class="nx">s</span><span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">emit</span><span class="p">(</span><span class="s1">'data'</span><span class="p">,</span> <span class="nx">msg</span><span class="p">.</span><span class="nx">data</span><span class="p">);</span>
<span class="nx">s</span><span class="p">.</span><span class="nx">ondata</span><span class="p">(</span><span class="ow">new</span> <span class="nx">Buffer</span><span class="p">(</span><span class="nx">msg</span><span class="p">.</span><span class="nx">data</span><span class="p">,</span> <span class="s1">'ascii'</span><span class="p">),</span> <span class="mf">0</span><span class="p">,</span> <span class="nx">msg</span><span class="p">.</span><span class="nx">data</span><span class="p">.</span><span class="nx">length</span><span class="p">);</span>
<span class="p">};</span>
</code></pre></div>
<p>Keep in mind that this code is a prototype only (please don't ship it -
I've left out a lot of error handling for the sake of readability ;),
but I thought it was interesting enough to share with a broader
audience. This implementation takes advantage of the task management and
message passing facilities of
<a href="http://github.com/pgriess/node-webworker" title="node-webworker">node-webworker</a>.
It should run out of the box on node-v0.1.100.</p>
<p>Anyway, the key to this is being able to replay the socket's data in the
worker. You'll notice in the code above that we're calling
<code>net.Stream.pause()</code> once we've received all necessary data in the
routing process. This ensures that this process doesn't pull any more
data off of the socket. If the kernel's TCP stack receives more data for
this socket after we've paused the stream, it will sit in the TCP
receive buffer waiting for someone to read it. Once the worker process
ingests the passed file descriptor and inserts it into its event loop,
this newly-arrived data will be read. In a nutshell, we use the TCP
stack itself to buffer data for us. If we really wanted to be clever, we
might be able to
use <a href="http://linux.die.net/man/2/recv" title="recv(2) - Linux Man Page"><code>recv(2)</code></a>
with <code>MSG_PEEK</code> to look at data arriving on the socket while leaving it
for the worker, but I'm not sure how this would play with the event
loop.</p>
<p>Finally, while I think this is an interesting technique, it's worth
noting that a typical production NodeJS deployment would be behind an
HTTP load balancer anyway, to front multiple physical hosts for
availability if nothing else. Many load balancers can route requests
based on a wide variety of characteristics like vhost, client IP,
backend load, etc. However, if one doesn't want/need a dedicated load
balancer, or needs very application-specific logic to make routing
decisions, I think the the above could be a useful tool.</p>Design of Web Workers for NodeJS2010-07-08T19:32:00-05:002010-07-08T19:32:00-05:00Peter Griesstag:betablog.std.in,2010-07-08:/nodejs-webworker-design/<p>The
<a href="http://github.com/pgriess/node-webworker" title="node-webworkers">node-webworker</a>
module aims to implement as much of the <a href="http://dev.w3.org/html5/workers/" title="HTML5 Web Workers">HTML5 Web Workers
API</a> as
is practical and useful in the context of
<a href="http://nodejs.org/" title="NodeJS">NodeJS</a>. Extensions to the HTML5 API are
provided where it makes sense (e.g. to allow file descriptor passing).</p>
<h2>Motivation</h2>
<p>Why bother to implement Web Workers for …</p><p>The
<a href="http://github.com/pgriess/node-webworker" title="node-webworkers">node-webworker</a>
module aims to implement as much of the <a href="http://dev.w3.org/html5/workers/" title="HTML5 Web Workers">HTML5 Web Workers
API</a> as
is practical and useful in the context of
<a href="http://nodejs.org/" title="NodeJS">NodeJS</a>. Extensions to the HTML5 API are
provided where it makes sense (e.g. to allow file descriptor passing).</p>
<h2>Motivation</h2>
<p>Why bother to implement Web Workers for NodeJS? After all, child process
support is already provided by the <code>child_process</code> module.</p>
<ul>
<li>A set of standard (well, emerging standard anyway)
platform-independent concurrency APIs is a useful abstraction.
Particularly as HTML5 gains wider adoption and JavaScript developers
are likely to familiar with Web Workers from doing browser
development. The set of NodeJS primitives for managing processes,
<code>child_process</code> provides a lot of utility, but is easily
misunderstood by developers who have not developed for a
UNIX platform before (e.g. <em>why does</em> <code>kill()</code> <em>not kill my
process?</em>).In addition, the error reporting APIs in the Web Workers
spec are more full-featured and JavaScript-specific than that
provided natively by <code>child_process</code> (e.g. one can get a stack
trace, etc).</li>
<li>Existing communication mechanisms with child processes
involve communicating over <code>stdin</code> / <code>stdout</code>. Use of these built-in
streams prevents <code>sys.puts()</code> and friends from working as expected.
Further, these are opaque byte streams and require the application
to implement their own framing logic to discern message boundaries.</li>
<li>HTML5 Shared Workers (also part of the same spec) provide a useful
naming service for communicating with other workers by name. Without
this, the application must maintain its own metadata for routing
messages between workers. <em>Note that shared workers are not yet
implemented.</em></li>
</ul>
<h2>Design</h2>
<p>The design that follows for Web Workers is motivated by a handful
of underlying assumptions / philosophies:</p>
<ul>
<li>Worker instances should be relatively long-lived. That is, it is
not considered an important workload to be able to create and
destroy thousands of workers as quickly as possible. Passing
messages to existing workers to dispatch work items is favored over
creating a new worker for each work item.</li>
<li>In the future, it will be desirable to run workers off-box, and
to implement workers in other application frameworks / languages.
This is particularly relevant in the choice of communication medium.</li>
<li>When practical, relevant standards and existing building blocks
should be taken advantage of, particularly those that are geared
towards JavaScript and/or HTTP. For example, this was one of the
motivators for selecting Web Sockets as a messaging layer rather
than rolling my own.</li>
</ul>
<h2>Worker processes</h2>
<p>Each worker executes in its own self-contained <code>node</code> process rather
than as a separate thread and V8 context within the master process.</p>
<p>The benefits of this approach include fault isolation (any worker
running out of memory or triggering some buggy C++ code will not take
down other workers); avoiding the complexity of managing multiple event
loops in a single process; and typical OSes are more likely to schedule
different processes on different CPUs (this may not always happen for
multiple threads within the same process), allowing the application to
utilize multiple CPUs.</p>
<p>Of course, there are drawbacks including the cost of context switching
between workers being more expensive when using a process-per-worker
model than it would be in a thread-per-worker model; passing messages
between processes typically requires a data copy and always requires
serializing data; and the overhead of spawning a new process.</p>
<h2>The worker context</h2>
<p>Each worker is launched by <code>lib/webworker-child.js</code>, which is handed
paths to the UNIX socket to use for communication with the parent
process (see below) and the worker application itself.</p>
<p>This script is passed to <code>node</code> as the entry point for the process and
is responsible for constructing a V8 script context populated with
bits relevant to the Web Worker API (e.g. the <code>postMessage()</code>, <code>close()</code>
and <code>location</code> primitives, etc). This also establishes communication
with the parent process and wires up the message send/receive listeners.
It's important to note that all of this happens in a context entirely
separate from the one in which the worker application will be executing;
the worker gets a seemingly plane-Jane Node runtime with the Web Worker
API bolted on. The worker application doesn't need to <code>require()</code>
additional libraries or anything.</p>
<h2>Inter-Worker communication</h2>
<p>The Web Workers spec describes a simple message passing API.</p>
<p>Under the covers, this is implemented by connecting each dedicated
worker to its parent process with a UNIX domain socket. This is lower
overhead than TCP, and allows for UNIX goodies like file descriptor
passing. Each master process creates dedicated UNIX socket for each
worker the path <code>/tmp/node-webworker-<pid>/<worker-id></code>, where <code><pid></code>
is the PID of the process doing the creating, and<code><worker-id></code> is an ID
of the worker being created. Although muddying up the filesystem
namespace doesn't thrill me, this makes the implementation easier than
listening on a single socket for all workers.</p>
<p>Message passing is done over this UNIX socket by negotiating an HTML5
Web Socket connection over this transport. This is done to provide
a reasonably-performant standards-based message framing implementation
and to lay the groundwork or communicating with off-box workers via HTTP
over TCP, which may be implemented in another application stack entirely
(e.g. Java, etc). The overhead of negotiating and maintaining the Web
Socket connection is 1 round trip for handshaking and the overhead of
maintaining HTTP state objects (<code>http_parser</code> and such). The handshaking
overhead is not considered an undue burden given that workers are
expected to be relatively long-lived and the HTTP state overhead
considered small.</p>
<h3>Message format</h3>
<p>The format of the messages themselves is JSON, serialized
using <code>JSON.stringify()</code> and de-serialized using <code>JSON.parse()</code>.
Significantly, the use of a framing protocol allows the Web Workers
implementation to wait for an entire, complete JSON blob to arrive
before invoking <code>JSON.parse()</code>. Although not implemented, it should be
possible to negotiate supported content encoding (e.g. to support
MsgPack, BERT, etc) when setting up the Web Socket connection. The
built-in <code>JSON</code> object is relatively performant
though <a href="http://github.com/pgriess/node-msgpack" title="node-msgpack"><code>node-msgpack</code></a>
is <a href="http://groups.google.com/group/nodejs/msg/1b8e3f4e64b4b061" title="node-msgpack vs JSON performance">quite a bit
faster</a>, particularly
when de-serializing.</p>
<p>Each object passed to <code>postMessage()</code> is wrapped in an array like so
<code>[<msg-type>, <object>]</code>. This allows the receiving end of the message
to distinguish control messages (<code>CLOSE</code>, <code>ERROR</code>, etc) from
user-initiated messages.</p>
<h3>Sending file descriptors</h3>
<p>As mentioned above, this Web Workers implementation can take advantage
of node's ability to send file descriptors using UNIX sockets. As a
nonstandard extension to the <code>postMessage(obj [,<fd>])</code> API, an optional
file descriptor can be specified. On symmetric API extension was made on
the receiving end, where the <code>onmessage(obj [,<fd>])</code> handler is passed
a fd parameter if a file descriptor was received along with the
specified message.</p>
<p>Unfortunately, UNIX sockets seem to allow file descriptors to arrive
out-of-band with respect to the data payload with which they were sent.
To tie a received file descriptor to the message with which it was sent,
all messages are wrapped in an array of the form <code>[<fd-seqno>, <obj>]</code>,
where <code><obj></code> is the object passed to <code>postMessage()</code>. The <code><fd-seqno></code>
parameter starts off at 0 and is incremented for every file descriptor
sent (the first file descriptor sent has a <code><fd-seqno></code> of 1). This
provides the receiving end with enough metadata to tie out-of-band
descriptors together with their originating message.</p>