Pipes

Pipes are functions that may be used when serving files to alter parts of the response. These are invoked by adding a pipe= query parameter taking a | separated list of pipe functions and parameters. The pipe functions are applied to the response from left to right. For example:

GET /sample.txt?pipe=slice(1,200)|status(404).

This would serve bytes 1 to 199, inclusive, of foo.txt with the HTTP status code 404.

There are several built-in pipe functions, and it is possible to add more using the @pipe decorator on a function, if required.

Note

Because of the way pipes compose, using some pipe functions prevents the content-length of the response from being known in advance. In these cases the server will close the connection to indicate the end of the response, preventing the use of HTTP 1.1 keepalive.

Built-In Pipes

sub

Used to substitute variables from the server environment, or from the request into the response.

Substitutions are marked in a file using a block delimited by {{ and }}. Inside the block the following variables are available:

{{host}}
The host name of the server excluding any subdomain part.
{{domains[]}}
The domain name of a particular subdomain
e.g. {{domains[www]}} for the www subdomain.
{{ports[][]}}
The port number of servers, by protocol
e.g. {{ports[http][0]}} for the first (and, depending on setup, possibly only) http server
{{headers[]}}
The HTTP headers in the request
e.g. {{headers[X-Test]}} for a hypothetical X-Test header.
{{GET[]}}
The query parameters for the request
e.g. {{GET[id]}} for an id parameter sent with the request.

So, for example, to write a javascript file called xhr.js that depends on the host name of the server, without hardcoding, one might write:

var server_url = http://{{host}}:{{ports[http][0]}}/path/to/resource;
//Create the actual XHR and so on

The file would then be included as:

<script src=”xhr.js?pipe=sub”></script>

This pipe can also be enabled by using a filename *.sub.ext, e.g. the file above could be called xhr.sub.js.

status

Used to set the HTTP status of the response, for example:

example.js?pipe=status(410)

headers

Used to add or replace http headers in the response. Takes two or three arguments; the header name, the header value and whether to append the header rather than replace an existing header (default: False). So, for example, a request for:

example.html?pipe=header(Content-Type,text/plain)

causes example.html to be returned with a text/plain content type whereas:

example.html?pipe=header(Content-Type,text/plain,True)

Will cause example.html to be returned with both text/html and text/plain content-type headers.

slice

Used to send only part of a response body. Takes the start and, optionally, end bytes as arguments, although either can be null to indicate the start or end of the file, respectively. So for example:

example.txt?pipe=slice(10,20)

Would result in a response with a body containing 10 bytes of example.txt including byte 10 but excluding byte 20.

example.txt?pipe=slice(10)

Would cause all bytes from byte 10 of example.txt to be sent, but:

example.txt?pipe=slice(null,20)

Would send the first 20 bytes of example.txt.

trickle

Note

Using this function will force a connection close.

Used to send the body of a response in chunks with delays. Takes a single argument that is a microsyntax consisting of colon-separated commands. There are three types of commands:

  • Bare numbers represent a number of bytes to send
  • Numbers prefixed d indicate a delay in seconds
  • Numbers prefixed r must only appear at the end of the command, and indicate that the preceding N items must be repeated until there is no more content to send. The number of items to repeat must be even.

In the absence of a repetition command, the entire remainder of the content is sent at once when the command list is exhausted. So for example:

example.txt?pipe=trickle(d1)

causes a 1s delay before sending the entirety of example.txt.

example.txt?pipe=trickle(100:d1)

causes 100 bytes of example.txt to be sent, followed by a 1s delay, and then the remainder of the file to be sent. On the other hand:

example.txt?pipe=trickle(100:d1:r2)

Will cause the file to be sent in 100 byte chunks separated by a 1s delay until the whole content has been sent.

Interface

wptserve.pipes.gzip(request, response)[source]

This pipe gzip-encodes response data.

It sets (or overwrites) these HTTP headers: Content-Encoding is set to gzip Content-Length is set to the length of the compressed content

wptserve.pipes.header(request, response, name, value, append=False)[source]

Set a HTTP header.

Replaces any existing HTTP header of the same name unless append is set, in which case the header is appended without replacement.

Parameters:
  • name – Name of the header to set.
  • value – Value to use for the header.
  • append – True if existing headers should not be replaced
wptserve.pipes.slice(request, response, start, end=None)[source]

Send a byte range of the response body

Parameters:
  • start – The starting offset. Follows python semantics including negative numbers.
  • end – The ending offset, again with python semantics and None (spelled “null” in a query string) to indicate the end of the file.
wptserve.pipes.status(request, response, code)[source]

Alter the status code.

Parameters:code – Status code to use for the response.
wptserve.pipes.sub(request, response, escape_type='html')[source]

Substitute environment information about the server and request into the script.

Parameters:escape_type – String detailing the type of escaping to use. Known values are “html” and “none”, with “html” the default for historic reasons.

The format is a very limited template language. Substitutions are enclosed by {{ and }}. There are several avaliable substitutions:

host
A simple string value and represents the primary host from which the tests are being run.
domains
A dictionary of available domains indexed by subdomain name.
ports
A dictionary of lists of ports indexed by protocol.
location

A dictionary of parts of the request URL. Valid keys are ‘server, ‘scheme’, ‘host’, ‘hostname’, ‘port’, ‘path’ and ‘query’. ‘server’ is scheme://host:port, ‘host’ is hostname:port, and query

includes the leading ‘?’, but other delimiters are omitted.
headers
A dictionary of HTTP headers in the request.
GET
A dictionary of query parameters supplied with the request.
uuid()
A pesudo-random UUID suitable for usage with stash

So for example in a setup running on localhost with a www subdomain and a http server on ports 80 and 81:

{{host}} => localhost
{{domains[www]}} => www.localhost
{{ports[http][1]}} => 81

It is also possible to assign a value to a variable name, which must start with the $ character, using the ”:” syntax e.g.

{{$id:uuid()}

Later substitutions in the same file may then refer to the variable by name e.g.

{{$id}}

wptserve.pipes.trickle(request, response, delays)[source]

Send the response in parts, with time delays.

Parameters:delays

A string of delays and amounts, in bytes, of the response to send. Each component is separated by a colon. Amounts in bytes are plain integers, whilst delays are floats prefixed with a single d e.g. d1:100:d2 Would cause a 1 second delay, would then send 100 bytes of the file, and then cause a 2 second delay, before sending the remainder of the file.

If the last token is of the form rN, instead of sending the remainder of the file, the previous N instructions will be repeated until the whole file has been sent e.g. d1:100:d2:r2 Causes a delay of 1s, then 100 bytes to be sent, then a 2s delay and then a further 100 bytes followed by a two second delay until the response has been fully sent.