Hands-on

HTTP Caching with Varnish

Bulgaria PHP Conference, September 25th, 2015

© David Buchmann

Step 1

apt-get install varnish

Step 1

apt-get install varnish

Step 2

Edit /etc/default/varnish, replace port 6081 by port 80
Edit webserver config to run on port 8080

service apache2 restart
service varnish restart

Step 3

What could possibly go wrong?

Nothing gets cached (HTTP Headers)
Too much gets cached (also HTTP Headers)
Editors see no changes (cache invalidation)
Caches get mixed up (personal content)

httpstatusdogs.com

Overview

HTTP refresher
HTTP cache control
Varnish
Advanced Topics
Wrap-Up

HTTP Refresher

What is a reverse proxy again?

HTTP is simple

GET /path

HTTP/1.1 200 OK
Content-Type: text/html

<html>...</html>

HTTP verbs

GET
HEAD
POST
PUT
DELETE
...

HTTP response codes

1xx hold on
2xx here you go
3xx go away
4xx you fucked up
5xx I fucked up

twitter.com/stevelosh/status/372740571749572610

Tools and helpers

firebug / chrome network tab in debug toolbar (copy URL as curl!)

                wget -Sq --spider varnish.lo/solutions/expiration.php
                curl -o /dev/null -sD - varnish.lo/solutions/expiration.php

HTTP Cache Control

Cache control headers

Cache Expiration Model
Cache Validation Model

HTTP 1.1, RFC 2616, Sections 13.2 and 13.3

Cache Expiration

Cache-Control: s-maxage=3600, max-age=900
Expires: Thu, 15 May 2014 08:00:00 GMT

s-maxage
max-age
Expires (HTTP 1.0 - avoid!)
Default to default_ttl if nothing specified

Hands on: Cache headers

noinfo.php
exercises/expiration.php

curl -sD - varnish.lo/noinfo.php
curl -sD - varnish.lo/exercises/expiration.php

Cache validation (I)

Application specific hash value on response
On request, cheap check if hash changed

ETag: 82901821233

If-None-Match: 82901821233

304 Not Modified

Hands on: Etag

exercises/etag.php

curl -sD - varnish.lo/exercises/etag.php
curl -H "If-None-Match: abc" -sD - varnish.lo/exercises/etag.php

Cache validation (II)

Last-Modified: Tue, 13 May 2014 08:13:20 GMT

If-Modified-Since: Tue, 13 May 2014 08:13:20 GMT

304 Not Modified

Hands on: If-Modified-Since

exercises/last-modified.php

curl -sD - varnish.lo/exercises/last-modified.php
curl -H "If-Modified-Since: {previous timestamp}" -sD - varnish.lo/exercises/last-modified.php

There are dog images too, but I won’t show the 304 one.

Validation: ETag or Last-Modified

Use ETag if possible to avoid clock skew hassle
If-Modified-Since in the future: 200, with content
Varnish only uses last modified if date is in GMT
gmdate('r') does not work as timezone is output as "+0000" instead of "GMT".
When using etag and date, both must match

Know your tools

Follow a link: use cache if not expired, If-None-Match / If-Modified-Since
ctrl-R or reload button in Firefox/Chrome: ignore local cache, but send If-... headers
Force reload: make sure the browser ignores all local caches
- Firefox: shift-ctrl-R
- Chrome: shift-ctrl-R or shift-F5
- Clear browser cache

Do not cache

Cache-Control: s-maxage=0, private, no-cache

By default, Varnish 3 only looks at s-maxage=0.
By default, Varnish 4 looks at s-maxage=0, private and no-cache.

www.varnish-cache.org/trac/browser/bin/varnishd/default.vcl?rev=3.0 (Varnish 3)
www.varnish-cache.org/trac/browser/bin/varnishd/builtin.vcl?rev=4.0 (Varnish 4)

Hands on: Do not cache

exercises/nocache.php

curl -sD - varnish.lo/exercises/nocache.php

Default Varnish behaviour

Only attempt to cache GET and HEAD request
Never cache request with cookies / authorization
Never cache response with set-cookie
Only cache safe responses (status 200, 203, 300, 301, 302, 307, 404, 410)

Hands on: Cookie

cookie.php

curl -sD - varnish.lo/cookie.php
curl -H "Cookie: x" -sD - varnish.lo/solutions/expiration.php

Keep variants apart

Content depending on request headers

GET /resource
Accept: application/json

GET /resource
Accept: text/xml

Vary: Accept

Hands on: Content negotiation

content-negotiation.php

curl -sD - varnish.lo/exercises/content-negotiation.php
curl -H "Accept: application/json" -sD - varnish.lo/exercises/content-negotiation.php

Let’s look into Varnish

Reverse proxy
Domain specific language: VCL
State machine
Hook function for each state
Configuration compiled to machine code
Loadbalancer

Warning!

Varnish does what you tell it

Think carefully and test thoroughly

Varnish Configuration Language

Read and write header values
If conditions, but no loops
Functions: state change, regexp and invalidation
No return values, only state changes
No variables, only store information in headers
Inline C code
//, # and /* foo */ for comments

Restart Varnish like a pro

Restart to empty the cache
Restart to test changed configuration

VCL: Debug hit or miss, TTL

sub vcl_backend_response {
    set beresp.http.X-TTL = beresp.ttl;
}

sub vcl_deliver {
    if (obj.hits > 0) {
        set resp.http.X-Cache = "HIT";
    } else {
        set resp.http.X-Cache = "MISS";
    }
}

VCL: Two applications

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}
backend legacy {
    .host = "127.0.0.1";
    .port = "8000";
}
sub vcl_recv {
    if (req.url ~ "^/archive/") {
        set req.backend_hint = legacy;
    } else {
        set req.backend_hint = default;
    }
}

VCL: Remove cookies

sub vcl_recv {
    if (req.http.Cookie) {
        if (req.url ~ "^/cache") {
            remove req.http.Cookie;
        }
    }
}

curl -H "Cookie: x" -sD - varnish.lo/solutions/expiration.php

Bad Practice: This is leaking knowledge about URLs to Varnish config

VCL: Custom TTL

header('X-Reverse-Proxy-TTL: 30');

C{ #include <stdlib.h>; }C
sub vcl_backend_response {
    if (beresp.http.X-Reverse-Proxy-TTL) {
        C{
            char *ttl;
            const struct gethdr_s hdr = { HDR_BERESP, "\024X-Reverse-Proxy-TTL:" };
            ttl = VRT_GetHdr(ctx, &hdr);
            VRT_l_beresp_ttl(ctx, atoi(ttl));
        }C
        unset beresp.http.X-Reverse-Proxy-TTL;
    }
}
...