We serve some of our assets directly out of s3 and while it is convenient it is not the speediest way to deliver content.  The crew over at Viximo worked out how to bolt Varnish on the side of Apache so that they can cache their S3 content and I was so smitten with the idea that I wanted to adapt what they worked out for our configuration so I asked Chris Chiodo reveal the secret sauce.  Below are the configuration files I munged from what he generously shared.

Nginx

This is pretty straightforward, what I’ve done is made varnish an upstream server and am intercepting any content in photos, avatars, kit, or caboodle and passing it the request to it.

upstream varnish {
server varnish01:7000 max_fails=3  fail_timeout=30s;
}

location ~ ^/(photos|avatars|kit|caboodle)/ {
proxy_pass http://varnish;
}

Varnish

This was my stumbling block until I talked to Viximo, the problem was how I defined the backend and that for whatever reason it did not like or AWS did not like the request to the bucket-name.amazonaws.com.

backend media {
.host = "s3.amazonaws.com";
.port = "80";
}

sub vcl_recv {
set req.url = regsub(req.url, "^", "/bucket-name");
set req.backend = media;
set req.http.host = "localhost";
remove req.http.X-Forwarded-For;
remove req.http.X-Forwarded-for;
remove req.http.X-Forwarded-Host;
remove req.http.X-Forwarded-Server;
set    req.http.X-Forwarded-for     = "127.0.0.1";
set req.grace = 30s;
lookup;
}

sub vcl_fetch {
set obj.http.X-Varnish-Url = req.url;
// set a 1 day ttl for avatars
set obj.ttl = 1d;
set obj.grace = 30s;

if (!obj.cacheable) {
pass;
}

set obj.prefetch =  -30s;
deliver;
}

That’s it.  Simple and it works.