18 KiB
+++ title = "hosting a static site on fly.io with nix and caddy" date = "2022-08-28" draft = true [taxonomies] tags = ["static-site", "nix", "caddy", "fly.io"] +++
Motivation
So, you've ditched Github and friends, set up your own Gitea instance, and there's just one (read: at least one) thing left for you to take care of—that snazzy static site you had up on Github Pages.
Pages and Static Site Hosting
I can remember how empowering it felt when I finally realized how much utility Github had packed into the Pages product. I went through a gleeful week or more of churning out Vue.js templates and half-baked styles for several different sites, all the while starry-eyed with how easy Pages made the deployment process. Eventually I migrated some of my work, including my static sites, to Gitlab, which has a virtually identical offering. I was quietly pleased with the ease of use afforded by both options.
Fast forward to today, and my migration away from Github and Gitlab has me once again pondering the question of an easy-to-use static site host. The tech industry has left us spoiled for choice here. Aside from Pages products from code hosting platforms, the major cloud infrastructure providers all have their own take on static site hosting. Companies like Netlify, Vercel, and Render offer a lot to hobbyist developers in terms of static hosting resources, with some going even further to provide Serverless-style "Functions-as-a-Service" products.
Some of these options were ruled out by requiring source code to be hosted on Github and friends, while others were much more heavyweight than what my requirements demanded. I was also motivated by an urge to consolidate: I already had projects running on Fly.io, as well as some important configuration and infrastructure on DigitalOcean, which made me hesitant to bring another third party into the mix.
As you may have guessed from the title, my ultimate decision was to take inspiration from a Fly.io tutorial document explaining how to deploy a very simple vanilla HTML site on Fly using a Go-powered webserver. I'm going to expand on their design a bit by introducing two major components: Nix, to give us the power to build our site with whatever static site generator (or other build process) we want to use; and Caddy, to give us a more flexible and extensible platform for actually serving the content.
For the purposes of this article, we'll assume you already have your static site ready to go. Whether you're writing pure HTML by hand, or using a cutting-edge Javascript framework that renders down to static resources, you'll be able to package it up with Nix and serve it with Caddy using Fly.io.
Before You Deploy: Nix
Much like static site hosts, static site generators are everywhere, and it seems like one of those things where people have particularly strong opinions on what workflow fits them best. Some people eschew a "generator" entirely and chain more purpose-built tools together to achieve the perfect bespoke output. Whatever your personal choice on generating the content for your static site, it helps to have a sort of "universal entry point" to actually kick off the build process. Github and Gitlab have their own custom Continuous Integration/Continuous Delivery systems where the build for the site can be configured. Makefiles can declare a set of commands to run in a generic way, but they don't help us pull in dependencies from the outside. Dockerfiles are not only generic, but also provide some primitives to make it easy to fetch any buildtime dependencies we need. Nix, a declarative package manager, is another option that we can use to specify our build AND dependencies, without requiring us to actually make a container. Nix also provides stronger guarantees around reproducibility and hermeticity than Docker. I already use Nix flakes for virtually all of my personal work, so that's how we'll specify our build step.
Nix is a very powerful tool, but it can be intimidating to use, and I've encountered many people saying that the documentation is too sparse to be useful. I'll keep the Nix details minimal, and try to include enough explanation for what we're doing that even unfamiliar readers can follow along, but I would encourage anyone who wants some more background on Nix and Nix flakes to check out these posts from Xe Iaso's blog.
If you don't already have Nix installed, we'll start with that:
# macOS
sh <(curl -L https://nixos.org/nix/install)
# Linux multi-user install (recommended by Nix)
sh <(curl -L https://nixos.org/nix/install) --daemon
# Linux single-user install (required when using SELinux)
# also Windows via WSL2
sh <(curl -L https://nixos.org/nix/install) --no-daemon
# Docker
docker run -it nixos/nix
Next up, we'll enable Nix flakes as a feature, since they are still disabled by default. You can do this the easy way, by editing one of ~/.config/nix/nix.conf or /etc/nix/nix.conf (if you're using a multi-user install, you'll also need to restart nix-daemon):
experimental-features = nix-command flakes
If you enjoy pain, you can do this the hard way by remembering to type this at the beginning of all your Nix commands, or setting an alias:
# one off
nix --experimental-features 'nix-command flakes' build
# sort of consistent but also not really
alias nix=nix --experimental-features 'nix-command flakes'
Now we can mosey on over to our site source. You might already be managing the source with Git, but if not, let's do that now:
cd site
git init
# Be careful not to commit anything secret! Git will keep a record of it
git add important/stuff but/no/secrets
git commit -m "TODO: pithy quip about starting a new endeavor"
Nix flakes require you to be using some form of source control that Nix understands, which means (to my understanding) either Git or Mercurial. Everything tracked by source control will end up going into the Nix store, so be doubly sure that you haven't committed any secrets, tokens, or manifestos that you don't want leaking out.
flake.nix: Entry Point
We finally have all the foundations in place to put our site's source into a flake. We can get a fresh flake by using the default template:
nix flake init
This will leave us with the following:
{
description = "A very basic flake";
outputs = { self, nixpkgs }: {
packages.x86_64-linux.hello = nixpkgs.legacyPackages.x86_64-linux.hello;
defaultPackage.x86_64-linux = self.packages.x86_64-linux.hello;
};
}
Hmm. Well that's a VERY basic flake. It's also seemingly out of date, as the latest advice I've seen recommends packages.${system}.default over defaultPackage.${system}. Let's scrap that and pull in a template from the flake-parts (Github), a flake that bills itself as the "Core of a distributed framework for writing Nix Flakes".
rm flake.nix
nix flake init --template github:hercules-ci/flake-parts
In my experience, flake-parts is an extremely helpful tool that provides some Nix functions which drastically reduce the amount of boilerplate you need for a working flake:
{
description = "Description for the project";
inputs = {
flake-parts.inputs.nixpkgs.follows = "nixpkgs";
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
imports = [
# To import a flake module
# 1. Add foo to inputs
# 2. Add foo as a parameter to the outputs function
# 3. Add here: foo.flakeModule
];
systems = [ "x86_64-linux" "aarch64-darwin" ];
perSystem = { config, self', inputs', pkgs, system, ... }: {
# Per-system attributes can be defined here. The self' and inputs'
# module parameters provide easy access to attributes of the same
# system.
# Equivalent to inputs'.nixpkgs.legacyPackages.hello;
packages.default = pkgs.hello;
};
flake = {
# The usual flake attributes can be defined here, including system-
# agnostic ones like nixosModule and system-enumerating ones, although
# those are more easily expressed in perSystem.
};
};
}
All we're going to need is systems and perSystem, so let's clean up the template to look something like this:
{
description = "Statically generated site";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
flake-parts.url = "github:hercules-ci/flake-parts";
flake-parts.inputs.nixpkgs.follows = "nixpkgs";
};
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
# modify these as needed if you're using a different system
systems = [ "x86_64-linux" "aarch64-darwin" ];
perSystem = { config, self', inputs', pkgs, system, ... }: {
devShells.default = import ./shell.nix { inherit pkgs; };
packages.default = pkgs.callPackage ./site.nix {};
};
};
}
This won't build right away, first we're going to have to add the two .nix files we used.
shell.nix: Reproducible Development Environment
shell.nix is a common feature of many nix builds, and typically specifies the packages that are needed for developing and changing a project. If you haven't used Nix before, you likely have the dependencies and tools for your static site installed directly onto your system, or using a language-specific build tool. By switching to shell.nix, you can decouple the project development environment from your local system, similar to using a Docker container for development. If you don't want or need any special tools to build your site, you can mostly ignore this file, but here's what this might look like for a simple static site:
{ pkgs ? import <nixpkgs> }:
pkgs.mkShell {
buildInputs = [
# build your static site with one of these
pkgs.hugo
# or pkgs.zola or pkgs.jekyll
# deploy with fly
pkgs.flyctl
];
}
shell.nix and mkShell can be extended even further.
site.nix: Reproducible Site Build
We'll write site.nix as a function from packages in nixpkgs to a derivation, which will let us call it more convneniently with pkgs.callPackage, as we did above in flake.nix. Here's an example of a site build for hugo:
{ stdenv, hugo, scour }:
stdenv.mkDerivation {
name = "static-site";
src = ./.;
nativeBuildInputs = [
# specify site build dependencies here
hugo
# optimize SVGs
scour
];
buildPhase = ''
# prepare and build the site
scour -i favicon-original.svg -o favicon.svg
hugo -D
'';
installPhase = ''
# install the Hugo output
cp -r public $out
'';
}
If we need to build any other auxiliary outputs, like Docker images (hint!), we can add them here. For now, let's just save our progress:
git add flake.nix shell.nix site.nix
nix flake lock
git add flake.lock
git commit -m "Initialize Nix flake"
Now we can test that our flake works with nix build:
nix build
ls result
index.html main.css ...
Serving: Caddy
We can reliably build our site, but now we need a way serve that onto the blagoblag. Let's use Caddy! The syntax is marginally less arcane than Apache or Nginx, and it has cool features like HTTPS-by-default!
Unfortunately, the first thing we're going to have to do in our Caddyfile is turn that off:
# fly.io handles https for us
{
auto_https off
}
:8080 {
root * {$SITE_ROOT}
encode gzip
file_server
# redirect to your custom 404 page
handle_errors {
@404 {
expression {http.error.status_code} == 404
}
rewrite @404 /404.html
file_server
}
}
You can test this out by adding caddy to your shell.nix file or else installing it locally, and running something like this:
nix build
env SITE_ROOT=result caddy validate
2022/08/27 19:40:48.724 WARN http automatic HTTPS is completely disabled for server {"server_name": "srv0"}
Valid configuration
env SITE_ROOT=result caddy run
You should be able to browse to your site at 127.0.0.1:8080 and load it, although some resources may load improperly or not at all. When we deploy to Fly, however, everything should be working. Add the Caddyfile to git:
git add Caddyfile
git commit -m "Add Caddyfile"
Deploying: Fly.io
We can scaffold a new Fly app in the usual way:
flyctl launch \
# something unique, doesn't matter if you're going to use a custom domain
--name seals-meander-daringly
# region where the app runs, don't supply this option if you want to interactively choose a region \
--region ewr \
# don't immediately deploy, we need to edit our fly.toml first \
--no-deploy
git add fly.toml
git commit -m "Initialize Fly app"
We need some way for Fly to build a VM from our application and Caddyfile. Fly supports Dockerfiles, so let's just go ahead and start with that:
FROM nixos/nix:latest
WORKDIR /code
ADD . /code
RUN nix \
--extra-experimental-features nix-command \
--extra-experimental-features flakes \
build
FROM caddy:latest
COPY Caddyfile /etc/caddy/Caddyfile
COPY --from=0 /code/result /var/www
ENV SITE_ROOT=/var/www
RUN caddy
We can use a multi-stage build to run the Nix build first, then copy that into a container with the Caddyfile and run that. We're set to deploy!
flyctl deploy
Now we can set up a custom domain, if we want:
flyctl ips list
VERSION IP TYPE REGION CREATED AT
v4 1.2.3.4 public global 2022-08-09T02:19:27Z
v6 aaaa:bbbb:1::a:cccc public global 2022-08-09T02:19:29Z
# add DNS A and AAAA records for the above addresses
# e.g., using doctl for digitalocean
doctl compute domain records create \
--record-type A \
--record-name my
--record-data 1.2.3.4 \
static.site
doctl compute domain records create\
--record-type AAAA \
--record-name my
--record-data aaaa:bbbb:1::a:cccc \
static.site
# get a certificate
flyctl certs add my.static.site
A small addition to the Caddyfile will be helpful, as well:
http://seals-meander-daringly.fly.dev {
redir https://my.static.site
}
Don't forget to flyctl deploy again!
Bonus Round: Configure Everything in Nix
"But mat!" you're shouting in anguish, "Why do we have to write a crummy Dockerfile to build our software! You said several sections ago that Nix could be the universal entry point, and that it was better than Docker for some important sounding reasons!"
I know. What's more is, you're entirely right. We DON'T have to settle for a Dockerfile! Nix has some tooling available to build our Docker images for us, and we can plug that right into our Fly.io application.
Let's add a container.nix file:
{ dockerTools, caddy, site }:
let
caddyfile = builtins.readFile ./Caddyfile;
in dockerTools.buildLayeredImage {
name = "static-site";
tag = "2022-08-28";
config = {
Cmd = [ "${caddy}/bin/caddy" "run" "-config" "${caddyfile}" ];
Env = [
"SITE_ROOT=${site}"
];
};
}
Add it to source control:
git add container.nix
git commit -m "Add Docker image build"
And then refer to that in our flake.nix:
{
# ...
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
# ...
perSystem = { config, self', inputs', pkgs, system, ... }: {
# ...
packages.container = pkgs.callPackage ./container.nix {
site = config.packages.default;
};
};
};
}
You can test that the build works like so, including running it if you have a local Docker installation:
nix build .#container
# if you have a working Docker installation
docker load < result
docker run -itp 8080:8080 static-site:2022-08-28
curl http://[::]:8080
Some people, myself included, don't enjoy the experience of running a Docker engine instance on their Macbooks, so the easiest way for us to test this would be to package up the deployment step into a script and run it on a Linux host with Docker, such as a container in a CI/CD server.
Let's add another callPackage friendly Nix file:
{ lib, docker, flyctl, formats, writeShellScriptBin, dockerImage }:
writeShellScriptBin "deploy" ''
set -euxo pipefail
export PATH="${lib.makeBinPath [(docker.override { clientOnly = true; }) flyctl]}:$PATH"
archive=${dockerImage}
image=$(docker load < $archive | awk '{ print $3; }')
flyctl deploy -i $image
''
And plug it into the flake:
{
# ...
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
# ...
perSystem = { config, self', inputs', pkgs, system, ... }: {
# ...
apps.deploy = pkgs.callPackage ./deploy.nix {
dockerImage = config.packages.container;
};
};
};
}
Now you should be able to use a command like this on a Docker-friendly host, and your site will be running before long:
nix run .#deploy
Future Directions
There is just a touch of boilerplate left in flake.nix, required to thread the correct nixpkgs context through (callPackage, passing site and dockerImage explicitly). We could easily package that up in a flake-module.nix and add it to the imports argument of flake-parts.lib.mkFlake. I'll probably try to make another blog post out of the process of packaging up the Nix glue, so keep your eyes peeled!
Thanks for reading!