sloane
/
mat.services
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

"docker" -> "container", add blog draft

pull/1/head
mat ess 2022-08-27 21:58:45 -04:00
parent d709014b31
commit 117790758b
7 changed files with 338 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Caddyfile
View File

@ -4,7 +4,7 @@
}
http://static-mat-services.fly.dev {
redir https://mat.services/
redir https://mat.services
}
:8080 {

2
content/posts/gitea-on-fly-io/index.md
View File

@ -184,6 +184,6 @@ I have been running my Gitea install on a 512MB instance since the first day I s
It's worth noting that scaling up to 512 MB means you will start accruing a Fly.io balance, but so far this has never been more than a $2 monthly bill for me. If that's too steep for you, consider that Fly.io doesn't (at the time of this writing) charge for monthly bills below $5, or else check out some of the competing PaaS options out there.
### Acknowledgements
## Acknowledgements
- [Thank you to Xe Iaso for xer blog post on Fly.io that inspired me to try it in the first place!](https://xeiaso.net/blog/fly.io-heroku-replacement)
- [Thank you to techknowlogick for their instructive post on running Gitea on Fly.io!](https://blog.gitea.io/2022/04/running-gitea-on-fly.io/)

329
content/posts/static-site-with-nix-and-caddy/index.md
View File

@ -1,6 +1,6 @@
+++
title = "hosting a static site on fly.io with nix and caddy"
date = "2022-08-20"
date = "2022-08-28"
draft = true
[taxonomies]
tags = ["static-site", "nix", "caddy", "fly.io"]
@ -21,7 +21,7 @@ As you may have guessed from the title, my ultimate decision was to take inspira
For the purposes of this article, we'll assume you already have your static site ready to go. Whether you're writing pure HTML by hand, or using a cutting-edge Javascript framework that renders down to static resources, you'll be able to package it up with Nix and serve it with Caddy using Fly.io.
## Before You Deploy: Nix
Much like static site hosts, static site **generators** are everywhere, and it seems like one of those things where people have particularly strong opinions on what workflow fits them best. Some people eschew a "generator" entirely and chain more purpose-built tools together to achieve the perfect bespoke output. Whatever your personal choice on generating the content for your static site, it helps to have a sort of "universal interface" to actually kick off the build process. Github and Gitlab have their own custom Continuous Integration/Continuous Delivery systems where the build for the site can be configured. Makefiles can declare a set of commands to run in a generic way, but they don't help us pull in dependencies from the outside. Dockerfiles are not only generic, but also provide some primitives to make it easy to fetch any buildtime dependencies we need. Nix, a declarative package manager, is another option that we can use to specify our build AND dependencies, without requiring us to actually make a container. Nix also provides stronger guarantees around reproducibility and hermeticity than Docker. I already use Nix flakes for virtually all of my personal work, so that's how we'll specify our build step.
Much like static site hosts, static site **generators** are everywhere, and it seems like one of those things where people have particularly strong opinions on what workflow fits them best. Some people eschew a "generator" entirely and chain more purpose-built tools together to achieve the perfect bespoke output. Whatever your personal choice on generating the content for your static site, it helps to have a sort of "universal entry point" to actually kick off the build process. Github and Gitlab have their own custom Continuous Integration/Continuous Delivery systems where the build for the site can be configured. Makefiles can declare a set of commands to run in a generic way, but they don't help us pull in dependencies from the outside. Dockerfiles are not only generic, but also provide some primitives to make it easy to fetch any buildtime dependencies we need. Nix, a declarative package manager, is another option that we can use to specify our build AND dependencies, without requiring us to actually make a container. Nix also provides stronger guarantees around reproducibility and hermeticity than Docker. I already use Nix flakes for virtually all of my personal work, so that's how we'll specify our build step.
Nix is a very powerful tool, but it can be intimidating to use, and I've encountered many people saying that the documentation is too sparse to be useful. I'll keep the Nix details minimal, and try to include enough explanation for what we're doing that even unfamiliar readers can follow along, but I would encourage anyone who wants some more background on Nix and Nix flakes to check out [these](https://xeiaso.net/blog/nix-flakes-1-2022-02-21) [posts](https://xeiaso.net/blog/nix-flakes-2-2022-02-27) from Xe Iaso's blog.
@ -65,6 +65,7 @@ git commit -m "TODO: pithy quip about starting a new endeavor"
Nix flakes require you to be using some form of source control that Nix understands, which means (to my understanding) either Git or Mercurial. Everything tracked by source control will end up going into the Nix store, so be doubly sure that you haven't committed any secrets, tokens, or manifestos that you don't want leaking out.
### `flake.nix`: Entry Point
We finally have all the foundations in place to put our site's source into a flake. We can get a fresh flake by using the default template:
```bash
nix flake init
@ -81,23 +82,337 @@ This will leave us with the following:
}
```
Hmm. Well that's a VERY basic flake. It's also seemingly out of date, as the latest advice I've seen recommends `packages.${system}.default` over `defaultPackage.${system}`. Let's scrap that and pull in a template from the [`flake-utils`](https://github.com/numtide/flake-utils), a very popular recommendation for taming boilerplate and glue code while authoring flakes.
Hmm. Well that's a VERY basic flake. It's also seemingly out of date, as the latest advice I've seen recommends `packages.${system}.default` over `defaultPackage.${system}`. Let's scrap that and pull in a template from the [`flake-parts`](https://flake.parts) ([Github](https://github.com/hercules-ci/flake-parts)), a flake that bills itself as the _"Core of a distributed framework for writing Nix Flakes"_.
```bash
rm flake.nix
nix flake init --template github:numtide/flake-utils#simple-flake
nix flake init --template github:hercules-ci/flake-parts
```
This template will create *three* files: `flake.nix`, `shell.nix`, and `overlay.nix`. Technically, we could squish this all into a single `flake.nix` file, but breaking it down like this will help to separate our concerns and keep things focused.
In my experience, `flake-parts` is an extremely helpful tool that provides some Nix functions which drastically reduce the amount of boilerplate you need for a working flake:
```nix
{
description = "Description for the project";
inputs = {
flake-parts.inputs.nixpkgs.follows = "nixpkgs";
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
};
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
imports = [
# To import a flake module
# 1. Add foo to inputs
# 2. Add foo as a parameter to the outputs function
# 3. Add here: foo.flakeModule
];
systems = [ "x86_64-linux" "aarch64-darwin" ];
perSystem = { config, self', inputs', pkgs, system, ... }: {
# Per-system attributes can be defined here. The self' and inputs'
# module parameters provide easy access to attributes of the same
# system.
# Equivalent to inputs'.nixpkgs.legacyPackages.hello;
packages.default = pkgs.hello;
};
flake = {
# The usual flake attributes can be defined here, including system-
# agnostic ones like nixosModule and system-enumerating ones, although
# those are more easily expressed in perSystem.
};
};
}
```
All we're going to need is `systems` and `perSystem`, so let's clean up the template to look something like this:
```nix
{
description = "Statically generated site";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
flake-parts.url = "github:hercules-ci/flake-parts";
flake-parts.inputs.nixpkgs.follows = "nixpkgs";
};
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
# modify these as needed if you're using a different system
systems = [ "x86_64-linux" "aarch64-darwin" ];
perSystem = { config, self', inputs', pkgs, system, ... }: {
devShells.default = import ./shell.nix { inherit pkgs; };
packages.default = pkgs.callPackage ./site.nix {};
};
};
}
```
This won't build right away, first we're going to have to add the two `.nix` files we used.
### `shell.nix`: Reproducible Development Environment
`shell.nix` is a common feature of many nix builds, and typically specifies the packages that are needed for developing and changing a project. If you haven't used Nix before, you likely have the dependencies and tools for your static site installed directly onto your system, or using a language-specific build tool. By switching to `shell.nix`, you can decouple the project development environment from your local system, similar to using a Docker container for development. If you don't want or need any special tools to build your site, you can mostly ignore this file, but here's what this might look like for a simple static site:
```nix
{ pkgs ? import <nixpkgs> }:
pkgs.mkShell {
buildInputs = [
# build your static site with one of these
pkgs.hugo
# or pkgs.zola or pkgs.jekyll
# deploy with fly
pkgs.flyctl
];
}
```
### `overlay.nix`: Reproducible Site Build
`shell.nix` and `mkShell` can be extended even further.
### `flake.nix`: External Interface
### `site.nix`: Reproducible Site Build
We'll write `site.nix` as a function from packages in `nixpkgs` to a derivation, which will let us call it more convneniently with `pkgs.callPackage`, as we did above in `flake.nix`. Here's an example of a site build for `hugo`:
```nix
{ stdenv, hugo, scour }:
stdenv.mkDerivation {
name = "static-site";
src = ./.;
nativeBuildInputs = [
# specify site build dependencies here
hugo
# optimize SVGs
scour
];
buildPhase = ''
# prepare and build the site
scour -i favicon-original.svg -o favicon.svg
hugo -D
'';
installPhase = ''
# install the Hugo output
cp -r public $out
'';
)
```
If we need to build any other auxiliary outputs, like Docker images (hint!), we can add them here. For now, let's just save our progress:
```bash
git add flake.nix shell.nix site.nix
nix flake lock
git add flake.lock
git commit -m "Initialize Nix flake"
```
Now we can test that our flake works with `nix build`:
```bash
nix build
ls result
index.html main.css ...
```
## Serving: Caddy
We can reliably build our site, but now we need a way serve that onto the blagoblag. Let's use Caddy! The syntax is marginally less arcane than Apache or Nginx, and it has cool features like HTTPS-by-default!
Unfortunately, the first thing we're going to have to do in our `Caddyfile` is turn that off:
```Caddyfile
# fly.io handles https for us
{
auto_https off
}
:8080 {
root * {$SITE_ROOT}
encode gzip
file_server
# redirect to your custom 404 page
handle_errors {
@404 {
expression {http.error.status_code} == 404
}
rewrite @404 /404.html
file_server
}
}
```
You can test this out by adding `caddy` to your `shell.nix` file or else installing it locally, and running something like this:
```bash
nix build
env SITE_ROOT=result caddy validate
2022/08/27 19:40:48.724 WARN http automatic HTTPS is completely disabled for server {"server_name": "srv0"}
Valid configuration
env SITE_ROOT=result caddy run
```
You should be able to browse to your site at `127.0.0.1:8080` and load it, although some resources may load improperly or not at all. When we deploy to Fly, however, everything should be working. Add the Caddyfile to git:
```bash
git add Caddyfile
git commit -m "Add Caddyfile"
```
## Deploying: Fly.io
We can scaffold a new Fly app in the usual way:
```bash
flyctl launch \
# something unique, doesn't matter if you're going to use a custom domain
--name seals-meander-daringly
# region where the app runs, don't supply this option if you want to interactively choose a region \
--region ewr \
# don't immediately deploy, we need to edit our fly.toml first \
--no-deploy
git add fly.toml
git commit -m "Initialize Fly app"
```
We need some way for Fly to build a VM from our application and Caddyfile. Fly supports Dockerfiles, so let's just go ahead and start with that:
```Dockerfile
FROM nixos/nix:latest
WORKDIR /code
ADD . /code
RUN nix \
--extra-experimental-features nix-command \
--extra-experimental-features flakes \
build
FROM caddy:latest
COPY Caddyfile /etc/caddy/Caddyfile
COPY --from=0 /code/result /var/www
ENV SITE_ROOT=/var/www
RUN caddy
```
We can use a multi-stage build to run the Nix build first, then copy that into a container with the Caddyfile and run that. We're set to deploy!
```shell
flyctl deploy
```
Now we can set up a custom domain, if we want:
```bash
flyctl ips list
VERSION IP TYPE REGION CREATED AT
v4 1.2.3.4 public global 2022-08-09T02:19:27Z
v6 aaaa:bbbb:1::a:cccc public global 2022-08-09T02:19:29Z
# add DNS A and AAAA records for the above addresses
# e.g., using doctl for digitalocean
doctl compute domain records create \
--record-type A \
--record-name my
--record-data 1.2.3.4 \
static.site
doctl compute domain records create\
--record-type AAAA \
--record-name my
--record-data aaaa:bbbb:1::a:cccc \
static.site
# get a certificate
flyctl certs add my.static.site
```
A small addition to the Caddyfile will be helpful, as well:
```Caddyfile
http://seals-meander-daringly.fly.dev {
redir https://my.static.site
}
```
Don't forget to `flyctl deploy` again!
## Bonus Round: Configure Everything in Nix
"But mat!" you're shouting in anguish, "Why do we have to write a crummy Dockerfile to build our software! You said several sections ago that Nix could be the universal entry point, and that it was better than Docker for some important sounding reasons!"
I know. What's more is, you're entirely right. We DON'T have to settle for a Dockerfile! Nix has some tooling available to build our Docker images for us, and we can plug that right into our Fly.io application.
Let's add a `container.nix` file:
```nix
{ dockerTools, caddy, site }:
let
caddyfile = builtins.readFile ./Caddyfile;
in dockerTools.buildLayeredImage {
name = "static-site";
tag = "2022-08-28";
config = {
Cmd = [ "${caddy}/bin/caddy" "run" "-config" "${caddyfile}" ];
Env = [
"SITE_ROOT=${site}"
];
};
}
```
Add it to source control:
```bash
git add container.nix
git commit -m "Add Docker image build"
```
And then refer to that in our `flake.nix`:
```nix
{
# ...
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
# ...
perSystem = { config, self', inputs', pkgs, system, ... }: {
# ...
packages.container = pkgs.callPackage ./container.nix {
site = config.packages.default;
};
};
};
}
```
You can test that the build works like so, including running it if you have a local Docker installation:
```bash
nix build .#container
# if you have a working Docker installation
docker load < result
docker run -itp 8080:8080 static-site:2022-08-28
curl http://[::]:8080
```
Some people, myself included, don't enjoy the experience of running a Docker engine instance on their Macbooks, so the easiest way for us to test this would be to package up the deployment step into a script and run it on a Linux host with Docker, such as a container in a CI/CD server.
Let's add another `callPackage` friendly Nix file:
```nix
{ lib, docker, flyctl, formats, writeShellScriptBin, dockerImage }:
writeShellScriptBin "deploy" ''
set -euxo pipefail
export PATH="${lib.makeBinPath [(docker.override { clientOnly = true; }) flyctl]}:$PATH"
archive=${dockerImage}
image=$(docker load < $archive | awk '{ print $3; }')
flyctl deploy -i $image
''
```
And plug it into the flake:
```nix
{
# ...
outputs = { self, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit self; } {
# ...
perSystem = { config, self', inputs', pkgs, system, ... }: {
# ...
apps.deploy = pkgs.callPackage ./deploy.nix {
dockerImage = config.packages.container;
};
};
};
}
```
Now you should be able to use a command like this on a Docker-friendly host, and your site will be running before long:
```bash
nix run .#deploy
```
Thanks for reading!

12
flake.lock
View File

@ -23,11 +23,11 @@
]
},
"locked": {
"lastModified": 1657102481,
"narHash": "sha256-62Fuw8JgPub38OdgNefkIKOodM9nC3M0AG6lS+7smf4=",
"lastModified": 1661009076,
"narHash": "sha256-phAE40gctVygRq3G3B6LhvD7u2qdQT21xsz8DdRDYFo=",
"owner": "hercules-ci",
"repo": "flake-parts",
"rev": "608ed3502263d6f4f886d75c48fc2b444a4ab8d8",
"rev": "850d8a76026127ef02f040fb0dcfdb8b749dd9d9",
"type": "github"
},
"original": {
@ -38,11 +38,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1659803779,
"narHash": "sha256-+5zkHlbcbFyN5f3buO1RAZ9pH1wXLxCesUJ0vFmLr9Y=",
"lastModified": 1661450036,
"narHash": "sha256-0/9UyJLtfWqF4uvOrjFIzk8ue1YYUHa6JIhV0mALkH0=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "f44884060cb94240efbe55620f38a8ec8d9af601",
"rev": "f3d0897be466aa09a37f6bf59e62c360c3f9a6cc",
"type": "github"
},
"original": {

13
flake.nix
View File

@ -17,7 +17,8 @@
systems = inputs.nixpkgs.lib.systems.flakeExposed;
perSystem = { config, self', inputs', pkgs, system, ... }:
let
inherit (pkgs.callPackage ./nix { }) deploy docker fonts optimize-images themes;
# TODO: move these to a flake-modules
inherit (pkgs.callPackage ./nix { }) container deploy fonts optimize-images themes;
inherit (fonts) copyFonts linkFonts;
inherit (themes {
theme = inputs.apollo;
@ -27,13 +28,13 @@
{
packages.default = with pkgs; stdenv.mkDerivation {
pname = "personal-site";
version = "2022-08-21";
version = "2022-08-27";
src = ./.;
nativeBuildInputs = [ optimize-images zola ];
configurePhase = copyTheme + copyFonts;
buildPhase = ''
optimize-images
zola build
zola build --drafts
'';
installPhase = ''
cp -r public $out
@ -43,12 +44,12 @@
packages = [ flyctl optimize-images zola ];
shellHook = linkTheme + linkFonts;
};
packages.docker = docker {
packages.container = container {
caddyfile = builtins.readFile ./Caddyfile;
site = self'.packages.default;
site = config.packages.default;
};
apps.deploy.program =
let deploy' = deploy { dockerImage = self'.packages.docker; };
let deploy' = deploy { dockerImage = config.packages.container; };
in "${deploy'}/bin/deploy";
};
};

0
nix/docker.nix → nix/container.nix
View File

2
nix/default.nix
View File

@ -1,6 +1,6 @@
{ callPackage }: {
container = { caddyfile, site }: callPackage ./container.nix { inherit caddyfile site; };
deploy = { dockerImage }: callPackage ./deploy.nix { inherit dockerImage; };
docker = { caddyfile, site }: callPackage ./docker.nix { inherit caddyfile site; };
fonts = callPackage ./fonts.nix { };
optimize-images = callPackage ./optimize-images.nix { };
themes = { theme, themeEnabled }: callPackage ./themes.nix { inherit theme themeEnabled; };