IT’S TAKEN ME 60 HOURS TO SAVE 2 MINUTES OF TYPING#

Git hooks are scripts of some kind that run when you do something in Git. Real developers (not me) use them to kick off a bunch of shit like QA tests, validation tools, sending messages to people, pretty much anything you can imagine. They do a ton of shit, but I am going to do the thing that I always do and barely scratch the surface of its potential. AND I am using it to run Bash scripts. If you are a proper dev, or DevOps type, this is your opportunity to look away.

Building modern web sites reminds me of my days in Seattle working for a small software startup. There, the QA team did tons of automation to run tons of tests on the product codebase. Mostly I wrote specifications, and helped the ex-Microsoft people (the whole department) do testing on Linux and Solaris. Back then, Real Devs™ wrote code that had to be compiled. Scripting languages and web pages were for weaklings.

Today, I am using Git like it was CVS, to check in what are essentially text files and kick off Bash shell scripts to produce a static website. It’s like using precision aircraft tools to open beer bottles.

The precision tool I am focued on today is the Git post-receive hook. Post-receive happens on the remote repo when you push an update. I have combed through Git documentation that was written Deveese and I have managed to kludge together a deployment script from other people’s code. I cannot begin to express how far out of my depth I am.

The Staging Server#

I am using a Linux container on my Proxmox server to push my MD files to. It runs Git, to serve the remote repo for my workstations, Hugo to build the static site, and Caddy to serve the test site. I don’t use any special features in Caddy, I just want to see what the site will look like before it gets deployed to my VPS where it’s visible to the Internet.

The Caddy server config is incredibly simple, since the staging server is only accessible from my home lab, I don’t need to worry about HTTPS, or domain names:

# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.

:80 {
        root * /var/www
        file_server
}

In the future, when I have proper DNS for my overlay network, and a bunch of other things, I will try to get clever with the Caddy config, but first I have to get this damn thing working.

The file Structure#

In my home directory on the staging server, I have some directories to hold the different files during the staging process. I wish I could just pipe everything right into Hugo, but that is beyond my skill to heal. So instead I use subdirectories. If this sounds overly complicated for a deploy script, that’s because it is. The file structure keeps reusing “example.com” because it can be used for multiple websites. Each website gets its own repo (foo.com.git, bar.com.git) and this same script can be run when new MD files are pushed.

/home/me
	-- auto <-- automation directory
		stage.sh
        deploy.sh
    -- build <-- the hugo site folder, required by hugo
		-- example.com <-- created with "hugo new site example.com"
        	-- archetypes
        	-- assets
            -- content <-- this is where the markdown files go
            -- data
            hugo.toml <-- the config file for your site, use it to set up your template
            -- i18n
            -- layouts
            -- static <-- anything that isn't markdown should go here, it gets copied to 'public'
            -- themes <-- git clone your hugo template to here
    -- git
		-- example.com.git <-- this is the bare repo for git remote created by 'git init --bare'
        	HEAD
            -- branches
			config
            description
            -- hooks
            	post-receive <-- edit this file to call stage.sh
            -- info
            -- objects
            -- refs
    -- markdown <-- the git repo gets cloned to here
    -- preview <-- the rendered static site goes here

There are separate “git” and “markdown” directories because the git repo is bare. The markdown files are kept in the git database somewhere (no clue where that is) and are not accessible to the file system. If I want to see the MD files on the local file system, I need to clone the Git repo. Git hates when you clone a repo into an existing folder, so the “markdown/example.com” folder gets deleted when stage.sh runs.

There are separate “markdown” and “build” directories because Hugo requires your MD files to sit in the “content” subdirectory, and Git won’t want the “build/example.com/content” to be pre-existing. Right now the script just uses rsync to copy the MD files from “markdown/example.com” to “build/example.com/content”. There is probably something I can do with hard links to avoid copying the files, or I could move the files instead of copying, or I could be brave and delete “build example.com/content” and clone the repo to there. But right now rsync and redundancy is my safety net.

There are separate “build/example.com/public” and “preview/example.com” directories so that I can rsync the generated static site to /var/www. Again, there is a more elegant way, but during the development process, I am being redundant.

What the script does:#

When I push my MarkDown files to the bare Git repo on my staging server, the hook (~/git/example.com.git/hooks/post-receive) calls a shell script and passes a command line argument which is the domain name of the site you are pushing. I keep the bare repo in my home directory in a directory named after the domain name (example.com in the example below). Git hooks have to be made executable before they run.
```
#!/bin/bash
/home/me/auto/stage.sh example.com
```

When stage.sh runs, it takes the commandline argument and does some stuff make the script fail if anything is off.

#  Strict bash, kills the script if anything goes wrong
set -euo pipefail

# You are supposed to "prep.sh example.com"
# if you put anything other than 1 argument, exit
if [[ $# -ne 1 ]]; then
echo "Usage: $0 <repo-name>"
exit 1
fi

It takes the command line argument (stage.sh example.com) and uses it to create variables for the different working trees:


	REPO_NAME="$1" # example.com
	SRC="$HOME/git/${REPO_NAME}.git" # your git repo is in ~/git/example.com.git
	MARKDOWN="$HOME/markdown/${REPO_NAME}" # you clone it to ~/markdown/example.com
	BUILD="$HOME/build/${REPO_NAME}" # ~/build/example.com/
	CONTENT="$BUILD/content/" #  ~/build/example.com/content
	PREVIEW="$HOME/preview/${REPO_NAME}" # ~/preview/example.com

The script then double checks that the “/git/example.com.git” directory exists, and if not, right to jail!

	
    # the source exists? no? exit
	if [[ ! -d "${SRC}" ]]; then
  	echo "Source repo not found: ${SRC}"
  	exit 1
	fi

Assuming the git repo is working, we now delete the “markdown/example.com” and clone it again.


    # delete the markdown folder, git clone hates existing files
	rm -rf "${MARKDOWN}"

	# clone repo to markdown directory
	git clone "${SRC}" "${MARKDOWN}"

Now that the MD files are available on the filesystem we can rsync them to “build/example.com/content”.


	# sync markdown/example.com build/example.com/content
	rsync -rvh "${MARKDOWN}/" "${CONTENT}/"

Now that the new MD files are part of the Hugo file structure, IT’S HUGIN’ TIME! Hugo now builds the site, but it changes the baseURL variable to a relative path. Also, “build/example.com” is the only place for the Hugo file structure. It does not go into Git, and it’s not present on my writing workstations.
This is because the example.com preview site is a subfolder of the “/var/www”, for for example http://staging.local/example.com/. Again, I can tighten things up by using subdomains in caddy (example.com.staging.local) and/or by using hard links. Notice how Hugo spits out the files to “preview.example.com” and they don’t get rsync’ed from “build/example.com/pubic”.


	# Hugo processes ~/build/example.com and builds in /preview/example.com
	hugo -s "${BUILD}" --baseURL="/${REPO_NAME}/" --destination="${PREVIEW}"

LOL/JK, here comes rsync again. This time rsync copies the files but also chowns the files to the caddy user so that the files have acceptable permissions for Caddy:


# sync preview/example.com to /var/www/example.com
rsync -rvh --chown=caddy:caddy "${PREVIEW}/" "/var/www/${REPO_NAME}"

I am not ready to make the script public on something like Github. This script is embarrassment in its current form. Also, there is a good change that I will get distracted by something shiny and never actually update it.

The Shell Script From Hell

IT’S TAKEN ME 60 HOURS TO SAVE 2 MINUTES OF TYPING#

The Staging Server#

The file Structure#

What the script does:#