git-check-assertions

I recently wrote two blogs posts arguing that there might be some value in writing verifiable claims, i.e. assertions, inside of our commit messages:

• Should we start writing verifiable claims in commit message?
• Writing the steps to validate a test in the commit message

This is a simple verifier for such assertions.

You include a small bash script inside your commit messages, and git-check-assertions will then check out every commit (from the point that your branch diverged from main), and verify that the script in the commit message runs successfully.

For a real-world example, check out the commits in this pull request, where git-check-assertions is used on itself.

⚠️ Only run this on repositories and branches that you trust, since the bash scripts in the commit messages can do whatever they want.

Installation

On most systems, clone this repository and add the bin directory to your PATH.

If you use Nix with flakes, you can simply add it to your program's devshell instead:

Instructions for using the flake

@@ -2,12 +2,18 @@
   inputs = {
     nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
     flake-utils.url = "github:numtide/flake-utils";
+    git-check-assertions = {
+      url = "git+https://codeberg.org/svenvanheugten/git-check-assertions.git?ref=main";
+      inputs.nixpkgs.follows = "nixpkgs";
+      inputs.flake-utils.follows = "flake-utils";
+    };
   };
   outputs =
     {
       self,
       nixpkgs,
       flake-utils,
+      git-check-assertions,
     }:
     flake-utils.lib.eachDefaultSystem (
       system:
@@ -17,7 +23,7 @@
       {
         packages.default = pkgs.callPackage ./default.nix { };
         devShells.default = pkgs.mkShell {
-          packages = [ ];
+          packages = [ git-check-assertions.packages.${system}.default ];
         };
       }
     )

What do you put in your commit messages?

Simply add a bash script enclosed in a git-check-assertions block to a commit message, e.g.:

```git-check-assertions
dotnet build
dotnet test --no-build
```

This script will be run with set -euo pipefail on the version of the code that is in the commit, and the commit will be considered correct if the script exits successfully.

You can technically assert that a command fails by writing ! ... || exit 1, or write assertions about a command's output by piping it to grep, but doing so won't lead to very useful error messages when things go wrong. To make those things easier, there are some helper functions included, which are inspired by bats and bats-assert:

• run <command>: run a command, capturing its exit status in status and combined stdout/stderr in output.
• assert_success: succeed if run produced a zero exit status.
• assert_failure: succeed if run produced a non-zero exit status.
• assert_output <string>: succeed if output exactly matches the string.
• assert_output --partial <string>: succeed if output contains the string.

I'm considering taking bats-assert as a dependency, but for now, this very minimal set of functions with a similar interface should get you on your way.

Multiple blocks in one commit message

You can include multiple git-check-assertions blocks in a single commit message. When you do, each individual block will run with a 'fresh' version of the commit, in which all changes that you made in the previous block have been undone.

Cache of successful commits

After a commit's assertions run successfully, git-check-assertions appends the commit hash to .git-check-assertions-cache in the repo root. On the next run, any commit listed there is skipped.

Be sure to add .git-check-assertions-cache to your .gitignore.

Examples of commit messages

Assert that a commit builds:

```git-check-assertions
dotnet build
```

Assert that a commit builds and that the tests succeed:

```git-check-assertions
dotnet build
run dotnet test --no-build
assert_success
```

Assert that a commit builds, but that the tests do not succeed:

```git-check-assertions
dotnet build
run dotnet test --no-build
assert_failure
```

Assert that a commit builds, and that the tests fail with exactly the error that you expect:

```git-check-assertions
dotnet build
run dotnet test --no-build
assert_failure
assert_output --partial "Invalid URL"
```

Assert that a specific change breaks the tests (as discussed here):

```git-check-assertions
run dotnet test
assert_success

sed -i '/crucial code/d' Main.fs
run dotnet test
assert_failure
```

Assert that a specific change in a commit is necessary for the tests to succeed:

```git-check-assertions
run dotnet test
assert_success

git checkout HEAD~ Main.fs
run dotnet test
assert_failure
assert_output --partial "Invalid URL"
```