git-check-assertions

I recently wrote two blog posts arguing that there might be some value in writing verifiable claims, i.e. assertions, inside of our commit messages:

• Should we start writing verifiable claims in commit messages?
• Writing the steps to validate a test that already passes in the commit message

This is a simple verifier for such assertions.

You write assertions in your commit messages, and git-check-assertions will then check out every commit (from the point that your branch diverged from main), and verify that the assertions in the commit message hold for the version of the code that is in the commit.

For a real-world example, check out the commits in this pull request, where git-check-assertions is used on itself.

⚠️ Only run this on repositories and branches that you trust, since the commands in the commit messages can do whatever they want.

Installation

On most systems, clone this repository and add the bin directory to your PATH.

If you use Nix with flakes, you can simply add it to your program's devshell instead:

Instructions for using the flake

@@ -2,12 +2,18 @@
   inputs = {
     nixpkgs.url = "github:nixos/nixpkgs/nixpkgs-unstable";
     flake-utils.url = "github:numtide/flake-utils";
+    git-check-assertions = {
+      url = "git+https://codeberg.org/svenvanheugten/git-check-assertions.git?ref=main";
+      inputs.nixpkgs.follows = "nixpkgs";
+      inputs.flake-utils.follows = "flake-utils";
+    };
   };
   outputs =
     {
       self,
       nixpkgs,
       flake-utils,
+      git-check-assertions,
     }:
     flake-utils.lib.eachDefaultSystem (
       system:
@@ -17,7 +23,7 @@
       {
         packages.default = pkgs.callPackage ./default.nix { };
         devShells.default = pkgs.mkShell {
-          packages = [ ];
+          packages = [ git-check-assertions.packages.${system}.default ];
         };
       }
     )

What do you put in your commit messages?

Simply add a git-check-assertions block to a commit message, e.g.:

```git-check-assertions
✓ dotnet build
✓ dotnet test --no-build
```

Each block is parsed line-by-line:

• A line starting with [success] or ✓ runs the rest of the line as a shell command and asserts that it exits with status 0.
• A line starting with [failure] or ✗ runs the rest of the line as a shell command and asserts that it exits with a non-zero status.
• Any following non-empty line belongs to the most recent command and asserts that the combined stdout/stderr from that command contains that string.

Blank lines are ignored. A new [success], [failure], ✓, or ✗ line starts a new command block.

Multiple blocks in one commit message

You can include multiple git-check-assertions blocks in a single commit message. When you do, each individual block will run with a 'fresh' version of the commit, in which all changes that you made in the previous block have been undone.

Cache of successful commits

After a commit's assertions run successfully, git-check-assertions appends the commit hash to .git-check-assertions-cache in the repo root. On the next run, any commit listed there is skipped.

Be sure to add .git-check-assertions-cache to your .gitignore.

Examples of commit messages

Assert that a commit builds:

```git-check-assertions
✓ dotnet build
```

Assert that a commit builds and that the tests succeed:

```git-check-assertions
✓ dotnet build
✓ dotnet test --no-build
```

Assert that a commit builds, but that the tests do not succeed:

```git-check-assertions
✓ dotnet build
✗ dotnet test --no-build
```

Assert that a commit builds, and that the tests fail with the error that you expect:

```git-check-assertions
✓ dotnet build
✗ dotnet test --no-build
Invalid URL
```

Assert that a specific change breaks the tests (as discussed here):

```git-check-assertions
✓ dotnet test
✓ sed -i '/crucial code/d' Main.fs
✗ dotnet test
```

Assert that a specific change in a commit is necessary for the tests to succeed:

```git-check-assertions
✓ dotnet test
✓ git checkout HEAD~ Main.fs
✗ dotnet test
Invalid URL
```