Introduce a DSL to replace the plain bash scripts

It's really important for `git-check-assertions` blocks to be 'obviously
correct', so that we don't need to... test the tests that test the tests.

Let's introduce a little DSL that is less error-prone than a plain bash
script.

README.md

@@ -7,11 +7,11 @@ I recently wrote two blog posts arguing that there might be some value in writin
 
 This is a simple verifier for such assertions.
 
-You include a small bash script inside your commit messages, and `git-check-assertions` will then check out every commit (from the point that your branch diverged from `main`), and verify that the script in the commit message runs successfully.
+You write assertions in your commit messages, and `git-check-assertions` will then check out every commit (from the point that your branch diverged from `main`), and verify that the assertions in the commit message hold for the version of the code that is in the commit.
 
 For a real-world example, check out the commits in [this pull request](https://codeberg.org/svenvanheugten/git-check-assertions/pulls/8), where `git-check-assertions` is used on itself.
 
-⚠️ Only run this on repositories and branches that you trust, since the `bash` scripts in the commit messages can do whatever they want.
+⚠️ Only run this on repositories and branches that you trust, since the commands in the commit messages can do whatever they want.
 
 ## Installation
 
@@ -56,26 +56,22 @@ If you use Nix with flakes, you can simply add it to your program's devshell ins
 
 ## What do you put in your commit messages?
 
-Simply add a bash script enclosed in a `git-check-assertions` block to a commit message, e.g.:
+Simply add a `git-check-assertions` block to a commit message, e.g.:
 
 ~~~
 ```git-check-assertions
-dotnet build
+[success] dotnet build
-dotnet test --no-build
+[success] dotnet test --no-build
 ```
 ~~~
 
-This script will be run with `set -euo pipefail` on the version of the code that is in the commit, and the commit will be considered correct if the script exits successfully.
+Each block is parsed line-by-line:
 
-You can technically assert that a command fails by writing `! ... || exit 1`, or write assertions about a command's output by piping it to `grep`, but doing so won't lead to very useful error messages when things go wrong. To make those things easier, there are some helper functions included, which are inspired by [bats](https://github.com/bats-core/bats-core) and [bats-assert](https://github.com/bats-core/bats-assert):
+* A line starting with `[success] ` runs the rest of the line as a shell command and asserts that it exits with status `0`.
+* A line starting with `[failure] ` runs the rest of the line as a shell command and asserts that it exits with a non-zero status.
+* Any following non-empty line belongs to the most recent command and asserts that the combined stdout/stderr from that command contains that string.
 
-* `run <command>`: run a command, capturing its exit status in `status` and combined stdout/stderr in `output`.
+Blank lines are ignored. A new `[success]` or `[failure]` line starts a new command block.
-* `assert_success`: succeed if `run` produced a zero exit status.
-* `assert_failure`: succeed if `run` produced a non-zero exit status.
-* `assert_output <string>`: succeed if `output` exactly matches the string.
-* `assert_output --partial <string>`: succeed if `output` contains the string.
-
-I'm considering taking `bats-assert` as a dependency, but for now, this very minimal set of functions with a similar interface should get you on your way.
 
 ## Multiple blocks in one commit message
 
@@ -93,7 +89,7 @@ Assert that a commit builds:
 
 ~~~
 ```git-check-assertions
-dotnet build
+[success] dotnet build
 ```
 ~~~
 
@@ -101,9 +97,8 @@ Assert that a commit builds and that the tests succeed:
 
 ~~~
 ```git-check-assertions
-dotnet build
+[success] dotnet build
-run dotnet test --no-build
+[success] dotnet test --no-build
-assert_success
 ```
 ~~~
 
@@ -111,20 +106,18 @@ Assert that a commit builds, but that the tests do not succeed:
 
 ~~~
 ```git-check-assertions
-dotnet build
+[success] dotnet build
-run dotnet test --no-build
+[failure] dotnet test --no-build
-assert_failure
 ```
 ~~~
 
-Assert that a commit builds, and that the tests fail with exactly the error that you expect:
+Assert that a commit builds, and that the tests fail with the error that you expect:
 
 ~~~
 ```git-check-assertions
-dotnet build
+[success] dotnet build
-run dotnet test --no-build
+[failure] dotnet test --no-build
-assert_failure
+Invalid URL
-assert_output --partial "Invalid URL"
 ```
 ~~~
 
@@ -132,12 +125,10 @@ Assert that a specific change breaks the tests (as discussed [here](https://sven
 
 ~~~
 ```git-check-assertions
-run dotnet test
+[success] dotnet test
-assert_success
 
-sed -i '/crucial code/d' Main.fs
+[success] sed -i '/crucial code/d' Main.fs
-run dotnet test
+[failure] dotnet test
-assert_failure
 ```
 ~~~
 
@@ -145,12 +136,10 @@ Assert that a specific change in a commit is necessary for the tests to succeed:
 
 ~~~
 ```git-check-assertions
-run dotnet test
+[success] dotnet test
-assert_success
 
-git checkout HEAD~ Main.fs
+[success] git checkout HEAD~ Main.fs
-run dotnet test
+[failure] dotnet test
-assert_failure
+Invalid URL
-assert_output --partial "Invalid URL"
 ```
 ~~~

bin/git-check-assertions

@@ -24,48 +24,6 @@
 
 set -euo pipefail
 
-# helper functions inspired by bats/bats-assert
-run() {
-  set +e
-  tmp_output="$(mktemp)"
-  "$@" 2>&1 | tee "$tmp_output"
-  status=${PIPESTATUS[0]}
-  set -e
-  output="$(cat "$tmp_output")"
-  rm -f "$tmp_output"
-  printf '%s\n' "$output"
-  return 0
-}
-assert_success() {
-  if [ "$status" -ne 0 ]; then
-    echo "Expected command to succeed, but it failed."
-    exit 1
-  fi
-}
-assert_failure() {
-  if [ "$status" -eq 0 ]; then
-    echo "Expected command to fail, but it succeeded."
-    exit 1
-  fi
-}
-assert_output() {
-  if [ "${1:-}" = "--partial" ]; then
-    local expected="$2"
-    if [[ "$output" != *"$expected"* ]]; then
-      echo "Expected output to contain: $expected"
-      echo "Actual output: $output"
-      exit 1
-    fi
-    return 0
-  fi
-  if [ "$output" != "$1" ]; then
-    echo "Expected output to equal: $1"
-    echo "Actual output: $output"
-    exit 1
-  fi
-}
-export -f run assert_success assert_failure assert_output
-
 # main flow
 if [ ! -z "$(git status --porcelain)" ]; then
   echo "Uncommitted changes. Refusing to run." >&2
@@ -121,8 +79,64 @@ for commit_hash in "${commits[@]}"; do
     check_assertions() {
       python3 - "$block" <<-EOF
 			import sys
+			import re
 			import subprocess
-			subprocess.run(sys.argv[1], shell=True, check=True)
+
+			block = sys.argv[1]
+			steps = []
+			current_step = None
+
+			for line_number, raw_line in enumerate(block.splitlines(), start=1):
+			    line = raw_line.rstrip("\\r")
+			    if not line.strip():
+			        continue
+
+			    match = re.match(r"^\\[(success|failure)\\]\\s+(.+)$", line)
+			    if match:
+			        current_step = {
+			            "expects_success": match.group(1) == "success",
+			            "command": match.group(2),
+			            "assertions": [],
+			        }
+			        steps.append(current_step)
+			        continue
+
+			    if current_step is None:
+			        print(
+			            f"DSL parse error on line {line_number}: "
+			            "assertion text must follow [success] or [failure]."
+			        )
+			        sys.exit(1)
+
+			    current_step["assertions"].append(line)
+
+			for step in steps:
+			    completed = subprocess.run(
+			        ["sh", "-lc", step["command"]],
+			        stdout=subprocess.PIPE,
+			        stderr=subprocess.STDOUT,
+			        text=True,
+			    )
+			    output = completed.stdout or ""
+
+			    if output:
+			        sys.stdout.write(output)
+			        if not output.endswith("\\n"):
+			            sys.stdout.write("\\n")
+
+			    if step["expects_success"]:
+			        if completed.returncode != 0:
+			            print("Expected command to succeed, but it failed.")
+			            sys.exit(1)
+			    elif completed.returncode == 0:
+			        print("Expected command to fail, but it succeeded.")
+			        sys.exit(1)
+
+			    for expected in step["assertions"]:
+			        if expected not in output:
+			            print(f"Expected output to contain: {expected}")
+			            print(f"Actual output: {output}")
+			            sys.exit(1)
 			EOF
     }
     if ! check_assertions; then

test/git-check-assertions.bats

@@ -43,7 +43,7 @@ commit_with_assertion() {
 }
 
 @test "should not run any assertion blocks when on main" {
-  commit_with_assertion "touch ../test"
+  commit_with_assertion "[success] touch ../test"
 
   run git-check-assertions
 
@@ -87,7 +87,7 @@ commit_with_assertion() {
 }
 
 @test "should not run any assertion blocks from main when on a feature branch" {
-  commit_with_assertion "touch ../test"
+  commit_with_assertion "[success] touch ../test"
   git checkout -b feature
   git commit --allow-empty -m "feature"
 
@@ -100,7 +100,7 @@ commit_with_assertion() {
 @test "should not run any assertion blocks from master when on a feature branch" {
   init_repo "$BATS_TEST_TMPDIR/repo-master" master
 
-  commit_with_assertion "touch ../test"
+  commit_with_assertion "[success] touch ../test"
   git checkout -b feature
   git commit --allow-empty -m "feature"
 
@@ -112,8 +112,8 @@ commit_with_assertion() {
 
 @test "should run all succeeding assertion blocks on the feature branch and finally return to the original branch" {
   git checkout -b feature
-  commit_with_assertion "touch ../test1"
+  commit_with_assertion "[success] touch ../test1"
-  commit_with_assertion "touch ../test2"
+  commit_with_assertion "[success] touch ../test2"
 
   run git-check-assertions
 
@@ -130,11 +130,11 @@ commit_with_assertion() {
 		test
 
 		\`\`\`git-check-assertions
-		touch ../test1
+		[success] touch ../test1
 		\`\`\`
 
 		\`\`\`git-check-assertions
-		touch ../test2
+		[success] touch ../test2
 		\`\`\`
 	EOF
 
@@ -159,11 +159,11 @@ commit_with_assertion() {
 		update
 
 		\`\`\`git-check-assertions
-		git checkout HEAD~
+		[success] git checkout HEAD~
 		\`\`\`
 
 		\`\`\`git-check-assertions
-		grep new readme
+		[success] grep new readme
 		\`\`\`
 	EOF
 
@@ -174,8 +174,8 @@ commit_with_assertion() {
 
 @test "should stop at the first failing assertion block and return to the original branch" {
   git checkout -b feature
-  commit_with_assertion "touch ../test1 && exit 3"
+  commit_with_assertion "[success] touch ../test1 && exit 3"
-  commit_with_assertion "touch ../test2"
+  commit_with_assertion "[success] touch ../test2"
 
   run git-check-assertions
 
@@ -188,7 +188,7 @@ commit_with_assertion() {
 
 @test "should restore worktree when finished" {
   git checkout -b feature
-  commit_with_assertion 'echo blah >> readme'
+  commit_with_assertion '[success] echo blah >> readme'
 
   run git-check-assertions
 
@@ -198,8 +198,8 @@ commit_with_assertion() {
 
 @test "should restore worktree between commits" {
   git checkout -b feature
-  commit_with_assertion 'echo blah >> readme'
+  commit_with_assertion '[success] echo blah >> readme'
-  commit_with_assertion 'cp readme ../test'
+  commit_with_assertion '[success] cp readme ../test'
 
   run git-check-assertions
 
@@ -209,7 +209,7 @@ commit_with_assertion() {
 
 @test "should restore worktree after a failure" {
   git checkout -b feature
-  commit_with_assertion $'echo blah >> readme\nexit 3'
+  commit_with_assertion $'[success] echo blah >> readme\n[success] exit 3'
 
   run git-check-assertions
 
@@ -219,7 +219,7 @@ commit_with_assertion() {
 
 @test "should restore index when finished" {
   git checkout -b feature
-  commit_with_assertion $'echo blah >> readme\ngit add readme'
+  commit_with_assertion $'[success] echo blah >> readme\n[success] git add readme'
 
   run git-check-assertions
 
@@ -229,8 +229,8 @@ commit_with_assertion() {
 
 @test "should restore index between commits" {
   git checkout -b feature
-  commit_with_assertion $'echo blah >> readme\ngit add readme'
+  commit_with_assertion $'[success] echo blah >> readme\n[success] git add readme'
-  commit_with_assertion 'cp readme ../test'
+  commit_with_assertion '[success] cp readme ../test'
 
   run git-check-assertions
 
@@ -240,7 +240,7 @@ commit_with_assertion() {
 
 @test "should restore index after a failure" {
   git checkout -b feature
-  commit_with_assertion $'echo blah >> readme\ngit add readme\nexit 3'
+  commit_with_assertion $'[success] echo blah >> readme\n[success] git add readme\n[success] exit 3'
 
   run git-check-assertions
 
@@ -250,7 +250,7 @@ commit_with_assertion() {
 
 @test "should skip cached commits on subsequent runs" {
   git checkout -b feature
-  commit_with_assertion "touch ../test"
+  commit_with_assertion "[success] touch ../test"
 
   run git-check-assertions
 
@@ -266,7 +266,7 @@ commit_with_assertion() {
 
 @test "should not cache failed commits" {
   git checkout -b feature
-  commit_with_assertion "touch ../test && exit 3"
+  commit_with_assertion "[success] touch ../test && exit 3"
 
   run git-check-assertions
 
@@ -282,8 +282,8 @@ commit_with_assertion() {
 
 @test "should skip cached commits when multiple commits are cached" {
   git checkout -b feature
-  commit_with_assertion "touch ../test1"
+  commit_with_assertion "[success] touch ../test1"
-  commit_with_assertion "touch ../test2"
+  commit_with_assertion "[success] touch ../test2"
 
   run git-check-assertions
 
@@ -303,10 +303,10 @@ commit_with_assertion() {
   git checkout -b feature
   echo commit1 >readme
   git add readme
-  commit_with_assertion 'grep commit1 readme'
+  commit_with_assertion '[success] grep commit1 readme'
   echo commit2 >readme
   git add readme
-  commit_with_assertion 'grep commit2 readme'
+  commit_with_assertion '[success] grep commit2 readme'
 
   run git-check-assertions
 
@@ -322,7 +322,7 @@ commit_with_assertion() {
   echo goodbye >src
   echo 'grep goodbye src' >tests
   git add src tests
-  commit_with_assertion $'git checkout HEAD~ src\nrun bash tests\nassert_failure'
+  commit_with_assertion $'[success] git checkout HEAD~ src\n[failure] bash tests'
 
   run git-check-assertions
 
@@ -330,18 +330,18 @@ commit_with_assertion() {
   assert_file_contains src goodbye
 }
 
-@test "assert_success should succeed if the executed command succeeded" {
+@test "a [success] command should succeed if the executed command succeeded" {
   git checkout -b feature
-  commit_with_assertion $'run exit 0\nassert_success'
+  commit_with_assertion '[success] exit 0'
 
   run git-check-assertions
 
   assert_success
 }
 
-@test "assert_success should fail if the executed command failed" {
+@test "a [success] command should fail if the executed command failed" {
   git checkout -b feature
-  commit_with_assertion $'run exit 1\nassert_success'
+  commit_with_assertion '[success] exit 1'
 
   run git-check-assertions
 
@@ -349,18 +349,18 @@ commit_with_assertion() {
   assert_output --partial "Expected command to succeed, but it failed."
 }
 
-@test "assert_failure should succeed if the executed command failed" {
+@test "a [failure] command should succeed if the executed command failed" {
   git checkout -b feature
-  commit_with_assertion $'run exit 1\nassert_failure'
+  commit_with_assertion '[failure] exit 1'
 
   run git-check-assertions
 
   assert_success
 }
 
-@test "assert_failure should fail if the executed command succeeded" {
+@test "a [failure] command should fail if the executed command succeeded" {
   git checkout -b feature
-  commit_with_assertion $'run exit 0\nassert_failure'
+  commit_with_assertion '[failure] exit 0'
 
   run git-check-assertions
 
@@ -368,47 +368,47 @@ commit_with_assertion() {
   assert_output --partial "Expected command to fail, but it succeeded."
 }
 
-@test "assert_output should succeed if the output matches the given string" {
+@test "an output line should succeed if the output contains the given string" {
   git checkout -b feature
-  commit_with_assertion $'run echo hello\nassert_output hello'
+  commit_with_assertion $'[success] echo hello\nhello'
 
   run git-check-assertions
 
   assert_success
 }
 
-@test "assert_output should fail if the output does not match the given string" {
+@test "an output line should fail if the output does not contain the given string" {
   git checkout -b feature
-  commit_with_assertion $'run echo hello\nassert_output goodbye'
+  commit_with_assertion $'[success] echo hello\ngoodbye'
 
   run git-check-assertions
 
   assert_failure
-  assert_output --partial "Expected output to equal: goodbye"
+  assert_output --partial "Expected output to contain: goodbye"
   assert_output --partial "Actual output: hello"
 }
 
-@test "assert_output should also check stderr output" {
+@test "an output line should also check stderr output" {
   git checkout -b feature
-  commit_with_assertion $'run sh -c "echo err 1>&2"\nassert_output err'
+  commit_with_assertion $'[success] sh -c "echo err 1>&2"\nerr'
 
   run git-check-assertions
 
   assert_success
 }
 
-@test "assert_output --partial should succeed if output contains the given string" {
+@test "multiple output lines should all be checked against the same command output" {
   git checkout -b feature
-  commit_with_assertion $'run echo hello\nassert_output --partial ell'
+  commit_with_assertion $'[success] printf "hello\\n"\nhell\nello'
 
   run git-check-assertions
 
   assert_success
 }
 
-@test "assert_output --partial should fail if output does not contain the given string" {
+@test "an output line should fail with the missing substring" {
   git checkout -b feature
-  commit_with_assertion $'run echo hello\nassert_output --partial xyz'
+  commit_with_assertion $'[success] echo hello\nxyz'
 
   run git-check-assertions
 
@@ -417,9 +417,9 @@ commit_with_assertion() {
   assert_output --partial "Actual output: hello"
 }
 
-@test "run should print command output" {
+@test "executed commands should print command output" {
   git checkout -b feature
-  commit_with_assertion $'run sh -c "echo -n hello; echo -n world"\nassert_success'
+  commit_with_assertion '[success] sh -c "echo -n hello; echo -n world"'
 
   run git-check-assertions