IndentationError on a line that is a comment, because a backtick in it ran as a shell command

The symptom

Every commit started printing this, from a git hook:

  File "<string>", line 12
    total 48
IndentationError: unexpected indent

Line 12 of the hook’s inline Python is a comment, and total 48 does not appear anywhere in the repository. It is the first line of ls -l output. Two facts that cannot both be true about the same program usually mean you are not looking at the program that ran.

What the hook looked like

The hook was a shell script that ran a short Python payload inline:

#!/bin/sh
python -c "
import subprocess, sys

def rebuild():
    # regenerate the graph; equivalent to running `ls -l` in the output dir first
    subprocess.run([...])

rebuild()
"

The comment mentions a command, and because the person writing it was writing something that reads like documentation, they put the command in backticks, the way you would in Markdown or a chat message. That is an ordinary thing to type, and it is the bug.

What was happening

The Python program is inside double quotes. Double quotes in a POSIX shell suppress word splitting and globbing, and leave three things active: $ and ${...}, so variable expansion still happens; backticks, so command substitution still happens; and \, so escapes still happen.

So before python ever started, the shell parsed that string, found `ls -l`, ran it, captured the output, and substituted the output for the backticked text. What python -c then received was:

    # regenerate the graph; equivalent to running total 48
drwxr-xr-x 2 mike mike  4096 Aug  9 11:02 graph
-rw-r--r-- 1 mike mike 91243 Aug  9 11:02 graph.json
 in the output dir first

The first line of the ls output landed inside the comment, where it is harmless. Every subsequent line landed on its own line, outside the comment, as Python source, and the first of those is indented because ls -l output is aligned. Hence IndentationError: unexpected indent on line 12, on text that was never in any file.

That also explains the thing that made it hard to reason about: the error changed depending on what the output directory contained. Add a file and you get a different line count and a different error; empty the directory and the whole thing might parse. A bug whose behaviour depends on unrelated files on disk does not feel like a syntax bug, so you do not go looking for one.

The shell does not tell you it substituted, and it has no notion of a command being mentioned rather than invoked. It has no idea it is inside a Python comment, because from where it stands it is inside a double-quoted string and Python does not exist yet.

The fix

Use single quotes for an inline program. Single quotes in a POSIX shell are literal, with no expansion, no escapes, and no substitution:

python -c '
import subprocess

def rebuild():
    # regenerate the graph; equivalent to running `ls -l` in the output dir first
    subprocess.run([...])

rebuild()
'

Now the backticks reach Python, where they are characters in a comment.

The tradeoff is that inside single quotes you cannot use a single quote, and there is no escape for one, so the payload must not contain apostrophes or '...' strings. Double-quoted Python strings inside a single-quoted shell argument is the arrangement that composes cleanly.

The better fix, once a payload is longer than a line or two, is to stop having a payload:

#!/bin/sh
exec python "$(dirname "$0")/rebuild_graph.py"

A real file has no quoting layer at all. It also gets syntax highlighting, a linter, a test, and a diff you can read, none of which apply to a program that lives inside a string.

If you are stuck with an inline payload, check what the shell is going to pass along before shipping it, by printing the string instead of running it:

printf '%s\n' "
import subprocess
    # ... `ls -l` ...
"

If the output is not byte-for-byte what you wrote, you have found the problem before it found you.

The wider version of this

Any code you pass as a shell string is shell source before it is anything else. That covers python -c, node -e, perl -e, sh -c, the command: field of a scheduled task or a CI step, a docker run argument, and the payload of an SSH invocation.

The characters that bite are $, backticks and backslashes, and they bite hardest in comments and documentation strings, because that is where people naturally write things that look like commands and prices and regexes without expecting them to mean anything. So prefer single quotes for anything inline, and when a program passed as a string starts behaving in a way that depends on the state of the machine rather than on its own text, stop debugging the program and look at what the shell handed over.

The lesson

When an error points at a line that cannot produce it, you are not looking at the source that ran. An IndentationError on a comment cannot be resolved by staring at the Python. The question at that moment is what the interpreter actually received, and every layer between your file and the interpreter is a suspect. Here there was exactly one layer, and it had quietly run a command that somebody had only meant to mention.