(One)-minute geek news

Debug your Bash scripts

Bash scripts sure are handy, but debugging them can become a nightmare. Fortunately, some tricks exist.

Stop at first fail

Bash is extremely resilient: unlike most languages, if a command fails, Bash will still try to execute the rest of the script.

While this behavior is often beneficial, it can easily become catastrophic, especially when dealing with files and deletions... (e.g. cp MD.out MD.out.bak;cp2k -i MD.in > MD.out).

One could use && and || operators for chaining commands, but that would lead to much writing efforts and scripts quite harder to read. Fortunately, there is a another, much simpler solution: simply add set -e at the beginning of your script, and that's it! Now, your script will stop after the first failure, preventing a cascade of potentially desastrous consequences.

Report undefined variables

If you use a variable (called parameter in Bash) that was not defined, Bash does not raise an error, and simply consider an empty variable.

This can lead to headaches in debugging and potentially catastrophic situations (e.g. dir_name="tests/Au_bad/";rm -r "research/$dir_neme").

Fortunately, Bash can treat the use of undefined variables as an error by adding set -u at the beginning of your script.

Note: you can combine the two set commands above: set -eu to make your script stop if an undefined variable is being used.

Debug mode

When debugging a script, it is common to put echo commands as milestones, in order to identify which part of your script is concerned by an error. In such cases, you might even want to put milestones at every line of your script, to better follow the steps taken.

Doing so manually would be extremely painful. Fortunately, there exist two mecanisms to achieve just that:

• Adding set -v at the beginning of your script will cause Bash to print every line (or group of lines, like a for/while/... loop) before executing it.
• Adding set -x at the beginning of your script will cause Bash to print every command (after parameter/pathname/brace/... substitutions/expansions) before executing it.

Errors reporting

When executing a script, multiple errors can occur, with different reporting mechanism.

If Bash cannot execute a command (command not found, syntax error, ...), then Bash will report it with some preceding information: /tmp/test.sh: line 7: syntax error near unexpected token `(' for example.

Note: Beware that Bash will not report where the error occurs, but where it finally becomes aware of the error... For example, if you forget a closing parenthesis on line 5, Bash might read your whole script until it finds a matching closing parenthesis and detect an error when it can't find one at the end of your script, line 42.

However, if Bash succeeds in executing a command that fails or raise a warning (like ls file_that_do_not_exist), it is the command itself that reports the error, not Bash. To be precise, the command writes messages on its standard error output (stderr), which is redirected to the stderr of your script, which is redirected to your terminal (unless you performed redirections: mon_script.sh 2> /dev/null for example). Since the command is likely not aware of being called by a script, there won't be any information on where in your script that command failed... (which is problematic if you have the same command at multiple locations)

Fortunately, traps can be used to handle failed commands (i.e. returned an overall non-zero exit status) in a more verbose manner. Simply add at the beginning of your script the command trap 'echo "Error detected line $LINENO"' ERR to display an error message with the line number ($LINENO parameter) everytime command failed (execpt if this command was part of a pipeline, list or compound command that did not failed (e.g. ls non_existing_file|echo ok).

Bonus note: You might want to make a pipeline fail if any of its commands fails, by running set -o pipefail beforehand.

Mixing it all

Make your scripts safer and easier to debug by adding at the beginning:

Bonus

The set command possesses another hidden gem for vi lovers: set -o 'vi' causes Bash to use a vi-style line editing interface for typing commands!

Last but not least, set -P causes Bash to follow symbolic links in cd commands (e.g. with set -x the command cd /links/link_to_tmp;cd ..;pwd would return /, while without this option (set +x to disable) the same command would return /links/).

References

man bash
man bc https://www.tldp.org/LDP/abs/html/debugging.html