Bash scripting cheat sheet

set -euo pipefail

Make a script stop at the first failing command, at an unset variable and at a failure anywhere in a pipeline.

-e

Exit when a command fails.

-u

Treat an unset variable as an error.

-o pipefail

A pipeline fails if any stage fails, not only the last.

-x

Print each command before it runs.

Gotcha set -e ignores failures in conditions: after if, while, && and ||, and under !. Test those on purpose.

"$var"

Double-quote every expansion, so a value with spaces or wildcards stays one word.

file="my report.txt"; touch "$file"; ls -1 my*

my report.txt

"$var"

Expands the variable, keeps the value as one word.

'text'

Nothing inside is expanded.

$'a\tb'

Backslash escapes such as \t and \n are interpreted.

Gotcha Unquoted, rm $file with file="my report.txt" removes "my" and "report.txt".

${var:-default}

Use a fallback when a variable is unset or empty, or stop the script when it must be set.

echo "${STAGE:-dev}"

dev

${var:-word}

word if var is unset or empty.

${var:=word}

The same, and assign it to var.

${var:?message}

Stop with message if var is unset or empty.

${var:+word}

word only if var has a value.

${var%suffix} ${var#prefix}

Trim, measure and change the case of a value without calling sed or basename.

f=backup.tar.gz; echo "${f%%.*} ${f#*.} ${f^^} ${#f}"

backup tar.gz BACKUP.TAR.GZ 13

${var#pattern}

Remove the shortest match from the start; ## the longest.

${var%pattern}

Remove the shortest match from the end; %% the longest.

${var/old/new}

Replace the first match; // every match.

${var^^} ${var,,}

Upper case and lower case.

${#var}

Length of the value.

$(command)

Put the output of a command into a variable or another command line.

today=$(date +%F); echo "backup-$today.tgz"

$(command)

The output, without its trailing newlines.

$(< file)

The content of a file, without running cat.

`command`

The old form. It does not nest; use $( ).

Gotcha The exit status of the substitution is lost in local x=$(cmd) and export x=$(cmd). Declare first, assign after.

$(( expression ))

Integer arithmetic with variables, without a dollar sign inside the parentheses.

count=7; echo $(( count * 3 + 1 )) $(( 16#ff )) $(( count % 2 ))

22 255 1

(( n++ ))

Change a variable; the status is 1 when the value is 0.

BASE#digits

A constant in another base: 2#1010, 16#ff.

let "n = n + 1"

The same arithmetic as a builtin.

Gotcha Integers only: 10 / 3 is 3. Use bc or awk for decimals.

exit N

End a script with a status: 0 for success, anything else for failure. The caller reads it in $?.

bash -c 'exit 3'; echo "status $?"

status 3

$?

Status of the last command.

exit

Without N: the status of the last command.

126 / 127

Not executable / command not found.

128 + N

Ended by signal N: 130 is Ctrl+C, 137 is SIGKILL.

[[ -f file ]]

Check what a path is before you use it: a file, a directory, readable, not empty.

[[ -f /etc/hostname && ! -d /etc/hostname ]] && echo "regular file"

regular file

-e / -f / -d

Exists / regular file / directory.

-r / -w / -x

Readable / writable / executable by you.

-s

Exists and is not empty.

-L

Is a symbolic link.

a -nt b

a was modified after b.

[[ $a == pattern ]]

Compare strings. Inside [[ ]] the right side of == is a wildcard pattern unless you quote it.

v=release-2.4; [[ $v == release-* && -n $v ]] && echo match

match

== / !=

Equal / not equal, with wildcards on the right.

-z / -n

Empty / not empty.

< / >

Sort before / after.

&& / ||

And / or, inside the brackets.

Gotcha [ ] is the old test command: quote every variable in it, and use = rather than ==.

[[ $s =~ regex ]]

Match an extended regular expression and pick the captured groups out of BASH_REMATCH.

v=release-2.14.3; [[ $v =~ ^release-([0-9]+)\.([0-9]+) ]] && echo "${BASH_REMATCH[1]}.${BASH_REMATCH[2]}"

2.14

BASH_REMATCH[0]

The whole match.

BASH_REMATCH[N]

Group N, counted by opening parenthesis.

Gotcha Do not quote the regex: quoted text is matched literally. Put a complicated one in a variable and use =~ $re.

if command; then ... fi

Branch on whether a command succeeded. No brackets needed: if runs the command and looks at its status.

if grep -q localhost /etc/hosts; then echo found; else echo missing; fi

found

grep -q

Quiet: only the status matters.

if ! command

Branch on failure.

elif

Another condition before else.

(( a > b ))

Compare numbers with the usual operators. Inside [[ ]] you would need -gt and -lt.

used=91; (( used > 90 )) && echo "disk almost full"

disk almost full

(( a >= b ))

True when the expression is not zero.

[[ $a -gt $b ]]

The same test with -eq, -ne, -lt, -le, -gt, -ge.

Gotcha [[ 10 > 9 ]] compares text, so it is false. Use (( )) or -gt for numbers.

case $x in pattern) ... ;; esac

Pick a branch by wildcard pattern: tidier than a chain of elif for file types, subcommands and yes/no answers.

case "report.csv" in *.csv) echo csv;; *.json|*.yaml) echo structured;; *) echo other;; esac

csv

a|b)

Either pattern.

*)

The default branch.

;;

End of a branch.

for x in list; do ... done

Run the same commands for each word of a list.

for host in web1 web2 db1; do echo "checking $host"; done

checking web1
checking web2
checking db1

"$@"

Loop over the arguments of the script.

{1..5}

A range, expanded before the loop starts.

break / continue

Leave the loop / go to the next round.

for f in *.log; do ... done

Loop over files with a wildcard. The names stay intact even when they contain spaces.

for f in /etc/host*; do echo "${f##*/}"; done

${f##*/}

The file name without its directory.

[[ -e $f ]] || continue

Skip the pattern itself when nothing matches.

Gotcha Never loop over $(ls): names with spaces fall apart. The wildcard does the same job safely.

while IFS= read -r line; do ... done < file

Read a file line by line. IFS= keeps leading spaces and -r keeps backslashes.

while IFS=: read -r user _ uid _; do echo "$user=$uid"; done < /etc/passwd

IFS=:

Split each line on colons into the variables.

done < file

Feed the loop from a file.

done < <(command)

Feed it from a command and keep the variables it sets.

Gotcha command | while read ... runs the loop in a subshell: variables set inside are gone after done.

for (( i = 0; i < n; i++ )); do ... done

Count with a number: a fixed count of attempts, or an index into an array.

for (( i = 1; i <= 3; i++ )); do echo "attempt $i"; done

attempt 1
attempt 2
attempt 3

until command; do sleep N; done

Repeat until a command succeeds: the usual way to wait for a service to come up.

until curl -fs http://localhost:8080/health; do sleep 2; done

until

Loop while the command fails.

while ! command

The same thing, spelled with while.

(( tries++ < 10 ))

Add a limit so the loop cannot run for ever.

Gotcha Always bound a retry loop. A service that never comes up should fail the script, not hang it.

name=(a b c)

Hold a list in one variable. "${name[@]}" gives every element as its own word.

hosts=(web1 web2 "db 1"); echo "${#hosts[@]} hosts, last: ${hosts[-1]}"

3 hosts, last: db 1

"${a[@]}"

Every element, each one word.

${#a[@]}

How many elements.

"${!a[@]}"

The indices.

a+=(x)

Append an element.

mapfile -t lines < file

Read every line of a file into an array in one step.

mapfile -t lines < /etc/passwd; echo "first account: ${lines[0]%%:*}"

first account: root

-t

Drop the newline at the end of each line.

-n N

Read at most N lines.

< <(command)

Read the lines of a command.

Gotcha command | mapfile fills the array in a subshell and it is empty afterwards. Redirect instead of piping.

name() { ...; }

Give a block of commands a name. Arguments arrive as $1, $2, ... and local keeps variables inside.

greet() { local who=${1:-world}; echo "hello $who"; }; greet; greet ops

hello world
hello ops

local

A variable that exists only inside the function.

return N

Leave the function with status N.

$FUNCNAME

The name of the running function.

"$@"

Pass every argument on unchanged, each one still a single word.

set -- deploy "web 1" prod; echo "$# arguments"; for a in "$@"; do echo "[$a]"; done

3 arguments
[deploy]
[web 1]
[prod]

$#

How many arguments.

$1 ... ${10}

One argument by position.

shift

Drop $1; the rest move down.

$0

The name of the script.

Gotcha "$*" joins everything into one word, and unquoted $@ splits words again. "$@" is almost always what you want.

while getopts "e:v" opt; do ... done

Parse short options. A colon after a letter means that option takes an argument.

set -- -v -e prod app; while getopts "e:v" o; do case $o in e) env=$OPTARG;; v) verbose=1;; esac; done; shift $(( OPTIND - 1 )); echo "$env $verbose $1"

prod 1 app

OPTARG

The argument of the current option.

OPTIND

Index of the next word; shift $((OPTIND - 1)) drops the options.

":e:v"

A leading colon: report errors yourself with the ? and : cases.

function && ... || ...

A function succeeds or fails like any command: its status is that of its last command.

known() { grep -q "$1" /etc/hosts; }; known localhost && echo up; known nowhere || echo "unknown ($?)"

up
unknown (1)

printf 'format' args

Print with exact control over columns and line endings. Unlike echo it behaves the same everywhere.

printf '%-8s %5d%%\n' root 42 data 7

root        42%
data         7%

%s %d %05.2f

Text, integer, padded decimal.

%-8s

Left-align in eight columns.

-v name

Store the result in a variable instead of printing it.

cat <<'EOF' ... EOF

Feed several lines to a command, or write a small file from a script.

cat > app.env <<'EOF'
STAGE=prod
PORT=8080
EOF

<<'EOF'

Quoted: nothing in the text is expanded.

<<EOF

Unquoted: $variables and $(commands) are expanded.

<<-EOF

Leading tabs are stripped, so the text can be indented.

<<< "text"

A here-string: one line of input.

trap 'command' EXIT

Run cleanup when the script ends, however it ends: at the last line, at exit, or when set -e stops it.

bash -c 'trap "echo cleaned up" EXIT; echo working; exit 3'; echo "status $?"

working
cleaned up
status 3

EXIT

The end of the script or subshell.

INT TERM

Ctrl+C and kill: handle them to shut down cleanly.

trap - EXIT

Remove the trap.

trap -p

List the traps that are set.

Gotcha A subshell does not inherit traps, and kill -9 cannot be trapped.

trap 'command' ERR

Report the command that failed. With set -e the ERR trap runs just before the script stops.

bash -c 'trap "echo failed with status \$?" ERR; false; echo continues'

failed with status 1
continues

set -E

Functions and subshells inherit the ERR trap.

$LINENO

In a real script: the line that failed.

mktemp

Create a scratch file or directory with a name nobody else can guess, readable only by you.

tmp=$(mktemp -d); stat -c "%a %F" "$tmp"; rmdir "$tmp"

700 directory

-d

A directory instead of a file.

-p DIR

Create it in DIR.

name.XXXXXX

Your own template: the X run is replaced.

Gotcha /tmp/myscript.$$ is predictable and unsafe. Use mktemp, and remove it in an EXIT trap.

timeout DURATION command

Give a command a time limit, so a hung network call cannot block the script for ever.

timeout 2 sleep 10; echo "status $?"

status 124

124

The status when the limit was hit.

-s KILL

Send SIGKILL instead of SIGTERM.

-k 5

Send SIGKILL five seconds later if it is still running.

30s 5m 2h

Durations take a unit.

exec > file 2>&1

Send everything the rest of the script prints, errors included, to a log file.

bash -c 'exec > /tmp/run.log 2>&1; echo started; date +%F'; cat /tmp/run.log

exec > file

Standard output from here on.

2>&1

Standard error to the same place.

exec "$@"

With a command: replace the shell with it (container entrypoints).

diff <(command) <(command)

Hand the output of a command to a program that wants a file name.

diff <(printf 'a\nb\n') <(printf 'a\nc\n')

2c2
< b
---
> c

<(command)

A file name that reads the output of the command.

done < <(command)

Feed a loop without a pipe, so its variables survive.

command & ... wait

Run steps side by side and carry on when all of them have finished.

sleep 3 & sleep 2 & wait; echo "both finished"

&

Start the command in the background.

$!

PID of the last background command.

wait PID

Wait for one job and get its exit status.

wait -n

Wait for whichever finishes first.

Gotcha Plain wait returns 0 even if a job failed. Wait for each PID when the result matters.

bash -x script.sh

Print every command with its values filled in before it runs: the quickest way to see what a script really did.

bash -x deploy.sh

set -x / set +x

Trace only part of a script.

bash -n script.sh

Check the syntax without running anything.

PS4='+ $LINENO: '

Put the line number in front of each traced command.