Bash Shell Scripting

Introduction

A shell script is a file with commands that will be executed by the shell. Shell scripts benefit from programming concepts such as if statements, loops, variables and functions. The aim of this article is to provide enough knowledge to comfortably read and write bash shell scripts.

Please note this article only looks at bash shell scripts.  You will be able to use most of the concepts described in this section with other shells, such as C shell, Korn shell, tcsh, zsh or older families of the bourne shell. You however need to be aware there are differences.

Creating a Shell Script

Let’s create our first script:

The first line of a shell script specifies the shell that will be used to execute the script.  This is done with the sha-bang #!  (also known as hash-bang) followed by the shell path. The sha-bang line provides portability, as it ensures your script will always be executed with the same shell regardless of environment configuration and *nix platform.

Before running the script, we need to give it execution permission:

And finally we execute it:

Note we need to prefix our script file with ./  if the current directory is not in the $PATH.

Variables

Setting Variables, Exporting and Sourcing

A variable stores a value that can be accessed or changed at a later time. This is the syntax to set variables in bash:

A variable will exist until the process ends or it is unset:

A variable is only known to the shell or script where it is defined. If you want to pass a variable to the child process, you need to export it:

or you can use the abbreviated form:

The  echovar.sh script below will help illustrate this behaviour.

When the var variable is set in the shell command line but not exported, it is not visible by the  echovar.sh script. Once we export it, it becomes visible:

Another source of confusion worth mentioning is when we set variables within a script, and then we expect those variable changes to be available in the parent shell. For example, look at the following  setvar.sh script:

If you execute this script from the shell, and then echo the variable from the command line, you will get an empty value. This is because the shell creates a new child process to execute the script, and child processes can never make changes to parent processes.

An alternative is to use the . or source commands to execute the script.  With the . and  source commands the shell executes the script in its current process, instead of creating a child process. This means that any changes will be applied directly to the shell process.

Accessing Variables

The simplest way to access a variable is to prefix it with the $ sign:

Alternatively, you can also use the ${ } form:

You can also include variables inside strings:

Parameter Expansion

Parameter expansion is a mechanism to access the value of the variable, but to return an alternative value depending on a test. For example, return a default value when the variable has not been set.

These are the main parameter expansion types:

Parameter ExpansionDescription
${var:-default}Expand to default value if the variable is unset or empty
${var-default}Same as ${var:-default}, but only applies when the variable is unset, and not when it is empty
${var:+alternative}Replaces the contents of the variable with an alternative value when the variable is set.  It does not apply when the variable is unset or empty.
${var+alternative}Same as ${var:+alternative}, but also applies when the variable is empty.

This is to say, we  use the alternative value when the variable is either set or empty, but not when it is unset.

${var:=default}Expands and sets the variable to a default value if unset or empty:

${var=default}Same as ${var:=default} , but  only applies when the variable is unset, and not when it is empty.
${var:?message}If variable is unset or emtpy, it prints the message to standard error and exits the script with status of 1.

${var?message}Same as ${var:?message}, but only applies when the variable is unset, and not when it is empty.
${#var}Returns the length of the variable value

 ${var:offset:length}Returns a substring of the the variable value.

The first character offset is 0.

If the offset is not provided, it then assumes it is 0.

You can also use a negative offset to count from the end of the string. Note that you need to leave a space between the : and the negative sign -  as otherwise if would conflict with ${var:-default}

If you do not provide the length, it then assumes it is until the end of the string:

${var//pattern/string}Replaces all instances of the pattern in the variable value with the string.  The regular expression is the same as pathname regular expresion. That is, ? matches any character, and * matches zero or more characters.

${var^}Replaces the first character to upper-case

${var^^}Replaces all characters to upper-case

${var,,}Replaces all characters to lower-case

Arrays

Arrays are programming structures that store multiple values under the same variable. Bash supports two types of arrays: integer-indexed arrays and associative arrays.

Integer-Indexed Arrays

Integer-indexed arrays are zero based and do not need to be contiguous. This is to say, you can store a value at index 0, and another at index 30.

You can reference any value in the array as follows:

You can return all values in the array using the indexes [*] and [@] .

When the array reference is double-quoted,   [<em>] and [@]  have a different meaning.   ${array[</em>]} returns a string with each element separated by space, while     ${array[@]} returns a separate string for each element.

You can return print the number of values in the array as follows:

You can also initialise arrays with a single command:

And you can also append to the end of the array using += :

Associative Arrays

Associative arrays are referenced using strings, similar to maps in other programming languages.

Before you can use a variable as an associative array, you need to declare it:

After this, you can add any items to the array:

You can reference to values in the array as usual:

You can also return all values of the array using the  [*] and [@]  index references.  When they are double-quoted, they have same behaviour as described in the integer-indexes arrays.

If you want to return the keys, you can use the ${!map[@]}  notation:

Positional Parameters

Positional parameters reference the arguments passed to the script by the shell. These arguments can be referenced as $1 , $2 , $3,…

For example, let’s use the following params.sh script

This is the result when executing it from the shell:

There are several types of positional parameters:

TypeDescription
$1, $2, $3,...Reference the first, second, third… argument provided by the shell
${10}, ${11},...Reference large number of arguments
$0The script name
$#The number of arguments
$*If not quoted, it returns all parameters as separate strings.

If quoted, it returns all parameters in the same string, each separated by a space.

$@If not quoted, it returns all parameters as separate strings (same as $*)

If quoted, "$@" returns the equivalent to "$1" "$2" "$3" ..., that it, separate strings.

Shift

The shift command removes the first positional parameter, and shifts the remaining parameters one place to the left.

For example, look at the following script where we shift the positional parameters twice:

This will be the output when passing 4 parameters to the script:

You can also shift several parameters at a time:

and you can also remove all parameters with $#:

Exit Status

All scripts return an exit status once they complete their execution. By convention, an exit status of 0 means success, and anything else means failure. The exit status is an integer between 0 and 255.

You can capture the exit status of the previous command with the $? special variable:

The exit command is used to exit the script and return an exit status to the parent process.

If the script does not have an exit command, then it returns the exit status of the last command that executed.

Comparisons: test, [ and [[ Commands

The test command evaluates a conditional expression and returns   when true, or  1 when false.  The example below uses test to check if the parameter passed to a script is 5.

The square bracket command  [ is a replacement for the test command that feels more natural when used in if/then statements.  The [  command supports the same conditions as the test command.  The syntax is   [ expression ].  The script below is equivalent to the previous one.

The double square bracket command [[ is an extension over the [  command. The  [[  command supports all conditions that  [ supports, plus string pattern matching.  The syntax is [[ expression ]] . The script below uses pattern matching to test whether the parameter starts with the letter a.

It is important to remember that both [ and [[ are commands. As any other command, you need to separate the command name from its parameters by a space. So you always need to leave a space after the opening square bracket, and another space before the closing square bracket.

The test[ and [[ commands support the following type of expressions:

  • Numeric comparisons
  • String comparisons
  • File comparisons
  • Pattern matching comparisons (only [[ )

Numeric Comparisons

These are the expressions used to compare integers. Valid for all test[ and [[ commands.

OperationDescription
a -eq ba is equal to b
a -ne ba is not equal to b
a -gt ba is greater than b
a -ge ba is greater than or equal to b
a -lt ba is less than b
a -le ba is less than or equal to b

String Comparisons

These are the expressions used to compare strings. Valid for all test[ and [[ commands. Note the differences between [  and [[ .

OperationDescription
str1 = str2str1 is equal to str2

Note that string comparison is just one equals symbol ( =).

[[  uses two equal symbols ( ==) for pattern matching.

 

str1 != str2str1 is not equal to str2
str1 < str2str1 is less than str2

test  and [  use standard ASCII ordering.

[[ uses the lexicography of the current locale.

str1 > str2str1 is greater than str2.
-n strlength of string is not zero
-z strlength of string is zero

File Comparisons

These are the expressions used to check the file status of files and directories. Valid for all test[ and [[ commands.

OperationDescription
-a fileTrue if file exists
-e fileTrue if file exists
-d fileTrue if file exists and is a directory exists
-f fileTrue if file exists and is a regular file
-r fileTrue if file exists and is readable
-w fileTrue if file exists and is writable
-x fileTrue if file exists and is executable
-s fileTrue if file exists and is not empty
file1 -nt file2True if file1 is newer than file2
file2 -ot file2True if file1 is older than file2

Below an example script that checks whether the file provided in the command line is a directory, regular file and whether it can be executed.

Pattern Matching

Below the pattern matching expressions. Only supported by the [[ command.

OperationDescription
str == patternTrue if str matches the pattern.

It uses the shell script pattern matching rules (e.g. only *  and ?  are supported)

 

str != patternTrue if str does not match the pattern

It uses the shell script pattern matching rules (e.g. only *  and ?  are supported)

str =~ patternTrue if str matches the pattern.

It uses extended regular expressions (it also supports + )

 

Command Substitution

pg 277

 

 

Bibliography

The following two tabs change content below.

Eduard Manas

Eduard is a senior IT consultant with over 15 years in the financial sector. He is an experienced developer in Java, C#, Python, Wordpress, Tibco EMS/RV, Oracle, Sybase and MySQL.Outside of work, he likes spending time with family, friends, and watching football.

Latest posts by Eduard Manas (see all)

Leave a Reply