Shell Scripting – Special Characters
Hello Everyone
Welcome to CloudAffaire and this is Debjeet.
In the last blog post, we have discussed some basics of shell scripting and how to create and execute your 1st shell script.
https://cloudaffaire.com/shell-scripting-getting-started/
In this blog post, we will discuss special characters in the shell. Every character (apart from alphabets and numbers) has a unique meaning in the shell and is interpreted differently based on the format and its quite essential to know most of them to be an effective shell scripter.
Shell Scripting – Special Characters:
Hash (#):
The general use case of hash (#) is to define a comment in the shell but hash (#) can also be used for shebang, parameter substitution, and base conversion.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
########################################## ## Shell Scripting | Special Characters ## ########################################## ## Prerequisites: One Unix/Linux/POSIX-compliant operating system with bash shell ##--------- ## Hash (#) ##--------- ## Create the script vi myshscript.sh ------------------- #!/bin/bash # hash used as shebang or hashbang above, not a comment # hash used as a comment here echo "The # hash used as normal character here" echo The \# hash used as normal character here echo The # hash used as a comment here echo ${PATH#*:} # hash used for parameter substitution, not a comment. echo $(( 2#101011 )) # hash used for base conversion, not a comment. -------------------- :wq ## Execute the script chmod +x myshscript.sh ./myshscript.sh |
Semicolon (;):
The single semicolon (;) is used to execute multiple commands in a single line, the double semicolon (;;) is used as terminator in a case option.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
##-------------- ## Semicolon (;) ##-------------- ## Create the script > myshscript.sh && vi myshscript.sh ------------------- #!/bin/bash ## Single semicolon (;) is used as command separator to execute multiple command in a single line of code. echo hello;echo world;ls -l ## Double semicolon (;;) is used as terminator in a case option choice=hello case "$choice" in hello) echo "$choice" ;; world) echo "$choice" ;; esac -------------------- :wq ## Execute the script ./myshscript.sh |
Dot (.):
The single dot (.) represent different use case depending upon the usage in your script. It can be used to represent a hidden file, the current working directory, a single character replacement in regular expression. The double dot (..) is used to represent the parent directory of current working directory.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
##-------- ## Dot (.) ##-------- # Create a directory and sub-directory mkdir -p my_paraent_directory/my_directory && cd my_paraent_directory/my_directory pwd # you should be in /some/path/my_paraent_directory/my_directory ## When working with filenames, a leading dot is the prefix of a "hidden" file, a file that an ls will not normally show. echo "echo 'hello world'" > .iamhidden ls # .iamhidden not visible ls -a # .iamhidden is visible ## Equivalent to source. This is a bash builtin. ## source .iamhidden . .iamhidden # Load a file. ## When considering directory names, a single dot represents the current working directory, ## and two dots denote the parent directory. cd . pwd # you should be in /some/path/my_paraent_directory/my_directory cd .. pwd # you should be in /some/path/my_paraent_directory ls -l # messages file is not available in current working directory sudo cp /var/log/messages . ls -l # messages file is available in current working directory ## When matching characters, as part of a regular expression, a "dot" matches a single character. grep wor.d my_directory/.iamhidden cd .. && rm -rf my_paraent_directory |
Quotes (“” and ”):
Double quotes (“”) represent string and preserves (from interpretation) most of the special characters within a string. The single quotes (”) represents string and preserves all special characters within a string. This is a stronger form of quoting than double-quotes.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
##------------------- ## Quotes ("" and '') ##------------------- ## Double quotes represent string and preserves (from interpretation) most of the special characters within string. name=user1 echo "hello $name" # prints hello user1 echo "! @ # $ % ^ & () [] {} ~ ; : - = + / $name \\ ' \"" # prints ! @ # $ % ^ & () [] {} ~ ; : - = + / user2 \ ' " ## Single quotes also represents string and preserves all special characters within string. ## This is a stronger form of quoting than double quotes. name=user1 echo 'hello $name' # prints hello $name echo '! @ # $ % ^ & () [] {} ~ ; : - = + / $name \ " ' # prints ! @ # $ % ^ & () [] {} ~ ; : - = + / $name \ " |
Comma (,):
Comma (,) can be used to links together a series of arithmetic operations where all are evaluated, but only the last one is returned. The comma can also be used to concatenate strings.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
##---------- ## Comma (,) ##---------- ## The comma operator (,) links together a series of arithmetic operations. All are evaluated, but only the last one is returned. let "sum = (a = 10, b = 20, 10 + 20)" echo $sum # returns 30 echo `expr $a + $b` # returns 30 ## The comma operator (,) can also concatenate strings. for file in /{,usr/}bin/*calc # ^ Find all executable files ending in "calc" in /bin and /usr/bin directories. do if [ -x "$file" ] then echo $file fi done |
Backslash (\):
Backslash (\) is used to escape other special characters in the shell.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
##-------------- ## Backslash (\) ##-------------- ## The backslash (\) is used to escape special charecters in shell name=cloudaffaire echo "welcome to $name" # prints welcome to cloudaffaire echo "welcome to \"$name\"" # prints welcome to "cloudaffaire" echo "welcome to \$name" # prints welcome to $name ## The backslash (\) is also used for multi-line command execution echo "hello \ world \ welcome \ to cloudaffaire" |
Forward slash (/):
The forward-slash (/) is used as a filename path separator (/some/path). It also represents the root directory. In arithmetic operation forward slash means division.
|
1 2 3 4 5 6 7 8 9 10 11 |
##----------------- ## Forwardslash (/) ##----------------- ## The forwardslash (/) is used as filename path separator (/some/path). pwd # returns /some/path cat /etc/motd # display the content of file motd located undet /etc directory sudo ls / # list root directory ## The forwardslash (/) is also used for division in arithmetic operations. echo $((20 / 5)) # returns 4 |
Colon (:):
In shell, a colon (:) can be used as a null command, endless loop, a place holder, for string evaluation, for field separator or to empty a file.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
##---------- ## Colon (:) ##---------- ## Colon (:) to do nothing : echo $? # returns 0 as exit code ## Colon (:) for endless loop while : do echo "hello" done #stop execution with control + c ## Colon (:) for placeholder in if/then test: if : # no condition then : # no action else echo "i am else" fi ## Provide a placeholder where a binary operation is expected : ${username=`whoami`} echo $username ## Colon (:) to evaluate string of variables using parameter substitution : ${HOSTNAME?} ${USER?} ${MAIL?} # Prints error message if one or more of essential environmental variables not set. ## Colon (:) to truncate a file ## In combination with the > redirection operator, truncates a file to zero length, without changing its permissions. ## If the file did not previously exist, creates it. echo "hello world" > hello cat hello : > hello # File "hello" now empty. # same as cat /dev/null > hello , # However, this does not fork a new process, since ":" is a builtin. cat hello ## Colon (:) may be used to begin a comment line, although this is not recommended and can break your script. : ' this is a multi line comment, however not safe to use ' ## Colon (:) to use as a feild separator cat $PATH |
Exclamation (!):
Bang (!) can be used to check the reverse of equality (not equal). Bang (!) can also be used to invert the exit status.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
##---------------- ## Exclamation (!) ##---------------- ## Bang (!) can be used to check reverse of equality (not equal) name=cloudaffaire if [ name != "debjeet" ] then echo hello $name else echo hello debjeet fi ## Bang (!) can also be used to invert the exist status echo "hello world" echo "exit status = $?" # returns 0 ! echo "hello world" echo "exit status = $?" # returns 1 |
Asterisk (*):
Asterisk (*) can be used as a “wild card” for filename expansion in globbing. By itself, it matches every filename in a given directory. Asterisk (*) also represents any number (or zero) characters in a regular expression. In arithmetic operation single asterisk (*) represents multiplication and double asterisk (**) represents exponentiation.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
##------------- ## Asterisk (*) ##------------- ## Asterisk (*) can be used as a "wild card" for filename expansion in globbing. ## By itself, it matches every filename in a given directory. ls * # list all files in present directory touch one.txt two.txt three.txt one.sh two.sh three.sh ls -l one* # list all files having one in its name ls -l *.sh # list all files having .sh as extension ## Asterisk (*) also represents any number (or zero) characters in a regular expression. echo "hello world! welcome to cloudaffaire" > hello grep cloud* hello ## Asterisk (*) also represents multiplication in arithmetic operations. echo $((5 * 2)) # returns 10 ## The double asterisk (**) represent exponentiation in arithmetic operations echo $((5 ** 2)) # returns 25 |
Question mark (?):
Question mark (?) or test operator is used within certain expressions to indicates a test for a condition. Question mark (?) or test operator is also used to test if a variable value has been set. Question mark (?) in the regular expression is used as a single-character wild card.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
##------------------ ## Question mark (?) ##------------------ ## Question mark (?) or test operator is used within certain expressions to indicates a test for a condition. ## condition?result-if-true:result-if-false let "greatest = (20 > 10?20:10)" echo $greatest # returns 20 let "smallest = (20 < 10?20:10)" echo $smallest # returns 10 ## Question mark (?) or test operator is also used to test if a variable value has been set name=debjeet : ${name?} echo "hello $name" : ${age?} # throws an error echo "you are $age years old" ## Question mark (?) in regular expression is used as a single-character wild card echo "hello world" > hello grep hell? hello ls -l on?.* # returns one.txt and one.sh |
Dollar ($):
A single dollar ($) prefixing a variable name indicates the value the variable holds. A single dollar ($) in a regular expression represents a match from the end of the line. “XXX$” matches XXX at the end of a line. A single dollar followed by a curly brace (${}) represents parameter substitution. A single dollar followed by a single quote ($”) can be used as a quoted-string expansion construct. A single dollar ($) followed by different character represents positional parameters and special meanings.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 |
##----------- ## Dollar ($) ##----------- ## A single dollar ($) prefixing a variable name indicates the value the variable holds. name=debjeet echo $name # returns debjeet ## A single dollar ($) in regular expression represent match from end of line. "XXX$" matches XXX at the end of a line. cat < welcome everyone to the exiting world of cloudaffaire world EOT grep world$ hello ## A single dollar followed by curly brace (${}) represents parameter substitution. fname=debjeet lname=bhowmik fullname=${fname}" kumar "${lname} echo $fullname # returns debjeet kumar bhowmik user1=debjeet echo ${user1-`whoami`} # returns debjeet echo ${user2-`whoami`} # returns the output of whoami as user2 not set ## A single dollar followed by single quote ($'') can be used as quoted string expansion construct ## This construct expands single or multiple escaped octal or hex values into ASCII or Unicode characters. echo $'\042hello \t world\042' # returns "hello world" uses octal value for "" echo $'\x22hello \t world\x22' # returns "hello world" uses hex value for "" ## A single dollar ($) followed by different charecter represents positional parameters and special meanings ## create a script > myshscript.sh && vi myshscript.sh ------------------- #!/bin/bash echo "1st parameter = $0" # 1st parameter = ./myshscript.sh echo "2nd parameter = $1" # 2nd parameter = 1 echo "3rd parameter = $2" # 3rd parameter = 2 echo "4th parameter = $3" # 4th parameter = 3 echo "no. of parameters = $#" # no. of parameters = 3 echo "all parameters = $*" # all parameters = 1 2 3 echo "all parameters = $@" # all parameters = 1 2 3 echo "PID of the script = $$" # PID of the script = 15172 echo "PID of last background job = $!" # PID of last background job = echo "exit status = $?" # exit status = 0 ------------------- :wq ## Execute the script with 1 2 3 as parameter ./myshscript.sh 1 2 3 |
Round brackets (()):
Round brackets (()) is used for grouping of multiple commands. Round brackets prefix by dollar ($()) represents parameter substitution. Round brackets (()) is also used to initialize an array. Round brackets (()) can also be used to initiate sub process.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
##-------------------- ## Round brackets (()) ##-------------------- ## Round brackets (()) can be used for grouping of multiple commands (echo 1; echo 2; echo 3) # returns 1 2 3 ## Round brackets prefix by dollar ($())represents parameter substitution name=$(echo debjeet) echo $name # returns debjeet ## Round brackets (()) can also be used to initialize an array myarray=(1 2 3 4 5) # initialize an array for no in "${myarray[@]}" do echo $no # returns 1 2 3 4 5 done ## Round brackets (()) can also be used to initiate sub process ## create a script > myshscript.sh && vi myshscript.sh ------------------- #!/bin/bash var1=debjeet echo $var1 # returns debjeet (var1=cloud; var2=affaire; echo $var1; sleep 60) # returns cloud, and this has a different pid. echo $var1 # returns debjeet echo $var2 # returns null ------------------- :wq ## Execute the script ./myshscript.sh ## Check pid for myshscript.sh in a new terminal window ps -ef | grep myshscript.sh |
Curly brackets ({}):
Curly brackets ({}) can be used to build a sequence using extended brace expansion. Curly brackets ({}) can also be used for parameter substitution. Curly brackets ({}) can also be used to create a function without any name.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
##-------------------- ## Curly brackets ({}) ##-------------------- ## Curly brackets ({}) can be used to build a sequence using extended brace expansion. echo {1..5} # returns 1 2 3 4 5 echo {5..1} # returns 5 4 3 2 1 echo {5..1..2} # returns 5 3 1 echo {1..5..2} # returns 1 3 5 echo {a..z} # returns all the alphabets a to z echo {a..c}{a..c} # returns aa ab ac ba bb bc ca cb cc ## Curly brackets ({}) can be used for parameter substitution. dectobin=({0..1}{0..1}{0..1}{0..1}{0..1}{0..1}{0..1}{0..1}) echo ${dectobin[10]} # returns equivalent binary value 00001010 of decimal value 10 ## Curly brackets ({}) can also be used for inline group which in effect creates an anonymous function (a function without a name). ## However, unlike in a "standard" function, the variables inside a code block remain visible to the remainder of the script. var1=debjeet { var2=bhowmik; echo $var1; echo $var2; } # returns debjeet bhowmik echo $var1; echo $var2; # returns debjeet bhowmik |
Square brackets ([]):
Square brackets ([]) is used for globbing in case of regular expressions. Square brackets ([]) is also used to evaluate a test condition. Square brackets ([]) is also used to reference array element. Dollar followed by square brackets ($[]) can be used for integer expansion. Double Square brackets ([[]]) is used to evaluate a test condition and is more flexible than single square brackets ([]).
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
##--------------------- ## Square brackets ([]) ##--------------------- ## Square brackets ([]) is used for globbing in case of regular expressions ls -l [o?e]* # returns one.txt and one.sh ## Square brackets ([]) is also used to evaluate a test condition [ 5 = 10 ] echo $? # returns 1 as above test fail [ 10 = 10 ] echo $? # returns 0 as above test success var1=10 if [ $var1 = 10 ] then echo "ten" # returns 10 else echo "not 10" fi ## Double Square brackets ([[]]) is also used to evaluate a test condition and is more flexible than single square brackets ([]) [[ a = a && b = b ]] echo $? # returns 0 as above test gets evaluated and its true [ a = a && b = b ] # syntax error echo $? # returns 2 ## Square brackets ([]) is also used to reference array element myarray=(1 2 3 4 5) # initialize an array for no in "${myarray[@]}" do echo $no # returns 1 2 3 4 5 done echo ${myarray[0]} # returns 1 echo ${myarray[1]} # returns 2 ## Dollar followed by square brackets ($[]) can be used for integer expansion a=1; b=5 echo $[ 1 + 5 ] # returns 6 echo $[ $a + $b ] # returns 6 echo $[ a + b ] # returns 6 |
Greater than (>):
Greater than (>) is used for redirection to file. Double Greater than (>>) can be used for append to file. Greater than (>) can also be used file truncation. Greater than (>) can also be used as a comparison operator in certain cases.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
##----------------- ## Greater than (>) ##----------------- ## Greater than (>) can be used for redirection to file echo "hello there" > hello ## Double Greater than (>>) can be used for append to file echo "hello again" >> hello cat hello ## Greater than (>) can also be used file truncation > hello # or : > hello cat hello ## Greater than (>) can also be used as comparison operator let "greatest = (20 > 10?20:10)" echo $greatest # returns 20 |
Less than (<):
Double less than (<<) is used as heredoc. Less than (<) can be used as a comparison operator in certain cases. Greater than and less than with prefix forward-slash (\<\>) is used as word boundary in the regular expression.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
##-------------- ## Less than (<) ##-------------- ## Double less than (<<) is used as heredoc cat << EOF The current working directory is: $PWD You are logged in as: $(whoami) EOF ## Less than (<) can be used as comparison operator let "smallest = (10 < 20?10:20)" echo $smallest # returns 10 ## Greater than and less than with prefix forwardslash (\<\>) is used as word boundary in regular expression echo "hello world" > hello grep '\ |
Pipe (|):
Pipe (|) is a method to send the output of one command as an input to another command. Pipe (|) can also be used to force redirection. Double pipe (||) is used as a logical OR operator in test construct.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
##--------- ## Pipe (|) ##--------- ## Pipeing (|) is a method to send output of one command as an input to another command cat hello | grep hello ## Pipe (|) can also be used to force redirection echo "hello again" >| hello cat hello ## Double pipe (||) is used as logical OR operator in test construct a=true; b=true if [[ $a == false || $b == false ]] then echo "fasle" else echo "true" # returns true fi |
Ampersand (&):
A single ampersand (&) is used to run a job in the background. A double ampersand (&&) represents the logical AND operator.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
##-------------- ## Ampersand (&) ##-------------- ## A single ampersand (&) is used to run a job in the background sleep 10 & ## shell returns the prompt immiditely jobs ## A double ampersand (&&) represents logical AND operator a=true; b=true if [[ $a == true && $b == true ]] then echo "true" # returns true else echo "false" fi ls -l && cat hello # executes both the command |
Plus (+):
Plus (+) represent an addition in arithmetic operations. Plus (+) can also be used to append string.
|
1 2 3 4 5 6 7 8 9 10 11 |
##--------- ## Plus (+) ##--------- ## Plus (+) represent addition in arithmetic operations echo $((5 + 2)) # returns 7 ## Plus (+) can also be used to append string a="hello" a+=" world" echo $a # returns hello world |
Minus (-):
Minus (-) represent subtraction in arithmetic operations. Minus or hyphen (-) is also used to go back to the previous working directory. The single or double hyphen (- or –) is also used to pass options to the command.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
##---------- ## Minus (-) ##---------- ## Minus (-) represent subtraction in arithmetic operations echo $((5 - 2)) # returns 3 ## Minus or hyphen (-) is also used to go back to previous working directory pwd # /some/path cd /var/log cd - # back to where we started /some/path ## Single or double hyphen (- or --) is also used to pass options to command ls -a ls --all |
Percent (%):
Percent (%) is used as a modulo operator in arithmetic operations.
|
1 2 3 4 5 6 |
##------------ ## Percent (%) ##------------ ## Percent (%) is used as modulo operator in arithmetic operations echo $(( 19 % 4 )) # returns 3 |
Tilde (~):
Tilde (~) represents your home directory. Tilde with plus suffix (~+) represents the current working directory and tilde with minus suffix (~-) represent the previous working directory.
|
1 2 3 4 5 6 7 8 |
##---------- ## Tilde (~) ##---------- ## Tilde (~) represent your home directory echo ~ # your home directory echo ~+ # your current working directory echo ~- # your previous working directory |
Equals to (=):
Equals to (=) is used for variable assignment. Equals to (=) is also used for comparison.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 |
##-------------- ## Equals to (=) ##-------------- ## Equals to (=) is used for variable assignment var=debjeet echo $var ## Equals to (=) is also used for comparision a=5; b=deb; if [[ $a = 5 ]] # equals to then echo "true" # returns true else echo "false" fi if [[ $a != 5 ]] # not equals to then echo "true" else echo "false" # returns false fi if(($a >= 2)) # greater than equals to then echo "true" # returns true else echo "false" fi if(($a <= 10)) # less than equals to then echo "true" # returns true else echo "false" fi if [[ $b == "deb" || $b = "deb" ]] # == and = represents same in most cases then echo "true" # returns true else echo "false" fi |
Caret (^):
Caret (^) represents the beginning of a line in the regular expression. Caret (^) can also be used for uppercase conversion.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
##---------- ## Caret (^) ##---------- ## Caret (^) represents beginning of line in regular expression echo -e "hello world \n welcome to cloudaffaire \n this is debjeet" > hello grep ^debjeet hello # nothing grep ^hello hello # returns hello world ## Caret (^) can also be used for uppercase conversion var="hello world" echo ${var} # returns hello world echo ${var^} # returns Hello world echo ${var^^} # returns HELLO WORLD |
Back quote (
The backquote () is used for command substitution.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
##---------------- ## Back quote (``) ##---------------- ## Back quote (``) is used for command substitution ## Create the script > myshscript.sh && vi myshscript.sh ------------------- #!/bin/bash script_name=`basename $0` echo "The name of this script is $script_name" ------------------- :wq ## Execute the script ./myshscript.sh # returns The name of this script is myshscript.sh |
Hope you have enjoyed this blog post. This blog post is inspired by the Advanced Bash-Scripting Guide written by Mendel Cooper. You can refer to his amazing series using the below link.
http://www.tldp.org/LDP/abs/html/