Bash tips

= Introduction = If you type in a rather long and complex command at the command line, you might want to save that command in a text file, so you can execute it later without having to type the whole long thing in again. If you add a line to the beginning of that file, like "#!/bin/bash" (a so-called "shebang" line), and then make the file executable with the chmod command, you've just created a shell script.

Here's an example shell script: echo "Hello world!"
 * 1) !/bin/bash
 * 2) Any line that starts with a hash, besides the shebang line, is a comment.

You run the shell script just like any other executable. Some folks create a bin directory in their home directory within which to place their scripts. Others drop them in /usr/local/bin.

Some key points to keep in mind about shell scripts:
 * They execute their commands, line by line, just as if you'd typed them in yourself at the command line.
 * The commands in the script run in a subshell of the shell from which you executed the script.
 * If you want the script to affect your present working directory or environment variables, source it with . scriptname.
 * Don't put spaces around equal signs.

You can check out the bash page and you'll also find lots of info in the Advanced Bash-Scripting Guide.

= Technologies =

Customized Prompts
Tip: dircolors is a bash command of great interest for folks who want to colorize the output of the ls command.

Using the information at the link below, you can colorize your prompt and have it show convenient things like the time of day, date and if you log in to a number of different machines, it might be good to have the name of the machine displayed so you can remember which one you are on. ;-) For more on customized prompts, check out Color Bash Prompt at ArchWiki.

Using quotes
Quotation marks, either double quotes (") or single quotes ('), are often needed in shell scripts to pass arguments that have spaces or other special characters. The two quotes have slightly different meanings.  Single quotes protect the exact text they enclose.

Inside double quotes, the dollar sign ($) is still special, so shell variables and other $ expressions are interpreted, as is the backquote (`) used for command substitution. Lastly, the backslash (\) is special in double quotes, so be cautious. Some examples and the shell output of each:

$ echo 'The location of the shell is stored in $BASH' The location of the shell is stored in $BASH $ echo "The location of the shell is $BASH" The location of the shell is /bin/bash $ echo '\\' \\ $ echo "\\" \

Return values
Using set -e in your script will cause it to drop dead on the first error. This is a good thing when manipulating an operating system, file system or if the script otherwise runs very long. Basically, when ever a return value that isn't 0 comes back from a command, it stops the script. Be careful though, this is not necessarily an error condition. The grep command, for example, returns 1 when it doesn't find anything... but then maybe that is what you are counting on! The consequence will be that your script will stop and you may not immediately understand why.

The return value, also known as error code, delivers the exiting state of the most recently executed program. It is defined to be 0 in case no error happened. It is stored in the variable $?: scorpio:~ # echo "hello world" hello world scorpio:~ # echo $? 0 scorpio:~ # ls /doesNotExist /bin/ls: /doesNotExist: No such file or directory scorpio:~ # echo $? 2

The [ ] operators deliver 0 if they contain a true expression, 1 if they contain a false expression: scorpio:~ # [ 5 = 5 ] scorpio:~ # echo $? 0 scorpio:~ # [ 5 = 6 ] scorpio:~ # echo $? 1 scorpio:~ # Here, [ is a command. A command requires a blank behind it - thus the following error is very common: scorpio:~ # [5 = 6] bash: [5: command not found Also the ] requires a blank in front of it: scorpio:~ # [ 5 = 6] bash: [: missing `]' The "=" sign requires a blank in front of it and after it. Otherwise, the bash only sees "something" within the brackets, and that is true (0), while "nothing" is false (1): scorpio:~ # [ aoei ] scorpio:~ # echo $? 0 scorpio:~ # [ 5=5 ] scorpio:~ # echo $? 0 scorpio:~ # [ ] scorpio:~ # echo $? 1 scorpio:~ # true scorpio:~ # echo $? 0 scorpio:~ # false scorpio:~ # echo $? 1 You can hand over your own error codes with the command exit. This command will exit from the current process. As the signs spawn out an own process, you can do the following: scorpio:~ # (exit 5); echo $? 5

Conditions
Conditions are handled with the if command, using the form if then command1 else command2 fi "fi" as the reverse of "if", closes the loop. bash allows you to substitute newlines with ";", so the following form is also legal: if ; then command1; else command2; fi Example: scorpio:~ # if true; then echo "yes"; else echo "no"; fi yes scorpio:~ # if false; then echo "yes"; else echo "no"; fi no You can omit the else-branch: scorpio:~ # if [ 5 = 5 ]; then echo "five is five"; fi five is five scorpio:~ # export name=Thorsten scorpio:~ # if [ $name = Thorsten ]; then echo "it's you"; fi it's you In this case, bash replaces $name with its content. That means, we get an error if $name is empty: scorpio:~ # export name= scorpio:~ # if [ $name = Thorsten ]; then echo "it's you"; fi bash: [: =: unary operator expected scorpio:~ # if [ = Thorsten ]; then echo "it's you"; fi bash: [: =: unary operator expected To overcome this problem, one can add an "x" to the left and to the right: scorpio:~ # export name=Thorsten scorpio:~ # if [ x$name = xThorsten ]; then echo "it's you"; fi it's you scorpio:~ # export name= scorpio:~ # if [ x$name = xThorsten ]; then echo "it's you"; fi scorpio:~ #

Negations
You can invert true/false values with the ! operator: $ ! true $ echo $? 1 That helps you if you want to do "not"-statements: if ! grep "network error" /var/log/messages then echo "There is no network error mentioned in the syslog" fi The ! is just like a command with arguments, that's why it is important to have a space behind it: $ !true bash: !true: event not found

Loops
You can loop on a condition: while [ 5 = 5 ]; do echo "This never stops"; done

until [ 5 = 6 ]; do echo "This never stops"; done

$-variables

 * $! : PID of last process that has been started in the background (e.g. firefox &)
 * $? : return value of last command

Counting
x=0; ((x++)) x=0; x=$((x+1)) x=0; let x=$x+1 for (( x=0; x<10; x++ )) ; do echo $x ; done

for x in $(seq 0 9); do echo $x; done x=0; x=$(expr $x + 1) x=0; x=$(echo $x + 1|bc) x=4; x=$(echo scale=5\; $x / 3.14|bc)

The first four demonstrate Bash' internal counting mechanisms, these will not use external programs and are thus safe (and fast). The last four use external programs. bc, as used in the last two examples, is the only one that supports numbers with decimals. For adding, subtracting and multiplying, you don't have to do anything special, but for division you need to specify the number of digits to keep (default is none) using the 'scale=n' line.

Combining multiple conditions of if statements
AND: if((a==1 && b==2)) OR: if(( a!=1 || b>10 ))

Conditional execution
As stated above, you can do something like scorpio:~ # if (( 5 == 5 && 6 == 6 )); then echo "five is five and six is six"; fi five is five and six is six We call 5 == 5 the left term and 6 == 6 the right term. Bash first evaluates the left term and if it is true, it evaluates the right term. If both are true, the result is true, otherwise, the result is false (boolean logic). It is an interesting effect that, if the left term is false, the right term is not even evaluated - the result will be false in any case. That means, if we replace the terms by commands, the right command will only be executed if the left command succeeded. One can use this effect: scorpio:~ # ls > /tmp/dir && echo "the command succeeded" the command succeeded scorpio:~ # ls > /proc/cmdline && echo "the command succeeded" /bin/ls: write error: Input/output error scorpio:~ # In the above example, bash tells you if the command succeded. The right command is conditionally executed. Considering a famous saying, you can even program a bird using this technology: eat || die This line says "eat or die" which is very useful if you want a program to exit in case it cannot perform a certain operation: touch /root/test || exit

Functions
You can define functions for structured programming. It is also possible to hand over parameters to a function. Example: function square \ {  echo "The square of your number is " echo $((a*a)) } echo "Input a number" read a square $a
 * 1) !/bin/bash
 * 2) This program calculates the square of a given number

= UseCases =

Renaming a set of files
The following example will rename all files ending in .jpeg to end in .jpg $ for file in *.jpeg; do mv $file ${file%.jpeg}.jpg; done

mmv is a really nifty tool for doing this sort of thing more flexibly too. The rename command also provides similar functionality, depending on your linux distribution.

If you have a load of files in all capital letters (common if you unzip old DOS programs for instance), you can use $ for file in *; do mv $file $(echo $file | tr upper: lower:); done to make all the files in the current directory lowercase.

Get your IP address
A bash example to get your IP address using http://www.whatismyip.com:


 * 1) TEST=$(lynx -dump http://www.whatismyip.com | grep -i "Your IP Address Is:" | awk '{ print $5; }')
 * 2) echo $TEST

Or use this: $ ifconfig ppp0 | awk '/inet addr/{print $2}' | cut -d: -f2

Or this: $ ifconfig ppp0 | sed -rn 's/^.*inet addr:([^ ]+).*$/\1/p' Note that ppp0 stands for the first point-to-point-protocol device which is usually a modem. If you are using an ethernet adapter instead, replace ppp0 with eth0. You can also leave it out all together to list all the addresses associated with your computer.

Converting several wav files to mp3
This statement will convert all .wav files in a directory to mp3 (assuming you already have the lame mp3 encoding package installed): for i in *.wav; do lame -h $i && rm $i; done The double ampersand prevents rm from deleting files that weren't successfully converted.

Full file name
Prints the full filename removing ./ and ../ and adding `pwd` if necessary.

fqn.sh
 * 1) !/bin/sh


 * 1) return the full filename, removing ./ ../ adding `pwd` if necessary

FILE="$1"

while true; do	case "$FILE" in		( /* ) # Remove /./ inside filename: while echo "$FILE" |fgrep "/./" >/dev/null 2>&1; do			FILE=`echo "$FILE" | sed "s/\\/\\.\\//\\//"` done # Remove /../ inside filename: while echo "$FILE" |grep "/[^/][^/]*/\\.\\./" >/dev/null 2>&1; do			FILE=`echo "$FILE" | sed "s/\\/[^/][^/]*\\/\\.\\.\\//\\//"` done echo "$FILE" exit 0 ;;		(*)		FILE=`pwd`/"$FILE" ;;	esac
 * 1) file		dot relative
 * 2) ./file	dot relative
 * 3) ../file	parent relative
 * 4) /file		absolute

done

Resolving a link
This little script follows symbolic links wherever they may lead and prints the ultimate filename (if it exists!) (note - it uses the previous script fqn.sh)


 * 1) !/bin/sh


 * 1) prints the real file pointed to by a link
 * 2) usage: $0 link

ORIG_LINK="$1" LINK=`fqn "$1"` while [ -h "$LINK" ]; do	DIR=`dirname "$LINK"`

# next link is everything from "-> " LINK=`ls -l "$LINK" |sed "s/^.*-> //"`

LINK=`cd "$DIR"; fqn "$LINK"` if [ ! -e "$LINK" ]; then echo "\"$ORIG_LINK\" is a broken link: \"$LINK\" does not exist" >&2 exit 1 fi done echo "$LINK" exit 0 This is a lot simpler, and just as useful (but doesn't do dead link checking) ORIG_LINK="$1" TMP="$(readlink "$ORIG_LINK")" while test -n "$TMP"; do   ORIG_LINK="$TMP" TMP="$(readlink "$ORIG_LINK")" done echo "$ORIG_LINK"
 * 1) !/bin/bash

Viewing a file according to its type
I save this as a script called v and it is my universal viewing script - it can be extended to any kind of file and to use your favourite tools for each type of file. It uses the file utility to determine what sort of file it is and then invokes the correct tool. It automatically adapts to being used on the system console or under X.

(note - this uses a prior script fqn.sh</tt>)
 * 1) ! /bin/sh
 * 2) View files based on the type of the first one:
 * 3) Relies partly on 'less' to invoke an appropriate viewer.
 * 4) Make sure than you have this set:
 * 5) LESSOPEN="|lesspipe.sh %s"

FIRST="$1" FULL=`fqn $FIRST`

[ -z "$FIRST" -o ! -r "$FIRST" ] && exec less "$@"
 * 1) if we are being piped to, just run less

case "$FIRST" in *.cpio.bz2) bunzip2 -c $1 | cpio -ctv 2>/dev/null |less; exit ;;  *.cpio) cpio -ctv $1 2>/dev/null |less; exit;; esac
 * 1) some file types beyond what /usr/bin/lesspipe.sh handles:

TYPE=`file -L -i "$FIRST" 2>/dev/null | \ awk '{len=length("'"$FIRST"'")+ 2; $0=substr($0, len); print $1;}'` echo "Type=\"$TYPE\"" case "$TYPE" in   image/jpeg | image/tiff | image/gif | image/x-xpm | image/*)        xzgv "$@" ;;    application/pdf) XPDFGEOM=`xdpyinfo |awk '/dimensions/ {print $2}'| \ awk -Fx '{printf("%dx%d+0+0", 0.60*$1, $2-35)}'` xpdf -geometry $XPDFGEOM -z width "$@" ;;   application/postscript)        gv "$@" ;;    application/msword) if [ -z "$DISPLAY" ] ; then antiword "$@" | \ less; else soffice "$@" & fi ;; text/html)       if [ -z "$DISPLAY" ] ; then lynx "$@"; else dillo "$FULL" & fi ;;    audio/mpeg) mpg123 "$FIRST" ;; *)       less "$@" ;; esac

Finding the 50 largest directories
du -S / | sort -nr | head -n50

Auto-generating a shebang line
(echo -n '#!'; which perl; tail -n+2 foo) > bar This will print "#!" and the path to perl</tt>, then everything except the first line (the original shebang) of the file foo</tt>; and redirect all that to go into a file bar</tt>.

This is really meant for use in install scripts; obviously you know where perl is on your system, but not necessarily on anybody else's system (though /usr/bin/perl</tt> is a safe guess). But you know bash is always at /bin/bash</tt>. So, instead of just saying in the documentation "change the first line .....", you can write a little shell script that puts the correct shebang line in the file and copies it to /usr/local/bin</tt>. Then, put this script in a .tar.gz file with your perl script and any documentation.

Creating an audio CD from .mp3 files
This little script requires the mpg123 and cdrdao packages. It uses mpg123 to create a .wav file from each mp3 in the current directory, and builds up a "TOC" file (Table Of Contents) as it goes along, using the &amp;&amp; notation to be sure only to include successfully-converted files. Finally, it starts writing the CD.

The TOC file is given a name based on the PID, and is placed in the current directory. Neither it nor the .wav files will be deleted at the end of the script, so you can always burn another copy of the CD if you want.

tocfile="cd_$$.toc" echo "CD_DA" > $tocfile for i in *mp3 do wav="`basename $i .mp3`.wav" mpg123 -w$wav $i\ && echo -en >>$tocfile "TRACK AUDIO\nFILE \"$wav\" 0\n" done echo -e "TOC file still available at $tocfile" cdrdao write $tocfile
 * 1) !/bin/sh

Note: in the for</tt> statement, you might think at first sight that it should be for i in *.mp3</tt>; but Linux doesn't care about filename extensions at all. * will match a. character; therefore, *mp3 will match everything ending in mp3 or .mp3. Those characters are unlikely to be found on the end of anything but an mp3 file.

Portable Indirection
Sometimes you need to use the value of a variable whose name is held in another variable. This is indirection. For example: for V in PATH LD_LIBRARY_PATH; do echo ${!V}; done but that isn't portable to older shells, eg. the old Bourne shell. A portable way is to use the following instead of ${!V}</tt>: `eval echo \\$"$V"` It's not pretty but at least it works on nearly all Unixes.

[ someone said - "It does require that the variable be exported." Actually no. It works without exporting V on bash and on Solaris sh ]

Matching Curly Brackets
If you indent your C, Perl or PHP code in what is commonly called "K&R style", which is to say something like this for ($i = 0; $i <= 6; ++$i) { for ($j = 0; $j <= $i; ++$j) { print "$i : $j "; };   print "\n"; }; i.e. where the opening curly bracket appears on the same line as the flow-control keyword  (if, while &c.)  and everything up to, but not including, the closing curly bracket is indented, then you may find this useful. I came up with it out of necessity, and thought it was just too important not to share. All it does is display the lines with a { but no following } (in Perl, these denote associative array indexes)  or a } with no preceding {. In other words, just the opening and closing lines of loops, not the content. This can help you to spot those annoying missing-bracket errors.
 * 1) nested loop example

awk '/({[^}]*$)|(^[^{]*})/{print}' foo

Replace foo with the filename to be examined. Or, of course, you can omit the filename and use the command in a pipeline.

The above example would appear as for ($i = 0; $i <= 6; ++$i) { for ($j = 0; $j <= $i; ++$j) { }; };

rpminfo
This is a small script that prints useful info about an rpm package. It was inspired by the ability of less to give almost useful info about an rpm file through lesspipe.sh</tt>.

echo INFORMATION: ; rpm -qi $* ; echo echo REQUIRES: ; rpm -qR $* ; echo echo PROVIDES: ; rpm -q --provides $* ; echo echo FILELIST: ; rpm -qlv $*
 * 1) !/bin/sh

Save it to a file called rpminfo</tt> in your path and chmod it so it's executable. It is often useful to pipe the output to less, as in the examples below.

To get info on an installed package, you can just give the package name like: rpminfo XFree86 |less For a package file that you have just downloaded, give the -p</tt> command option, just like rpm</tt>: rpminfo -p XFree86-4.1.0-25.i386.rpm |less

Finding All Files That Start With a '.' (period)
To list all the files in your current directory whose file names start with a '.') ls -A | grep -E "^\.\.*"

The following command will copy, overwriting existing files, the hidden contents of the present directory (files and directories that start with a period) into the folder, '/tmp/testing' (assuming that you've already made the target directory):

(Useful for backing up hidden files and directories in your home folder; be sure to change the target directory to something more sensible, though.)

cp -vfr `ls -A | grep -E "^\.\.*"` -t /tmp/testing/

Connecting to a wireless access point through Bash (no encryption key)
To connect to a wireless access point through the shell, first determine the designation of your wireless adapter:

cat /proc/net/wireless

This should give you a similar output:

Inter-| sta-|  Quality        |   Discarded packets               | Missed | WE face | tus | link level noise |  nwid  crypt   frag  retry   misc | beacon | 22 wlan0: 0000  57. -53. -87.       0      0      0      0      0        0

In this case the designation of the wireless adapter is wlan0. Now you can proceed to the next step, scanning the available wireless points:

sudo iwlist wlan0 scan

An example of the output you should get:

wlan0    Scan completed : Cell 01 - Address: 00:1D:8B:52:2C:9C Channel:6 Frequency:2.437 GHz (Channel 6) Quality=57/70 Signal level=-53 dBm Encryption key:on ESSID:"costa reina" Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s 9 Mb/s; 12 Mb/s; 18 Mb/s Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s Mode:Master Extra:tsf=000000208e5ce217 Extra: Last beacon: 110ms ago IE: Unknown: 000B636F737461207265696E61 IE: Unknown: 010882848B960C121824 IE: Unknown: 030106 IE: Unknown: 2A0104 IE: Unknown: 32043048606C IE: WPA Version 1 Group Cipher : TKIP Pairwise Ciphers (1) : TKIP Authentication Suites (1) : PSK IE: Unknown: 0706495420010E10 Cell 02 - Address: 00:1C:A2:B0:DC:54 Channel:1 Frequency:2.412 GHz (Channel 1) Quality=36/70 Signal level=-74 dBm Encryption key:on ESSID:"FASTWEB-1-001CA2B0DC54" Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s 9 Mb/s; 12 Mb/s; 18 Mb/s Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s Mode:Master Extra:tsf=00000000ddf7c3e4 Extra: Last beacon: 3240ms ago IE: Unknown: 0016464153545745422D312D303031434132423044433534 IE: Unknown: 010882848B960C121824 IE: Unknown: 030101 IE: Unknown: 2A0104 IE: Unknown: 32043048606C IE: WPA Version 1 Group Cipher : TKIP Pairwise Ciphers (1) : TKIP Authentication Suites (1) : PSK IE: Unknown: 0706495420010E10 Cell 03 - Address: 00:1C:A2:C3:DF:CC Channel:6 Frequency:2.437 GHz (Channel 6) Quality=32/70 Signal level=-78 dBm Encryption key:on ESSID:"FASTWEB-1-001CA2C3DFCC" Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s 9 Mb/s; 12 Mb/s; 18 Mb/s Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s Mode:Master Extra:tsf=00000000087f0232 Extra: Last beacon: 2850ms ago IE: Unknown: 0016464153545745422D312D303031434132433344464343 IE: Unknown: 010882848B960C121824 IE: Unknown: 030106 IE: Unknown: 2A0104 IE: Unknown: 32043048606C IE: WPA Version 1 Group Cipher : TKIP Pairwise Ciphers (1) : TKIP Authentication Suites (1) : PSK IE: Unknown: 0706495420010E10

The important bits of information are: ESSID, Channel, Encryption key, and Mode. ESSID is the name given to the wireless point. Now you select which one you want to connect to. Let's say you want to connect to an access point that has the following information:

Cell 03 - Address: 00:1C:A2:C3:DF:CC Channel:6 Frequency:2.437 GHz (Channel 6) Quality=32/70 Signal level=-78 dBm Encryption key:off ESSID:"connect here" Bit Rates:1 Mb/s; 2 Mb/s; 5.5 Mb/s; 11 Mb/s; 6 Mb/s 9 Mb/s; 12 Mb/s; 18 Mb/s Bit Rates:24 Mb/s; 36 Mb/s; 48 Mb/s; 54 Mb/s Mode:Master Extra:tsf=00000000087f0232 Extra: Last beacon: 2850ms ago

The command to connect would be:

sudo iwconfig wlan0 essid "connect here" mode managed channel 6 key off

Followed by:

sudo dhclient


 * Please note that if there is an encryption key encrypted with WPA, things become slightly more complicated. You can refer to a script I wrote for WPA encrypted connections.

Acquire root access from within a BASH script, using sudo
Since some Linux distributions do not offer to set a root password upon installation, prompting a user for root access from within a script can be a bit of a challenge. One option is to use sudo. This works well, except that if the commands to be executed take longer than the sudo timeout, it could require prompting the user, again--which is unacceptable if the script is intended to run non-interactively, after the initial prompt. Additionally, one cannot execute BASH functions using sudo, since it runs as its own process and doesn't export functions. If you can simply invoke your script as root (example: "sudo myscript.sh"), then that's preferable, and the below method is not required. As an example of where you might need to use this, some GUIs (like MATE) currently do not give you the option of running something as root, using just a double-click, so the below can allow the user to "Run in Terminal" and then type in a password if required. (There may be additional software you can install to do this, such as GKSUDO, but one might not want to assume that the target machine has this software installed.) I personally use this method in an automated script for using rsync with Google Drive.

Method 1
The below is a workaround that involves creating a temporary file in /tmp/, and then deleting it when the script is done.

(from within your BASH script)  </tt>

The "EOF" at the end is important, because that tells cat that everything after "EOF" doesn't go into the temporary file, and should be treated as part of the original script. "EOF" will NOT end up in the temporary script. If, for any reason, you need to include the string, "EOF" in the text of the temporary script, then you can change what cat is waiting to see, like so:

 </tt>

Method 2
It's possible to avoid having to create a temporary file by using EOF input directly into a "sudo bash" command, like this:

<tt> </tt>

I don't know if this method has some limits, such as number of characters in a single command, or the like.

Sample Script
This script uses Method 1. If you want to use Method 2, instead, just change the "cat ..." stuff to "sudo bash <<'EOF' ...", as above, and omit the file execution and management commands at the bottom.

<tt> </tt>

Note that in this sample script, instead of doing "bash /tmp/gsyncbackup.tmp", I first change its permissions to be executable, and then invoke it without the "bash" command. Both methods produce essentially the same result. Do note, however, that if you do it without invoking "bash", you have to make sure that the first line of your temporary file is, "#!/bin/bash".

= See also =
 * ShellScripts