Franske ITC-2480 Lab 14

Introduction

In this lab you will create and modify some basic python shell scripts.

Lab Procedure

Prerequisites

1. Have completed the reading on BASH and python scripting. If you haven't done this you will almost certainly have difficulty completing the lab!
2. Have completed the BASH scripting lab, this lab will take you through several similar tasks at a much faster rate
3. Open an SSH console to your Linux system using the PuTTY software, login with your standard user account

Creating Simple Python Scripts

1. Just like with BASH scripts we want to specify the interpreter at the beginning of our script with a sha-bang. Create a new text file called "hello.py" (note that we are now using the .py extension so we can remember this is a Python instead of a BASH script) with the following text:

   #!/usr/bin/python
   print "Hello World from Python!"
   

2. Set the hello.py file to be executable by all users on the system so that we will be able to run it and then try running the script.
3. Something to note is that you are now printing text to the screen using the Python print command and not the BASH echo command. If you try running the command print "Hello World!" from your command line you will get an error. This is what will make scripting in Pythona bit different than BASH scripting. When scripting in BASH every line in your script file could be entered directly on the command line, in Python the commands need to be run through the Python interpreter to be executed.
4. That said, it's still possible to run system commands from inside of a Python script as well (that's what ultimately makes them useful for shell scripting purposes after all. To do this we will need to import a special library of functions that Python can use to interact with the underlying system. Try modifying your hello.py file like this:

   #!/usr/bin/python
   import os
   print "Hello World from Python!"
   os.system('echo "Hello World from BASH inside of Python!"')
   

5. In this example we have imported the os library and then run a special function called os.system to execute a command on the system (in this case the echo "Hello World from BASH inside of Python!" command). This gets a little tricky because of the double and single quotations which are needed because we need to specify the command we want to run through os.system inside of quotation marks but we also want to pass some quote marks through to the echo command itself. There are a few ways to accomplish this but the one used in this case is to use single quotes for Python to know what command to run and then double quotes inside of those for the echo command to get it's argument. Try running this new script and see how it works.
6. Now try using what you have learned to create and test another script called simple-backup.py which runs the command "tar -czf /var/jsmith-backup.tar.gz /home/jsmith/"

Using Variables and Conditionals in Python Scripts

1. Argument parsing in Python is not quite as simple as it is in BASH but with practice it actually allows for much more complex scripts to be developed. Arguments are accessed as entries in an array through the sys library. First we need to import the sys library and then arguments will be accessed through the variables sys.argv[0] (for the script name) sys.argv[1] (for the first argument) and so on. Try adjusting your hello.py script to look like this:

   #!/usr/bin/python
   import sys
   print "The script you are running is: " + sys.argv[0]
   print "Hello " + sys.argv[1] + " from Python!"
   

2. What do you think will happen if you run the script like this "./hello.py yourfirstname"? Try it and find out if you were correct in your hypothesis!
4. Wait, what if someone doesn't know they are supposed to include their name as an argument though? Try running "./hello.py" and see what happens. Unlike BASH Python is a little more particular about things and if you try to access a variable that doesn't exist it will give you an error. This isn't very user friendly so let's add a conditional to check and see if an argument was supplied or not. Try updating your hello.py script again and adding a conditional like this:

   #!/usr/bin/python
   import sys
   if len(sys.argv) > 1:
           name = sys.argv[1]
   else:
           name = "World"
           print "Try running " + sys.argv[0] + " yourfirstnamehere to get a personal greeting!"
   print "Hello " + name + " from Python!"
   

5. See if you can predict what these changes will do and then try it out to see if you are correct. Because python stores all of the arguments into a special type of variable which can hold multiple pieces of data, called an array, we can use the len() function to check the length of the sys.argv array. If it's more than one (because sys.argv[0] always holds the script name so there is always at least one) we know arguments have been added. If not we know they have not been added. You'll also notice that indenting properly is important in Python scripts this is how python knows if a line is part of the if/else statement or something that comes after it.
6. It's also possible to capture the standard output of a command and put that into a variable for use. For example try running the "date +%Y%m%d" command. We can use the output of this command in a variable to make our backup script a little bit nicer. Save this script as better-backup.sh with the appropriate permissions to run it as a script.

   #!/bin/bash
   USER="jsmith"
   TODAY=$(date +%Y%m%d)
   BACKUPFILE="/var/"$USER"-backup-"$TODAY".tar.gz"
   echo Beginning backup for $USER on $TODAY
   tar -czf $BACKUPFILE /home/$USER/
   echo Backup completed for $USER saved to $BACKUPFILE
   

7. See if you can predict what this script will do and then try it out to see if you are correct. Once you have figured out how this script works see if you can modify it so you can specify a username as an argument instead of as a fixed variable. Can you do this with a change to only one line?
8. After successfully doing that try modifying the script so that it gives instructions about how to use the script if you forget to give a username as an argument.
9. It is important to note that while this script works, it's just a simple example and not well written because it is easily broken by giving a username which doesn't actually exist on the system, or trying to backup files you don't have permission to access, etc. A better script could be written to check for all of these things as well as other errors. If the script is to be used by more people than the original author (and in the best cases, even then) it is important for usability and security purposes to do these sorts of checks, particularly if the script accepts user input such as an argument.

Using Loops and Reading User Input

1. Sometimes you want a program to do something over and over again several times. This is typically done using a loop. Let's say you wanted to create a bunch of sample files to do some further practice with in the current directory named "sample-fileX.extension" where X was a number that would keep incrementing by 1 each time and you could specify any extension you wanted. We could certainly use arguments to capture that user input but it is also possible to accept input from the user directly while the script is running. This would create what we call an "interactive" script because the user is interacting with it.
2. Create a file-generator.sh script on your system like the one below and see if you can figure out how it works before testing it:

   #!/bin/bash
   BASENAME=sample-file
   echo "Welcome to the sample file generation utility!"
   echo "Files will be created using the filename format "$BASENAME"X.extension"
   echo "Enter the number of files you wish to create, then press Enter:"
   read NUM
   echo "Enter the extension (without the leading period) to put on the files"
   read EXTENSION
   for CURNUM in `seq 1 $NUM`; do
       touch $BASENAME$CURNUM.$EXTENSION
   done
   echo "Created "$NUM" file(s)."
   

   There are a few things to note about this script. First, we have used one type of loop, a FOR loop, but there are others and other ways to make this work. Second you can see several more examples of calling another program inside your script. In this case we have used the read, seq, and touch programs to accomplish our goal. The seq program is especially useful while writing scripts. It can be used to generate a series of numbers from a starting number (1 in this case) to a stopping number ($NUM in this case). There are many other helpful programs you can use while writing scripts including some you know about already such as grep and find, and others you don't such as sed and awk. You can even use programs like wget to get information from the Internet and process it for use in a script instead of using it for just downloading a file. For example, if you know of a web site that lists the sunrise and sunset times you can use a combination of wget and grep/awk/sed to pull that time out of the webpage and use it as part of a script. The possibilities are almost endless, time spent learning about the ins and outs of these little utilities is time well spent if you will be writing shell scripts!
3. Try running the script and see if you were correct about what it does and how it works.
4. Also, it is important to note that while this script works, it's just a simple example and not well written because it is easily broken by the user entering something other than a number for the number of files to be created, etc. A better script could be written to check for all of these things as well as other errors. If the script is to be used by more people than the original author (and in the best cases, even then) it is important for usability and security purposes to do these sorts of checks, particularly if the script accepts user input! This is how security bugs are born.
5. See if you can modify the script so you can specify a starting and an ending number for the files rather than just a number of files to be created.
6. See if you can add a check to the script which will display an example of a filename that will be created and asking the user if that looks correct before actually creating all the files. If the user enters N then do not create the files.

Functions and Experimenting

1. One final programming structure we should discuss is the function. In shell scripting you can think of a function like something you could have put into a separate script so that you could call it to do something multiple times from inside of your script but instead of making it a completely separate file you decided to include it in a single script. Here is a simple sample:

   #!/bin/bash
   function helloname {
       echo "Enter your name, followed by enter:"
       read NAME
       echo "Hello "$NAME"!"
   }
   echo "This program will welcome you twice to be extra nice!"
   helloname
   echo "At this point we could be doing lots of other stuff in our script or be in a loop, etc."
   helloname
   echo "Thanks for playing!"
   

   See if you can figure out what will happen when you run it, then try it out and see if you were correct. Note that functions need to be declared (listed) before you use them in your script so it would be common to see them at the beginning of the file as in this example. Also, it's possible to pass information to a function using arguments just like you would with a separate script.
2. Now that you have a grasp on the fundamentals of BASH scripting see if you can write a script which can be used to modify the name of all files in the current directory which share the same extension such as those created by file-generator.sh. The goals for your script should be:
   • The user has a choice to enter the original and the new extension for the files either using command line arguments or interactively by answering questions your script asks.
   • The script should display each old file name and then the new file name next to it so you can see what is going on
   • You will want to use conditionals to figure out if the user entered arguments to use as the old and new extensions or should be prompted for them interactively.
   • You should use functions and loops in order to make the most efficient script possible. Hints: The actual renaming of the files and displaying old and new names would be a good thing to put in a function because you will be calling it over and over again. Also, loops can be used to operate on a list of things such as a list of filenames.
   Don't be afraid to try using search engines for help with this. The goal of this lab is not to make you into a professional programmer but to introduce you enough to shell scripting so that you can create useful scripts with resources such as search engines available to you.