Trick or Treat #1

#0 | #1 | #2 | #3 | #4

Inline editing

One of the things I like about Perl, and possibly why it is still very popular, is its versatility.

By that I mean that it handles jobs not only from a shell via the command line, but also through a web browser via a web server.

On these pages I'll try and show you some of the things I use Perl for, and that may be unique to it. Perhaps another page (or few) will deal with web programming and site development.

Since I work on an iMac, this is easy to get going, as every iMac already comes with Perl installed.

Not to mention some other *nix goodies like awk, sed, and an Apache web server.

During the course of writing all these pages, I realized that I wanted to change the <title> on all the HTML pages - over 20 files.

It used to be 'Perl Tutorials' but I wanted to change it to 'Perl Refreshments'.

I spend a lot of time in a text editor and am quite comfortable at it, but changing 1 word in about 20 files seemed a little ridiculous. Granted I could use an HTML authoring tool, but I roll all my own code - HTML, JavaScript, and Perl. So here is how I managed this little editing job.

Perl can be run with certain arguments on the command line to make it do different things. One of things it can do is called 'inline file editing'. What that means is that Perl can open a file, do some editing, and close the file - all without an editor of any kind.

In the directory with all my HTML files I ran this 1-liner:

perl -pi'.old' -e 's/Tutorials/Refreshments/' *.html

You might recognize part of that code to do some string substitution.

Basically it tells Perl to walk through all '.html' files in the current directory, copy them to a new file with a '.old' extension, replace 'Tutorials' with 'Refreshments', and do this for each file.

Very nice! And very quick too.

That's one of the things I like about Perl :)

After I checked to see everything was fine (all 20-some files), I deleted all the files ending in '.old':

rm *.old

Remove blank lines

Many times I need to remove blank lines from a text file. Of course I could open it up in my text editor, and page through it looking for blank lines to remove. But ...

perl -ne "print unless /^$/;" oldfile > newfile
# if a line contains just spaces:
perl -ne "print unless /^\s+$/;" oldfile > newfile

... this seems so much easier!

Swap Variables

If you've already done some programming, you have probably had to swap 2 or more variables at some point. In most programming languages, to swap 2 variables, you actually need a 3rd one:

my ($alpha,$bravo) = ("alpha","bravo");
my $temp = $alpha;
$alpha = $bravo;
$bravo = $temp;

But Perl lets us do away with the $temp variable:

print "A double swap:\n";
my ($alpha,$bravo) = ("alpha","bravo");
print "Old:\n\t\$alpha:$alpha\n\t\$bravo:$bravo\n";
($alpha,$bravo) = ($bravo,$alpha);
print "New:\n\t\$alpha:$alpha\n\t\$bravo:$bravo\n";

We can even swap more than 2 variables ... no kidding.

print "Time for a triple swap\n";
my ($alpha,$bravo,$charlie) = ("a","b","c");
print "Old:\n";
print "\t\$alpha:$alpha\n\t\$bravo:$bravo\n\t\$charlie:$charlie\n";
print "New:\n";
($alpha,$bravo,$charlie) = ($bravo,$charlie,$alpha);
print "\t\$alpha:$alpha\n\t\$bravo:$bravo\n\t\$charlie:$charlie\n";

In those examples, you may notice something strange in the print statements.

Some of the variables have a back-slash ahead of the $:

\$alpha \$bravo \$charlie

That's not a typo - since $ is a special character in Perl, to actually print one, we need to 'escape' it with a back-slash. It's one of those 'miscreants' mentioned earlier.

So to print the name of a variable (and not its value) we use this technique. This applies to arrays and hashes as well.

Logging script errors

When you run a script with a syntax error, typo, or other mistake, Perl will usually stop and give you some kind of error message on the monitor (you DO have use strict; and use warnings; included at the top of ALL your scripts, right?).

#! /usr/bin/perl
use strict;
use warnings;

Most times when an error happens, you can zero in on the line or section of code that is causing the problem and fix it. Perl is very helpful with debugging - it tells you the problem (as it sees it) and the line number where it first saw the problem.

But sometimes, you get a whole page of 'warnings' and 'errors' - it can be a real bear trying to go through.

This can be especially annoying if you have a web script that goes awry, and instead of 'failing gracefully', blurts everything out to the user - not what you want.

Here is some code that I always include in any scripts I make. It goes right after use warnings;:

BEGIN
{
	open (STDERR,">> $0.txt");
	print STDERR "\n", scalar localtime, "\n";
}

This is a BEGIN block, and as the name suggests, it executes this code BEFORE compiling any code in your script. In this block, I set it to send any messages to a file, which I can then check for errors.

The first line opens a standard file handle and redirects the output to a file. You could choose any name you like for this file, but I choose to give it the same name as the script, with a '.txt' extension. That's what ">> $0.txt" is doing. The $0 is another special variable meaning the name of the current script.

>> is a way of telling the system to ADD any new lines to the end of the existing file, thus creating a continuous log.

I chose this particular naming since the log file is now very easy to find - it's named the same as the script that created it, and in the same directory.

So a script named findNames.pl would create a log file named findNames.pl.txt.

The second line of the block writes the date and time on a line by itself - a very handy clue to debugging.

Here is a typical log file created with this method:

Tue Mar 31 19:52:50 2015
DBD::mysql::st execute failed: Column 'pubyear' cannot be null at 
/Users/user/Documents/httpd/cgi-bin/kb/mysql/kbcommit.pl line 60.

Tue Mar 31 19:53:15 2015
DBD::mysql::st execute failed: You have an error in your SQL syntax; 
check the manual that corresponds to your MariaDB server version 
for the right syntax to use near '"NULL","How the function works 
with arrays or hashes.","A",1,"Perl",198)' at line 1 at 
/Users/user/Documents/httpd/cgi-bin/kb/mysql/kbcommit.pl line 60.

Tue Mar 31 19:53:43 2015

This is from a script accessing a MySQL database - after finding the 2 errors logged, the code was fixed and only a date/time was logged the next time the script was run.

The beauty of creating a log file of errors becomes immediately apparent if you write scripts for web sites. Your script may be executed hundreds of times a day (or minute) and any errors are usually sent to 1 file on the server - along with all the other errors of other scripts.

Trying to find the errors created by your script in this log file is beyond a nightmare.

If you use this method on web-based scripts, you would be wise to comment it out AFTER you are confident it works properly, and BEFORE you make it public.

Every time the script is run it prints something to this log file, and if you forget about it, it can grow VERY HUGE.


#BEGIN
#{
#	open (STDERR,">> $0.txt");
#	print STDERR "\n", scalar localtime, "\n";
#}

If at some point in the future the script again begins misbehaving, you only have to 'uncomment' the block to get a log file.

Our next trick will be storing data inside your script.