Teaching Programming; Part 5: RAMing an F?

Now we get to the fun part of what I had to teach my Algonquin college students about the C programming language. In the last couple of talks I laid out some of the basic principles. I pointed out that we can store a list of names by using symbols to stand in place of the data—and that these symbols act like containers. We used name(x) to get the individual entries in the list, where x is a number describing where in the list the data we want is located.

What are we actually talking about when we talk about data and symbols? Simply this: everything a computer does or works on is located in Random Access Memory (RAM). When we tell a program to start running (by double-clicking on its icon in Windows), the computer has to locate the code for the program on a disk-drive and copy it into RAM. From then on, everything that the program references is an address in RAM. For example, your program is loaded into memory starting at address 435A6B7E. All the symbols in your program can now be interpreted in relation to that first address. When your program needs to read some data, it fetches it from the disk drive and loads it into RAM starting at a different address and tells your program what that address is so that it can find and use it. So, your symbol “name” might be at address 345A6C89 in your program code, but the data it is referencing could begin at address 4563EE12. When we tell our program to work on the data in name(x) we are actually telling it to go to the memory address that name(x) has calculated and start working on the data found there.

Clear as mud, right?

That’s how my student felt, too.

The point is: data and the place (address) in RAM where it is temporarily stored are two different things. It is like telling a window-cleaner to go to 546 Main Street and clean the windows there. Whatever he finds there is not the same thing as the address and he will clean the windows found at that address, not clean the address itself.

We have to keep that straight when programming or else we might inadvertently tell our window-cleaner to go to “windows” and clean the “546 Main Street”—whatever that is. And, this is a very common programming error. We see it whenever a Windows program “crashes” (i.e. stops working abruptly and returns control to the operating system). It has either run into data that it is not prepared to handle (for example, a window-less building where the window-cleaner has been ordered to clean the windows); or it is trying to locate an address in RAM that it has no business accessing (such as where the operating system instructions are stored, or, more commonly, address “0”). Competent programmers are not supposed to mix up what is data and what is an address, but they do. I got caught by that once after I had been writing programs professionally for many years; they can be nasty—and sometimes very subtle—errors to track down and fix.

My students had to know the syntax for fetching an address and the syntax for fetching the data at that address. Sometimes, to further complicate things, we might find another address at the address where we are looking for data—and we have to know the difference so we can instruct our program to go to the second address instead of treating it like ordinary data.

I know it’s difficult to understand when you first run into it. But we worked at it and most students managed to pass the course.

However, one young woman had a rather unique way of handling the situation. I gave a review quiz every second week of class. I would take them home to grade, then return them and go over the answers in the next class. This young woman was consistently sick whenever we had a quiz, but always showed up for the next class where I’d be going over the answers. She would ask for a copy of the quiz so she could follow along. Fair enough. After we had finished our review of the test, she would bring her paper to me, asking me to grade it so she could make sure she had gotten all the information correctly so she could study from it. So, I did. She usually still had a few errors even though we had just discussed the answers.

I thought nothing of it and gave her an F for the course on the grounds that she had not completed any of the course work. (I believe she was also sick for the final exam—a terrible, unpredictable disease that always struck at quiz or exam time.) In any case, after the students had received their grades she phoned me at home to complain. How could I have given her a failing grade? She had all the tests and had “passed” them all. I tried to explain that writing down the answers during the review was not the same thing at all as writing the exams when they were scheduled. She was adamant and phoned me several times, making a pest of herself. Finally I gave her the phone number for the college ombudsman, asked her to take her case to him and said I could not discuss it any further with her.

The ombudsman called and asked me for the story. “Okay, I get it,” he said when I finished. I never heard any more about the case, but I am confident that my F had stood.

Need to Know

In computer programming letters or words are used as symbols standing for information. For example: the symbol “name” can stand for “Mary Jones,” “Jack Smith,” or “Thomas Mann.” The symbol is “name;” the data is “Mary Jones,” “Jack Smith,” or “Thomas Mann.” Don’t worry yet about how we get there: just think of “name” as a container that can hold any one of the given and family names of our individuals. If I arrange my people in a list then I can refer to them as name(1), name(2), name(3).

Alternatively, the symbol “x” can stand for any number. If I want to count from 1 to 10, I can increment (raise by step) the value of a symbol that stands for a number. A common way of notating that in many languages is: x = x + 1. In other words, you take the number that is represented by “x,” add 1 to it, then store the result back in the symbol “x.”

Putting these two ideas together we can tell the computer to fetch all the names in our list by using the two symbols “name” and “x” as in “name(x)”. We don’t know—or even care—the content of any of our containers. We just know that if we tell the computer to print whatever is in container “name” in row number “x” it will print out one of our names—depending on what the value of x is.

Now you know everything there is to know about computers. Almost.

In the two examples above, our symbols represented actual data, whether it be names or numbers. If you tell the computer “print name(2)” it will not print out literally “name(2)” but will, instead, print out the data represented by those symbols. In other words, it will print “Jack Smith.” A computer can do thousands or even millions of these kinds of data retrievals in a second (though it will take much longer to actually print the list). If you had a list of the name of every person in Canada you could have a computer fetch any name, sort the list by any key you wished, find all the people named “Smith,”—or whatever you can think of doing with such a list—in the blink of an eye.

But, what if you wanted to manipulate the data in another section of a program? For example, you might want to reverse given and family names. So, you would write a routine that would parse (split into parts) the contents of name(x), then put the parts back together in reverse order. Such commands might look something like this:

first, last = parse(name(x),” “);
name(x) = last & first;

(We are assuming that the computer language has some sort of “parse” command built in that will split a string of characters using whatever delimiter (in this case a blank space) you tell it to use. And we are also assuming the language uses the symbol “&” to concatenate two individual pieces data.)

Did you notice what we did? We changed the data stored in our symbols “name,” row “x”. I mentioned that our procedure that did the name reversal was “in another section of a program.” We do this so that we have to write the commands to reverse the names only once. If we call our name swapping section “switch,” for example, then, in the main part of our program all we need to do is tell the computer to go to section “switch” and execute its operations on the data we have selected. And, because we are fetching our data by using a symbol called “name” and the information in “name” is in rows called “x,” we can write a compact little program. Sort of like this:

x = 0;
while (x <= 10)
x = x + 1;
print name(x);
switch (name(x));
print name(x);
end while;

The section we have called “switch” would be along the lines of the two lines of code that we wrote above. (Here we are assuming that the characters “<=” would mean “less than or equal to” in the language we are using.)

When we passed our data to the section called “switch” we were telling it to use the data that is represented by the symbols “name” and “x”. The “switch” routine then did its thing and changed the data stored at that location. So when x equals one, our program would print:

Mary Jones
Jones Mary

As the value of x is incremented it will do the same thing with the other names in our list.

But now, what if another program wanted to use our little “switch” procedure and, instead of using the symbols “name” and “x” to stand for data, it uses the symbols “person” and “number”? It will make no difference to our “switch” routine. It would still split the data into two parts then reverse them. The data represented by the symbols person(number) in the new program would be exactly the same as the data represented by name(x)in our procedure.

So, do you get it?

When I worked at Corrections Canada I wrote programs that fetched the data about their clients. I did not care what the actual data was—nor did I want to know it—all I had to worry about was the format that the data was stored in so that I could extract whatever part of the entire information field I needed. For example, if my manager told me that a senior official wanted to know how many persons are in federal prisons who were convicted of non-violent crimes and have less than six months left in their sentence, I could write program that would get the data based on that information (non-violent; less than 6 months) and count them. Bingo! Zip! As fast as that I would get an answer that I could email to my manager (who would pass it to his supervisor, who would pass it to his department head, who would send it to the department head where the senior official worked, who would send it …on and on until the information reached its destination.) My job would be in serious jeopardy if I had emailed the answer to the senior official directly.

In any case, the distance I had from the data about our government’s long-term guests was called “Need to Know.” The only thing I needed to know about was the format of the information about the prisoners. I didn’t need to know their names or any of the myriad of personal details that the government stores about its guests. No matter what projects I worked on, whether at a military research lab, a national library, or the prime minister’s office, the format of the data was all I ever needed to know. I really didn’t want to know more than that.

Teaching Programming; Part Four: Finding Things the Computer's Way

There are a few basic concepts one has to get across if students are ever going to understand and work with computer languages. I’ll get to one of the most difficult in another story, but, first, an easy one.

Databases, indexed files, and the like, keep track of where everything is by using indexes, sort of like the index in a book. Keywords are stored separately from the actual data and stored with each keyword is a number; that number being the address of where the complete record is stored. Make sense? You want to find out something about “blogs” so, you look up the word “blog” in the index of a book which lists the numbers of the pages in the book where you will find information about blogs. Computers handle information pretty much the same way. I want the record for “Smith, John” who lives on “King Street” so the computer will look those terms up in its index and come up with the address of the memory location where the complete record that matches those two criteria resides.

Every location in computer memory has an address associated with it. A typical address might look like: 1A78C4DF. If you’ve been following these entries, you should recognize that as a hexadecimal number where each digit represents four bits of information. In other words, 1A78C4DF represents a group of 32 1’s and 0’s, and 32 bits can represent 4,294,967,296 integer numbers—which is why Windows computers based on 32-bit architecture can address a maximum of four Gigabytes of RAM (actually about 3.6 GB). Folks running computers with XP, for example, waste their money if they buy more than 4GB RAM because their machines simply can not address the excess storage.

The point of all this being: each 32-bit piece of information has a unique number associated with it so that the processor can find it when needed. (In reality it has an "offset" number rather than a fixed one.) Having said that: what is the fastest way to locate a number from a list where you know the end points (the lowest and highest numbers)? That’s what a computer has to do when it looks up data for you. It could start at the lowest address and work its way through the list one at a time: is it location 11111111? No? How about location 11111112? And so on. It can take a very long time to find a specific number (or address) using that method. Starting at the other end won’t help. Theoretically if you start at the lowest or highest number and work your way one at a time through the list you might have to actually look at every number in the list. If your list comprises the numbers 1 through 100 and you start at 1, if your target number is 100 you will have to look at 100 numbers to find it. There’s got to be a faster way—and there is.

I would ask a student to write a number from 1 to 100 on a piece of paper and then say I was going to tell her what the number is in seven guesses or less. All she had to do was answer “higher” or “lower” until I got the number. I started at 50. (Let’s assume all her answers were “lower” to make this description easier.) 50, 25, 12, 6, 3, 2, 1: yes. As you can probably tell, all I did was split the difference between the guess number and the closest known number in the direction (higher or lower) that the student told me to go. Let’s say I’m trying to find 36. Here’s how it would go: 50: lower; 25: higher; 37: lower; 31 higher; 34: higher; 35: higher ;36! Seven guesses maximum (less if I had picked 36 instead of 35 as the difference between 34 and 37).

This called a binary search (binary meaning two: you divide the difference between your guess number and the closest known number in the given direction by two.) Computers might be fast but when you compare seven guesses to find a number to potentially one hundred guesses out of a list with 100 items, a binary search comes out ahead almost every time—and computers usually have a lot more than 100 numbers to search when looking for information. A binary search will not always be the fastest way, but, if you do thousands of searches through thousands of numbers, overall, statistically, a binary search is the winner.

Students were usually delighted to discover this trick and I’d give them time to play with it between themselves. Like most tricks, it’s not really a trick at all: it’s applied logic. Data often has implied information associated with it and if you can figure out how to identify that implied information and apply it to your problem you’ve gone a long ways towards becoming a programmer. In this case, each number in our list has a relationship with our target number: It is the target number itself, or it is either higher or lower than it, and by exploiting that information we can make our search for it much quicker and more efficient.

By learning how to teach computers how to solve problems we are sometimes teaching ourselves how to solve our problems in other ways. It’s all in how you look at it.

Teaching Programming; Part Three: A Colossal Blunder

After the Christmas break, I was to teach my class the basics of the C language. Now this was more like it; I had spent the previous three years as a full-time C programmer so I was very comfortable with the language. However, in the first class of the New Year I made my first mistake.

I announced that we were going to teach a computer how to play Poker. I thought this might generate some excitement; instead, I got a number of puzzled looks. No one said anything, so I began by explaining that we first had to break the program down into discrete data and procedures. The logical place to start was to define what a deck of cards is to the computer. So, I wrote the playing card sequence on the board in four columns, each column headed by an initial to indicate the suit. Already I sensed that I had lost some of the class.

A group of women wearing dark clothing were whispering together, so I thought this a logical place to step in and clarify what was going on. I asked them if they had any questions. They shook their heads no and giggled.

I sketched out a two-dimensional array where we would store the deck of cards. I assigned the “2” cards to row 1 and worked my way up to the Aces, which was in row 13. I then wrote the numbers 1 to 4 at the tops of the columns (which I had cleverly arranged in the suit hierarchy of Clubs, Diamonds, Hearts, then Spades, so that Clubs was in the lowest numbered column and Spades the highest.) I now had all the cards in a deck arranged in a two dimensional pattern that happened (by design) to mirror the relative value of the cards and suits. The two of Clubs, the lowest ranking card in a full deck, was in position 1,1. The Ace of Spades occupied the highest value box: 13,4.

I explained that with this array, not only did we know what a deck of cards looked like, from a computer’s point of view, but we had, at the same time, established the relative values of each card in the deck simply by its position in the array (by concatenating the two numbers that made up the array coordinates. So, the two of Clubs would be worth 11 (from position 1,1) and the Ace of Spaces comes out to 134. The values of the cards with face values of 7 would be 61, 62, 63, 64; while the cards with a face value of 9 would be 81, 82, 83, 84 respectively. Thus any card with a face value of 9 was worth more than any card with a face value of 7, but a 9 of spades was worth more than a 9 of diamonds.)

Before going on to define the relative value of different combinations of five cards by assigning values to different configurations, it was time to ensure that they got the concept. So, I asked them to evaluate different pairs of cards and determine which had the higher value. I thought this was pretty straight-forward. All they had to do was find the card by its numeric order and suit on the array, create its “value” by concatenating the coordinates and compare the result to the value of another card. For example: which is worth more: the Ace of Hearts (133) or the Jack of Diamonds (112)? They were hopelessly lost.

I gave them a number of such questions and asked them to work in groups while I went around to see what was going on. I was startled to get questions like, “What’s a J?” and “What’s the difference between spades and clubs?” The light slowly turned on in my head: Arabs do not play cards. Most of them had probably never seen a deck of cards before, let alone become familiar with the values and suits.

Realizing my god-awful mistake and knowing that I had just wasted a good hour of class time as well as probably offended the students who I hadn’t simply baffled, I sent them on a break while I tried to gather my thoughts and figure out a different approach to teaching them the purpose and some of the applications of arrays. I was thankful that the curriculum covered only two-dimensional arrays and not 3-, 4-, 5-, or x-dimensional ones.

Teaching Programming, Part Two: Y2K

Amazing how disappointed people appeared to be when planes did not start dropping out of the skies and governments didn't stop functioning. The Y2K crisis was then dismissed as a big hoax, a fuss about nothing. However, if they had any idea of what was going on in the computer-world world-wide for the previous 18-24 months people wouldn't have been so blasé about it. The reason we averted a major disaster was that computer programmers had been working their butts off to identify and fix as many Y2K bugs as they could uncover. That Windows and its applications were not affected was a matter of good planning, at best, or dumb luck, at worst. And, that's what most people think of when they think of computers. Didn't affect my computer games, so why should I bother? Also, large companies do not admit publicly that they have computer difficulties; it makes clients nervous. Rumors were that at least one major bank had serious problems. Other failures, such as some American slot machines, were regarded as trivial and so were not reported widely in the press.

At Algonquin I had to get across the reason for the bug and how to identify and fix it to a group of students who had virtually no computer experience. To do this I had to teach them the rudiments of an obsolete computer language that was meant to handle applications of a sort they probably had not seen before. COBOL was a business-oriented language. It was good for adding up expenses and tracking inventory and sales. It was a major job in itself to show someone who did not know how to operate a cash register how businesses calculate profit-loss and capital depreciation, let alone how to do it in a language they had never seen before that ran on computers they had almost no experience with. Add to that language and cultural barriers and you have a task almost impossible to complete in about 25 3-hour classes.

But, I tried.

The Y2K bug was really quite simple. Generally speaking, computers track time by incrementing a counter beginning at a given start time. Unix systems start counting seconds beginning at 1 January 1970 00:00:00 UT. Windows NT counts the number of 100-nanosecond ticks since 1 January 1601 00:00:00 UT. (One nanosecond equals one billionth--or .000000001--of a second. For example, light takes 3.33564095 nanoseconds to travel one meter in a vacuum). These numbers are stored in a group of 128 bits--sometimes 256 bits depending on the accuracy needed. There is no problem inherit in that. Where the problem comes in is that programs get the date, or time, from that group of bits and then manipulate it. Different computer languages have different degrees of accuracy. For example, COBOL can get time from the internal clock to the second while Java Script can fetch time to the nanosecond.

Often programmers want to display the year as a two-digit number. This works okay when you are in mid-century, but, when you get to both ends of a century you are going to run into dates that in are another century--and that is going to cause problems. To make matter worse, sometimes those 2-digit numbers are used mathematically to calculate things like the number of days between events or to forecast a future date (eg., what's the date 90 days from now?) As a result, some computers could not tell the difference between 1900 and 2000; others insisted that the year after 1999 was 19100. Some programs calculated that the year two years before the year "01" is a negative number(01 - 2 = -01). Where these kinds of errors occur, systems crash, or display bizarre results in place of the year. (For example, instead of “00” for 2000 you might see “A&” or some other random collections of characters.)

There were two main methods that programmers used to reduce the year to a two-digit number. One way was simple to subtract 1900 from the date so that
1998 - 1900 = 98. However, if you subtract 1900 from 2001 you get 101 which takes us right back to the original problem. The computer might display this as “10” (taking the first two digits) or as random characters.

Another way was to turn the year into a character string and simply chop off the first two characters. However, you can’t do arithmetic with characters, so this method was not very popular. The solution I used for one Y2K contract I had was to do the arithmetic needed using a four-digit year and, only as the final step when the year was to be displayed, did I switch it into four characters then truncate the first two characters.

These were generally the kinds of solutions employed. To make the field reserved for the year four places could easily be done in new programs as it was just a policy decision. Older programs required a great deal more effort and programmer hours. An ingenious solution was to change six-digit dates (11/31/87) from three two-digit fields to two three-digit fields by converting the month and day to a three digit ordinal number (so 02/28 (February 28th) became 059) and the year could expand to three places, sometimes by dropping the second digit (making 1998 into “198” and 2001 into “201”). Another way of fixing it was to retain the year as two digits and include the century as a separate piece of information that could be fetched when arithmetic calculations were required.

If your head is spinning by now imagine that you are hearing this in a language you barely comprehend, from someone who looks and acts oddly to you (he expects females to address him directly and he wears pants instead of robes). Add to that the unfamiliar setting, unfamiliar weather, strange foods and customs, and you might be able to comprehend what my students were struggling with.

However, we stuck with it and most of them passed the course, though I wouldn’t have hired any of them to actually do any work on the problem.

Teaching Programming, Part One

In 1998-99 I took on teaching an evening course at Algonquin Community College. Algonquin has a huge campus; it grants diplomas in a very wide variety of fields, mostly work-related. In other words, it's not where you would go if you want to study the humanities. In the fall term I was to teach an introduction to programming in COBOL, with emphasis on the Y2K problem. COBOL was obsolete, but a lot of businesses were still dependent on code written in that language so there was a perceived need for people who could read and fix COBOL programs. In the spring term I taught an introduction to C.

My students were mainly in their 20's. Very many were fairly recent immigrants. I had a largish group from Latin America who worked together, but the overwhelming majority were from mid-eastern countries. The women wore heavy multi-layered dark clothing, and some wore hijabs. The men wore western dress, usually jeans and open-necked buttoned shirts. Almost all sported mustaches.

My previous teaching experience had been in secondary school teaching adolescents. Though many were Algonquin, they were all western (Christianized) in attitude. I had also taught seminars and short courses to people in the high tech field. No culture clash there. But, I was totally unprepared for some of the issues I faced with adults from the mid-East.

First, many regarded it as rude for me to address a female directly. If I did direct a question at a woman, she would hide her face and her companions would start giggling. I would not get an answer. Males, on the other hand, were hyper sensitive about perceived challenges to their masculinity. I had to tread as carefully with them as I had to with 14-year-old boys when I taught in high school.

Generally, we got along. I tried to adjust to their expectations and eventually most seemed to realize that I was not there to convert or harass them in any way. But, I did have a serious problem: none of them were prepared for the concepts that I had to impart. Computer languages are not just sets of keywords and rules; they have a syntax and a style unique to each language. When approaching problems in programming you need to be able to see the solution broken down into a step-by-step process with nothing taken for granted.

How computers recognize data is another problem. Bits are either on (set) or off (clear). Bits are grouped together so that eight of them in a row make up a byte; bytes make up words, longwords, integers, boolean expressions, floating point numbers, and so on. Early personal computers could process four or eight bits simultaneously. Then 16-bit computers reigned for a long time before 32-bit processors became standard. Now personal computers that can process 64 bits at a time are available. Every increase like this represents an exponential increase in computer power and speed. You have to know this and understand it so that your programs will make sense and work.

Getting people to see the difference between the number 1234 and the characters 1234 is often difficult. Numbers can be manipulated mathematically; characters cannot. Human don't care whether an integer they see on a page is a character or a number because they can treat it either way as needed. Computers cannot do this. It has to be a number or a character and the computer cannot simply switch from viewing it one way to another as needed. We have to do it for them. To complicate matters the basic operations of computers are done in powers of 2, as pointed out above. The most efficient way to treat numbers, then, from the computer's point of view, is as hexadecimal numbers. What this means is that a number that can be represented in four bits can be precessed as a unit. Two four-bit units can make up a two-digit number and can be expressed in a byte--which a computer can process in one step.

Look at it this way: we can use only 0's and 1's to represent all data. So, when we count, using only 1's and 0's we get: 0, 1, 10, 11, 100, 101, 110, 111, and so on. We call this series, binary numbers ("binary" meaning two, because we have only two digits to work with.) By "counting" in the numbers we usually use (decimals), we can determine that binary 10 equals decimal 2, and binary 111 equals decimal 7. (It helps avoid confusion if we do not name binary numbers as if they were decimal. We name them: one, one-zero, one-one, and so on.) If we use four bits the highest number we can make is 1111, or 15, (in decimal counting.) So, with four bits we can make numbers from 0 to 15. If we go above 15 we are going to have to start adding more bits. So, 16 = 10000 (made up of five bits). As said above, the most efficient way for the computer is to see numbers as groups of 4-bits.

Here's how we count in hexadecimal: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F.
"F," as we should be able to figure out by now, is the same as our decimal number 15. What happens when we want to represent 16 as a hexadecimal number? We do the same thing as we do with decimal numbers: we increment the "10" position. So, decimal 16 in hex is 10. Now we can count using the same method: 11, 12, 13, 14, 15, 16, 17, 18, 19, 1A, 1B, 1C, 1D, 1E, 1F. 1F (which in binary is: 11111) is equivalent to the decimal number 31. The next number after 1F would be 20, and so on. Now, by using two groups of four bits, we can make an 8-bit number which is 11111111 in binary, FF in hex, and 255 in decimal.

Now we have arrived at a situation where any combination of two hexadecimal "digits" can be used to represent 256 possibilities (counting 0). All characters of the alphabet, upper and lower case, as well as digits and punctuation can easily be represented by 256 2-digit numbers (from 00 to FF). Or, we can use 2-digit hexadecimal numbers to represent 256 colours. We can use a pair of such numbers to represent coordinates on a screen (giving us 65536 unique points, from 0,0 to FF,FF). If you worked with early personal computers you probably saw a lot of 2-place hexadecimal numbers.

This is just one example of the kinds of changes I had to make in the students' thinking and processing. When they saw something like A1, they had to see it as a number (decimal 161). If they saw 1554 they had to determine whether it was a group of characters or a decimal number (or, possibly, a hexadecimal number). They had to recognize immediately that a set bit (value 1) always represents on, yes, true; while a clear bit (value 0) represents off, no, false. They had to get used to ideas like "flipping bits" (changing a 1 to a 0 or a 0 to a 1). They had to understand basic logic sentences like "If a then b else c." and "If not a then b = c."

I had thirteen weeks, two evenings a week, to not only get all this firmly in their minds, but to teach them enough COBOL that they would be able to recognize dates that were going to cause problems when the clocks went from 1999 to 2000 and how to fix them.

What Does an Analyst Do All Day?

When I first took over the position of project manager with a team of programmers answering to me, there was a "Bring Your Kid to Work Day." So, I took my eldest with me thinking, erroneously, that he might see first-hand what his old man did for a living. At the end of the day he concluded that I did "nothing" at work and he wasn't shy about letting everyone he knew know that.

Well, I received a pretty good salary for "doing nothing." In a sense, "doing nothing" was a sign of success; it meant I had no crises to attend to and it happened there were no client or status meetings scheduled for that day. However, I was "working;" it's just that he didn't see my seemingly casual conversations with different folks around the office, both with my employees and with other analysts, as "work." He didn't notice that we were discussing programming strategies, working out solutions to non-urgent problems, and the way that all the various parts of application development fit together. There is a lot more involved than someone sitting at a computer writing lines of code.

First, there's a needs analysis. There's no point in spending hundreds of hours developing something if it already exists, or, if it is not commercially viable. "Commercially viable" does not mean a top seller. What we did was business-to-business sales. In other words, we had to have a client or two who was willing to pay for the product before a single line of code was written. Cost-benefits calculations also worked into the development plan. You could not afford to take on a project that would involve potentially thousands of man-hours if the potential revenues did not at least cover your salaries and other expenses.

Preliminary technical outlines of the final solution were needed by the sales folks because they would have to sell something that did not yet exist. They would also need comparisons to similar products, or products in a related field, to demonstrate where your proposal was superior and worth the effort. Analysts were required for all this work. In theory nothing should be done in the area of development until a signed contract was in hand, but, in reality the preliminary work had to be well underway by that point.

Both general and specific goals of the product were needed as they would get turned into, or were driven by, "deliverables." That is, the description of the end-result that the buyer would have to "accept" before the project could be called complete and final payment was due. The deliverables were also worked into the quality acceptance plan that would be developed. Quality acceptance meant careful definitions of what, exactly, the program was to produce and within what guidelines or parameters. Tests would be developed to measure this. This is called "scoping" the project. One thing you wanted to avoid was some programmer getting a bright idea and spending time on a feature that the purchaser did not ask for.

The actual plan of approach to the writing of the code needed to have a general outline with delivery dates (that is, when various parts were supposed to be done to the purchaser's satisfaction) and detailed plans so that the various programmers assigned to the job knew precisely what was expected of them. Someone had to oversee their work and keep them on track, making adjustments to assignments and personnel as required. That was my job in this case. So, chats with the head of quality control, with the database designers, with the writers responsible for delivering documentation, with project directors, with sales folks, with client liaison, with networking and operations (the back-room folks), and with senior management were all part of the job. Just keeping in touch to insure that everything was in sync and on schedule.

I got my position by default. I had quickly become the top C programmer on the project as well as a specialist in other odd parts, such as a routine that had to be written in LISP (an obscure and difficult language to work with). However, the original project manager proved to be not up to the job and he was demoted to managing the programming team. He couldn't handle that job either. There was no one left in the organization who was specifically qualified, so I just started doing the job. Then the president made my position official and I was given hiring and firing rights. The first person I fired was my former boss. I then set about building a team of highly skilled programmers.

I had a scheduled meeting with my entire team once a week. That's where I'd pass on any news from higher up in the company, discuss points of company policy, and go over any topics of general interest to the team. We'd then go around the table and each member would describe what they had accomplished in the past week and were encouraged to bring any problems they had encountered to the table so the rest of us could make suggestions or offer solutions. I'd also go over the plans for the next week and lay out what was expected from each member. I would make sure that they agreed with their job assignments. I did not want an unhappy, grumbling, and backstabbing team. I think I succeeded in that regard.

I would meet with the company president once a week to bring him up to speed on what was going on in my area. Also at that meeting was the fellow responsible for co-ordinating everything with the client. However, outside of that meeting he never had anything to pass on to me. He spent almost his entire day, every day, on the phone with client reps going over their requirements. It would have been very helpful if he had delivered information from the client to me, but, he never did. So, in the meeting with the president he would go over the charts he had prepared, and then I would report on what was actually going on. It might as well have been two different meetings, as his charts had nothing to do with what we actually did. I did not deliver charts--I verbally reported on what was happening in a conversational mode.

So, that's what I did in one "analyst" position. I also worked as a performance and capacity planning analyst for clients during my career and that had very different job requirements. Ironically, I spent my last five years in a "support" role, which, in a sense, was a demotion for me, but, it was also the job that made the least demands on me and paid the highest income I'd received during my career. I had started in the field as an operator in 1983 making $11,000 a year and wound up in 2004 with a fairly good reputation and an annual income of well over $120,000--in other words, more than 10 times what I had started out with. Not bad for a job where I did "nothing."

Morning Prayers

My first contract as a self-employed consultant was to begin when the Ice Storm of 1998 struck. It was just as well I hadn't had my contract approved yet and so could not begin work until the proper committee met because I needed to be home during the next two weeks during long periods with no power, no telephone, and three children unable to attend school. My wife, meanwhile, braved the bizarre weather and highly dangerous road conditions to drive to work, combined to and fro, up to four hours each day. She did not miss a day at the office, unlike probably hundreds of thousands of others.

After two weeks, the worst of the storm damage cleaned up, I was due to start at Transport Canada at 7:30 on the Monday morning. Despite my best efforts, I did not arrive at the downtown office tower until 7:35. The meeting I was to attend was already in progress and I was told that I could not interrupt it. I waited. The meeting ended some twenty minutes later and the attendees filtered out. My contract manager was a ferocious-looking fellow with blazing red hair and huge mustache. He always seemed to be angry about something. When he saw me he said, "Perhaps I didn't make myself clear. You are to be here at 7:30, not 7:31."

I mumbled apologies and set about learning about my new position. Being self-employed I had a lot of overhead expenses that employees do not have. For example, I had to pay both the employee contribution and the employer contribution to the federal CPP plan. I had to pay the Goods and Service Tax (GST) on any money I made. The really expensive part that employees probably never think about was that I had to buy my own medical and dental coverage. Individual plans are far more expensive than the group plans that companies get. After income taxes (which I also had to pay out of my income--I had no such thing as a kindly employer holding back funds to cover that sort of thing), medical insurance was my biggest expense. Though Canada has Medicare, it does not cover subscription drugs, eye glasses, dental work, and other expensive procedures. I also paid for long-term disability insurance (which resulted in a long legal battle a few years later that I sort-of won. In other words, I never got the coverage I paid for, but I did get a sizable settlement from the insurance company.)

For the rest of my self-employed career I heard grumblings from regular employees that I worked alongside about how much money I was making. But, when you got right down to it, after all my expenses, I had about the same after-tax income that they did. Add to that that I was not entitled to Employment Insurance which meant that when my contract ended I had no government-backed income to depend on. And, self-employment contracts were much, much easier to break than employment contracts. At any moment I could be told, "Sorry, Ron, but we don't need your services any more. Please clear out your desk." I had no union or government regulations to protect me from arbitrary decisions. Oh, and I had to be very careful about how I ran my business so that Revenue Canada never got the idea in their heads that I was really an employee in disguise (which it did in a major crackdown after I was out of the business--a move that cost both sides of the contracts a lot of money.)

So, the title of this piece Morning Prayers could refer to the insecurity of being self-employed, but it really refers to the meetings at 7:30 at Transport Canada. At that meeting representatives from the department across Eastern Canada "met" in a conference call to discuss any technical problems encountered the day before. It was called "Morning Prayers" because one (especially me) had to be prepared to discuss any event in detail covering a multitude of related issues. If you weren't fully versed in all aspects of the problem, you had better have said your prayers.

That, in a nutshell, was my job: to know in detail about any technical problems and issues and what was being done to rectify them and ensure that they did not repeat. I spent my day getting ready for the next morning's meeting. To do this, I was to monitor two independent IBM systems watching for new problems, and, progress on existing problems. One system was for the (contracted out) engineers who filled a warehouse on King Edward Avenue; and the other system was for Transport Canada's employees. I had to ensure the two systems were in sync--except I could not make entries in the King Edward system. I had to get the engineers to make the appropriate entries in their system first so that I could then update the Transport Canada system. (I was told that someone previously in my position had been fired on the spot for violating that protocol.)

In other words, my job was to nag. Engineers generally hate having to put things down in writing--and, fair enough--sometimes they are just too busy. Though I became friendly with many of them, they still did not look forward to hearing my voice on the phone because it meant that they would have to do something they hated to do. I tried to keep everything light and casual, but I had enormous pressure behind me. Transport Canada was one of those government departments that ran on rules and procedures. I rarely heard a friendly word on the floor I worked on; employees were overly-stressed from watching their backs and worrying about inadvertently crossing a line somewhere. An old friend of mine worked in the same floor in a different section. I was told firmly that I was to have nothing to do with him during working hours because his section somehow rivaled the section I was in. (There were other issues with my friend, but they aren't relevant to this story.)

However, once the Prayer meeting was over and the problem-tracking systems were up-to-date no one cared what I did (unless it was to chat with my friend). By 3:00 in the afternoon, a lot of employees had disappeared for the day. Fair enough, as they had to be at work before 7:30 am. I soon learned that I had time to join the gym in the hotel next door and work out for an hour in the early afternoon (by calling it my "lunch break"), and, after showering and changing, return to my desk to check the postings on the two tracking systems. If everything was okay, bugger off.

I hated the job. First, I do not like the kind of pressure I faced where I was responsible for what someone else was doing but I had no control over them. Secondly, I had to use IBM systems which I had never been comfortable with. Everything to do with IBM was overly-complex and obscure and intuition simply did not work. Third, I missed programming. In my previous position I had been a full-time C programmer for almost two years before being appointed Project Manager (with programmers reporting to me) for six wonderful months before the company self-destructed. I kept in touch with my team and held meetings with them in a coffee shop, listening sympathetically and giving them advice as if I were still their boss. I guess you could say that I was grieving after finally making it to a dream position.

So, when an opportunity to came along for a contracted systems analyst with programming skills in a non-IBM environment, I jumped at it. Six weeks had been more than enough Morning Prayer Meetings for me.