In the early 1990’s I was sent to a military research station to see if I could help out with a performance issue they were having. The site consisted of several buildings surrounded by a very high steel-link fence (fifteen feet?) and you entered by stopping at a barrier; a commissionair (the government’s standard security; actually retired members of the military) in a glassed-in cage would ask to see your identity card. After my first few days I just held my ID card up to the car window and the guard waved me through. I quickly learned that the base was off-limits to all non-security personnel between the hours of 6:00 pm and 6:00 am. No overtime there.
After I settled in I took a look at the system—which was running very slowly indeed. It didn’t take long to spot a batch job running at an elevated priority; high enough that it was blocking interactive users from accessing the system. The batch job was the daily backup procedure. Two questions: 1) why was the backup running during prime time? and 2) why was its priority raised?
The operator responsible told me it took two 9-track reel-to-reel tapes to perform a complete backup. He arrived at work at 6:00 am to start it, but, as each tape took about two hours, backup time ran into work time. He raised the priority of the job so that it would execute faster.
My first response: it should not take a 9-track tape two hours to fill with backed-up files. Raising the priority of that job was counter-productive because a job cannot run faster than the device it is writing to. Tape drives are slow, but not that slow. Lowering the priority of the job to a more reasonable level and increasing the buffer size, so that it wrote more data at a single time, cut the execution time down closer to an hour per tape.
My second response: why are you waiting until 6:00 am to start the backup? Why not mount a tape before leaving the previous day and have the job start executing at, say, 4:00 am. That way, by the time humans were allowed on-site, at 6:00 am, the job would be waiting for its second tape mount—and that should take less than an hour, thus finishing the daily backups by 7:00 am well before any scientists arrived to start their daily shift.
Both steps were simple common-sense solutions. I sometimes irritated people I worked with by asking: “why are you doing it that way? Why not…” It didn’t matter where I worked or in what field, I usually saw a more efficient way of doing something and ran into defensiveness and resentment when I pointed it out. The only way I was ever truly happy at work was when I was either the “expert” who they had to listen to, or the boss (who they also had to listen to). That way I could implement what, to me, looked like obvious improvements that apparently weren’t so obvious to others. The times when I was in the kind of position where I could make improvements in the work-flow without running into snarling and snapping mother wolves protecting the old ways, I always received praise and complements from higher ups, the reason being that my improvements were genuine improvements: just like resetting job parameters to speed up writing to tape and starting the job before humans arrived to mount the second tape.
They kept me around at the research facility for a while; first, writing documentation, then, when that was done, writing a program that would automatically take the data stored on old 9-track tapes and write it to cassette tapes. Three 9-tracks could fit onto one cassette and cassettes required a lot less storage space. That was fun. I got to explore the actual physical and logical structures that controlled tapes because I had to over-ride them to write three tapes onto one. Of course the program kept track of everything and printed out labels to put on the cassette covers listing the contents.
While I was working on that the system suddenly slowed down one afternoon. What the? I identified an interactive session that was running at an elevated priority. Even though I could do anything I wanted to on a system, having full access to everything, I would never consider raising the priority of my session except in a few exceptional emergency situations. I knocked the priority of the run-away session back to where it was supposed to be. Ten minutes later, there it was, running at a high priority again. Time for a closer look. The account in question was assigned all system privileges. Impossible: I was the only one who was supposed to have such access. Even operators were restricted in what they could do. I fixed the account by stripping away all the privileges that a common user had no right to. End of problem.
Until a few days later. There was that same account running at an elevated priority, locking everyone else out of the system. And, the account once again had full system access. There were only two ways this could have happened: either my—or the system account that was reserved for upgrades—had been broken into; or, someone had unauthorized access to the computer room. It was the latter. In those primitive days of computing we did not have monitors and keyboards connected directly to the computer in the computer room, we had teletype machines. In other words, every command and response was printed on fan-fold paper. That system solved a few mysteries during my career.
Because the console was wired directly into the machine it had access to everything. That was intentional so that a technician could work on a failed machine. Someone, presumably the owner of the account, had entered the computer room and typed the commands necessary to give his account full system access. Now most computer rooms were locked with a card-scanner or a cipher lock so that only those who needed to be there could actually enter the room. You would think that at a secure military research site that this basic security feature would be in place. It was not. The computer room door was left wide-open at all hours.
I reported the incident to the security team who immediately had the computer room locked. The employee was given a stern dressing-down and warned. I heard from the people working at the site a few months after I had moved on that the culprit had been caught trying his old trick again and had been fired on the spot. So, what had he gained at the expense of his career? a few minutes of uninterrupted access to a CPU. It wasn’t as if he was doing important work at the time; he had been playing a computer game.
Monday, 5 September 2011
Friday, 2 September 2011
The Commodore-64 and contemporary programming.
I was just starting out in my high tech career in 1982 when the Commodore-64, the first commercially successful home computer, was introduced. It was cheap, at under $600, and a large number of applications and games were produced for it. Though I wanted one, I couldn’t afford it with a young family while transitioning careers. However, the schools were anxious for programs that could be used as teaching aids and were generous about lending them to anyone who could write a few lines of code. I took full advantage of that opportunity and wrote many little games that could be used in the classroom. Eventually, I was given a retired Commodore-PET—a much more limited version of the C-64—and so happily wrote little programs for my boys to play with.
Like the Apple II, released about the same time, the Commodore did not have an operating system as we know the term today. It came with a BASIC interpreter and, by using low-level “poke” and “peek” commands, programmers could access the very limited monitor and sound chip. Magazines, like Byte, regularly printed the hex codes for accessing various functions of the C-64, and printed out entire programs for users to type into their computers. The C-64 did not come with a disk drive; Programs had to be stored on cassette tapes. If you were a fast typist you could input most of an entire program in the time it took for the tape drive to load its contents into RAM.
Though there was a huge gap between the home computer and the serious mainframe computers I used at work, I did learn some valuable programming lessons on the C-64 and PET. Because it was such a primitive and limiting interface programmers had to be constantly aware of what they were putting into RAM. A program would simply run out of room at about 32 KB as the rest of RAM was given over to the BASIC interpreter and some low level functions, such as communicating with the keyboard. In other words, from the programmer’s point of view, every bit counted.
This meant that one had to learn how to write efficient code. However, it reinforced another bad programmer habit: it discouraged the imbedding of comments in the code to explain what was going on because comments took up valuable RAM. There was an entire class of programmers who could write dense complex code that was unintelligible to anyone else. If it broke, you were stuck because chances were good that you would not be able to unravel its secrets without a lot of help from the original programmer.
The monitor was wide enough to display 40 characters and characters were generally 8 by 8 bits. So, we never had reason to count pixels. But, within the limitations of the chunky slow graphics all sorts of small applications could be written. Most of the games I wrote involved a moving graphic, either under control of the user through keyboard commands, or control of inner logic. Given the type of programming limitations, the most effective way to discover where the graphic was in relation to other “objects” on the screen was to “peek” ahead to the next character position. If you had planned correctly you could figure out which was the bat or paddle and which was the edge of the screen. Movement was controlled by incremented or decrementing simple x and y coordinates.
It didn’t take very much code at all to replicate some of the commercially-available games. A “pong” program (bouncing ball with two paddles) could be written in less than 20 lines of code. I wrote a successful copy of the “Breakout” game (a bouncing ball smashes its way through a brick wall of differently-coloured and -valued bricks.) It was fun. But here’s something important that I learned.
I mentioned that to determine where one object on the screen was in relation to other objects on the screen, I could use a “peek” command to see what was coming next. As long as I was “peeking” ahead only one character position at a time, there was no problem. I would “collide” with another object or the screen border and reverse my coordinates to move the ball, or other object, away from the “barrier.” No problem.
But, something odd would happen if, in order to speed up my moving object, I were to increment or decrement by twos—or even some larger number. The program would “work” for a few moments and then the ball would disappear from the screen. After another few moments, the program would freeze, usually with an annoying whiny sound.
The only way out was to “break” into the code and go looking for the problem. The first few times I did this, without really understanding what was going on, I was annoyed to discover that my code had been changed. Sections of lines had been erased and some characters had been turned into unrecognizable junk. I fixed the code, ran it again, same result, same changes in the code.
Hum….
I eventually figured out what was happening. The Commodore, in its primitive state, did no error-checking on its own. Whatever you wrote, it would do—or die trying (which it often did). When I started peeking ahead by more than one character position at a time, there was a good chance that I would miss the actual screen boundary—and my “ball” would keep moving in the direction it had been going. No, the virtual ball would not emerge from the side of the monitor, but it would leave the area of code that described the monitor to the system. There wasn’t a heck of a lot more in RAM in those days, so the ball would “move” through the area of RAM where the program code resided. As it bashed into areas it had no business being, it made changes until the program could no longer function.
Modern programs cannot behave like that. The operating system makes sure that code instructions do not access areas in RAM where they shouldn’t be. The operating system itself is protected. The areas allotted to other users or applications are out of bounds. The program code itself is out of bounds. If a program does increment or decrement outside of the area it is supposed to be confined to, an immediate fatal error is issued bringing the program to a halt. That’s why programmers are supposed to always know where their program is operating and what, exactly, it is doing inside of RAM.
Of course, they usually don’t have a clue. Programming has become automated and the layers of complexity between the program code and the underlying machinery have become so thick that no one can really know. “Code generation” programs abound so that the programmer doesn’t need to know anything about the physical nature of the computer. “Drivers” (which seem to be permanently out-of-date) wrap every piece of hardware in nice thick mushy layers of code so that no metal is seen by the programmer—or user. A result: users aren’t the only ones with no understanding of the physical processes that make their computers function.
Like the Apple II, released about the same time, the Commodore did not have an operating system as we know the term today. It came with a BASIC interpreter and, by using low-level “poke” and “peek” commands, programmers could access the very limited monitor and sound chip. Magazines, like Byte, regularly printed the hex codes for accessing various functions of the C-64, and printed out entire programs for users to type into their computers. The C-64 did not come with a disk drive; Programs had to be stored on cassette tapes. If you were a fast typist you could input most of an entire program in the time it took for the tape drive to load its contents into RAM.
Though there was a huge gap between the home computer and the serious mainframe computers I used at work, I did learn some valuable programming lessons on the C-64 and PET. Because it was such a primitive and limiting interface programmers had to be constantly aware of what they were putting into RAM. A program would simply run out of room at about 32 KB as the rest of RAM was given over to the BASIC interpreter and some low level functions, such as communicating with the keyboard. In other words, from the programmer’s point of view, every bit counted.
This meant that one had to learn how to write efficient code. However, it reinforced another bad programmer habit: it discouraged the imbedding of comments in the code to explain what was going on because comments took up valuable RAM. There was an entire class of programmers who could write dense complex code that was unintelligible to anyone else. If it broke, you were stuck because chances were good that you would not be able to unravel its secrets without a lot of help from the original programmer.
The monitor was wide enough to display 40 characters and characters were generally 8 by 8 bits. So, we never had reason to count pixels. But, within the limitations of the chunky slow graphics all sorts of small applications could be written. Most of the games I wrote involved a moving graphic, either under control of the user through keyboard commands, or control of inner logic. Given the type of programming limitations, the most effective way to discover where the graphic was in relation to other “objects” on the screen was to “peek” ahead to the next character position. If you had planned correctly you could figure out which was the bat or paddle and which was the edge of the screen. Movement was controlled by incremented or decrementing simple x and y coordinates.
It didn’t take very much code at all to replicate some of the commercially-available games. A “pong” program (bouncing ball with two paddles) could be written in less than 20 lines of code. I wrote a successful copy of the “Breakout” game (a bouncing ball smashes its way through a brick wall of differently-coloured and -valued bricks.) It was fun. But here’s something important that I learned.
I mentioned that to determine where one object on the screen was in relation to other objects on the screen, I could use a “peek” command to see what was coming next. As long as I was “peeking” ahead only one character position at a time, there was no problem. I would “collide” with another object or the screen border and reverse my coordinates to move the ball, or other object, away from the “barrier.” No problem.
But, something odd would happen if, in order to speed up my moving object, I were to increment or decrement by twos—or even some larger number. The program would “work” for a few moments and then the ball would disappear from the screen. After another few moments, the program would freeze, usually with an annoying whiny sound.
The only way out was to “break” into the code and go looking for the problem. The first few times I did this, without really understanding what was going on, I was annoyed to discover that my code had been changed. Sections of lines had been erased and some characters had been turned into unrecognizable junk. I fixed the code, ran it again, same result, same changes in the code.
Hum….
I eventually figured out what was happening. The Commodore, in its primitive state, did no error-checking on its own. Whatever you wrote, it would do—or die trying (which it often did). When I started peeking ahead by more than one character position at a time, there was a good chance that I would miss the actual screen boundary—and my “ball” would keep moving in the direction it had been going. No, the virtual ball would not emerge from the side of the monitor, but it would leave the area of code that described the monitor to the system. There wasn’t a heck of a lot more in RAM in those days, so the ball would “move” through the area of RAM where the program code resided. As it bashed into areas it had no business being, it made changes until the program could no longer function.
Modern programs cannot behave like that. The operating system makes sure that code instructions do not access areas in RAM where they shouldn’t be. The operating system itself is protected. The areas allotted to other users or applications are out of bounds. The program code itself is out of bounds. If a program does increment or decrement outside of the area it is supposed to be confined to, an immediate fatal error is issued bringing the program to a halt. That’s why programmers are supposed to always know where their program is operating and what, exactly, it is doing inside of RAM.
Of course, they usually don’t have a clue. Programming has become automated and the layers of complexity between the program code and the underlying machinery have become so thick that no one can really know. “Code generation” programs abound so that the programmer doesn’t need to know anything about the physical nature of the computer. “Drivers” (which seem to be permanently out-of-date) wrap every piece of hardware in nice thick mushy layers of code so that no metal is seen by the programmer—or user. A result: users aren’t the only ones with no understanding of the physical processes that make their computers function.
Saturday, 27 August 2011
Teaching Programming; Part 5: RAMing an F?
Now we get to the fun part of what I had to teach my Algonquin college students about the C programming language. In the last couple of talks I laid out some of the basic principles. I pointed out that we can store a list of names by using symbols to stand in place of the data—and that these symbols act like containers. We used name(x) to get the individual entries in the list, where x is a number describing where in the list the data we want is located.
What are we actually talking about when we talk about data and symbols? Simply this: everything a computer does or works on is located in Random Access Memory (RAM). When we tell a program to start running (by double-clicking on its icon in Windows), the computer has to locate the code for the program on a disk-drive and copy it into RAM. From then on, everything that the program references is an address in RAM. For example, your program is loaded into memory starting at address 435A6B7E. All the symbols in your program can now be interpreted in relation to that first address. When your program needs to read some data, it fetches it from the disk drive and loads it into RAM starting at a different address and tells your program what that address is so that it can find and use it. So, your symbol “name” might be at address 345A6C89 in your program code, but the data it is referencing could begin at address 4563EE12. When we tell our program to work on the data in name(x) we are actually telling it to go to the memory address that name(x) has calculated and start working on the data found there.
Clear as mud, right?
That’s how my student felt, too.
The point is: data and the place (address) in RAM where it is temporarily stored are two different things. It is like telling a window-cleaner to go to 546 Main Street and clean the windows there. Whatever he finds there is not the same thing as the address and he will clean the windows found at that address, not clean the address itself.
We have to keep that straight when programming or else we might inadvertently tell our window-cleaner to go to “windows” and clean the “546 Main Street”—whatever that is. And, this is a very common programming error. We see it whenever a Windows program “crashes” (i.e. stops working abruptly and returns control to the operating system). It has either run into data that it is not prepared to handle (for example, a window-less building where the window-cleaner has been ordered to clean the windows); or it is trying to locate an address in RAM that it has no business accessing (such as where the operating system instructions are stored, or, more commonly, address “0”). Competent programmers are not supposed to mix up what is data and what is an address, but they do. I got caught by that once after I had been writing programs professionally for many years; they can be nasty—and sometimes very subtle—errors to track down and fix.
My students had to know the syntax for fetching an address and the syntax for fetching the data at that address. Sometimes, to further complicate things, we might find another address at the address where we are looking for data—and we have to know the difference so we can instruct our program to go to the second address instead of treating it like ordinary data.
I know it’s difficult to understand when you first run into it. But we worked at it and most students managed to pass the course.
However, one young woman had a rather unique way of handling the situation. I gave a review quiz every second week of class. I would take them home to grade, then return them and go over the answers in the next class. This young woman was consistently sick whenever we had a quiz, but always showed up for the next class where I’d be going over the answers. She would ask for a copy of the quiz so she could follow along. Fair enough. After we had finished our review of the test, she would bring her paper to me, asking me to grade it so she could make sure she had gotten all the information correctly so she could study from it. So, I did. She usually still had a few errors even though we had just discussed the answers.
I thought nothing of it and gave her an F for the course on the grounds that she had not completed any of the course work. (I believe she was also sick for the final exam—a terrible, unpredictable disease that always struck at quiz or exam time.) In any case, after the students had received their grades she phoned me at home to complain. How could I have given her a failing grade? She had all the tests and had “passed” them all. I tried to explain that writing down the answers during the review was not the same thing at all as writing the exams when they were scheduled. She was adamant and phoned me several times, making a pest of herself. Finally I gave her the phone number for the college ombudsman, asked her to take her case to him and said I could not discuss it any further with her.
The ombudsman called and asked me for the story. “Okay, I get it,” he said when I finished. I never heard any more about the case, but I am confident that my F had stood.
What are we actually talking about when we talk about data and symbols? Simply this: everything a computer does or works on is located in Random Access Memory (RAM). When we tell a program to start running (by double-clicking on its icon in Windows), the computer has to locate the code for the program on a disk-drive and copy it into RAM. From then on, everything that the program references is an address in RAM. For example, your program is loaded into memory starting at address 435A6B7E. All the symbols in your program can now be interpreted in relation to that first address. When your program needs to read some data, it fetches it from the disk drive and loads it into RAM starting at a different address and tells your program what that address is so that it can find and use it. So, your symbol “name” might be at address 345A6C89 in your program code, but the data it is referencing could begin at address 4563EE12. When we tell our program to work on the data in name(x) we are actually telling it to go to the memory address that name(x) has calculated and start working on the data found there.
Clear as mud, right?
That’s how my student felt, too.
The point is: data and the place (address) in RAM where it is temporarily stored are two different things. It is like telling a window-cleaner to go to 546 Main Street and clean the windows there. Whatever he finds there is not the same thing as the address and he will clean the windows found at that address, not clean the address itself.
We have to keep that straight when programming or else we might inadvertently tell our window-cleaner to go to “windows” and clean the “546 Main Street”—whatever that is. And, this is a very common programming error. We see it whenever a Windows program “crashes” (i.e. stops working abruptly and returns control to the operating system). It has either run into data that it is not prepared to handle (for example, a window-less building where the window-cleaner has been ordered to clean the windows); or it is trying to locate an address in RAM that it has no business accessing (such as where the operating system instructions are stored, or, more commonly, address “0”). Competent programmers are not supposed to mix up what is data and what is an address, but they do. I got caught by that once after I had been writing programs professionally for many years; they can be nasty—and sometimes very subtle—errors to track down and fix.
My students had to know the syntax for fetching an address and the syntax for fetching the data at that address. Sometimes, to further complicate things, we might find another address at the address where we are looking for data—and we have to know the difference so we can instruct our program to go to the second address instead of treating it like ordinary data.
I know it’s difficult to understand when you first run into it. But we worked at it and most students managed to pass the course.
However, one young woman had a rather unique way of handling the situation. I gave a review quiz every second week of class. I would take them home to grade, then return them and go over the answers in the next class. This young woman was consistently sick whenever we had a quiz, but always showed up for the next class where I’d be going over the answers. She would ask for a copy of the quiz so she could follow along. Fair enough. After we had finished our review of the test, she would bring her paper to me, asking me to grade it so she could make sure she had gotten all the information correctly so she could study from it. So, I did. She usually still had a few errors even though we had just discussed the answers.
I thought nothing of it and gave her an F for the course on the grounds that she had not completed any of the course work. (I believe she was also sick for the final exam—a terrible, unpredictable disease that always struck at quiz or exam time.) In any case, after the students had received their grades she phoned me at home to complain. How could I have given her a failing grade? She had all the tests and had “passed” them all. I tried to explain that writing down the answers during the review was not the same thing at all as writing the exams when they were scheduled. She was adamant and phoned me several times, making a pest of herself. Finally I gave her the phone number for the college ombudsman, asked her to take her case to him and said I could not discuss it any further with her.
The ombudsman called and asked me for the story. “Okay, I get it,” he said when I finished. I never heard any more about the case, but I am confident that my F had stood.
Friday, 26 August 2011
Need to Know
In computer programming letters or words are used as symbols standing for information. For example: the symbol “name” can stand for “Mary Jones,” “Jack Smith,” or “Thomas Mann.” The symbol is “name;” the data is “Mary Jones,” “Jack Smith,” or “Thomas Mann.” Don’t worry yet about how we get there: just think of “name” as a container that can hold any one of the given and family names of our individuals. If I arrange my people in a list then I can refer to them as name(1), name(2), name(3).
Alternatively, the symbol “x” can stand for any number. If I want to count from 1 to 10, I can increment (raise by step) the value of a symbol that stands for a number. A common way of notating that in many languages is: x = x + 1. In other words, you take the number that is represented by “x,” add 1 to it, then store the result back in the symbol “x.”
Putting these two ideas together we can tell the computer to fetch all the names in our list by using the two symbols “name” and “x” as in “name(x)”. We don’t know—or even care—the content of any of our containers. We just know that if we tell the computer to print whatever is in container “name” in row number “x” it will print out one of our names—depending on what the value of x is.
Now you know everything there is to know about computers. Almost.
In the two examples above, our symbols represented actual data, whether it be names or numbers. If you tell the computer “print name(2)” it will not print out literally “name(2)” but will, instead, print out the data represented by those symbols. In other words, it will print “Jack Smith.” A computer can do thousands or even millions of these kinds of data retrievals in a second (though it will take much longer to actually print the list). If you had a list of the name of every person in Canada you could have a computer fetch any name, sort the list by any key you wished, find all the people named “Smith,”—or whatever you can think of doing with such a list—in the blink of an eye.
But, what if you wanted to manipulate the data in another section of a program? For example, you might want to reverse given and family names. So, you would write a routine that would parse (split into parts) the contents of name(x), then put the parts back together in reverse order. Such commands might look something like this:
first, last = parse(name(x),” “);
name(x) = last & first;
(We are assuming that the computer language has some sort of “parse” command built in that will split a string of characters using whatever delimiter (in this case a blank space) you tell it to use. And we are also assuming the language uses the symbol “&” to concatenate two individual pieces data.)
Did you notice what we did? We changed the data stored in our symbols “name,” row “x”. I mentioned that our procedure that did the name reversal was “in another section of a program.” We do this so that we have to write the commands to reverse the names only once. If we call our name swapping section “switch,” for example, then, in the main part of our program all we need to do is tell the computer to go to section “switch” and execute its operations on the data we have selected. And, because we are fetching our data by using a symbol called “name” and the information in “name” is in rows called “x,” we can write a compact little program. Sort of like this:
x = 0;
while (x <= 10)
x = x + 1;
print name(x);
switch (name(x));
print name(x);
end while;
The section we have called “switch” would be along the lines of the two lines of code that we wrote above. (Here we are assuming that the characters “<=” would mean “less than or equal to” in the language we are using.)
When we passed our data to the section called “switch” we were telling it to use the data that is represented by the symbols “name” and “x”. The “switch” routine then did its thing and changed the data stored at that location. So when x equals one, our program would print:
Mary Jones
Jones Mary
As the value of x is incremented it will do the same thing with the other names in our list.
But now, what if another program wanted to use our little “switch” procedure and, instead of using the symbols “name” and “x” to stand for data, it uses the symbols “person” and “number”? It will make no difference to our “switch” routine. It would still split the data into two parts then reverse them. The data represented by the symbols person(number) in the new program would be exactly the same as the data represented by name(x)in our procedure.
So, do you get it?
When I worked at Corrections Canada I wrote programs that fetched the data about their clients. I did not care what the actual data was—nor did I want to know it—all I had to worry about was the format that the data was stored in so that I could extract whatever part of the entire information field I needed. For example, if my manager told me that a senior official wanted to know how many persons are in federal prisons who were convicted of non-violent crimes and have less than six months left in their sentence, I could write program that would get the data based on that information (non-violent; less than 6 months) and count them. Bingo! Zip! As fast as that I would get an answer that I could email to my manager (who would pass it to his supervisor, who would pass it to his department head, who would send it to the department head where the senior official worked, who would send it …on and on until the information reached its destination.) My job would be in serious jeopardy if I had emailed the answer to the senior official directly.
In any case, the distance I had from the data about our government’s long-term guests was called “Need to Know.” The only thing I needed to know about was the format of the information about the prisoners. I didn’t need to know their names or any of the myriad of personal details that the government stores about its guests. No matter what projects I worked on, whether at a military research lab, a national library, or the prime minister’s office, the format of the data was all I ever needed to know. I really didn’t want to know more than that.
Alternatively, the symbol “x” can stand for any number. If I want to count from 1 to 10, I can increment (raise by step) the value of a symbol that stands for a number. A common way of notating that in many languages is: x = x + 1. In other words, you take the number that is represented by “x,” add 1 to it, then store the result back in the symbol “x.”
Putting these two ideas together we can tell the computer to fetch all the names in our list by using the two symbols “name” and “x” as in “name(x)”. We don’t know—or even care—the content of any of our containers. We just know that if we tell the computer to print whatever is in container “name” in row number “x” it will print out one of our names—depending on what the value of x is.
Now you know everything there is to know about computers. Almost.
In the two examples above, our symbols represented actual data, whether it be names or numbers. If you tell the computer “print name(2)” it will not print out literally “name(2)” but will, instead, print out the data represented by those symbols. In other words, it will print “Jack Smith.” A computer can do thousands or even millions of these kinds of data retrievals in a second (though it will take much longer to actually print the list). If you had a list of the name of every person in Canada you could have a computer fetch any name, sort the list by any key you wished, find all the people named “Smith,”—or whatever you can think of doing with such a list—in the blink of an eye.
But, what if you wanted to manipulate the data in another section of a program? For example, you might want to reverse given and family names. So, you would write a routine that would parse (split into parts) the contents of name(x), then put the parts back together in reverse order. Such commands might look something like this:
first, last = parse(name(x),” “);
name(x) = last & first;
(We are assuming that the computer language has some sort of “parse” command built in that will split a string of characters using whatever delimiter (in this case a blank space) you tell it to use. And we are also assuming the language uses the symbol “&” to concatenate two individual pieces data.)
Did you notice what we did? We changed the data stored in our symbols “name,” row “x”. I mentioned that our procedure that did the name reversal was “in another section of a program.” We do this so that we have to write the commands to reverse the names only once. If we call our name swapping section “switch,” for example, then, in the main part of our program all we need to do is tell the computer to go to section “switch” and execute its operations on the data we have selected. And, because we are fetching our data by using a symbol called “name” and the information in “name” is in rows called “x,” we can write a compact little program. Sort of like this:
x = 0;
while (x <= 10)
x = x + 1;
print name(x);
switch (name(x));
print name(x);
end while;
The section we have called “switch” would be along the lines of the two lines of code that we wrote above. (Here we are assuming that the characters “<=” would mean “less than or equal to” in the language we are using.)
When we passed our data to the section called “switch” we were telling it to use the data that is represented by the symbols “name” and “x”. The “switch” routine then did its thing and changed the data stored at that location. So when x equals one, our program would print:
Mary Jones
Jones Mary
As the value of x is incremented it will do the same thing with the other names in our list.
But now, what if another program wanted to use our little “switch” procedure and, instead of using the symbols “name” and “x” to stand for data, it uses the symbols “person” and “number”? It will make no difference to our “switch” routine. It would still split the data into two parts then reverse them. The data represented by the symbols person(number) in the new program would be exactly the same as the data represented by name(x)in our procedure.
So, do you get it?
When I worked at Corrections Canada I wrote programs that fetched the data about their clients. I did not care what the actual data was—nor did I want to know it—all I had to worry about was the format that the data was stored in so that I could extract whatever part of the entire information field I needed. For example, if my manager told me that a senior official wanted to know how many persons are in federal prisons who were convicted of non-violent crimes and have less than six months left in their sentence, I could write program that would get the data based on that information (non-violent; less than 6 months) and count them. Bingo! Zip! As fast as that I would get an answer that I could email to my manager (who would pass it to his supervisor, who would pass it to his department head, who would send it to the department head where the senior official worked, who would send it …on and on until the information reached its destination.) My job would be in serious jeopardy if I had emailed the answer to the senior official directly.
In any case, the distance I had from the data about our government’s long-term guests was called “Need to Know.” The only thing I needed to know about was the format of the information about the prisoners. I didn’t need to know their names or any of the myriad of personal details that the government stores about its guests. No matter what projects I worked on, whether at a military research lab, a national library, or the prime minister’s office, the format of the data was all I ever needed to know. I really didn’t want to know more than that.
Sunday, 21 August 2011
Teaching Programming; Part Four: Finding Things the Computer's Way
There are a few basic concepts one has to get across if students are ever going to understand and work with computer languages. I’ll get to one of the most difficult in another story, but, first, an easy one.
Databases, indexed files, and the like, keep track of where everything is by using indexes, sort of like the index in a book. Keywords are stored separately from the actual data and stored with each keyword is a number; that number being the address of where the complete record is stored. Make sense? You want to find out something about “blogs” so, you look up the word “blog” in the index of a book which lists the numbers of the pages in the book where you will find information about blogs. Computers handle information pretty much the same way. I want the record for “Smith, John” who lives on “King Street” so the computer will look those terms up in its index and come up with the address of the memory location where the complete record that matches those two criteria resides.
Every location in computer memory has an address associated with it. A typical address might look like: 1A78C4DF. If you’ve been following these entries, you should recognize that as a hexadecimal number where each digit represents four bits of information. In other words, 1A78C4DF represents a group of 32 1’s and 0’s, and 32 bits can represent 4,294,967,296 integer numbers—which is why Windows computers based on 32-bit architecture can address a maximum of four Gigabytes of RAM (actually about 3.6 GB). Folks running computers with XP, for example, waste their money if they buy more than 4GB RAM because their machines simply can not address the excess storage.
The point of all this being: each 32-bit piece of information has a unique number associated with it so that the processor can find it when needed. (In reality it has an "offset" number rather than a fixed one.) Having said that: what is the fastest way to locate a number from a list where you know the end points (the lowest and highest numbers)? That’s what a computer has to do when it looks up data for you. It could start at the lowest address and work its way through the list one at a time: is it location 11111111? No? How about location 11111112? And so on. It can take a very long time to find a specific number (or address) using that method. Starting at the other end won’t help. Theoretically if you start at the lowest or highest number and work your way one at a time through the list you might have to actually look at every number in the list. If your list comprises the numbers 1 through 100 and you start at 1, if your target number is 100 you will have to look at 100 numbers to find it. There’s got to be a faster way—and there is.
I would ask a student to write a number from 1 to 100 on a piece of paper and then say I was going to tell her what the number is in seven guesses or less. All she had to do was answer “higher” or “lower” until I got the number. I started at 50. (Let’s assume all her answers were “lower” to make this description easier.) 50, 25, 12, 6, 3, 2, 1: yes. As you can probably tell, all I did was split the difference between the guess number and the closest known number in the direction (higher or lower) that the student told me to go. Let’s say I’m trying to find 36. Here’s how it would go: 50: lower; 25: higher; 37: lower; 31 higher; 34: higher; 35: higher ;36! Seven guesses maximum (less if I had picked 36 instead of 35 as the difference between 34 and 37).
This called a binary search (binary meaning two: you divide the difference between your guess number and the closest known number in the given direction by two.) Computers might be fast but when you compare seven guesses to find a number to potentially one hundred guesses out of a list with 100 items, a binary search comes out ahead almost every time—and computers usually have a lot more than 100 numbers to search when looking for information. A binary search will not always be the fastest way, but, if you do thousands of searches through thousands of numbers, overall, statistically, a binary search is the winner.
Students were usually delighted to discover this trick and I’d give them time to play with it between themselves. Like most tricks, it’s not really a trick at all: it’s applied logic. Data often has implied information associated with it and if you can figure out how to identify that implied information and apply it to your problem you’ve gone a long ways towards becoming a programmer. In this case, each number in our list has a relationship with our target number: It is the target number itself, or it is either higher or lower than it, and by exploiting that information we can make our search for it much quicker and more efficient.
By learning how to teach computers how to solve problems we are sometimes teaching ourselves how to solve our problems in other ways. It’s all in how you look at it.
Databases, indexed files, and the like, keep track of where everything is by using indexes, sort of like the index in a book. Keywords are stored separately from the actual data and stored with each keyword is a number; that number being the address of where the complete record is stored. Make sense? You want to find out something about “blogs” so, you look up the word “blog” in the index of a book which lists the numbers of the pages in the book where you will find information about blogs. Computers handle information pretty much the same way. I want the record for “Smith, John” who lives on “King Street” so the computer will look those terms up in its index and come up with the address of the memory location where the complete record that matches those two criteria resides.
Every location in computer memory has an address associated with it. A typical address might look like: 1A78C4DF. If you’ve been following these entries, you should recognize that as a hexadecimal number where each digit represents four bits of information. In other words, 1A78C4DF represents a group of 32 1’s and 0’s, and 32 bits can represent 4,294,967,296 integer numbers—which is why Windows computers based on 32-bit architecture can address a maximum of four Gigabytes of RAM (actually about 3.6 GB). Folks running computers with XP, for example, waste their money if they buy more than 4GB RAM because their machines simply can not address the excess storage.
The point of all this being: each 32-bit piece of information has a unique number associated with it so that the processor can find it when needed. (In reality it has an "offset" number rather than a fixed one.) Having said that: what is the fastest way to locate a number from a list where you know the end points (the lowest and highest numbers)? That’s what a computer has to do when it looks up data for you. It could start at the lowest address and work its way through the list one at a time: is it location 11111111? No? How about location 11111112? And so on. It can take a very long time to find a specific number (or address) using that method. Starting at the other end won’t help. Theoretically if you start at the lowest or highest number and work your way one at a time through the list you might have to actually look at every number in the list. If your list comprises the numbers 1 through 100 and you start at 1, if your target number is 100 you will have to look at 100 numbers to find it. There’s got to be a faster way—and there is.
I would ask a student to write a number from 1 to 100 on a piece of paper and then say I was going to tell her what the number is in seven guesses or less. All she had to do was answer “higher” or “lower” until I got the number. I started at 50. (Let’s assume all her answers were “lower” to make this description easier.) 50, 25, 12, 6, 3, 2, 1: yes. As you can probably tell, all I did was split the difference between the guess number and the closest known number in the direction (higher or lower) that the student told me to go. Let’s say I’m trying to find 36. Here’s how it would go: 50: lower; 25: higher; 37: lower; 31 higher; 34: higher; 35: higher ;36! Seven guesses maximum (less if I had picked 36 instead of 35 as the difference between 34 and 37).
This called a binary search (binary meaning two: you divide the difference between your guess number and the closest known number in the given direction by two.) Computers might be fast but when you compare seven guesses to find a number to potentially one hundred guesses out of a list with 100 items, a binary search comes out ahead almost every time—and computers usually have a lot more than 100 numbers to search when looking for information. A binary search will not always be the fastest way, but, if you do thousands of searches through thousands of numbers, overall, statistically, a binary search is the winner.
Students were usually delighted to discover this trick and I’d give them time to play with it between themselves. Like most tricks, it’s not really a trick at all: it’s applied logic. Data often has implied information associated with it and if you can figure out how to identify that implied information and apply it to your problem you’ve gone a long ways towards becoming a programmer. In this case, each number in our list has a relationship with our target number: It is the target number itself, or it is either higher or lower than it, and by exploiting that information we can make our search for it much quicker and more efficient.
By learning how to teach computers how to solve problems we are sometimes teaching ourselves how to solve our problems in other ways. It’s all in how you look at it.
Friday, 19 August 2011
Teaching Programming; Part Three: A Colossal Blunder
After the Christmas break, I was to teach my class the basics of the C language. Now this was more like it; I had spent the previous three years as a full-time C programmer so I was very comfortable with the language. However, in the first class of the New Year I made my first mistake.
I announced that we were going to teach a computer how to play Poker. I thought this might generate some excitement; instead, I got a number of puzzled looks. No one said anything, so I began by explaining that we first had to break the program down into discrete data and procedures. The logical place to start was to define what a deck of cards is to the computer. So, I wrote the playing card sequence on the board in four columns, each column headed by an initial to indicate the suit. Already I sensed that I had lost some of the class.
A group of women wearing dark clothing were whispering together, so I thought this a logical place to step in and clarify what was going on. I asked them if they had any questions. They shook their heads no and giggled.
I sketched out a two-dimensional array where we would store the deck of cards. I assigned the “2” cards to row 1 and worked my way up to the Aces, which was in row 13. I then wrote the numbers 1 to 4 at the tops of the columns (which I had cleverly arranged in the suit hierarchy of Clubs, Diamonds, Hearts, then Spades, so that Clubs was in the lowest numbered column and Spades the highest.) I now had all the cards in a deck arranged in a two dimensional pattern that happened (by design) to mirror the relative value of the cards and suits. The two of Clubs, the lowest ranking card in a full deck, was in position 1,1. The Ace of Spades occupied the highest value box: 13,4.
I explained that with this array, not only did we know what a deck of cards looked like, from a computer’s point of view, but we had, at the same time, established the relative values of each card in the deck simply by its position in the array (by concatenating the two numbers that made up the array coordinates. So, the two of Clubs would be worth 11 (from position 1,1) and the Ace of Spaces comes out to 134. The values of the cards with face values of 7 would be 61, 62, 63, 64; while the cards with a face value of 9 would be 81, 82, 83, 84 respectively. Thus any card with a face value of 9 was worth more than any card with a face value of 7, but a 9 of spades was worth more than a 9 of diamonds.)
Before going on to define the relative value of different combinations of five cards by assigning values to different configurations, it was time to ensure that they got the concept. So, I asked them to evaluate different pairs of cards and determine which had the higher value. I thought this was pretty straight-forward. All they had to do was find the card by its numeric order and suit on the array, create its “value” by concatenating the coordinates and compare the result to the value of another card. For example: which is worth more: the Ace of Hearts (133) or the Jack of Diamonds (112)? They were hopelessly lost.
I gave them a number of such questions and asked them to work in groups while I went around to see what was going on. I was startled to get questions like, “What’s a J?” and “What’s the difference between spades and clubs?” The light slowly turned on in my head: Arabs do not play cards. Most of them had probably never seen a deck of cards before, let alone become familiar with the values and suits.
Realizing my god-awful mistake and knowing that I had just wasted a good hour of class time as well as probably offended the students who I hadn’t simply baffled, I sent them on a break while I tried to gather my thoughts and figure out a different approach to teaching them the purpose and some of the applications of arrays. I was thankful that the curriculum covered only two-dimensional arrays and not 3-, 4-, 5-, or x-dimensional ones.
I announced that we were going to teach a computer how to play Poker. I thought this might generate some excitement; instead, I got a number of puzzled looks. No one said anything, so I began by explaining that we first had to break the program down into discrete data and procedures. The logical place to start was to define what a deck of cards is to the computer. So, I wrote the playing card sequence on the board in four columns, each column headed by an initial to indicate the suit. Already I sensed that I had lost some of the class.
A group of women wearing dark clothing were whispering together, so I thought this a logical place to step in and clarify what was going on. I asked them if they had any questions. They shook their heads no and giggled.
I sketched out a two-dimensional array where we would store the deck of cards. I assigned the “2” cards to row 1 and worked my way up to the Aces, which was in row 13. I then wrote the numbers 1 to 4 at the tops of the columns (which I had cleverly arranged in the suit hierarchy of Clubs, Diamonds, Hearts, then Spades, so that Clubs was in the lowest numbered column and Spades the highest.) I now had all the cards in a deck arranged in a two dimensional pattern that happened (by design) to mirror the relative value of the cards and suits. The two of Clubs, the lowest ranking card in a full deck, was in position 1,1. The Ace of Spades occupied the highest value box: 13,4.
I explained that with this array, not only did we know what a deck of cards looked like, from a computer’s point of view, but we had, at the same time, established the relative values of each card in the deck simply by its position in the array (by concatenating the two numbers that made up the array coordinates. So, the two of Clubs would be worth 11 (from position 1,1) and the Ace of Spaces comes out to 134. The values of the cards with face values of 7 would be 61, 62, 63, 64; while the cards with a face value of 9 would be 81, 82, 83, 84 respectively. Thus any card with a face value of 9 was worth more than any card with a face value of 7, but a 9 of spades was worth more than a 9 of diamonds.)
Before going on to define the relative value of different combinations of five cards by assigning values to different configurations, it was time to ensure that they got the concept. So, I asked them to evaluate different pairs of cards and determine which had the higher value. I thought this was pretty straight-forward. All they had to do was find the card by its numeric order and suit on the array, create its “value” by concatenating the coordinates and compare the result to the value of another card. For example: which is worth more: the Ace of Hearts (133) or the Jack of Diamonds (112)? They were hopelessly lost.
I gave them a number of such questions and asked them to work in groups while I went around to see what was going on. I was startled to get questions like, “What’s a J?” and “What’s the difference between spades and clubs?” The light slowly turned on in my head: Arabs do not play cards. Most of them had probably never seen a deck of cards before, let alone become familiar with the values and suits.
Realizing my god-awful mistake and knowing that I had just wasted a good hour of class time as well as probably offended the students who I hadn’t simply baffled, I sent them on a break while I tried to gather my thoughts and figure out a different approach to teaching them the purpose and some of the applications of arrays. I was thankful that the curriculum covered only two-dimensional arrays and not 3-, 4-, 5-, or x-dimensional ones.
Wednesday, 17 August 2011
Teaching Programming, Part Two: Y2K
Amazing how disappointed people appeared to be when planes did not start dropping out of the skies and governments didn't stop functioning. The Y2K crisis was then dismissed as a big hoax, a fuss about nothing. However, if they had any idea of what was going on in the computer-world world-wide for the previous 18-24 months people wouldn't have been so blasé about it. The reason we averted a major disaster was that computer programmers had been working their butts off to identify and fix as many Y2K bugs as they could uncover. That Windows and its applications were not affected was a matter of good planning, at best, or dumb luck, at worst. And, that's what most people think of when they think of computers. Didn't affect my computer games, so why should I bother? Also, large companies do not admit publicly that they have computer difficulties; it makes clients nervous. Rumors were that at least one major bank had serious problems. Other failures, such as some American slot machines, were regarded as trivial and so were not reported widely in the press.
At Algonquin I had to get across the reason for the bug and how to identify and fix it to a group of students who had virtually no computer experience. To do this I had to teach them the rudiments of an obsolete computer language that was meant to handle applications of a sort they probably had not seen before. COBOL was a business-oriented language. It was good for adding up expenses and tracking inventory and sales. It was a major job in itself to show someone who did not know how to operate a cash register how businesses calculate profit-loss and capital depreciation, let alone how to do it in a language they had never seen before that ran on computers they had almost no experience with. Add to that language and cultural barriers and you have a task almost impossible to complete in about 25 3-hour classes.
But, I tried.
The Y2K bug was really quite simple. Generally speaking, computers track time by incrementing a counter beginning at a given start time. Unix systems start counting seconds beginning at 1 January 1970 00:00:00 UT. Windows NT counts the number of 100-nanosecond ticks since 1 January 1601 00:00:00 UT. (One nanosecond equals one billionth--or .000000001--of a second. For example, light takes 3.33564095 nanoseconds to travel one meter in a vacuum). These numbers are stored in a group of 128 bits--sometimes 256 bits depending on the accuracy needed. There is no problem inherit in that. Where the problem comes in is that programs get the date, or time, from that group of bits and then manipulate it. Different computer languages have different degrees of accuracy. For example, COBOL can get time from the internal clock to the second while Java Script can fetch time to the nanosecond.
Often programmers want to display the year as a two-digit number. This works okay when you are in mid-century, but, when you get to both ends of a century you are going to run into dates that in are another century--and that is going to cause problems. To make matter worse, sometimes those 2-digit numbers are used mathematically to calculate things like the number of days between events or to forecast a future date (eg., what's the date 90 days from now?) As a result, some computers could not tell the difference between 1900 and 2000; others insisted that the year after 1999 was 19100. Some programs calculated that the year two years before the year "01" is a negative number(01 - 2 = -01). Where these kinds of errors occur, systems crash, or display bizarre results in place of the year. (For example, instead of “00” for 2000 you might see “A&” or some other random collections of characters.)
There were two main methods that programmers used to reduce the year to a two-digit number. One way was simple to subtract 1900 from the date so that
1998 - 1900 = 98. However, if you subtract 1900 from 2001 you get 101 which takes us right back to the original problem. The computer might display this as “10” (taking the first two digits) or as random characters.
Another way was to turn the year into a character string and simply chop off the first two characters. However, you can’t do arithmetic with characters, so this method was not very popular. The solution I used for one Y2K contract I had was to do the arithmetic needed using a four-digit year and, only as the final step when the year was to be displayed, did I switch it into four characters then truncate the first two characters.
These were generally the kinds of solutions employed. To make the field reserved for the year four places could easily be done in new programs as it was just a policy decision. Older programs required a great deal more effort and programmer hours. An ingenious solution was to change six-digit dates (11/31/87) from three two-digit fields to two three-digit fields by converting the month and day to a three digit ordinal number (so 02/28 (February 28th) became 059) and the year could expand to three places, sometimes by dropping the second digit (making 1998 into “198” and 2001 into “201”). Another way of fixing it was to retain the year as two digits and include the century as a separate piece of information that could be fetched when arithmetic calculations were required.
If your head is spinning by now imagine that you are hearing this in a language you barely comprehend, from someone who looks and acts oddly to you (he expects females to address him directly and he wears pants instead of robes). Add to that the unfamiliar setting, unfamiliar weather, strange foods and customs, and you might be able to comprehend what my students were struggling with.
However, we stuck with it and most of them passed the course, though I wouldn’t have hired any of them to actually do any work on the problem.
At Algonquin I had to get across the reason for the bug and how to identify and fix it to a group of students who had virtually no computer experience. To do this I had to teach them the rudiments of an obsolete computer language that was meant to handle applications of a sort they probably had not seen before. COBOL was a business-oriented language. It was good for adding up expenses and tracking inventory and sales. It was a major job in itself to show someone who did not know how to operate a cash register how businesses calculate profit-loss and capital depreciation, let alone how to do it in a language they had never seen before that ran on computers they had almost no experience with. Add to that language and cultural barriers and you have a task almost impossible to complete in about 25 3-hour classes.
But, I tried.
The Y2K bug was really quite simple. Generally speaking, computers track time by incrementing a counter beginning at a given start time. Unix systems start counting seconds beginning at 1 January 1970 00:00:00 UT. Windows NT counts the number of 100-nanosecond ticks since 1 January 1601 00:00:00 UT. (One nanosecond equals one billionth--or .000000001--of a second. For example, light takes 3.33564095 nanoseconds to travel one meter in a vacuum). These numbers are stored in a group of 128 bits--sometimes 256 bits depending on the accuracy needed. There is no problem inherit in that. Where the problem comes in is that programs get the date, or time, from that group of bits and then manipulate it. Different computer languages have different degrees of accuracy. For example, COBOL can get time from the internal clock to the second while Java Script can fetch time to the nanosecond.
Often programmers want to display the year as a two-digit number. This works okay when you are in mid-century, but, when you get to both ends of a century you are going to run into dates that in are another century--and that is going to cause problems. To make matter worse, sometimes those 2-digit numbers are used mathematically to calculate things like the number of days between events or to forecast a future date (eg., what's the date 90 days from now?) As a result, some computers could not tell the difference between 1900 and 2000; others insisted that the year after 1999 was 19100. Some programs calculated that the year two years before the year "01" is a negative number(01 - 2 = -01). Where these kinds of errors occur, systems crash, or display bizarre results in place of the year. (For example, instead of “00” for 2000 you might see “A&” or some other random collections of characters.)
There were two main methods that programmers used to reduce the year to a two-digit number. One way was simple to subtract 1900 from the date so that
1998 - 1900 = 98. However, if you subtract 1900 from 2001 you get 101 which takes us right back to the original problem. The computer might display this as “10” (taking the first two digits) or as random characters.
Another way was to turn the year into a character string and simply chop off the first two characters. However, you can’t do arithmetic with characters, so this method was not very popular. The solution I used for one Y2K contract I had was to do the arithmetic needed using a four-digit year and, only as the final step when the year was to be displayed, did I switch it into four characters then truncate the first two characters.
These were generally the kinds of solutions employed. To make the field reserved for the year four places could easily be done in new programs as it was just a policy decision. Older programs required a great deal more effort and programmer hours. An ingenious solution was to change six-digit dates (11/31/87) from three two-digit fields to two three-digit fields by converting the month and day to a three digit ordinal number (so 02/28 (February 28th) became 059) and the year could expand to three places, sometimes by dropping the second digit (making 1998 into “198” and 2001 into “201”). Another way of fixing it was to retain the year as two digits and include the century as a separate piece of information that could be fetched when arithmetic calculations were required.
If your head is spinning by now imagine that you are hearing this in a language you barely comprehend, from someone who looks and acts oddly to you (he expects females to address him directly and he wears pants instead of robes). Add to that the unfamiliar setting, unfamiliar weather, strange foods and customs, and you might be able to comprehend what my students were struggling with.
However, we stuck with it and most of them passed the course, though I wouldn’t have hired any of them to actually do any work on the problem.
Subscribe to:
Posts (Atom)