Did that title catch your attention? I hope so.
Writing computer code requires a certain perspective, a way of looking at problems that most are not used to. I taught a programming course evenings at a local community college and the biggest problem I faced was getting students to look at problems from a computer's point of view. It is, as I have pointed out earlier, a very elementary system of breaking problems down into component parts. To have a computer print the numbers from 1 to 10 involves several steps. You need to set aside a memory location, you need to initialize the value at that memory location, you need to loop in order to increment the value at that location, you need to tell the computer to print out the result of that incrementation, you need a condition to tell the computer when to stop looping.
In 1993 I was assigned to a two-day project at the Department of External Affairs. I had been reading about a developing scandal in the department for the past few weeks, but hadn't paid particular attention. Numbers of between $200,000,000 and $300,000,000 were being tossed around.The COSICS project at External Affairs was supposed to be a world-wide network designed to link all of Canada's foreign embassies and consulates into one secure communications system. I was asked to take a look at the state of the acceptance testing and to make recommendations for improving the performance of the system.
Security at External Affairs is serious business. I've worked in a Department of Defense Research Facility and the Prime Minister's Office, but had never experienced anything like this. I was met by a security officer who accompanied me throughout the day. At no time was I allowed to get close enough to a computer keyboard to be able to reach out and press a few keys. Instead I had to dictate what to type to a security officer and have him read me back the results. An interesting way to analyze a system. Still it took only a few minutes to identify a problem program. I don't recall the name of the program but its purpose was to scan all mailboxes on the system looking for the keyword; ***Urgent***. On finding it, it was supposed to send an alert to the intended recipient of the message, using a technique that would flash the message on the user's screen. So, let's call it "Sentry." It took almost five minutes to complete its task and then it would start over again--and while it ran the system slowed to a crawl. We've seen this before. It had a raised priority so that it had almost exclusive use of the CPU. Users got whatever free cycles the program didn't need. Interesting....
The security agent couldn't answer any questions about the program. It was his job to type and read back the results.
I was then led to a room where I was given a pile of computer printouts. This represented all of the acceptance testing to date. Acceptance testing is something like a game. The buyer of the system or program has a number of specified targets the product was supposed to meet, and both the buyer and the vendor agree before-hand on specific tests to be run to demonstrate that the system meets those requirements. I have seen this system abused and manipulated, but, such did not appear to be the case here. The thing that jumped out at me as I read through pages after pages of boring tables was that part way through the testing cycle the vendor had made significant changes to the core program. I looked back. Sure enough, those changes would impact earlier tests and change the results. There was no doubt in my mind that the testing process had to be started over.
An External Affairs contractor then took me to the offices of the engineering company who had designed and was implementing the system. You can read all about this company in StevieCamerons' book: On the take: crime, corruption, and greed in the Mulroney years (Random House Canada). I was not at all interested in the sales and analysts explanations that the money spent so far was intended only for half the project, not the entire thing as the press was claiming. It would take another $200,000,000 - $250,000,000 to actually finish the project, but this had been the plan all along. (You might want to go back and read my Amicus article on contract completion and acceptance.) That was all irrelevant to me. I wanted to talk to whoever was responsible for that "Sentry" program. Finally, I was introduced to a very nervous young man, still in his 20's, wearing blue jeans. That's what I wanted.
First of all, I asked him why his program was running at an elevated priority. Did he not realize that this was killing the overall system's performance? His stammered answer was that the contract said that no more than five minutes could pass between the reception of the urgent message and the warning issued to the target's computer screen. Okay, how does the program work?
He had taken the approach of opening each account's email directory and reading through it, one character at a time, looking for ***Urgent***. All email accounts? I asked. "Yes." "Why?" In case there's an urgent message. What if the account has not been used for months? Are you still opening it? He blushed at this point and said he had not taken that into account.
I gave him a hard long look. He was working for a competitor. How much help should I give him? I offered a token. Why did you write this in DCL? (DCL was a scripting interpretive language--which meant that every line of code had to be translated into machine language by the CPU before it could process it; compiled languages, on the other hand, were translated to machine language at the time of their creation, significantly speeding up their execution. Generally speaking, production applications are always written in compiled languages.)
He said he was in the process of re-writing the program, hoping to speed it up.
I wrote up my report the next day and handed it to the project head. First, I told him that the acceptance testing had to be restarted because the core program had been changed. I like the way this guy thinks, he said to a colleague at the meeting. What about performance? he asked. I handed him a page long list of bullet points indicating methods for getting better performance. Uppermost was getting rid of that Sentry program. As long as it was run in the form it was in, they could never achieve a satisfactory level of performance.
What about the requirement for urgent email handling? I was asked. It can be done, I answered, but not with the approach that the programmer is using. I was thanked, hand-shakes all-round and a security agent escorted me from the building. A few months later I read that the entire project had been scrapped.
What I didn't tell the young programmer was that he was thinking backwards. Look at the problem: you want to know when a message with the string ***Urgent*** in it has been received by the system. Then why on earth would you go off looking at messages that have already been received? Especially, why read the same messages, character by character, on each cycle of the program? You already know that they are of no interest. Are you that insecure that you need to verify that no old messages contain that string every five minutes? That's the sort of clarity of thinking that programmers need--and which is very difficult to teach. You have to understand what the computer is doing.
If |I want to know when an email that contains a certain string of characters arrives at the system, then I want to look at each message at the time of its arrival. There is absolutely no requirement for the monitoring program to be cycling every five minutes. It should simply wait--and when a message arrives, it wakes and does its job. I wrote that type of code fairly often so I know that it works; the process has a name: Asynchronous System Trap (AST.)The Sentry program if written in C (and I would have preferred Assembler for this task) would wake, check for the magic string, split off another process to send the message to the user. A dozen lines of code executed in microseconds and only when needed.
Maybe the overall system would have worked in the end if there were experienced analysts and programmers working on it, or maybe it never would have worked. It is hard to know at this stage. International communications, especially secure ones, were just being developed in the early 1990's--this after a decade of out-standing scandals involving computer hackers waltzing through supposedly secure military systems. But thinking backwards will bring you down every time.
No comments:
Post a Comment