Low Balling

The federal government is supposed to put all requests, with a few exceptions, for materiel and services out to tender. They do this by describing the details in what is called a Request for Proposal (RFP). The RFPs are published and venders submit their bids called Proposals. The process is a formal one overseen by PWGSC.

The RFP lists in detail what is required and expected (called deliverables), listing the both the mandatories and the desirables.  The vender must show in his proposal that he meets all of the mandatory requirements and list as many desireables as he can manage. The proposal also includes cost and delivery times of the deliverables. The idea is that the company or individual who meets all of the mandatories and can produce the deliverables is granted the contract, provided his is the lowest bid of all other proposals that meet the same requirements. In theory. That's the basic idea.

Writing both RFPs and Proposals is an art form practiced extensively in Ottawa. You might want your cousin Joe to land the contract to deliverer cement to one of PWGSC's building projects, but you can't list Joe as a mandatory--or even as a desireable. You have to word your RFP very carefully so that no one other than Joe is likely to be able to meet the requirements, while at the same time giving the appearance of a fair and unbiased request. The same for proposal writing. You might not be able to meet all the mandatories so you have to word your proposal to avoid promising to deliver something that you can't, while giving yourself an out. Typical for large projects is a clause that says the contract may be extended under the same terms. So, if you can't complete the inter-provincial bridge within the time frame of the RFP, you say you can, win the contract, complete as much of the work as  can be done within your means then ask for an extension of the contract. After all, no one wants to be stuck with a half-completed bridge. That's the sort of thing that gets media attention and the last thing any senior public servant or political appointee wants is publicity, unless it is celebrating the project completion. You know, pictures of the department minister wearing a hardhat, shovel in hand, grinning with a .group of company presidents and vice-presidents, none of whom have ever lifted a shovel in their lives.

This practice is a natural setting for what is called low-balling.  Companies deliberately propose the lowest price possible knowing full-well that they cannot meet the delivery requirements with that restriction.  That's why I found the situation I did when I first went to Supply and Services Canada. DEC had proposed a solution and then could not meet the specifications of the RFP because they were counting on selling additional hardware and expensive performance management software--something they could not do for the original price. They were confident that once it became apparent that the system could not carry the load that SSC would be willing to renegotiate. After all, no one wants to be stuck with a half-completed bridge, right? (However, the fact that I could get the system to meet the specifications with a bit of basic knowledge put a roadblock in that plan, if that's what it was.) The same principle applied in the case I related about the translation department at Energy, Mines, and Resources: the proposed sized VAX could never support the 25 or so users it was meant to. I saw this principle being played out again and again in my career, sometimes involving hundreds of millions of dollars.

One such example I see almost daily. I do basic research for an editor of scientific articles. Basically, it involved looking up references to articles to ensure their accuracy. Sometime a book is referenced. For that, my first stop is Amicus, the online catalog of all publications held by the National Library. I was there when Amicus was created.

Prior to the early 1990's the library's catalog had been maintained on an IBM mainframe. Typical of IBM at the time, access to the catalog was difficult and too arcane for the average user. So, the National Library put out an RFP to replace the system with something a little more human-friendly. The contract was won by a consortium of companies (not unusual for large projects) overseen by a large computer consulting company based in Montreal. The proposal was to use VMS on VAXes as the platform. There were many good reasons for this at the time, among them was that VAXes could be linked together to act as one larger machine (a process called clustering) and so scaling, always an important consideration when looking at future growth, was simply a matter of adding more machines to the cluster.

I was brought in about six months into the project, primarily to prepare the operations procedures and to help National Library staff adjust to the new environment. I gave a few seminars on VMS and on programming in that environment and wrote up all the jobs the operators would need to maintain the system. Because I was the only one who could do it, I was given the task of translating all the data on the IBM tapes (in EBCDIC format) into the industry-standard ASCII.  That was the sort of job I loved. I'd creep my way along an IBM tape, translating one record at a time, and whenever I came across something unusual I'd go ask the IBM specialist what on earth that was. The header (the part of the tape where information about the data on the tape is stored) was organized completely differently than any other operating system. He would answer with the IBM sneer, "That's that standard way of..." always stressing the word standard, even though the feature was unique to IBM systems. An aside, one thing that blew me away was that at the end of some records I'd find a string of characters like this: ˆD. I had recognized that as the ASCII translation of the EDCDIC end-of-record mark, but why would I find so many strung together? Didn't make sense. The IBMer condescended to tell me that because IBM tape drives were so fast they might miss a record shorter than 50 bytes; hence the extra end-of-record marks were used to pad short records. (That's what he said!)

In any case, the project was to be ready for release within a year, but, as the deadline approached the developers were nowhere near ready.  Talk among themselves was that they needed another year. The night before the press conference at which the Minister was going to announce the competition of the project I was sitting with a programmer who had a routine that would simulate users; he could adjust it to imitate the activity of any number of users. I was adjusting operating system parameters so that the system could adjust as he bumped up the number of users. The project manager hovered, wringing his hands, watching us. One thing the system had to do was support up to 1,000 simultaneous users. We finally hit that marker after an hour or so of careful adjustments. The project manager was able to breath again.

There must have been a lot of talk and consultation overnight because the next day, instead of the planned announcement of the completion of the project, the Minister presented the outline of a contract extension in a style and manner that suggested that this had been the plan all along. It did take another year. However, no sooner was the project done than the National Library decided that they wanted to replace VMS with Unix. (Sigh! Another story...).

Every time I look up a publication on Amicus I am transported back to that project.

Porn and the Workplace

While I'm still thinking about my experience at Stentor, the topic of porn comes up. Yeah, I know! The Internet was slowly arriving throughout my career. When I started in 1983 computer to computer communication was...rather primitive. Generally all communication was done through lines leased from telephone companies, or, as it was called: hardwired. I took a computer course at Carleton University in 1984 and I was able to log into my account at the university from home via an acoustic coupler. This was a device that plugged into a keyboard and monitor (personal PCs were still extremely rare and expensive at the time). Then a telephone handset was placed in the receptacles on the coupler. By stuffing towels around the area where the headset and coupler met, I was able to communicate at 200 BAUD--with a lot of errors. I'd go to Carleton early before my class and go through my programs. When lines of code would not compile I knew that the coupler had transmitted noise and so I'd delete the line and type it over.

A year or two later, at Energy, Mines, and Resources, we linked one of my VAXes with an IBM mainframe at the University of Ottawa through dedicated telephone lines. From the U of O we could exchange messages  with other computers. Generally, it was a university to university and scientific network. There was also a military equivalent. Interest Groups were starting to form, though they were all text-based. You could subscribe to a group and add your comments to the discussion and receive a file containing all the previous day's comments every morning.

Ten years after my Carleton experience, the Internet was starting to come into its own. PCs had replaced dumb terminals and keyboards in the workplace. There was something called FreeNet in Ottawa--a cost-free Internet portal. Internet Service Providers (ISP) just weren't around. Personal Computers were still expensive. But, between the time I had linked our first VAX with the University of Ottawa's mainframe and I worked at Stentor, an all-text Internet had morphed into an image-based Internet. Internet Porn could now come into its own. PCs in the work place were soon beeping every time a user clicked on a new porn image.  At Stentor, it was an obsession. The system manager, a female Stentor employee, and the contracted system manager (JackSys, remember him?) spent their days together as JackSys downloaded image after image. I could overhear their comments. Things like, Oh look, you can see his thing! and Hey, look at those knockers! Real mature professional workplace behavior.

Of course I looked at it when back at my office--but never at a client site. I stopped abruptly when I realized that everything I typed and saw on my computer could be recalled and reviewed by anyone in the computer room where the communications servers were located. Still, after all those years and experiences, I've seen employees looking at porn when at work as recently as a year ago. To me, cruising porn sites on an employer's computer is like masturbating in public. Just a wee bit uncomfortable as well as unethical.

When I think back on all the ugly things that JackSys did during the time I knew him (and there is much, much more to tell), the one image that comes immediately to mind is of him sitting in a client's office, the middle-aged female employee looking over his shoulder giggling like a six-year-old. And, to top it off, he tried to take credit for the work I did while he was collecting obscene images.

A Security Hole Big Enough to Drive a School Bus Through

In 1995 I was assigned to a short-term contract at Stentor. Stentor was a conglomerate of several Canadian telephone companies working together. It disbanded in 1999. In any case, I was asked to take a look at their security setup and recommend/make changes to secure it.

I nearly choked with disbelief when I logged into their system and started looking around. There simply was no security. Any kid with a modem anywhere in the world could have logged in and wrecked havoc with this highly confidential and sensitive information. Just to make it easier for the kid with the modem, the system account and password were still set to their default. The system account was the master account on VMS systems. Log into it and you could go anywhere and do anything, including messing with the internal setups of the machine. The default password for the system account was widely known. The first thing one was told to do when receiving a new VAX was to change the system password. That didn't take very long for me to fix. Plug hole number one.

There were a number of smaller issues, but the major problem with the system setup was that the entire accountability system was compromised. Anyone with any account and its password could mess up the data files and there would be no way to trace the damage back to an individual. Instead of individual accounts with real people associated with them, there were group accounts. Everyone in the accounting department, for example, used the same username and password when they logged into the system. The same for each of the other departments in the company. I wrote up a report outlining the potential damage and suggested how I would change it. They could still have group accounts, but instead of using a single username-password, every individual would have a unique username-password. They would still have access to the same programs and data as before, but now, if someone deleted a database, they could not shrug and say, I don't know who did it. It could have been anyone.

The head of the computer services department (who seemed to be troubled by my pony-tail, as he referred to it often during our interactions), gave me the go-head to fix the system. I was sensitive to user concerns and so I moved slowly. I met with representatives of each of the group accounts and explained the problem to them and what the remedy was. They agreed. So, I set about writing up a set of programs that would make the necessary changes to the system. I again met with representatives of each of the group accounts, explained the changes and how their users would log in to the system in the future and gave them a date when I would make the change. Everyone was on board.

Then I did a stupid thing, something that no programmer or analyst should ever do: I set the programs up to run overnight when I knew I was not going to be back for a couple of days because I had another contract to attend to. When I did return three days later it was to face a furious group consisting of the head of the computer services department (the one with the problem with pony-tails), the in-house system manager, and a contracted-out system manager who I am going to give a (false) name to because his path crossed with mine several times over the next few years. Let's call him JackSys--that name should stick with you when you encounter it later in my stories. Apparently when my programs ran users panicked. They claimed they couldn't log in (mainly because they were trying to use accounts that no longer existed) and both system managers panicked.  It didn't matter how often I had told everyone that this was going to happen and that from now on users would have to use their real last name as their account name and would be forced to change the password on their first login.

The systems managers spent the entire night trying to restore everything to the way it had been before my changes kicked in. They finally gave up trying to undo the damage and restored the system from backup tapes. The computer services head told me I had two days to make sure that all the damage was undone and to write up a report on the state of their security. And so, I did.

A few days later JackSys phoned and asked me to forward to him an electronic copy of my report. I didn't want to do it. Electronic documents can easily be changed. But, I felt I had no choice. I was in everyone's doghouse and could not depend on anyone to back me up if I refused to cooperate. And so, I did.

Footnote: four years later (in 1999) I was working on contract at Corrections Canada when they took on another VMS professional. This guy knew me, though I did not know him and hadn't met him before. He told me of how he had seen my name on programs over the years and was very impressed with my technical skill. He also told me that he had been contracted to Stentor, some time after my banishment, in order to address their security concerns. He found the same mess I had when I was there. But, in looking around he came across the programs I had written to correct the situation and well as my security report. JackSys was still there and said that he had written the code and the report--his name was on the programs.  However, my new-found friend knew JackSys and that he, JackSys, didn't have the talent or the command of written English evidenced--and, he recognized my style of writing programs. He went to the same head of computer operations that I had reported to and told him that the security situation was absolutely untenable--and if they didn't implement my solution, then they were leaving themselves open to damage and lawsuits. My programs were run and the system secured as I had recommended.

I Screw Up

As I tell these stories, I'm somewhat concerned you might think that I'm arrogant and I'm sneering at others. First: yes, I do have a problem walking the line between self-confidence and arrogance. When things come easily to one that others struggle with it's hard not to feel somewhat...elevated. Secondly, I came across a lot of good, hard-working, and intelligent people during my career, both inside and outside of the public service. It's just that the screw-ups make more interesting stories. So, in an attempt to balance things out, here's a story where I screwed up.

I told the story earlier of how I helped land a multi-million dollar contract  at SSC (Supply and Services Canada, now PWGSC). I had, at that time, been establishing myself as an expert in VMS performance and capacity planning. I was invited to and presented talks at three national conferences on the subject (in Calgary, Edmonton, and Toronto). So, I was getting pretty sure of myself. But this story doesn't involve detailed technical details: I think anyone can understand that spreading the work-load evenly over several devices is a more efficient way to operate. This is true for computers as well. The slowest part of computer performance is communication to and from disk drives. To give you an idea: to find something that is present in RAM (memory) is equivalent to a carpenter picking up his hammer and driving a nail. To find something that is on a disk drive is like being that same carpenter, but, when he reaches for the hammer, it isn't there. He goes home to get it. On the way he stops at a bar, gets arrested for drunkenness, spends the night in jail, gets home the next day to retrieve the hammer and goes back to the work-site to drive the nail. And, if a computer has to go to disk again, it's as if the poor carpenter had to go through that same routine for every nail that was to be hammered. As you can surmise, it is going to take that carpenter a long time to finish building a house. That should give you an idea of the relative speeds when we are talking about billions of operations a second.

Now, when I got to SSC one of the first things I discovered was that the policy was to keep adding to a disk until it was full, then starting in an another. Empty disks were left spinning uselessly waiting for the day when other disks filled up. You don't need an advanced degree in computer science to figure out that this is not only wasting resources, but is directing all the traffic along one or two paths. I set about reorganizing disk usage so that all the drives were used and with the load spread fairly evenly amongst them. I didn't make a secret of what I was doing--I kept the head of operations informed every step of the way and, as it was his job to make sure that the disk drives were backed up to tape every night I assumed he was adjusting the backup schedules accordingly.

The main part of assume is ass, but you already know that.

Disk drives failed fairly regularly in the 1980's. Swapping out a failing drive and replacing it with a new one was a common procedure carried out by a technician from the manufacturing company (in this case, DEC) with an operator to assist. The operator's part could be carried out by any junior operator; he had to make sure that the data was backed up before giving the technician the go-ahead, restore the data to the new disk, and run a few basic tests to make sure everything was okay. Inevitably one of our drives started to fail and the call went out to have it replaced. As I left that evening the operator who would be staying with the technician (such work was always done outside of regular working hours, unless it was an extreme emergency), asked me what was up. I told him that the tech was going to replace the HDA and told him which specific disk.

The next morning I arrived at the operations center to find a very glum group staring helpless at one of the VAXes. There was a handful of operators, the head operator, and the client project manager. They told me that the minister's email had gone missing. In the government, department ministers are political appointees to positions roughly equivalent to that of the president of a large corporation. They are powerful figures and everyone treads lightly around them. You don't mess with them. Missing email was a serous security problem as well as a prelude to several people losing their jobs.

A quick look at the minister's account told me that her files were on the disk that had been replaced the night before. So, get the backup tape and let's go. There was no backup tape. The operator did not realize first: that the disk was in active production; and secondly: that when I said HDA, I was using jargon for the "Head-Disk-Assembly"--in other words, the whole darn disk, except for the casing. Okay, let's get the regular backup tape from the previous night. There was no backup tape from then either. The drive had not been included as part of the regular backup schedule. The head operator assumed that I had been changing the backup schedules while I was moving files around; I assumed that he had been adjusting the backup schedule because it was his responsibility and I had kept him informed of what I was doing. The operator had not only not realized what the HDA was, but had also assumed that the drive was not in production and so no backup-restore operation was necessary.

With all these asses around you'd think there was blame to share, but, not really. When I took on the job of reorganizing the systems it was up to me to make sure that I didn't cause any harm. Despite all the apologies I heard, I knew it was my screw-up and no one else's.

The only option we had left was to call the technician who had replaced the drive and hope that he had not taken the old one to scrap processing yet. Whew! he had not. The drive was still in the trunk of his car. But, it had been very much below freezing overnight, so we had no idea if the disk was now even readable after being bounced around in the trunk and frozen. He brought it in--I should mention that in those days disk drives weighed abut 200 pounds and were about the size of a large file cabinet drawer. He swapped the drives while we all held our breaths. I mounted it, checked it out, and everything was still there and intact. We backed it up, swapped it out again, and restored the files to the new disk. Did I mention that all through this the phone was ringing every few minutes with people from the minister's office wanting to know what was going on and why couldn't the minister read her email? We simply could not tell the real story. Everyone of us would have been instantly escorted out of the building and told never to return. And, if the story got to the press we would be instantly fired again. We all knew the consequences which is why we all agreed that it was all just a temporary technical fluke that we had heroically managed to correct.

A Balloon Bursts

Every now and again in the workplace we run across someone who suffers from a "Napoleon Complex."  This is a personality that grasps at any pretense of power, no matter how small, and inflates it and himself. Just like a balloon. Sometimes they burst just like a balloon. The public service is not immune to this persoanlity disorder. In fact, in some ways it encourages it.

After two years at Energy, Mines, and Resources  I had established myself as the VMS expert. I fell into the position by default; no one else wanted the position because, after all, VAXes weren't real mainframes and so were beneath the dignity of most of the 200 or so employees in the computer support centre. (A few years later about 180 of those mainframe experts were let go in a massive layoff--the mainframe day had come and gone--another story.) Besides, I had taken three five-day courses in system management in Toronto and so was also the only one in the department who was qualified for the position. I had two machines to look after when a third appeared on the horizon. The translation bureau of EMR bought a VAX to use for document tracking. The machine was delivered to the office of the self-appointed project manager. Let's call him Sammy, as he certainly would be very embarrassed if the real person were associated with this story. He should be.

Though VAXes were not mainframes they still required a controlled environment. That meant air-conditioning and basic cleanliness (isolated from the dust and debris of normal office life). Besides, this particular machine was about the size of a modern large freezer chest. An office definitely was not the right place for it. Still, Sammy wanted it to be set up in his office. One of the senior operators and I appealed to his boss who quickly agreed that the machine had to be placed in the computer centre. Sammy was not happy.

I went about setting up the system: creating accounts and printer queues, setting up the main application and the basic management routines, as well as writing up procedures for the operators to follow. One day Sammy showed up my office with a fellow who offered me a 9-track tape. Sammy was excited saying that the wonderful program on the tape would speed the system up. Right away my warning lights flashed on. I looked at the tape. It had a hand-scrawled and much-smudged label on the casing. I politely told Sammy and friend that we had established procedures to be followed and that I could not even mount the tape on a tape drive without proper authorization. Sammy was insistent that it was "his" system and so I should do as he said and install the program that was on the tape. I stood my ground. Later that day the head of the computer department came to see me and asked about this tape that my client wanted installed. I was surprised at how far up the totem pole the disagreement had reached.  I sketched out what had happened and he left, satisfied with my account.

Communications in the pre-PC days was usually by using lines leased from Bell Telephone to link a user's terminal (composed of a monitor, a keyboard, and a modem) to a large electronic switchboard in the computer centre. The switch would route the user to the correct mainframe and thus establishing a link for input (from the keyboard) and output (to the monitor). One day Sammy phoned me to tell me to raise the speed-limit on his modem. He would not listen or understand when I explained to him that what he was seeing, "9600 Baud," was it. It was a description, not a variable that could be manipulated. He started shouting that I had to cooperate. I firmly told him it had nothing to do with "cooperation;" he was asking for the impossible. Really, it was a 9600 Baud modem and there was nothing I could do about it. He may as well have been asking me to make his car float in the nearby lake.

Another feature of pre-PC days was that users shared resources on a central computer. Methods had been established to ensure that each user got his fair share of CPU access. On VAXes this was managed by a round-robin interrupt-driven preemptive system. What this meant was that access to the CPU was determined by priority. If the system lost power, it required the highest priority in order to ensure a safe shutdown as the power drained away (I was amazed to discover how much a CPU could accomplish in the micro-second of a power failure.) The highest was priority 31 in a list that began at 0. As events occurred priorities of "processes" (batch jobs, interactive sessions) were raised or lowered. For example, if an interactive session issued an I/O request (usually access to a device) it would be granted another two position points on the priority scale. With each cycle through the CPU the priority would drop by one. Interactive sessions were given a base priority of four--the position below which they could not drop. This meant a batch queue could be set up at, say, priority two, to use up CPU cycles when no interactive users required CPU cycles. This is important to get because this issue is going to come up again and again as I relate my experiences.

So, one day when the system was particularly sluggish I check the jobs running on the system and found an interactive session with a base priority of six. That meant that that process was hogging access to the CPU. It was, you guessed it, Sammy's account. I lowered its priority to four and system functioning returned to normal. A short time later Sammy's session was again at a base priority of six. I phoned him and told him that he was hurting the system's performance and asked him to please stop tinkering with the running of the system. He was upset and refused. I went to my boss, who went to Sammy's boss. I was then told to take away the access to the system (privileges) that allowed Sammy to adjust CPU priorities. Sammy was livid. Apparently he locked himself in his office and when anyone knocked he shouted, "No entry! Insufficient privileges!"

VAXes came with a communications port, usually with 16 slots. That way the computer room switch could route up to 15 sessions to the machine (one slot was reserved). There was space for only one communications port on this particular model. There were more than 15 employees in the translation bureau. The machine couldn't support more than about 12-13 users at any one time in any case because of RAM limitations (they would spend most of their computer time having their sessions written out to and read from disk.) In a meeting to discuss solutions to the problem, Sammy suggested getting another communications unit and plugging it into one of the free slots on the existing one. I assume the reader has enough experience with basic logic to realize how ludicrous this suggestion was. You have 15 apples. Cutting one of them into 15 pieces is not going to give you more apples. Fortunately no one needed my expertise to respond to that suggestion.

And so it went. These are just a few of the examples of his behavior.  Sammy became more and more frustrated as his dream of controlling the computing for the bureau he worked for melted away. His behavior got more bizarre. He attempted to copy the entire operating system files to his own account. He edited batch jobs required for the running of the department in order to "speed them up." (His changes would only slow the jobs down.) He phoned me with more absurd demands.  At last, I was asked to document Sammy's requests and behavior. He disappeared from my life--and the life of the translation bureau. His replacement was a reasonable and pleasant woman and we got along well for the duration of my stay at EMR, and later, when I was consulting to the department.

Comment: On re-reading this I realize that I didn't make clear why "Sammy" was so concerned with speeding up the computer's operations. The system was undersized from the beginning. The absolute maximum number of simultaneous users it could support was 15, but, performance fell off sharply after 12-13 users. There were 25-30 employees in the department it was intended to support. Some simple arithmetic could have predicted this. They bought a VAX 11-750, but needed, at a minimum, a VAX 11-780 (roughly twice the speed and capabilities). As Sammy had been involved in preparing the original specifications for the system he was somewhat anxious to prove that he was right and the reason that the system did not meet their requirements was because I was uncooperative.

How I Stole a Multi-Million Dollar Contract

Six years after I started my high tech career in the Department of Energy Mines and Resources (Natural Resources Canada), I was working for a private company (let's call them XYZ because they still exist), as a VMS Specialist. VMS was the name of the proprietary operating system ("Virtual Memory System") that Digital Equipment Corp (DEC) had created for its line of mini-computers.  These computers, called VAXes, were much smaller and cheaper than mainframes and could handle the computing needs of a small company or a department within a larger company. They sold like the proverbial hotcakes (in fact, both the Russian and Chinese governments were run by VAXes using stolen copies of the VMS operating system, an outcome of the first major hacking crime. But, that's another story.)

Remember what I said last time about the basic operations of computer programming? Recursion (looping) and memory manipulation are what it's all about. Get those basic things straight (and I am still amazed at the number of programmers I came across during my career who didn't get those things straight) and you're off and running. Operating systems are basically just as simple. The CPU is the master of the system; it reads its instructions, not only from  from a disk, tape, or other physical medium, but from the computer's dynamic memory (usually referred to as RAM, for "Random Access Memory.") The CPU can also store its instructionss and data in RAM for later use. What happens when the CPU has so much to "remember" that all of RAM is used up? You get a system crash. Plain and simple. The machine just quits working. So, there were many strategies worked out in the early days of computing to make sure that that didn't happen. Most broke at some point. Then along came the engineers of DEC.

When we run of of physical memory why don't we just pretend that we have more? That way the CPU can continue merrily computing away.  If you are using a PC or a MAC to read this, their solution, in the late 1970's, is the core idea behind your computer's operating system. Suppose we reserve an area on a disk that the CPU can write to and read from when it runs out of physical RAM? It might be a bit slower (actually, it is a lot slower), but we will avoid crashing the system. The system was called VMS (for "Virutal Memory System"). I took to it immediately. I love simplicity.

I had been working for XYZ for about a year as a contracted-out consultant to their clients running VMS systems when my boss said, "Why don't you take a run over to Supply & Services Canada (now called PWGSC, for "Public Works  and Government Services Canada") and see if you can help them out? They've been having terrible problems for a year now." XYZ had a contract to supply operations support for the department's VAXes while DEC held the master contract that included system management.

Okay-dokey. Sounded like something right up my alley. I arrive at the operations centre of SSC and the chief operator, a young man we will call Greg (we became fast friends until his tragic death a few years later) told me to go ahead and create myself an account on the system while he went out for a cigarette break. (Security?) He gave me the password to the system so I could do this. I created my account, then logged into it. "There must be something wrong," I thought. The system was sluggish, almost impossibly slow.  So, I checked my account settings and was appalled by what I saw. My memory access was restricted to a tiny portion of RAM. Even personal computers at the time came with more RAM than what I had access to. So, I fixed it, giving myself some reasonable quotas. That was much better.  The system was now responding as it should.

But why did I get such low quotas? was my next line of inquiry. I checked the system's "Default" account (from where it got the values not specified when an account is created) and saw it had the same tiny, impossible-to-work-with quotas. When Greg returned from his break I asked him who had set the system up? The answer: DEC. They were responsible for installing and managing the system and they had an "expert" on site. I told him that they were nuts (using professional language, of course) and that I could fix it. "Go ahead," he said. I typed one command line to give all accounts on the system access to reasonable amounts of RAM. Like that. That easy. The phones started ringing.

Users of the system were calling to find out what had happened. The system was now operating as it should instead of the slow, like walking underwater, experience they were used to. "You fixed it!" they were saying elatedly.

I found out that for the past year DEC had been telling the folks at SSC that they needed to buy more hardware and some fancy (expensive) software packages to solve the performance problem. SSC (quite rightly) were refusing, saying that DEC had delivered a system that they had claimed would meet the performance requirements, so, any shortfall was on DEC's account.

Within a half hour I was called to the director's office. He had one simple question: "Was what you did something any system manager should be able to do?"  "Yes sir." "Thank you." (I was not exaggerating: memory management and managing user quotas is a basic responsibility of a system manager and it is included in any introductory course.) An hour or so later the DEC employee appointed as on-site system manager came to see Greg and told him, face pale and voice trembling, that SSC was tearing up the contract. I had nothing to say to this "expert" who had been screwing around for a year, probably being paid far more than I, who apparently hadn't even taken "System Management 101." As I was to discover over the years I was in the business the situation at SSC was not unique. In fact, I was involved, in a small way, in killing a project that had cost the taxpayers some $300,000,000.00 to date. The reason? The people designing and implementing the system didn't have a clue what they were doing.  I'll tell that story another time.

Meanwhile what the "system manager" had reported about the contract was true. All DEC employees were escorted from the building and the XYZ company I worked for landed a very cozy five-year multi-million dollar contract  to manage the systems at SSC. I was there for three very important (to me) years.

In the Beginning

I got into this consulting programming/analysis business in 1982. I had been teaching English Lit in a secondary school in Maniwaki Quebec for seven years and decided it was time for a change. Besides, my wife could not find suitable work (she's an editor) in that remote predominantly French-speaking community. As part of a program to reduce spending the government of Quebec offered a deal: if someone with seniority resigned, thus saving a younger employee's job (who made much less money), then they would give you six months' salary and return your pension contributions. I took it.

I plowed the financial break into retraining. I paid one of those private training companies a fortune for a three-month course in computer operations. My wife, I,  and baby son moved into a slummy attached house in Ottawa so we could get resettled. My wife landed a full-time job with a company she had done some freelance for and I finished the course and started looking for a job.

I had no idea what a "computer operator" actually did. All I knew was that it was the cheapest and shortest program. The two other courses offered were: "computer programming" (4 months) and "computer technical support" (five months), with correspondingly larger fees. I learned the basic architecture of computers and trained in running a mainframe (a CDC Cyber in this case). Actually operating the mainframe meant putting programmers' punch cards through a reader then returning the printout to the programmer. That took about two minutes to master.

Part of the course was learning a computer language; in this case: BASIC, which was a very commonly used language at the time. I was supposed to take 6 weeks to master the basics of BASIC; it took me two weeks. So, in order to fill my pre-purchased class time, I took a second programming course; this one in COBOL (a language designed for and primarily used by business applications). We wrote our programs by hand on forms designed with the basic layout of the language and then punched the code into card-punch machines that spat out our programs, a separate card for each line of code, max 80 characters per line/card. We would then hand the stack of cards to an operator who would run them through a card-reader and then retrieve the resulting print-out. That's when you found your errors. If you were lucky the computer didn't throw up and refuse to continue after encountering your first line of code. Back to the drawing board (pencil and paper), repunch the card deck and try again. We were limited to three computer runs per day.

I went into some detail here because that was the state of the computing world at the time. There were only large mainframes that were very expensive to run (hence the limited number of runs per day); there was a hierarchy on the human side that meant only specially trained operators were allowed to actually touch the physical equipment--and those operators were very poorly paid and at the bottom of  the heap. I mentioned above that it took me about two minutes to master the basic skills of an operator. Programmers were kept at a distance from the machines and essentially worked with pencil and paper. The code they wrote was line by line instructions for solving simple problems. Here's a sample:

10 A =0

20 A = A + 1

30 PRINT A

40 IF A .EQ. 10 THEN GOTO 60

50 GOTO 20

60 PRINT "FINISHED!"

70 END

That told the computer to print the numbers from 1 to 10, then print the word "FINISHED!" Pretty basic stuff, eh? It took me a while to realize that "A = A + 1" was not an illogical algebraic equation, but was a command to take the contents of A, add 1 to it, then store the result in A. Once I got my head around the following concept:

C = B

B = A

A = C

I was away and running. (What that fragment does is swap the contents (numbers, words, addresses, etc.) of A and B.

And with those two ideas lies the basis of all computer programming.

It took me about six months to land a job in my new profession. I stayed home with the baby while my wife worked and I searched. Then in June of 1983 I received a call asking me to come in for an interview at a government department. I was so excited I forgot to ask which department and location. I spent an afternoon on the phone with the civil service personnel department tracking down my appointment site.

I was given a written test comprised of 10 questions about computer architecture (I got 9 out of 10 because I had never heard of a "burster")* and interviewed by two middle-aged men wearing blue-jeans and cotton shirts who were impressed by the fact that I was a former teacher and that I had trained on CDC Cybers (the brand in their computer room). In my training for operations I was to be trained in only one computer language, but I was fortunate that I had been bright enough to learn a second language because the job required that applicants have two languages. I was hired and began my career as a civil servant working in the computer room of a large government department ("Energy. Mines, and Resources" at the time, now called "Natural Resources Canada") as a junior operator paid $11,000/year, down slightly from my salary as a teacher (about $28,000/year at the time). Still, it was a beginning.

*Burster: a machine through which an operator fed continuous-feed computer printout. The burster separated the perforated pages and spat them out in an orderly stack. You might be able to see one in a computing museum.

The Background

Between 1983 and 2004 I worked for and was a consultant to the Government of Canada. I was a computer analyst/programmer for the most part and, during that 21 years, I was inside almost every government department. I held Secret clearance which meant I could travel fairly freely within department and government buildings. I worked on a need-to-know basis. That meant that, for example, I might have to know the structure of a database, but not the actual contents. That was fine with me. I really did not want to know details about Corrections Canada's guests. If  I did see information as a result of testing, I had no interest in reading it in any detail or attempting to recall what I had seen. Suffice to say that government employees with the right access can find out when an inmate used the toilet in his cell. (That is an excellent reason to stay on the right side of the steel doors and concrete walls, in my opinion.)

I am not going to expose any state secrets and, what was considered classified twenty or thirty years ago, has no relevance now (such as the brands, operating systems, and locations of government computer installations). However, I did see a lot of stupidity, waste, unethical behavior, and childish temper tantrums. Don't get me wrong: government employees are no different than you and me. Most simply want to do a good and honest job and take home a reasonable salary on which to support their families. But, when computers started making their way into the workplace in the late 1970's and early 1980's, there was a vacuum of skills and knowledge into which any con artist with the right vocabulary could make a power grab.

In these stories I will disguise identities as much as possible to avoid causing anyone personal embarrassment. And, remember, I usually did not have access to the Big Picture and saw only how things played out.

To start off: this is a story I like to tell because it says a lot about our country. In the late 1980's I worked on a three-month contract at the Prime Minister's Office. My job was to help set up the computers in the newly created Department of Inter-Provincial and Government Affairs. This was Brian Mulroney's solution to the question of what to do with Joe Clark. I was part of a small team who assembled the computers at the Prime Minister's Office in the Blackburn Building, and then carted them to an office tower two blocks away where Joe Clark was intended to rule. In case you are wondering, the computers were PC's with an amazing 8kb RAM (I had to add 4kb memory chips to the four that shipped installed in the machines), and the new staff were getting color monitors and laser-jet printers--in other words, the latest and greatest. When the machines were ready, I'd load the computers, monitors, and printers on a dolly and push it along the Sparks Street Mall, crossing downtown Ottawa streets. I'd head to the location inside the office tower where the equipment was to go and then set it up and test it. When done, I'd push the empty dolly back to the PMO and prepare another batch of equipment for the next trip.

One day, as I was descending an elevator with my empty cart, the elevator dinged, stopped, the doors opened, and Joe Clark himself stepped into the elevator. Alone. He was Acting Prime Minister of Canada at the time (Brian Mulroney was out of the country). I could have been anyone with an identity badge. He is a very tall man (I'm of average height); he looked down at me and said, "Hello" in a deep voice. I looked up and said, in my best Ottawa valley accent, "G'Day."

We exited the building together. I fell a bit behind while pushing the cart and watched as Joe Clark walked up the short hill towards the Parliament Buildings. It was almost 2:00 pm, traditionally Question Period in the House. In other words, he was going to work, just like most of the people milling about the busy streets. No security, no body-guards, no "handlers." Just Joe. And it struck me: in how many countries in the world can the leader of the government walk alone through busy streets, just going about his job?  I felt a swell of nationalist pride.