Ed Healey
Interviewee: Rear-Admiral (Ret’d) E.J.(Ed) Healey
Interviewers: Tony Thatcher, Mike Saker & Don Wilson
Date of Interview: 2 September 2009
Location of Interview: Ottawa, Ontario
Transcribed by: Sue Easterman
INTERVIEWER: Admiral Healey has had a remarkable career in the Canadian Navy within the Department of National Defence. He has had experience with many ship procurements from HMCS PROVIDER in 1963 through to the Canadian Patrol Frigate (CPF) and the Tribal Class Update and Modernization (TRUMP) in the 1980s. He was Project Manager for the CPF Project in the first half of the 1980s and later became Assistant Deputy Minister Materiel ADM (Mat) and was responsible for all procurements within DND. In this interview Admiral Healey has kindly offered to review the Naval Procurement Process over this period - a period where the process changed considerably and hence had a considerable affect on Canadian Naval industry. I would ask Admiral Healey first to introduce himself and describe his career as it relates to this interview.
HEALEY: Thank you very much, Tony. I suppose my first involvement with Naval procurement was as a young officer in the Engineering Department of the HMC Dockyard in Halifax where after my time at sea I was sent ashore to work in the Engineering Department and I was responsible for large ships and odds and sods including getting rid of surplus ships. But in that context I was exposed to a ship that was just out of the shipyard building HMCS PROVIDER and that started me on the process of my first involvement with the procurement processes that we have seen over the years and it ended up, as you have already mentioned, when I was ADM (Mat) and looking after all procurements from the DND prospective. So that spanned 1962-63 up until 1990, so it spanned a good period of time and all the various shipbuilding projects that occurred during that period I had some involvement with, some more than others.
Well, Tony, let me start by talking about PROVIDER which was built in Davie Shipyard and it arrived in Halifax, I think, ’63. I’m not quite sure whether that was correct, ’62 or ’63 and it fell to me to fix it because it was far from complete. And there was a list as long as your arm of defects and “I forgots” not insignificant ones some pretty major ones. A lot of which, on reflection, it occurred to me had resulted directly from the procurement process. Now when the Navy acquired the SAINT LAURENT and follow up classes, the Navy was the designer, they had a design engineering team in Montreal (NEDIT) and they had people in Ottawa who did parts of the design. And the Navy procured most of the equipment and the systems for those ships and then the government including the Navy had a Procurement Allocation Board or shipyard where they allocated ships to various shipyards and the shipyards received the government supplied equipment, the government supplied design and they built these ships essentially on a cost plus basis. That worked well as far as the Navy was concerned because it was a … they got what they wanted and they got the ships completed. But we also should remember that those ships [were] like night and day compared to the ships that we need today. They were simple, they had a gun that you operated the gun separately, operated the radar separately, you didn’t have any integration, you didn’t have a lot of complicated systems in terms of today. So they were able to do that without great difficulty. When it came time for the PROVIDER there was a lot of, I understand, “to-ing and fro-ing” in Ottawa and a lot of the various newly minted DSS (Department of Supply and Services) thought this was an opportunity to exert themselves because these ships like the PROVIDER was essentially a commercial vessel. Therefore, we should go, the government should go with a commercial procurement with an overlay of requirements such as the replenishment systems and the various other military systems that were needed to make the ship a multipurpose replenishment and support ship. That was the genesis, I think, of much of the problems because the specification, the requirements, were pretty confusing and the shipbuilder, of course, bid on the basis of what he thought he could minimally produce and, in fact, that is what he did. So the result was a ship that was far from complete and I personally spent night and day for three months doing a tremendous amount of work and a tremendous amount of back-of-the-envelope engineering to get the ship to minimally function safely at sea. In other words, if you could look at it from another perspective what happened was the problem that was created in Ottawa was transferred to the East Coast to the then dockyard where we had some control over spending and we fixed the problems and the whole thing went away.
Let me just tell you about some of the problems. The ship was rather innovative and probably ahead of its time in some ways in that it had a replenishment system that was fairly novel and new … probe system … and in order to support that replenishment system there was a low pressure hydraulics pumps that controlled rope accumulators, which kept the tension on the lines between the ship and the ship being replenished. The rope accumulators were charged with high pressure air so that the tension could be held, but somebody had forgotten to install an air compressor so there was no method of replenishing the high pressure air on the rope accumulators and I think if I remember rightly there were four or six of them … rope accumulators onboard. So when I first saw the ship coming into Halifax harbour here some of these hydraulic winches were being used to pump the rope accumulators up and down which was the only method they could figure out to put some air into the rope accumulators. The low pressure hydraulic system had maybe a half a dozen pumps or more, maybe 12 pumps all together, if I remember rightly. And there was only local control. So in order to operate two or three stations at once you would need half a dozen people or a dozen people to operate them. And, of course, you didn’t have that capacity within the crew. So we had to install a remote control or a pneumatic control system, foot controllers and cages that we built on decks, so that we could control several of the winches with one operator.
There were other problems in the system for blanketing the tanks with inert gas and we had to repair the inert gas generator which turned out to have a land based set of “doughnuts” in its cooling tower, which disintegrated on the first wave coming down from Davie or in sea trials and, of course, plugged all the air passages from the generation of the seal to the blanket the waves in the cooling tower. And we had to … the cooling tower was built under the focs’l deck and the top of it was unable to be removed because there wasn’t enough clearance between the focs’l and the thing, so we had to cut the deck open about 10 feet diameter to get the lid off and discover what the problem was.
But there were also other problems with quality. The endurance of some of the commercial type equipment wasn’t what we had expected and they tended to last the warranty period and that was about it. But those are just some of the typical problems. There didn’t seem to be any recourse other than the fact that the process was so muddled and so confused between commercial and military that the end result was, for at least the initial phase, a poorly outfitted ship that did not operate properly and couldn’t really be used until we have spent a lot of money, a lot of time and a lot of effort to repair it. Unfortunately, I don’t know as the lesson really got fed back but that…this had been such a big problem because I think it was fairly quietly done with not a lot of fanfare. In those days, of course, you didn’t have the press attention that you get these days and you didn’t get the people transmitting brown envelopes. So any sort of attempt these days to cover something up would not last and rightly so but we would be castigated in the media.But come the next procurement of similar vessels, PROTECTEUR and PRESERVER, by God, if we didn’t use the same process again at the behest of DSS. We thought some of the problems were cured and they certainly were and the specification for the replenishment system had been updated and changed but again we had this difficulty with building to Lloyd’s, not having a very complete comprehensive specification and as a consequence we had lingering problems with those ships as time went on. And, if memory serves me, I think we ended up having to pay a claim by the shipbuilder even for the additional work that he claimed he required because of naval insistence. But… maybe just reiterate that, you know, the shipbuilders in those days and this PROTECTEUR and PRESERVER Saint John Shipbuilding which was a pretty rudimentary shipyard didn’t have much history or much … this was a pretty big project for them. I think I once described it as three men and a pile of mud and a couple of dogs was all they had to start with and this was a pretty big procurement that they … not made easier by what was laid upon them to do. So those two ships, of course, are still operating but we have had to change out a lot of the equipment, a lot of the materials since that time.
Unfortunately, I think, we have not been tempted to go back to that type of procurement … the Lloyd’s type of procurement. Someone asked me if they were fairly up to date ships and, I think they were. They were …the methods of … the multi-purpose aspect of them was unique and most other navies, the US and British Navy had single-purpose ships, which they could afford, carrying equipment and troops and others just strictly for replenishment. Whereas we had designed these ships and outfitted them so that they could not only, in a limited way, carry troops and equipment but also aviation gas, water and regular bunker oil and diesel oil and so on. So they were multi-purpose and they had this rather up to date system of replenishment which was fairly quick and you could do it in a fairly high sea state whereas the old method you couldn’t accomplish that other than in fairly calm weather.
Well, in terms of … I think there was some progression between PROVIDER and PROTECTEUR and PRESERVER in terms of how the package was presented for bidding by the shipbuilders. But essentially, as I remember, I understand the outline of the ship and the basic concept, of course, were provided by the government and the Navy, but the detailed design and drawing and equipment selection, with probably some exceptions, were essentially done by the shipbuilders, so that the shipbuilder was responsible for selection of equipment such as basic stuff as steering gear, engines, propellers, gear boxes … oh I guess there weren’t any in the steam [ships] … But things like the Navy having specified a level of remote operation for the propulsion plant, for example, and it was the requirement that the shipyard designed to or contracted out the design and provided the material. The Navy, I think, supplied such things as any weapon systems and probably designated some other equipments, but essentially that whole process was one that the naval design community did not get much involved with. So it was…
Now my involvement at that time, I was on PROTECTEUR / PRESERVER, I was the head of the Propulsion Section in the DMEE and it fell to me to do the trials when those ships were ready and, of course, as a consequence I saw them in the shipyard and I saw all the problems they had and I saw the problems we had in arguing with the shipbuilder to get some of those problems fixed. And it was the usual sort of case of finger pointing and ‘not my problem’ that sort of thing. But again we faced later on in those ships some of the problems of substandard equipment which didn’t have great longevity. Probably, I think, having said that the ships have been gradually improved over the years and they have lasted a long time so they probably in the end they have turned out to be pretty valuable assets in the Navy.
Vivid in my memory is being called to the trials of the PRESERVER which was the last of the two in Saint John Shipbuilding. And Saint John at the time, of course, had little or no follow-on work. Their labour contract had expired and there was a threat of strike and they were running late on the ship and ready for trials so they … it was their call and they called us early and after a day or two in the yard we were ushered aboard, if I remember, I think, there were armed guards on the jetty and the workers were put aboard and tugs arrived and towed us out into the harbour and the anchor was dropped and from the labour perspective we were now on trials; except we were two or three days, if not more, before we could actually commence trials because the ship wasn’t there.
The other thing I remember vividly was during the trials it was the foggy season in the Bay of Fundy, we had a civilian master aboard hired by the shipyard and we did all the trials including the full power ahead and for some reason the civilian master wouldn’t do the full power astern until the fog lifted which we had completed all the other trials and it was two or three days hence we sat out there waiting for the fog to lift to go astern. In the process of that we managed to hog the main turbine because it wasn’t when we did get the go ahead they ramped it up too quickly and we were in danger of having to remove and repair the main turbine. In some ways that was an interesting evolution in that it was the Trials Officer’s call if he had cause to require that the main turbine be opened and so at the conference, the wrap-up conference, where we discussed all the outstanding defects …. I took … I was the spokesman … the person on the government side and dealt with my opposite number on the shipyard side, John Shepherd and he wanted to deal with whether I wanted the main turbine opened as the first item. And I said “no” I need some more information, so we will deal with it later. I think the conference took 2 days, if I remember rightly, the 250 or so defects most of them shipyard were very co-operative on until we got to the opening of the main turbine. In the meantime, I had discussed it with the GE, General Electric rep who was onboard, who was, of course, the representative of the maker of the turbine, we decided that it wasn’t necessary for the conference. But he agreed to keep that quiet and I had agreed to keep it quiet, so we had a very successful defect reconciliation conference and we didn’t need to open the main turbine, look through the inspection ports, bearings and clearance strips and so on. We got a lot of defects fixed that way.
Okay I just to move on now to talk a little about more military vessels such as GP (General Purpose) Frigate which never really came about; the 280 procurement; the BONAVENTURE final refit, which I had some involvement with in my dockyard role, I was in charge of the specification for that; and the Aurora procurement, which was going on about the same time as the 280 procurement. Now, I think a lot of people don’t understand that the perception of these things from the outside is sometimes quite different and maybe even unfair from the perception of them from the inside but the 280 following on the heels of what was considered a debacle in the BONAVENTURE where costs were out of control and I suppose the ultimate crowning cap on that one was that the ship was scrapped not long after the refit.
The 280 was piecemeal procurement which significantly had design creep … it had design involvement during its life and the government rightly or wrongly; senior officials and ministers saw this as people in DND attempting to sneak things by to get things surreptitiously that they would not have necessarily got otherwise had they been upfront about the total costs. This was partially the case with the Aurora procurement as well and a lot of this BONAVENTURE, Aurora and CPF. You remember the first CPF Project Manager got fired and his DSS counterpart got fired and that was the days when the whole project was reporting to DSS. It was not reporting to DND and that was the enthusiasm of the time and, of course, the problem … the fundamental problem with that was that DSS didn’t have the money and DND had the money and, of course, that whole process collapsed and the procurement nearly stopped at one stage.
But out of that came a government sponsored review called the Pennefather Commission which looked at the structure of DND and how it was set up which was the first major review of DND organization and structure since unification and the result of the Pennefather Commission was … on the procurement side was ADM (MAT) was born. And that was the genesis of ADM MAT, prior to that it had been split into two groups coming out of unification and the other item coming out of the Pennefather Commission was a whole new set of rules and procedures for procurement and it’s led to this idea of Major Crown Projects and Total System Responsibility, joint management through Boards of Directors from various government departments on major projects such that … and the responsibility … the major responsibility being clearly returned to the funding department in this case DND, so that Chairman of the Board of Directors, I think, it has changed its name many times, was from the originating department of the process. So the whole process from that time and how significant the procurement process was, or the “misprocurement” process of 280 BONAVENTURE and Aurora resulted in a major, major change which, of course, led to the ultimate pinnacle of this procurement process and the CPF project which was a total package put out to industry, total responsibility to industry and included every imaginable cost including all training, all buildings, every spare nut and bolt that we were going to need for the life of the project all of that had to be… And that, or course, led to another set of problems, but that process did result in a project that I think judging by public opinion and that in Canada lack of public awareness or lack of criticism means that it must have been a success. And CPF was a big success because no one goes around saying what a terrible thing that was. People still remember the BONAVENTURE and so on and so forth. So it came in eventually under budget and ahead of schedule and the last ship got delivered ahead of schedule and I think from a fundamental point of view the ships actually worked. Right out of the shipyard the ships actually functioned which I think in the case of the 280s for example, the procurement process …it took after the ships were delivered from the shipyard maybe another 4 or 5 years before the ships actually could do useful work for which the whole purpose of them was to go fight a war and they couldn’t do that for 4 or 5 years after because of the process.
So I just wanted to emphasize again that what I notice today is that people seem to be getting away from this process of total package procurement and making sure that the government DND … military is not involved in its own procurement in a fashion that could come back to haunt it because we went through that. We went through that in PROTECTEUR / PRESERVER. We went through it in the 280 Project and so on and so forth. And the tolerance for those sort of things seems to me to be less and less each day. Media scrutiny is higher and higher and so the more we can have comfort in the media, in government and in senior bureaucrats the more likely we are to get projects approved. The more uncomfortable they are then it will take more and more time and be more and more difficult and will probably have more controls put upon us.
I just wanted to say a few words about the technological evolution over my period in naval procurement. I think it goes without saying that procurement of new ships was a pretty intermittent type affair. We didn’t have that many so we ended up unlike other navies, the USN and the British Navy, tending to revolutionize what we did as opposed to the USN in particular which would evolutionize, they would move slowly ahead because they were building many more ships. Now this revolutionized concept, let’s just take one example, we went from steam destroyers on the SAINT LAURENT to follow-on classes which were an evolution of that first design and not much radically changed in terms of the basic ship and the basic systems and later ships had flight decks and hangars added but the 280 Project was a revolution. Even in ship design we had a ship that depended totally on gas turbines for its propulsion and almost totally for its electrical generation. We had nine gas turbines in that ship and not many navies, if any, at that stage had that reliance on gas turbines. Most of them had at least propulsion diesels and fairly significant power generation diesels as well. We made that revolutionary step to have minimal manning in the propulsion systems and we were in the age of …we didn’t have the electronics and the digital capability we have today. We had pneumatics. We had a combination of pneumatics and electronics to control the propulsion system and somehow we made it work. These presented enormous challenges of course to industry because when we would contract and do these things industry was faced with a prospect of doing something that they had not seen before. Acquiring equipment that might or might not be totally mature and certainly contributed to some of the subsequent problems and teething problems that some of our procurements had.
Even in CPF we had competitors bidding for the CPF that incorporated as much Canadian technology as they could because they knew that doing that meant their chances of winning the project were much better. If they left them out and went with old proven technology which would have been safer. That would have been great except they probably wouldn’t have won the day and so they went and they ended up with items that had been sponsored by the Navy over the years, such as machinery control systems [SHINMACS]. And the integrated communications … internal and external communication systems [SHINCOM]. Both of these systems were not really all that mature and would cause subsequent problems. One of them, I think, the machinery control system has gone on now to be, as I read it, a great success for CAE and they sell them around the world and the genesis of that was a Navy project and it caused some difficulties in CPF. But that was a quantum jump ahead that digital method of machinery control and damage control and information system, which was a quantum move.
Now, whether we can think other navies are now beginning to look at the idea of starting with a clean sheet of paper to maybe make some significant advances because they too are finding the that evolutionary process may be a little slow. But it does, as I say, have its difficulties and it tends to fly in the face of enthusiasm that people get like buying off the shelf which seems to be the mantra from time to time and very difficult to have a development project that hasn’t been fitted with somebody else’s thing and have it put into your ship if the people are claiming that everything has to be off the shelf per se. These things work themselves out and they tend to bring us more up to date than we would otherwise get and I think the pinnacle of that was the CPFs which are, though dated now, were pretty competitive in terms of capability in what they could do particularly in the combat management system and in terms of controlling the sensors and weapons systems. So it is the only way we could get there and it was … added an element of risk to all our projects but I think it was worth the effort.
My previous remarks convey the impression that a lot of the problems with the procurement processes were in-house within DND and more particularly within government both in the bureaucratic level and quite often at the ministerial level. Things get changed at the last minute, decisions get modified or which have a significant impact on the possibility of success. What isn’t really understood in a lot of cases is that many of these come back to haunt the department because those changes that occur are long forgotten who precipitated them and it looks like the Navy has screwed up yet again … when a project goes bad or the Defence Department has screwed up yet again, when in fact the problem may well be the one that’s precipitated by government.
I just mentioned one in particular the Trump Project which I was instrumental in getting approved at the same time as the first batch of CPFs but a requirement was suddenly laid upon us by Ministers that one of the losers in the CPF contest should get the shipbuilding portion of the Trump Project and that shipbuilder happened to be Davie. And that I think led to all sorts of downstream problems, because Davie had the work and Litton was trying to control them in a package procurement and of course it ended up in a schmozzle and costs escalated and delays occurred etc. etc. And the rebound of that was that DND and the Navy didn’t look very good, whereas indeed had that intervention not been there we might have escaped a lot of that problem.
I wanted also to talk a bit about the contractors, particularly shipbuilders and combat systems suppliers and say they too have their problems, particularly with this technology jump that I talked about. Some of them have difficulty stepping up and understanding what they did themselves. We certainly saw that in CPF. We saw it where both the major contractors, Saint John Shipbuilding and its subsidiary or its sister contractor Sperry which was called Paramax in Canada really tried to build something different than they had signed up for only a few short months ago. They seemed to have difficulty with the group they had now assembled to do the implementation, …didn’t understand what the groups who had done the bidding… that bid and we hear that … this piece of equipment wasn’t working and we had to get it changed out and so on and so forth. Well, I say, you bid it you make it work. Don’t blame someone else when you bid it. Don’t blame someone else when you get ahead of yourself and the detailed design should be done up front before you start cutting steel. And no, Sperry, we are not buying an FFG7 American Combat System. We want something that we signed up for and that’s why we had a 35,000 page contract that you had produced to make sure that this is what we got. So they have their problems as well. But there are enough problems to go around in these processes so I don’t think you can say it’s all their problem or all our problem and all DND’s problem or all someone else’s, but certainly it takes a fair amount of understanding of the likelihood that you are going to get into trouble and in designing the procurement process, I certainly believe you should design it in such a fashion that you have some chance of success. And if you design it so narrowly or so tightly without flexibility in your process, that can take a few hits here and there, then you are doomed to have a difficult procurement.
INTERVIEWER: Well, thank you very, very much Admiral Healey for that interview. A fascinating account of the procurement process and the various aspects throughout the years. And you filled in a lot of the blanks there. We appreciate it very much. Thanks.
TRANSCRIPTION ENDS