THE STANDING SENATE COMMITTEE ON NATIONAL FINANCE
EVIDENCE
OTTAWA, Tuesday, March 27, 2018
The Standing Senate Committee on National Finance met this day, at 9:30 a.m., to continue its study on such issues as may arise from time to time relating to federal estimates generally, including the public accounts, reports of the Auditor General and government finance (Topic: Phoenix pay system problems).
Senator Percy Mockler (Chair) in the chair.
[Translation]
The Chair: My name is Senator Percy Mockler from New Brunswick. I am the Chair of this committee.
[English]
I wish to welcome all of those who are with us in the room and viewers across the country who may be watching on television or online. As a reminder to those watching, the committee hearings are open to the public and also available online at sencanada.ca.
[Translation]
Now I would like to ask the senators to introduce themselves, starting at my left.
Senator Moncion: Senator Lucie Moncion from Ontario.
Senator Pratte: André Pratte from Quebec.
[English]
Senator Jaffer: Mobina Jaffer, British Columbia.
Senator Black: Doug Black, Alberta.
Senator Marshall: Elizabeth Marshall, Newfoundland and Labrador.
Senator Eaton: Nicky Eaton, Ontario.
Senator Andreychuk: Raynell Andreychuk, Saskatchewan.
[Translation]
The Chair: I would like to recognize Gaëtane Lemay, the clerk of the committee, and our two analysts, Sylvain Fleury and Alex Smith, who team up to support the work of this committee.
[English]
Today, honourable senators, the committee continues its study on the Phoenix pay system in Canada. We have before us two officials from Goss Gilroy Inc., a management consulting firm hired by the Treasury Board Secretariat and Public Services and Procurement Canada in early 2017 to conduct an independent study on the management and implementation of the Transformation of Pay Administration Initiative. To discuss the findings of their report of last October, we welcome Mr. Jim Alexander, Associate; and Ms. Sandy Moir, Managing Partner, Ottawa Office.
For information, Goss Gilroy Inc., known as GGI, was founded in 1981. It has a staff of over 25 in its Ottawa, St. John’s and Vancouver offices. GGI has completed over 3,300 consulting projects in all areas of Canada and throughout the world for a variety of clients in both the public and private sectors.
GGI was hired by the federal government to conduct a study on the lessons learned from the management and implementation of the Transformation of Pay Administration Initiative, known as the TPA Initiative. The scope of the study covers activities from 2008 to April 2016. The study was launched in late February 2017, and data collection was completed in July 2017.
I have been made aware, Mr. Alexander and Ms. Moir, that you each have comments, which will be followed by questions from the senators.
Sandy Moir, Managing Partner, Ottawa Office, Goss Gilroy Inc.: Good morning, and thank you for the opportunity to appear before you to discuss the findings of our lessons learned report related to the TPA.
I’d like to start by stating that this was an independent assessment. While government officials enabled our work and granted us access to documents, contact information for key informants, workshop participants and so on, the findings in this report are ours alone.
We looked at the Transformation of Pay Administration Initiative, as you mentioned, the TPA, and this included both pay modernization, which we all know as Phoenix, as well as pay consolidation, which included transferring pay services from a subset of departments and agencies to a pay centre located at Miramichi. It was these two projects together that we looked at.
I would also like to emphasize that this was not an audit. We did not come to the study with predetermined lines of inquiry, nor were we trying to point fingers or lay blame. Rather, the methodology emphasized consultations undertaken in an open and sharing environment.
We conducted 37 interviews with over 50 individuals from various perspectives, both within and outside the Government of Canada. We also conducted eight workshops with over 100 individuals from 39 departments and agencies. These consultations were supported by a targeted document review, and we reviewed these documents to understand the context and to clarify and validate what people were telling us through the consultations.
We further validated the report by applying our own expertise, as well as through meetings with each of Treasury Board Secretariat’s and Public Services and Procurement Canada’s Departmental Audit Committees and targeted in-person meetings. The report was also reviewed by several government officials to make sure we got our facts right.
I’ll turn it over to Jim now to go through a high-level description of the lessons.
Jim Alexander, Associate, Goss Gilroy Inc.: Good morning. In all, we developed 17 lessons. These lessons are organized into six major areas. There is a hierarchy to these lessons, which I’d like to take you through now.
The first major area of lessons is initiative definition. This is the “what” of the initiative. We identified two lessons in this area. One pertains to scoping, where we felt it is imperative to properly define what is changing. In the case of the TPA, the initiative was too narrowly defined as changing a software system and moving people to a pay centre. In fact, the service model was changing, business processes were changing, skills and know-how were changing, and the culture around HR management, human resources, was changing as well.
The second lesson in this initiative definition is to develop a road map that articulates the discrete projects and how they interrelate. So while all these aspects were changing, how did they relate to each other?
If that is the first major area of lessons, the second one is governance and oversight. This is the “who” of the initiative. Here we identified three lessons. The first is about accountability and that accountability and authority for the initiative need to be assigned to one single office. While PSPC had accountability for TPA, they did not have the accountability for what occurred in departments and agencies, for example, and no one office had the authority for TPA, that is, the ability to tell others what to do.
The second lesson in this area is about governance and that you need to establish broad and inclusive governance. It needs to include the process owner — that is, the unit who leads the revitalization of the business processes associated with this end-to-end change — as well as the organizations leading the initiative and other different types of stakeholders, such as the affected departments and agencies, functional communities and the policy centres.
The third lesson in this area pertains to oversight. It is critical that oversight be more than checking in and providing updates. There needs to be a strong challenge function. Drawing upon independent organizations with significant international experience is a best practice in this area.
Those two areas — initiative definition, and governance and oversight — are foundational to the success of any initiative. They ensure that the big picture is properly defined, understood and managed. If these two foundational areas are addressed effectively in the earliest stages of the initiative, the transformation has a greater chance of success.
The additional four areas of lessons learned greatly influence the probability of success of the transformation initiative. While they cannot overcome flaws in the two major foundational areas I have just discussed, the lessons learned in the other four areas are each essential to get right for transformation initiatives.
The three lessons within the change management major area focus on the “who,” the people involved with and impacted by the transformation initiative. Change management must surround and underpin the whole initiative and be treated as a fundamental component, being critical for buy-in from those affected by the change, and must support staff and managers to help them adapt to their changing roles.
There is an important lesson on culture within the change management area in terms of fully assessing the culture and the leadership environment, and then either adapting that culture or proactively managing the initiative within the existing culture to ensure success of the transformation.
The next major area of lessons learned is business case and outcomes management. That addresses the specifics of the system itself and the “what we want to get out of the transformation.”
This includes lessons about engaging with stakeholders regarding the anticipated functionality of the new IT solution, as well as taking steps to minimize the need for extensive and risky customizations. For example, the fact that the Government of Canada conducts a large proportion of its human resource transactions retroactively greatly complicated the customizations.
Outcomes management is another important topic and formed two of our lessons. Understanding and then monitoring the “why” of the initiative should be addressed throughout the transformation, quarter by quarter and year by year, and not just at the beginning and hoping that something good happens at the end.
The other lesson under outcomes management pertains to when benefits, including cost savings, can realistically be expected. In the case of TPA, cost savings were actually expected in the first year of the launch.
A final lesson under this major area is about support to affected stakeholders. Departments and agencies played an extremely critical role in the transformation, but they did not have the capacity, resources or capabilities in areas such as change management.
The fifth major area of lessons is initiative and project management, which highlights aspects of “how” the transformation initiative and the projects within it are managed day to day over the years of the initiative. Lessons in this area have to do with adequate testing, collaboratively engaging with the private sector, and regularly updating and revisiting the business case.
The sixth and final major area of lessons is capacity management, which addresses the need to explicitly manage the transformation of the workforce, the “who,” but emphasizing the knowledge and how that knowledge must be stewarded and developed as a critical resource.
I’ll turn it back now to Sandy for our conclusions.
Ms. Moir: In conclusion, while we have identified 17 lessons, it’s our view that fundamentally it was the underestimation of the initiative’s scope and complexity that led to its downfall.
It is also important to emphasize the importance of having the appropriate culture within which to undertake these complex transformations. Agility, openness and responsiveness are key features of a culture that needs to be aligned with the magnitude and challenge of the transformation.
We would like to highlight that, at the time of writing our report last summer, these were lessons that we felt were yet to be learned; they were not lessons that had been learned throughout the implementation of TPA.
Finally, we would also like to note that these are not simple lessons with quick fixes. Learning these lessons, and then operating in a manner reflective of having learned the lessons, is actually a very difficult and significant challenge for any organization, especially one as large and complex as the Government of Canada.
There are a number of large and not-so-large transformations under way or contemplated within the federal government. Learning and applying these lessons will be a necesssary requirement to increase the probability of success of these transformations.
The Chair: Thank you.
Senator Marshall: Thank you very much for being here this morning and also for the report, which I found to be very helpful.
As a committee, we’ve been putting a lot of attention on how the government is dealing with the problems with Phoenix now and how it’s going about fixing it, and whether in fact it will be fixed.
I want to reflect on a few things that both of you said. You said there was an underestimation of the complexity of the project. You talked about the need for a road map, and I read into that there wasn’t one. There was an indication that it was actually more than one project, that it should have been several projects and they should have been set up as separate projects. You talked about no one individual or body being assigned the authority to ensure the overall success. I think, Ms. Moir, you talked about the culture wasn’t open to risk and does not reward speaking truth to power. I know that Phoenix wasn’t fully tested as it should have been, that there was a restriction on the complexity of the transactions to be tested. There was no contingency plan.
So now the government is in a situation where they recognize that they have major problems now with the entire system, and they’re trying to fix it.
My question is this: Given the major problems that you’ve identified in your report and the situation before government now — and I notice, Ms. Moir, you said lessons yet to be learned, and we’ve had individuals here from government talking about how they’re going about fixing it — how confident are you that government can actually fix the problems with Phoenix and with the overall system that they have now if the lessons you’ve identified haven’t been learned?
You’re reading the media reports. We’ve had the people in. I’m not getting a lot of confidence in what people are saying, seeing that the fix project is so big that you still need to have these steps. Could you talk about where you think this is going to lead?
Ms. Moir: Sure. Confidence. Well, I can tell you that there’s been a lot of interest in the report. I can tell you that the officials have read it; they’re listening to it. They refer to having the lessons up in their offices and referring to them frequently. We’ve been invited to speak at various fora with senior decision makers and other people leading transformations, including Phoenix and the fixes to Phoenix. We are aware that there’s a new governance system in place. However, we haven’t been privy to the specifics that have been undertaken in order to address it.
At the time of last summer, these were lessons yet to be learned. To what extent they have been learned today, in March, it’s very difficult for me to say, aside to say that there is interest in fixing it, there’s a lot of visibility of the lessons, and we are aware of some steps having been taken to address them.
Senator Marshall: So the government now, in its budget, indicated that it’s setting aside funding to look at options, as they call it, to move away from Phoenix. It seems that we’ve gone through a certain process and didn’t get the results that we wanted. It looks like we’re stepping back in disbelief and saying, “Well, let’s try it again.” With another project of the same magnitude, won’t there have to be major changes in order for a new project to be successful?
Ms. Moir: I would say absolutely, yes. We can only hope that they will take these lessons to heart. As Jim mentioned, the foundational problems were around understanding the complexity. They didn’t understand what compensation advisors did every day, and they didn’t want to hear it. They didn’t understand that there were people working in cubicles physically making changes to the pay system, every moment of every day, to keep on top of employees’ pay. They didn’t understand all the various sub-projects that had to be changed: culture, business process, skills, capabilities. All kinds of things had to be changed.
As they think about the future, as they think about the next generation of pay systems, we can only hope that they will think broadly enough that they can understand everything that’s changing and moving as they move forward. It’s more than just a new system to replace Phoenix. It’s so much more.
Senator Marshall: Given the two options — we have the Auditor General saying, “You can’t scrap Phoenix now,” and then you have the unions saying, “Scrap it” — based on what you know, what option would you follow? If you were making the decision now, would you work with what you have and try to fix it, or would you go back and start from scratch again?
Ms. Moir: Well, I’m not privy to the in-depth analyses to understand how broken Phoenix is. I suppose there is a threshold at which it is so broken that, yes, you need to look at a new system, but I’m not in a position to comment on even what that threshold is. I’m not an IT person. I don’t know where that threshold lies and I don’t know how close Phoenix is to that threshold. I would certainly need to understand those aspects before I could make a recommendation about whether they need a new system or whether they need to fix Phoenix.
Senator Pratte: Thank you for being here. I’m interested in everyone’s role in this. You paint a pretty clear picture of what the public sector’s role was in what happened. If I understand correctly what you’re saying, IBM was not really responsible for the problems. In fact, they did what they were asked to do, and maybe they would have liked to do more but they didn’t have a chance to do more. Is my understanding correct?
Mr. Alexander: From the chance that we had to look at the various documentation and discuss with people, it was pretty clear that the government spent quite a long period of time, multiple years, developing extremely detailed requirements that specified exactly what any system integrator should do if they won the contract. The challenge with that is that it’s very difficult for any government entity to really figure out exactly what it is that should be done in that the people who know best how to do it, they’ve done it many times, for example, the IBMs of the world. Overly specifying and spending a lot of time overly specifying results in a very black and white contract that says, “You must do this, you must do this, and you must do it this way.”
The discussions that we had both with officials and with the private sector indicated that leading practices — and it’s our experience as well — would be that you engage the private sector much earlier in terms of possible approaches and solutions so that then the contract that is eventually entered into is one that both sides can feel very confident of success on.
The short answer is IBM did what we understand is pretty well precisely what the government specified and kept changing through multiple change orders through the length of the contract.
Senator Pratte: But I guess we would have liked IBM still to maybe at one point or another take the initiative and say, “Wait a minute. What you’re asking us to do is not working,” or “This is going to be a dead end;” no?
Mr. Alexander: One of the comments that we made that we referred to was the nature of how the private sector was engaged in governance and in problem solving, and not just IBM but the other private sector firms that were involved. The indications we had from the discussions were that the private sector firms did not have the latitude to discuss with each other how things were going or let’s sit back and figure out how we could do things differently. It was a very structured relationship that didn’t really allow for the creativity that at times can be there and can be very beneficial in public-private sector dealings.
Senator Pratte: The other thing that struck me in the report is that this all seems to be totally apolitical. I mean, this is purely bureaucratic. There’s no minister involved. There’s no politician. There were ministers that made some decisions there, either to launch the initiative and eventually to decide to go online, but there are no fingerprints of any minister or politician anywhere there. Is it because it was purely bureaucrats, is it because you decided not to get involved in the political side of things, or what?
Mr. Alexander: As Sandy had indicated, we were trying to avoid finger pointing either with officials or with politicians who may have been involved. We did refer to decisions that were made by cabinet committees, Treasury Board and so on, and to the extent that we were able to talk to the impact of those early decisions and how foundational those were, clearly those early decisions involved very senior officials, with ministers being involved in decisions that were made back there in the mid-2000s and so on. But we really tried to use this as lessons learned for future ones as opposed to diving in and saying exactly what happened at which cabinet committee or anything like that.
Senator Pratte: It’s clear, at least with the benefit of hindsight, that when the decision was made to go online, testing had not been sufficient. There are problems, which apparently are significant problems. Is it only because we have hindsight and therefore it seems evident to us, or should it have been evident to the people then taking the decision that they should not have gone online?
Ms. Moir: It was definitely evident to people involved at the time, except their concerns were ignored, essentially. They planned to do a pilot, for example, with NRCan, which was cancelled. They never had a sandbox set aside, an independent testing environment. At the time, we were shown all the testing records, all the testing reports, and yet, as you say, in hindsight obviously it wasn’t enough. But even at the time, there were concerns expressed, particularly by departments and agencies but even those within PSPC to say we don’t think we’re quite ready, we’re still having concerns. In our report, we mentioned that we reviewed a testing report from January 2016, so only a month before going live, and there were concerns that we felt were sufficiently critical that would have suggested a pause. So, yes, testing was not adequate, and certain people certainly knew about it.
Senator Eaton: Thank you very much. This is very interesting. I’d like to clarify something from a question Senator Pratte asked about the private sector re the government doing the programming. Are you saying that instead of the government laying out what each ministry needed, the private sector should have brought in, SoftPay and perhaps anybody else who was looking for the contract, and allowed to sit down in each of the ministries and determine what should be in the program?
Mr. Alexander: To some extent, it’s that, but it’s even further back. It would be, “So, government, what’s the problem that you’re trying to solve?” And then keep asking the question of, “What’s the problem you’re trying to solve?” Given that we start to understand the problem you want to solve, that you want a modernized pay system and you want to save some money and so on, then have the private sector propose different ways of doing it as opposed to we’re going to take a particular software package that the government has specified and we’re going to configure and customize that package. So it’s more: Is there a different role for the private sector? Is there a way that risk could be shared between the public and the private sector?
Other jurisdictions have experimented, have piloted and implemented ways of doing procurements that get one or more private sector firms involved in initial work and then almost a winning approach is eventually developed jointly between the government and the private sector.
Senator Eaton: So they almost put the cart before the horse?
Mr. Alexander: They presumed they knew what the solution would be and spent two years specifying it in great detail, and then asked someone to implement it.
Senator Eaton: Having looked at the system — and obviously it’s going to need upgrading or changing — the minister was with us last week, and she was very interesting. She said that she hoped to stabilize it. She wasn’t hopeful about how much she could fix it; she hoped just to stabilize it.
Would your opinion be that because of the complexity and differences in each ministry, that perhaps each ministry should have its own pay system as opposed to trying to have one pay system fit everybody over the whole spectrum of the public service?
Mr. Alexander: Senator, it’s a great question. At the heart of that is this: How much diversity is there in which parts of the public service; and which parts are similar to each other and which parts are quite unique?For example, one of the instances that’s often raised is the shipboard crew on Coast Guard and the complexity of their pay and how that works. Trying to fit everyone into one box is very challenging, especially given the complexity of the business processes.
Senator Eaton: We heard from the Corrections department last week.
I found everything very interesting, but one of the things that stood out for me was a quote from your study:
. . . the study team heard from those consulted from both within and outside PSPC that there was a culture . . . that is not open to risk and does not reward speaking truth to power. Additionally, we heard that this culture of discouraging briefings with bad news was particularly strong at PSPC during the time frame of the initiative.
What kind of steps does a ministry take to change that culture, to make it safe for people to come forward and say, “This isn’t working” or “This is a bad idea”?
Ms. Moir: Thank you for the question. It’s a good one.
Often, when it comes to changing a culture, it needs to be led from the top. The leaders need to model the fact that failures are okay as long as they’re flagged early. A failure such as Phoenix is not okay because nobody caught it and nobody said anything until it was too late. The idea is encouraging a culture where bad news is accepted — and even welcomed, encouraged even — and that has to happen at the most senior levels of the organization, I would suggest from the minister right on down.
While we said that PSPC was particularly strong, perhaps it was the personalities leading the initiative at the time, and yet it was even more than that; it was government-wide, to a large extent. We heard about cases in departments and agencies, people who said, “I briefed up to my deputy,” and then they came back from the meeting and nothing changed.
It’s a pervasive culture in the Government of Canada — risk aversion, the tendency to want project dashboards to be green: “Heaven forbid we have a yellow; never shall we have a red on our dashboard.” It’s not just a PSPC factor.
To address your question, to change the culture, it has to be led from the top. There have to be examples that can be communicated broadly, so not just one-on-one relationships where it’s encouraged, but that the rest of the organization sees that this happened, that it was accepted and that we made a course correction and we are on the right track again.
Senator Eaton: I wish I had known that before we had the minister last week.
Senator Moncion: One of the things I found surprising in the report is that there was only one tender. There was only one company who presented a solution to the government, which was PeopleSoft from IBM.
My question to you is this: How unusual is that? Because when you go out to tender, you usually get more than one company who will provide an RFP or whatnot, or a proposition. Here there is only one, and there had been bad reviews on this software. That’s the first hurdle or flag that we get.
There are two other things that I did not see in your report. I looked at your report, but I didn’t see the project management component — how the project was managed and the different steps that were going to be taken — and the project analysis, the risk analysis. I did not see the risk analysis either. I’m thinking that when you’re looking at pay, it’s probably your biggest risk, because it’s your reputation that will take the beating right off the bat if your people are not paid according to what they’re supposed to be paid. Your report doesn’t look at these things. Did you get any hindsight on these things and how it was done?
Ms. Moir: To respond to your first question, senator, regarding the one bidder, obviously it’s not an ideal situation when you have one bidder. You want a competitive process. But, as Jim mentioned, the Government of Canada came out with a very prescribed: “This is exactly what we want.” They had already decided they wanted to go with PeopleSoft. It wasn’t IBM that suggested it. They were looking for a system integrator that could implement PeopleSoft in the Government of Canada in the following ways. I believe the bidding community considered the requirements, considered the ask and simply felt that they didn’t want to undertake the project. We didn’t speak with other potential bidders, so I’m hypothesizing at this point.
With respect to your questions around project management and the steps, they followed typical best practice project management in terms of having all the various tools. All the various plans were in place and they had the committees in place that were required for large complex projects at the time. However, you raise a good point. I think our lessons would suggest that the Government of Canada should consider some of their project management policies for very large, complex, enterprise-wide initiatives such as this.
In terms of risk, it was identified as a risk. Again, they had all the risk management plans, they had all the risk registers and they were tracking against the risks. Paying employees, as you say, is extremely important and not something to be taken lightly. However, again it was this culture of not reporting bad news. While the risks were identified and understood, they weren’t managed appropriately because nobody accepted the fact that perhaps we were in trouble.
I think, Jim, you would like to say a few words.
Mr. Alexander: If I could. It gets back to the foundational challenges with this. When the project was defined as two separate projects — the consolidation of pay advisors into Miramichi, and then there’s another project, the new system that we’re going to do, called Phoenix — all of the other things that were changing weren’t regarded as projects or interrelated, like the roles of departments and changes to the HR practices within departments.
The risk registers and the project management was actually focused just on the software solution. It wasn’t focused on this massive change that was happening. It gets back to once you start off on the wrong foot, with the scope being too narrow, then you can apply really good project management practices to too narrow a project and miss the really big change going on. That’s the foundational lesson that we started this with.
Senator Andreychuk: Following up on many of the questions already asked, I’m still confused as to where the buck stopped in a bureaucratic sense. We know where it goes; it goes to a minister, ultimately. But was it a committee or was there one person in charge? In reading it, I thought I got the answer, and then I didn’t when it went through.
Mr. Alexander: It’s a really good question, and it should have a really simple answer.
When Treasury Board granted project authority to do the Phoenix project, it gave the responsibility of that to PSPC, or PWGSC at the time. Within PWGSC, there was an official who would be responsible overall for that project.
There was a committee that was used to disseminate and possibly gather information from various levels of committees from deputy minister down to director level, but those were information committees and often used to broadcast information from the project team out to departments. They were definitely not decision-making committees.
It gets right back to, as soon as you say this is a project, that one department will be in charge of and they can determine how to go forward. That’s how the accountability flowed, and in some ways there’s no exit off that track. It’s pretty well set when you set the scope so narrowly and give it to one organization to do without a collaborative governance system put into place.
Senator Andreychuk: Following up on that, this project would have been initiated and the person who would have taken charge at the committees or the department had some confidence that they understood what they were doing. For example, I would walk away. I can’t even get my computer to work. A lot of bureaucrats I talk to are in my position. We’re learning and know our limitations. But there seems to have been some pervasive confidence they could understand both the bureaucracy and what they were doing to it through the Phoenix system.
When you talk about being risk averse, I’m not sure you could delve into it from my point of view, but yes, there’s a culture of being risk averse. You don’t get your bonus if you say, “I screwed up, to be quite honest.” You have to prove you did a good job to get a bonus. It’s human nature.
But there’s another kind of risk averse, which is, “I think I know it better than you.” That’s what I’m getting from this. Somewhere at the start, someone really thought they could manage this whole thing, and what they didn’t understand was their own limitations.
Mr. Alexander: I think that’s an excellent observation. Sandy indicated in the briefing that one of the things that was totally misunderstood was the role of compensation advisors, these 2,500 or so compensation advisors who worked on the old system. The old system, we understood, made lots of mistakes, was complex and kludgy to work with, but these compensation advisors would know the community they were working with, they were embedded in the department or agency and they made things work. If you had a problem, you could go to the compensation advisor and he or she would work through the problem, get your pay fixed and everything was fine.
This underestimation of the knowledge that was resident in all of these compensation advisors was probably one of the other really big keys there, because once those compensation advisors disappeared, whether it was the old system or the new system, no one could figure out how to get pay to work because the system doesn’t do enough. You needed these people and the knowledge they had gained over 25 or 30 years.
The consolidation of pay administration into Miramichi and the age profile of the compensation advisors meant the government lost a large number of them. They retired or went to other jobs, and all that knowledge of how to make pay work for corrections officers, shipboard officers and nurses in the north walked out the door.
Ms. Moir: I would like to address the comment about capabilities of understanding pay, managing projects and seeing changes.
I think we’re explicit in our report that the project team did not have the necessary capabilities to manage an enterprise-wide project of this nature, but they thought they did. That was evident in the RFP that was issued and laid out the requirements. It was evident in the fact that they were not open to advice in involving the private sector and governance and getting their views on the plans. It was evident in reactions departments and agencies got when they voiced their concerns with the project team. They were pretty much dismissed. They said, “Don’t worry, we understand, we’ll fix it.” It was always, “It will be fine, it will be fine.” And it’s evident in the fact they were reporting up constantly good news.
As I say, we say in the report we don’t know whether they were reporting up good news because they didn’t understand it was actually bad. Was it because they thought they could fix it in time? We don’t know why they weren’t doing that. Was it because they were worried about their bonus? We don’t know why they weren’t willing to communicate the bad news. Part of it, at least, boils down to capabilities to manage a project of this nature.
Senator Andreychuk: We’re in the middle of Phoenix. It’s a struggle. The old system is no longer there, except in certain departments that weren’t part of it. So if you are sitting in cabinet, do you move forward? You can’t move back, I don’t think. Am I reading between the lines that we’ll have to bump along, but we should be starting today in developing what the government really needs in a pay system that takes into account all of the things you said? That isn’t going to be a short-term enterprise. It’s a project that will take time. It also needs political oversight as well as a competent person who may have to be hired for the purpose, rather than promoting someone up to take on the project exclusively.
Mr. Alexander: I think those are good observations. I’m sure you know the rough rule of thumb that if it takes a given number of years to create a problem, it’s going to be something similar to that to fix the problem as well.
Senator Moncion: I think what is happening is that a lot of problems are being managed on a day-to-day basis now and corrections are being put in place as they come up. I might not understand it well, but I think what they have right now is they’ve put enough people there to manage the pay system right now, so now they have a lot of corrections to do, but I think they’re trying to stabilize the weekly or biweekly pays so that staff get paid properly. I think with Phoenix, after a couple of years, they understand how it works. A lot of people are being paid well and according to the contracts.
Yes, the problem is here right now, but to start with another system now is a very big project. I think they’re just trying to work with and stabilize what they have. When we spoke with Statistics Canada, that’s what they were telling us. They’re managing as they’re working forward. I’ve asked the question at the Senate, and it’s the same because they took the problems right off when they started occurring and corrected them, but it’s a small group with people in place to do the work. Am I correct in what I’m saying?
Ms. Moir: We’re not currently involved in fixing it, so it’s hard for us to say.
Senator Black: I want to thank you both for being here. I find your report to be very open, very honest and, ultimately, devastating.
From the evidence you’ve given the committee, I have certainly formed an understanding of the government’s role and failings. I think that’s very clear from what you’ve shared with us.
I am less clear as to the role that IBM has played through this saga. You’ve made some helpful comments about the restrictiveness of the process, and I understand that. But I am puzzled that a firm of their reputation and depth would be, if I follow your suggestions through to their conclusion, so intimidated by their masters that they would not either recognize or identify that the project was broken. I would have thought, as an outside consultant, they would have had some obligation or responsibility to do so.
How constrained was IBM? Do you believe those constraints were so effective that they basically neutered them? Let us have it.
Mr. Alexander: Well, it’s a really good question. In doing the lessons learned report, we really tried to focus on trying to understand enough of what seemed to happen around transformation of pay administration that we could come up with lessons learned for future transformations. Although we had discussions with current IBM officials who were leading Phoenix, we didn’t, for example, go back and talk to individuals at IBM who were there in the 2012 time frame or in the initial bid or talk to other firms as to why they didn’t bid and why there was only one bidder.
Given the scope of what we had to do, we didn’t really dive into, “How constrained did you feel,” but in talking with the firms that were involved in this in various roles besides IBM, we very definitely got the feeling that it was a very strict command and control. “This is the contract, and we’ve asked you to do this. We haven’t asked you to do these other things, so don’t do them, please.”
Having worked in Ottawa and worked with the federal government, procurement has a lot of authority and a lot of power. When they ask you to do things and the contract says this is exactly what you’re supposed to do and not supposed to do, you can have some discussions, but in the end there is a black and white contract. The federal government is a very large contracting entity.
Senator Black: Why did the Government of Canada, given that point of view, even hire an outside consultant?
Mr. Alexander: Generally, system integrators are hired because the government doesn’t have the experience configuring PeopleSoft or SAP or any of these large enterprise resource planning systems. There is expertise that would be imported by these firms. They have done it before. Whether this team had actually configured PeopleSoft for a complex pay situation, I’m not too sure, but they would have a lot of intellectual property as a firm that would say how can we do this quickly in terms of translating requirements into configurations and so on.
Senator Black: Do you find any contradiction between that point of view, which I completely agree with, and the fact that they were completely put on a short leash? Is there any contradiction there? “We’re hiring you because you’re the best, you can help us integrate, but by the way, we know how it’s done, so we’re going to tell you.”
Mr. Alexander: Those are always challenging dynamics to work through. It would be interesting to discuss with various firms why there was only one bidder. This was a large contract that would provide a revenue stream for a number of years, and most private sector firms would go after that. The Government of Canada pays its bills as well. It’s not as if something would happen to the Government of Canada. It’s good, reliable money. The fact that only one firm is there is probably worth musing about, but it’s not something we were able to do.
Ms. Moir: This trend away from advice and toward task authorizations or change orders is government-wide. Certainly PSPC does it, but other departments do too. There’s this feeling that the private sector is there to do our bidding. “We know what we need. We know what’s best. We’re going to hire you to write this report, to do this set of tasks, to build this system the way we want it built.” That’s essentially what happened. It’s pervasive of most contracts that come out from the federal government. It’s prescriptive. The advisory function of the private sector, of the consulting firms, has been decreasing over time. Strategic advice from outside is not as valued as it used to be, quite frankly. It’s not a surprise to me that IBM, having received this contract, when they tried to say, “Well, would you like us to review that plan for you?”, it was, “No thank you; we're all set,” and then, “Okay.” As Jim said, the contract is in black and white. “I’m getting paid to do A, not B. They told me not to do B. They think they know what they’re doing. I’m going to do A.”
Senator Cools: I would like to thank the witnesses. In a special sort of way, I admire you. You’ve said some very daring things that touched me deeply. You talk about the pervasive cultures in government. Risk aversion was one of them. The other thing you raised was the disinclination to report. I assume if they were reluctant to report, they were also reluctant to hear bad news.. When I put this in my mind and grind it around and look for the results to come out, the only way to describe this is a colossal failure of leadership and management at every level.
I don’t understand how the Government of Canada could be in this position. I still don’t. I've been grappling with this for a few days. If we were in a developing Third World country, you might be able to understand it and accept it, but we’re not there. We’re in one of the most modern and progressive states of all time, known for its proper dealings all over the world. Do you have any idea as to how these terrible events could have happened? It’s incredible. It’s like we’re in outer space.
Ms. Moir: Thank you for your comments and observations. You’re correct. Reading the report, it's damning, for sure.
Coming back to the fundamental underestimation of the complexity, they thought they were doing two things: moving compensation advisors and centralizing and building. We heard the phrase, “We’re replacing a calculator.” That’s what they thought was happening. “We are moving people around. We do that all the time. We centralize services, we workforce adjust our staff, that’s regular business and we build systems. We’ve done that before. IBM has done that before. Contracts look like this. This is what the tasks are.”
If they had properly assessed the complexity and the size and the scope, and said that they were changing business processes, changing culture, or, “We’re really bad at this. How do we manage around that culture?” — they needed more private sector oversight, quite frankly. Hire someone to come in and do a large study halfway through and see if you’re on track. But they didn’t understand the scope, and therefore they didn’t apply those necessary tools to ensure that things were on track.
Senator Cools: Could you attempt to even suggest to us what the total cost of this failure has been? It’s a calamity. Have you any idea what the total cost will be to government at the end of the day?
Ms. Moir: I’m not in a place to guess. I’m sorry.
Senator Cools: I have no doubt it will be huge. It is quite a magnitude.
Senator Black: Senator Cools, the minister told us last week it was $642 million and counting.
Senator Marshall: I’m back to thinking about the future. I found the report really interesting because it tells what happens. Unless the lessons learned were really learned in government, they’re not going to be much benefit.
The government is trying to fix Phoenix. The system itself was so huge and complex, and you’re saying that one of the reasons for the problems that they had was they underestimated the complexity. It seems to me that it’s also possible now, because of the problems they’re encountering, that they are underestimating the complexity of the problems that they have to fix.
You talk in your report about all of these individuals or departments — Treasury Board, Chief Information Officer — all of these positions that should have had bigger involved in the project. You’re talking about the culture. Isn’t it still a problem? I mean, the culture hasn’t changed, and now we’re looking at the possibility of a new system.
The public service has to cope with the fixing of the system, and now it looks like they might have to cope with an entirely new system. You were in there talking to people. How capable is the public service now of going forward and even doing another project of this magnitude? They aren’t even getting paid right now, or they’re not getting paid accurately. That’s a problem. They have this risk-averse culture. What’s your assessment? How capable would they be to take on another big project, even to fix this big problem?
Ms. Moir: Well, it certainly depends on the extent to which they have read these lessons and started applying them, for sure. There are lots of very smart, intelligent, effective people within the federal government. Hopefully, as they move forward, they will engage private sector experts early and often, and different experts, not just experts from one firm or even experts only from Canada, but look outside the country and look for people that can guide them to make sure that whatever comes next is fully scoped and properly thought out.
In terms of the stabilization of the system, if this committee had called the project responsibles before them even two or three years ago, you probably would have had a couple of representatives from PSPC before you. On February 6, you had representatives from OCRO, Comptroller General and PSPC, so they are starting to work together. They are starting to understand that it’s not just the people building the system that need to be accountable for it.
I don’t know if that answers your question fully, senator.
Senator Marshall: I’m more interested in the public service as a whole having to implement a new system or even having to implement the fixes to the current system.
You spoke to the people. You interviewed them for your report, so you must have some type of assessment. I would think there would be people now in the public service who, if you told them, “By the way, we’re scrapping Phoenix. We did this multi-year project over 10 years, and listen, it didn’t work out, so we’re going to do it all over again,” I can imagine some public servants throwing up their hands and saying, “I give up. I’m not even getting paid right. I just give up.”
I’m just trying to get a handle on what’s the feeling in the public service? What do you think? Do you think they would be capable of another 10 years of doing a similar project? It’s the same people.
Ms. Moir: It’s the same people, and there’s no doubt that when we did workshops with compensation advisors and employers and line managers, the incredible frustration, obviously, but also fatigue, and that was even a year ago. We haven’t spoken with public servants on Phoenix since then, but I can only imagine that that sense of frustration and fatigue has only deepened. However, we have a system. It’s been two years since launch and we haven’t addressed things. So it wouldn’t surprise me if there was a sizeable portion of the public sector community feels we need something different because what we have isn’t working, but that would be a supposition on my part.
Senator Marshall: “Fatigue” is a good word. Thank you very much.
Senator Pratte: I want to go back to what you said about the loss of hundreds of compensation advisors. That was, of course, a result of the consolidation project.
I’m a bit confused as to the linkage between the modernization, the Phoenix project, and the consolidation project. Did both projects just happen to be launched at the same time, or is there really a link between the two? Did they need to go together, or did it just happen and they thought they could do both at the same time?
Mr. Alexander: Our understanding from review of documents and from discussions with individuals who were involved at the time is that these two initiatives were linked conceptually, that the modernization of the pay system, the actual software solution, would be such that a lot more pay files could be handled by one person and therefore we didn’t need as many people. Similarly, consolidating people into one location would allow for a large centre, critical mass and hopefully some efficiencies out of that.
The order in which they were done was — we spoke to that in our report, that to actually harvest the savings by releasing the people and consolidating into Miramichi in advance of the system being up and being properly utilized was backwards from what anyone would do. Similarly, anticipating that you would get savings as soon as it went live, that never happens. There’s always a drop in productivity and then you will eventually come back up again.
So they were linked, but they appear to have been done in the wrong order, for possibly — well, I won’t speculate why they were done in the wrong order.
Ms. Moir: I would say not only the wrong order, but they weren’t even managed together. They were part of the same Treasury Board submission. They were talked about together as TPA, these things together, but you can’t find a report or a plan, a change management plan, that talks about them together. It’s always pay modernization or pay consolidation. The senior project advisory committee, the SPAC, was focused on pay modernization exclusively. They had separate committees for the first several years of the initiative. It wasn’t until 2013 that the senior project advisory committee started getting updates about pay consolidation. But they were updates. It wasn’t a meaningful linking. So while they were under the same TPA umbrella, they were not managed together in any way.
As Jim says, in our opinion they were implemented in the wrong order, but they were running on such completely parallel tracks that nobody thought about, “Let’s do this and then that,” because the pay modernization team was thinking about replacing a calculator and the pay consolidation team was thinking about establishing a pay centre in Miramichi.
Senator Pratte: Is my understanding correct from your report that once they let the 600 compensation advisors go, in a way, they had to go live because they didn’t have those advisors anymore. They just had to go live with the new system because they didn’t have the old system to rely on anymore.
Ms. Moir: That’s correct. It’s in our report, as you say. That’s certainly what everybody told us. “Even if we knew how much trouble we were in, we still had to go live because we didn’t have compensation advisors in departments and agencies any longer.”
However, our report also states that it’s not impossible to reverse these sorts of decisions if it’s dire enough. I’m not an expert in these things, but I believe it would have required ministerial sign-off to say, “We changed our mind, we don’t want to workforce adjust these people. Please come back. Your job isn’t actually up.”
It's not that it was impossible to reverse that decision, but it would have been a major decision and it would have required everybody to have a full grasp and understanding of how much trouble we were in at the time, which wasn’t fully understood or accepted.
Senator Pratte: Actually, what they’re doing now is calling these people back.
Ms. Moir: That’s correct.
Senator Eaton: Just to get back to your study again, union officials told us, when they came here, that there had been no interaction about the pay problems between Phoenix and the officials in Australia, the Queensland Health Department, which also implemented a similar system from IBM to manage its 78,000 employees. In your report, there was no mention of the Queensland experience. Why? Because you didn’t feel it was instructive? That’s one question. Do you think government itself consulted with other organizations with pay problems?
Ms. Moir: We didn’t include it in our study. Our mandate was fairly clear: talk to the people involved in what happened over the last eight years or so and try to find out what happened wrong. We certainly looked at international best practices. We sort of referred to those in our report — “As leading practice would suggest . . .” So you’re correct; we didn’t specifically make mention of the Australia case.
As to whether government officials consulted them or met with them, it was not mentioned in our consultations. We didn’t hear about the meeting they had with Australia or anything like that. Why didn’t they meet with them? I don’t know. Although, recall that they thought they were simply replacing a calculator, so perhaps they didn’t see the need.
Senator Eaton: Are there countries in the G7 who have centralized pay systems for their public services?
Ms. Moir: I don’t know.
Senator Eaton: Do the U.S., France, Germany, England?
Mr. Alexander: When we asked that question, we were given examples of changes that have happened in, for example, Alberta. I think they went to a centralized pay system.
Senator Eaton: I’m not laughing at you, but Alberta doesn’t really compare with its population.
Mr. Alexander: It’s not the right scale at all.
We were not able to find examples of that. We were told, as we dug in a little bit, that one of the things that was very unique about the Canadian solution that IBM was asked to put into place is that PeopleSoft is a software package that does all of human resources and all of compensation and things like that. It will actually even do finances and pieces like that. But what PSPC or PWGSC asked to be implemented was only the compensation portion of PeopleSoft. What we had heard is that no other organization that anyone was aware of had ever split off HR from compensation and then tried to stitch them together afterwards. Because all the departments ran HR systems, some of them being PeopleSoft, and then PSPC ran Phoenix. They come out of the box sort of talking to each other. They split them apart and then tried to knit them together again.
That’s, again, one of those things that, in retrospect, you look at and wonder: Why was that split done? Because it just increases risk and complexity so much.
Senator Eaton: That’s one of the things that Minister Qualtrough said last week, that they were trying to put back together again Humpty Dumpty. HR systems.
Mr. Alexander: Exactly.
Senator Eaton: One last question: Were you given complete, free access to everybody? Did both ministers, Treasury Board and Procurement Services, everybody, help you? Did everybody speak? Were they open? So you had no problems?
Ms. Moir: We had no problems. We didn’t speak at the political level, so we didn’t speak with the ministers or the President of the Treasury Board. We did speak with the deputy minister community, however, right down to employees. If someone suggested a name in an interview, for example — “Make sure you talk to this person” — we would make an effort to talk to that person. Same thing with documents. “Have you seen the plan that came out in 2013?” We would ask to see the plan from 2013, and we had full access.
Senator Moncion: My question is about the project analysis that was done. I don’t know if you’ve assessed that, but how knowledgeable were the people who were working on the software solution or making comment or bringing forward the expectations of the system? How knowledgeable were they on HR and pay issues? Because it’s complex. A lot of times people think that pay is easy. It’s not. It’s a very complex department to run because there are so many checks and balances, and there are so many things that need to be integrated. I’m just wondering: How knowledgeable were the people who were putting this project forward? How knowledgeable were they about pay systems?
Mr. Alexander: Our understanding is that there was a lot of input from those who would be involved in PSPC and the regional pay system and the role of Public Works at the time in doing the pay process. What there wasn’t necessarily as much knowledge on was the nuances of pay that would happen in Corrections or in Health Canada for nursing in the North or for Coast Guard ships and so on. I think there was really good knowledge of a lot of the pay that would happen that PSPC would see, but, as I mentioned earlier, so much of it was in the minds of those 20- and 30-year veterans of compensation who would work in every department. I remember, when I started off way back when, when I was doing shift work within the government, I would go and talk to the compensation advisor, and she’d help me with a problem and explain it to me or fix it. That’s the part, that complexity and diversity, that to our understanding wasn’t really well understood.
The Chair: Before we close, honourable senators, I would like to ask a few questions of our witnesses.
For the record, going back to February 21, we have a total exact number of 626,000 outstanding issues relating to documents to be shared, documents to be corrected, and also the pay system itself within that context.
According to your report, in one of the lessons, lesson 14, you say that the government should learn from the TPA initiative to launch any required new IT solution only after it has been fully tested, with end-to-end real-life simulation — and you touch a bit on it — using a broad spectrum of real users.
Were you surprised to have to remind the federal government to sufficiently test such a complex technological solution before launching it, keeping in mind that many departments and agencies did bring to the attention of government that they were going to have problems?
Ms. Moir: Yes. Were we surprised? Absolutely. Pretty much everybody we talked to talked about the testing and the fact that it was inadequate. We talked to people who actually participated in the tests. They did some testing in departments, but it was done over weekends. It was done at the last minute. It was done, for example, in Parks, at the wrong time of year, when they didn’t have any students on the roster. The whole student issue has been a huge complicating factor because of the seasonality and the fact that they have to do terminations and hires and the rest of it. As I say, we reviewed that report. It was dated January 2016. We were surprised that things were moving forward despite the state of that report that talked about outstanding issues and coded them in terms of how critical they were, things like acting, which we know is part of the sticking points that we’re still living with today.
The Chair: You’re professionals. I’m not putting an iota of doubt on the professionalism of your company and what I’ve seen this morning. However, the question I’d like to ask is this: How long do you think, with the outstanding issues that we have now, it will take for the present system, if it can, to find solutions? How many years do you think it will take?
Mr. Alexander: Let me just go back to a rule of thumb. I would say that if it took from 2008 when this was conceived, or maybe 2012 when the contract was there, to maybe you could say 2016 or 2018, where we are now, those numbers of years, whether it be six years or whatever, if it takes that long to get yourself into a hole, it will take something like that. If you can do it in half the time, you’re doing great, but it’s not going to be months when it has taken multiple years to get into a problem of the magnitude of this. This is an incomprehensible problem in terms of how complex it is and how many people are affected in so many ways.
The Chair: To the two witnesses from Goss Gilroy Inc., thank you very much for your professionalism and the information you shared with us. It was no doubt very informative and even educational.
Honourable senators, I want to thank you very much. If you permit me, I would like to have a short meeting, an in-camera discussion, in order to share with you the agenda coming in the next few weeks.
(The committee continued in camera.)