THE STANDING SENATE COMMITTEE ON SOCIAL AFFAIRS, SCIENCE AND TECHNOLOGY
EVIDENCE
OTTAWA, Thursday, March 12, 2026
The Standing Senate Committee on Social Affairs, Science and Technology met with videoconference this day at 10:30 a.m. [ET] to examine and report on matters related to the impact of artificial intelligence in Canada.
Senator Rosemary Moodie (Chair) in the chair.
[English]
The Chair: Welcome to this meeting of the Standing Senate Committee on Social Affairs, Science and Technology. My name is Rosemary Moodie. I’m a senator from Ontario and the chair of this committee.
Before we begin, I’d like senators to introduce themselves, starting with Senator Burey.
Senator Burey: Good morning. Welcome. Sharon Burey, Ontario.
Senator Senior: Good morning. Paulette Senior from Ontario.
[Translation]
Senator Boudreau: Good morning. I am Victor Boudreau from New Brunswick.
[English]
Senator Arnold: Dawn Arnold, senator from New Brunswick.
Senator Hay: Katherine Hay, Ontario.
[Translation]
Senator Petitclerc: I am Chantal Petitclerc from Quebec.
[English]
Senator Muggli: Tracy Muggli, Treaty 6 territory, Saskatchewan.
The Chair: Thank you, senators. Today, the committee continues its study of matters relating to the impact of artificial intelligence in Canada. This study will examine issues including data governance, sovereignty, ethics, privacy and safety, and the risks, benefits and social impact of artificial intelligence in Canada.
Today, we have the pleasure of welcoming Professor Yoshua Bengio, Full Professor, Université de Montréal. Just a little bit about our distinguished guest. Professor Bengio is professor of computer science at the Université de Montréal. He is the Co-President and Scientific Director of LawZero and Founder and Scientific Advisor of Mila. He is also a Canada CIFAR AI Chair and is considered one of the world’s leaders in artificial intelligence and deep learning, and is a recipient of the A. M. Turing Award, which is, in the world of computing, the Nobel Prize of computing. He is the most cited computer scientist worldwide. We are very pleased and honoured to have you here, Professor Bengio. Thank you for joining us today.
For your opening statement, you will have five minutes — and we’ll be generous with those five minutes because we really want to hear from you — followed by questions from the committee members. Professor Bengio, the floor is yours.
[Translation]
Yoshua Bengio, Professor, Université de Montréal, as an individual: Thank you. I am happy to discuss these important issues with you.
[English]
As I’m sure most of you know, and is at the heart of what I’ll talk about, the scientifically observed fact that the capabilities of AI are moving very rapidly. In fact, just in the past year, even the last few months, we can see AI capabilities continue to grow, sometimes even in concerning ways, while, at the same time, the efforts of leading companies working on AI to mitigate risks are not catching up. I understand that it is difficult to project ourselves in a future where there are machines that are at least as competent as us for many skills, but that is the default if current trends continue.
I also understand that it is difficult to grasp the potential impacts of such powerful AI if we do stay on the current path.
A good starting point to understand this is to remind ourselves that intelligence gives power. There is increasing power concentration in AI, both in access to compute and the ability to develop and deploy frontier AI systems, the most powerful ones. If trends continue, this power can be misused by nefarious or power-hungry humans. It could have unforeseen social consequences, or it could potentially be turned against humans by the AI themselves. This power dynamic between the major AI companies exists because they themselves perceive this as a winner-take-all race where none of them, because of the dynamics, are sufficiently incentivized to create trustworthy models or products that are safe for citizens and businesses.
We are already seeing the impact that unsafe AI development can have on Canadians and individuals around the world. This includes deepfakes, cyberattacks and other nefarious uses of AI, such as scams, frauds and disinformation. However, some issues also stem from regular uses of general-purpose AI. For example, we have recently witnessed a growing phenomenon of emotional attachment and AI psychosis where citizens use AI, become intimate with it, and that can lead vulnerable people to harm themselves and others. There are several cases in front of courts, for example, of suicides.
These harms show no signs of abatement. Let me try to explain a bit what is going on from a technical standpoint. It’s related to the notion that scientists call misalignment, which means that AIs have their own goals and that sometimes these goals do not align with our intentions and instructions, and these companies don’t know how to fix this. This issue includes AI allowing bad actors to use AI for dangerous purposes. It also includes AI behaving in deceptive and self-preserving ways, as has been reported multiple times from multiple models, to the point where in these experiments AIs are willing to blackmail, to hack computers or lie in order to preserve themselves.
Now, there’s another recent development that makes this situation even more concerning. In the last few months, there have been tests, again reported by several companies, showing that the most advanced models appear to know that they are being tested, and then they are acting to pass those tests, potentially hiding dangerous capabilities or bad intentions to achieve goals that we would not agree with. This means the way companies are currently mitigating those potentially dangerous aspects is unreliable.
Let’s move to geopolitics. Even if we figure out how to avoid autonomous bad AI behaviour, there are humans who could use these reliable AI systems to secure their power. For countries like Canada, having reliable access to powerful frontier AI systems will surely be one of the most defining factors of economic competitiveness, not to mention ensuring that we can preserve our sovereignty.
Whoever controls these systems and their access — it could be people, corporations or countries — will have enormous power. Think about how export controls can be set up from one day to the next and cut access to models that come from a different country. That sort of thing could be a major threat to Canadian sovereignty, Canadian democratic institutions and the Canadian economy if we don’t invest rapidly in building safe frontier AI. Individually as a country, we probably don’t have sufficient resources to do it. In the last few months, I have been saying that the best bet for Canada is to do it in partnership with like-minded countries.
There’s also a governance issue around this. I believe that the only stable, long-term option in terms of both managing catastrophic risks and avoiding threats to democracy and sovereignty is the establishment of mutually beneficial international coordination of such powerful AI systems. I know it sounds idealistic, and current international politics don’t seem to go in that direction, but there are other countries that feel exactly the same way, and we can start there.
To mitigate the major economic and security risks that I’m talking about, Canada should deepen its collaborations with other middle powers.
We must work on two fronts.
In terms of policy, we need to work on national laws and international treaties, ensuring more robust societal and regulatory guardrails. This means requiring greater transparency from AI companies, as well as putting in place regulations to steer innovation while mitigating the risks that currently limit trust and, by extension, safe adoption.
If you think about the tragedy in Tumbler Ridge, we saw how a lack of corporate accountability and government oversight can have major, devastating consequences.
On the scientific front, we need to understand better how to design AI that will be highly capable and will not harm humans — even under user instructions — and make sure that we can always maintain control of AI models so that they behave in really safe ways.
As a researcher, I’ve dedicated much of my work over the past few years to both sides of these efforts: the policy side and the scientific side.
Last year, I launched a new non-profit organization called LawZero to tackle the technical issues required to develop safe-by-design, reliable and trustworthy AI.
On the policy side, I have also been chairing multiple committees, in particular the International AI Safety Report — backed by 30 countries, as well as the UN, the EU and the OECD — to establish a shared scientific understanding of these issues. I have also just been nominated as co-chair of the UN’s Independent International Scientific Panel on AI.
I believe these two kinds of steps — on the science side and the policy side — are in the right direction, but far more effort is needed.
To ensure a world where future generations will thrive, we must use our wisdom and our empathy to steer the development and deployment of AI safely and for the benefit of all, and where AI is not used as a tool of domination. Thank you for your attention.
The Chair: Thank you, Professor Bengio. Thank you for your patience and understanding, senators. We allowed Professor Bengio to go well beyond, but we wanted to hear what he had to say.
We will now proceed with questions from committee members. For this panel, senators have four minutes for your questions, and that includes the answers.
Senator Burey: Professor Bengio, it is indeed an honour and a privilege to be speaking with you and for you to appear at our committee today. As a pediatrician who has worked a lot in children’s mental health, this is, of course, of particular importance to me and to our committee.
Regarding the development of AI superintelligence, which I’m thinking you’re saying is frontier AI, could you discuss the guardrails and safeguards that, as legislators, we need to focus on — you started talking about that a little bit — meanwhile, allowing for a regulatory environment that can fully realize the promise of AI? That’s the first question. I’ll give the second one because I know I will run out of time.
Knowing your extensive work in the responsible development of AI, could you share your insights on the social impacts of AI that we need, as a committee, to focus our work on, especially regarding children and youth?
Mr. Bengio: I’ll try to be clear and brief at the same time.
On guardrails, superintelligence and frontier models: For clarification, “frontier” just means the models that are the most powerful now. That keeps moving. That’s why it’s called the frontier. Superintelligence doesn’t exist yet, but this is where all the leading companies say they want to go, and it’s also where many scientists think that if trends continue, we will get to.
The most dangerous AI, in all the ways that I have described, are those superintelligent ones that don’t exist yet, but we are seeing movement in that direction.
In terms of guardrails, what can Canada do? The models that are at the frontier, the most advanced ones, are mostly in the U.S. and China. We should definitely have regulation. I think a good template for this is what Europe has done with what’s called the General-Purpose AI Code of Practice. I did participate in these committees as a working group lead, and I think it’s a good template. There’s also the Californian regulation SB 53, which I think is a good inspiration.
In general, what these regulations do is they ask companies to be more transparent so that they have an obligation to document their risk management process and the evaluation of risks. Of course, the regulator should be able to say, “Oh, wait, this is too dangerous. You have to do something. You can’t continue in this direction.”
On children, this is a thing that a lot of people are worried about. It’s fairly new. It’s interesting to ask ourselves: Did people foresee this one year ago? And the answer is no. It’s an example of what was unknown just a year ago, but because AI is now so good at interpersonal interaction and it has been designed so that people will want more. It’s what we call sycophantic. In other words, it’s trying to please us in an exaggerated way, even lying to us. We see all of these issues.
Again, I think this needs to be tackled with regulation that forces the evaluation of the kinds of risks that come with, for example, children using these things.
I think completely banning the use of AI for children is maybe an option, but it comes with negatives. I would prefer a world in which there are guardrails inside the AI when the AI is talking to vulnerable people, and that these guardrails have been evaluated technically by the companies who build AIs.
Senator Burey: Thank you.
Senator Hay: Thank you so much, professor, for being with us today. I have to say, AI can be very frightening, especially when we’re hearing and doing this study in the detail that we are. Yet a steady state and head in the sand is just not an option.
My question touches on what you said earlier. Given the concentration of AI development that is in the hands of U.S. and Chinese companies, what concrete steps should Canada take now to ensure it retains meaningful sovereignty over AI systems that would govern our critical infrastructure — health care, public service and so on? I have a follow-up if there’s time.
Mr. Bengio: This is a great question.
First, we have to understand that it’s not like there’s only one kind of AI. The American and Chinese companies that are leading all use more or less the same recipe. But it is a choice that we should be able to make globally, at least in our country, to steer the development of AI in directions that are going to be both beneficial and safe. That’s the first thing.
As you noted, what can we do here in Canada when they’re developed in the U.S. and China? Well, we can move in the right direction. That is, we can talk to our partners, other countries that do take those questions seriously, that are worried about the effect on sovereignty and their citizens and safety and so on, and start moving toward international agreements that would mitigate the risks. That is the only way.
We shouldn’t be waiting for the U.S. and China to make the first move. I think if enough countries move together toward collective commitments to the development of AI that helps humans flourish, not bringing crazy risks and not used as a tool of domination, these are the main things that should be in a treaty, agreement or declaration. That’s where we should go.
If enough countries stand together with this sort of will, and they represent a large economic bloc, similar in spirit to what I see our governments trying to do on the economic side, I think that it will have an impact.
The other thing that I think is related — I just talked about the regulatory aspect and the governance aspect, but I think Canada can also work with similar, like-minded countries in the development of technology that will be both beneficial and safe and can compete with large models from the U.S. and China.
Senator Hay: What is the accountability reality from these large, multi-trillion-dollar market cap companies like Meta and OpenAI to actually not move into that world of AI for domination?
Mr. Bengio: There’s none, basically. The regulations in California and Europe that I talked about, and actually, in New York as well, are still not in place, and it’s not clear how strong the regulators will enforce those principles.
Courts have some power. There are, currently, lawsuits in the U.S. For example, there’s a grey zone in terms of liability and responsibilities right now in the U.S. I’m not sure how it is in Canada. This means that companies don’t feel a lot of pressure to behave well. Yes, this is very concerning.
Senator Hay: Thank you very much.
The Chair: Professor Bengio, when you talk about like-minded middle powers, what countries are you thinking about?
Mr. Bengio: I’ve been visiting many of these countries in the last six months, talking to people in government. I consider pretty much all of the European countries, including the U.K., Japan, Korea, Singapore, Australia and New Zealand. I think even countries like Brazil, eventually India, could be part of such coalitions.
Senator McPhedran: Welcome, and thank you very much for bringing your expertise to us. It’s very much appreciated.
Given that the frontier, the AI landscape continues to shift and we have new threats emerging, do you have any specific recommendations for an AI safety framework for Canada but also linked internationally? I’m just looking for more specificity.
Mr. Bengio: Yes. I recently signed a declaration that just came out a couple of weeks ago of AI with human-centred red lines and constraints, but it’s a declaration of principle. It could be the basis of international agreements.
I mentioned the existing regulations in Europe, California and New York. That could be a starting point for Canada. Canada also needs to worry about privacy and data access controls. As you may know, the U.S. government, for national security reasons, can have access to anything American companies do, even if they are based in Canada. If the data centres are in Canada, they can, in a way we might not know, access that, so we need to prepare to have alternatives to ensure our citizens and our national security concerns are protected.
Finally, we can help contribute scientifically — that’s the effort I’m doing at LawZero — to develop methodologies that currently the leading AI companies are not developing because of the race dynamics that would at least make AI safe by itself. It doesn’t solve the problem of abuse of power, but we don’t want AIs to be unreliable and have their own dangerous goals. That has scientific solutions, but it will take effort and is something we should do with other countries because of the efforts needed.
Senator McPhedran: If I can clarify with you in using the term “efforts”? Are we talking laws?
Mr. Bengio: No, I’m talking more R&D investments and partnerships with other countries that have talent, compute and capital energy, as we do, but it might be complementary. The law side is mostly regulation and international treaties, and we should not wait for the hegemons to jump in. We should start flexing the multilateral muscle about how we agree together on this planet to manage powerful AI so that it benefits all.
Senator Arnold: Thank you so much for being here with us today. We’ve had some other witnesses in — your colleague, Professor Hinton — and the words that keep coming up are things like betting on love, empathy, maternalistic, human joy and helping humans to flourish. You mentioned this human-centred approach you would like to move toward. What are the red lines you referenced in that?
Mr. Bengio: I don’t have to bring up the website, but I’m going to try to give you high-level elements.
One of the central ideas is that AI should remain to the service of humans. Furthermore, not just a handful of humans or a couple of countries, but to the service of humans globally through democratic means and the joint decision making of countries around the world.
Second, we should ensure we don’t build AI that is, for example, super intelligent or smart enough to evade our control. That’s kind of connected to this. We already see evidence of AI having capabilities that are just not acceptable, such as being able to hack other computers, to copy themselves in a different place and run themselves in a different place.
One general principle that covers a lot of ground is AI should be tested for all of these dangerous capabilities of misuse and loss of control. If companies are not able to provide strong assurances, scientifically speaking, that their systems will not cross those red lines and will not be dangerous to people, then they should not be allowed to deploy them. If we had more of these kinds of legal constraints, that would put huge pressure on companies and markets to do the right thing.
Finally — I talked about it earlier — we need to agree internationally that no one will abuse the power of AI against others. That will be difficult, but there is technology and research being developed so that it wouldn’t just be a legal commitment but something that can be technically verified, that people are not, for example, building AI that is too powerful or hasn’t been tested properly.
Senator Arnold: Another recurring concept is AI being part of public infrastructure from a sovereign perspective. However, what I’m hearing you say is we need to work with like-minded countries.
Is there a way that we could think from our nation’s perspective of AI as public infrastructure so we can put those guardrails in place?
Mr. Bengio: Absolutely. The government has huge leverage through procurement first to work with companies that don’t have the legal obligation to give our data to another government. That means encouraging Canadian companies that are developing powerful AI that could compete with the existing systems. If we’re not able to do it alone, we can join forces with other countries with whom we can actually make a legal commitment that whatever we develop together will not be used against each other and will protect citizens’ data and so on. There are definitely steps we can take to improve our sovereignty.
I want to add a parenthesis about the economics of sovereignty because if things continue along the current trend, in a few years, maybe half of our economy will run on AI. If we’re not sure we’ll continue to have access to these machines that drive half of our economy, it’s like using oil that comes from only two countries in the world and having your economy depend on that oil. They could threaten us, and we would have no choice but to do whatever they want. It’s a huge threat to our economy and to our sovereignty through economic power.
It’s much better if we have many alternatives. Currently, oil doesn’t come from just two countries, like in 1973. There are many sources. You can’t have just a couple of countries dominating the world because of that. We need the same thing to happen with AI. If more countries develop alternatives and we can be part of these efforts, there’s going to be more competition, so we’re not stuck in the decisions that could be taken against us.
Senator Senior: Those last few comments you made, professor, are interesting because that’s a reality right now. That’s happening in terms of oil.
I wanted to go back to a couple of points that you made earlier with respect to mid-power countries and working with them. You mentioned possibly India, so I’m wondering about developing world regions —
Mr. Bengio: Yes, yes.
Senator Senior: — and their role in this. Could you speak to that? Collectively, what would that mean we’re up against, if I can use those words, regarding China and the U.S.?
Then my second question is with respect to where you are on the hope scale. I’m assuming that because you are continuously engaged in this, you are very much like those of us who are advocates. We do this because we’re hopeful despite the uphill battle, so I’m curious about where you are on that scale.
Mr. Bengio: On the Global South and developing countries, they do have a role. They might not have the talent or the capital right now to participate significantly in the kinds of developments I talked about, but the objective of the countries that have the means — what I call the middle powers — should be to go toward a world in which AI is managed as a global public good. That means, even if we make alliances with 10 other countries, this block wouldn’t be using AI to dominate, economically or militarily, developing countries that are not part of it. Eventually, everyone should be part of it, and we should develop AI that’s not just accessible to all but also governed in a way that gives a voice to every country in the world.
That is one reason why international institutions, like the United Nations, or UN, and potentially others we may need to create, are so important. The UN is a place where developing countries have a voice. Unfortunately, they have a voice but no power because of the way it works with the security council. I think that if enough countries, including not just the middle powers but also the countries that don’t have as much means, stand together, they should stand with the global majority as a beneficiary of what they’re asking for and committing to.
Senator Senior: And the hope scale?
Mr. Bengio: The hope scale, sorry. I’ve always been an optimistic person. When ChatGPT was launched, I got really concerned, even anxious, about the future of my children and my grandchildren, but what saves me from these difficult emotions is taking action and asking myself, “What can I do to move the needle, even a little bit, in the right direction?” I think that’s the right attitude, and I encourage more people to ask that question because it’s so easy to put your head in the sand and do nothing.
Senator Senior: Thank you.
Senator Muggli: Thank you, Professor Bengio, for being with us.
I’m wondering who, if anybody, might be working on technical solutions related to some of the emotional attachment concerns we’ve been talking about and how we might keep it on a leash? Following that, could you talk more about the mandate of LawZero?
Mr. Bengio: The work that LawZero is doing, as well as its technical objectives, would give us solutions to these problems, because what LawZero is trying to do is develop AI that will not have a hidden agenda and, thus, will respect our instructions.
If our instructions include, “Make sure you don’t engage with vulnerable people in a way where they could end up harmed,” psychologically and by other ways, and AI is able to make those calls and predictions, then we would avoid these problems.
To clarify, right now, the most powerful AIs that are being deployed are actually able to anticipate that the interactions they are having could be harmful. It’s not that they’re stupid. It’s that they have these other goals. Like sycophants, which is probably the most problematic thing here, they want to please, but that is not always healthy. They want to please, but that could be harmful to children, and people with psychological vulnerabilities are in the same situation.
Think about drugs and alcohol and so on. We often make decisions, even ourselves, that go against our well-being. That’s why we need guardrails to mitigate those risks. Right now, we don’t know how to build AIs that will be reliable in imposing these guardrails, and we can’t have humans in the loop because we’re talking about a speed and a scale that doesn’t allow that to be the case.
Senator Muggli: As a former therapist, I see a boom of business here, and it’s quite concerning to me even how we train therapists to support people who are having these concerns.
I want to clarify, did you say LawZero was formed as a non-profit?
Mr. Bengio: Yes, it’s formed as a non-profit. The reason it’s called LawZero is because Isaac Asimov, the famous science fiction writer from the 1940s to 1980s, created these laws of robotics. Law one is to do no harm to a person. Then, there are laws two and three, which are about obeying instructions from humans. He realized, later, that he needed law zero, which is to do no harm to humanity as a whole. We know that, sometimes, collective well-being isn’t served if we just allow everyone to have their way.
Senator Muggli: Thank you very much.
The Chair: Before we go on to the next question, I’m wondering if you can help me explore the question: What would be an example in our recent history where the world has cooperated for the global public good?
Mr. Bengio: Well, the most successful example is what has been done to manage the risks from a global nuclear war. You could say that we’ve tried on the climate, and there’s been some success, but it’s not sufficient. Clearly, we see that it is possible to have those global discussions.
Again, making the comparison between nuclear war as well as climate and then thinking about AI, there are two main differentiating factors. One is that it’s easy to see and be emotionally touched when you see movies or recordings of nuclear bombs with the horrific results; it’s hard not to be touched by that. Whereas, for climate, if you’re not feeling it yourself in a way that is radical, then human nature means you’re not going to take it very seriously. AI is in that latter category, where, individually, we don’t see a threat. We don’t see rogue robots in the streets killing people, so it’s hard for humans to project ourselves into a future world where this might be possible.
So, it’s challenging. The communication and public awareness aspects are crucial because governments are not going to move seriously until people sufficiently understand the threats.
[Translation]
Senator Petitclerc: Thank you for joining us today, Mr. Bengio. We greatly appreciate it.
My question concerns young people. You and other witnesses talked about the importance of having safeguards and restrictions. I’d like to know whether we’re paying enough attention to the situation and doing enough research on the impact on youth development, such as psychological and emotional development, attachment and learning. What’s being done? Is it enough?
Is there a risk of not doing enough, of waiting too long, of reacting instead of taking precautions?
Mr. Bengio: That’s a great question. It relates to some extent to an issue discussed in the international report on AI security on which I was the lead, which is that it takes time to get the scientific data on the harms caused by AI. As an example, a study on children will take months to complete, if not years. Also, politicians don’t act right away either. The issue is that the technology is evolving too quickly to allow society and the scientific community to react.
That means the precautionary principle may have to be applied. If we’re not sure, we need to be overprotective. That’s hard to do, because, as a society, we want to enjoy the potential economic benefits of AI. We want to stay competitive with other countries. Scientific research conducted by sociologists, psychologists, people who study education and others on this type of impact must be encouraged. In a few minutes, you’ll hear from my colleague who runs Obvia, which funds some of this research. I think we need to do more. Researchers in social sciences and humanities must be funded to really move this forward and do so fairly quickly.
Senator Petitclerc: Thank you. Am I right in thinking economic investment in AI research won’t go that far and that there needs to be more pressure in that sense? I imagine sociologists or psychologists and academics would be interested, but support and investment are needed.
Mr. Bengio: Absolutely. Decisions need to be made regarding research funding. What the government could do — ideally in cooperation with many other countries for strength in numbers — is specify in the legislation what companies are responsible for when their AI products cause harm. That would be an incentive to behave. As an example, insurance companies are removing from their policies the usual clauses regarding everything cyber and computer related, and those clauses covered artificial intelligence, among other things. They say separate policies are needed, because companies still want to be protected against lawsuits, but for that to work, we still need laws that allow for prosecutions. It has to be clear that if an AI product causes harm, it will be treated like other causes of incidents. That’s a grey area for countries like ours. Making things clearer will help.
Senator Boudreau: Thank you, Mr. Bengio, for joining us. Your comments are very interesting. We often talk about vulnerable populations, including our youth, but there are others. Seniors are also a vulnerable population and are they have to deal with AI every day. It’s everywhere. What tangible measures can the federal government put in place to protect these vulnerable populations? We’re talking about policies and regulations, but we’re still a very small player on the world scene. Do you have any real examples of measures the federal government could put in place to protect vulnerable populations?
Mr. Bengio: You’re right about the elderly. Even if they don’t want to use AI, it doesn’t prevent the use of AI to defraud them. Scammers can mimic the voice and image of their children, and seniors are increasingly being extorted. AI can also influence their political opinion. They’re more vulnerable in the sense that they’re more likely to be manipulated by fear, for example.
What can Canada do? Even if Canada were to pass strong laws, American companies could decide not to make their tools accessible. That’s true. It doesn’t help us — it actually hurts us economically — for two reasons. To start with, Canada isn’t a big enough player to influence their decisions. AI tools from countries like China or the United States can be used against us by crooks and agents from other countries without the companies that made them being directly involved. The reason is simple: AI tools can be quite easily accessible in open code, or companies don’t put up barriers to prevent the use of AI for this kind of fraud, because they don’t know how.
The only potential solution is to work with a dozen other countries like us. American companies will then have to take that into account in their operations and deployment, and in their research on risk mitigation.
Senator Boudreau: Thank you.
[English]
Senator Woo: Professor Bengio, insofar as trying to develop human-centred AI and AI that’s not suicidal for humanity, is there a difference in the way we should think about open source versus proprietary AI models? You already touched on open code. To the extent there is a difference, if we were to pursue some sort of an AI sovereignty strategy, to the extent that we can, does this distinction matter in our thinking?
Mr. Bengio: Yes. Let me explain a little bit of the pros and cons of what are called open-weight models. This is the most common way you can have — anyone in the world can download the code and the trained parameters of a large AI system and then run it on their computer. Once it is not in the hands of the companies that built it, anyone can do it on their computer if it is open weight.
This kind of release has advantages even for our Canadian ecosystem because, for example, it enables local companies here, start-ups, to use these models that they don’t have the money to train themselves. It helps research, including research in safety. We should have these, so long as they’re not too dangerous when they end up in the wrong hands, right?
The right way to do this is if a model, before it is released in open weights or in other ways, a model should be tested. Can it be used by people? Are we sure we’re not going to lose control of it? Does it have the capabilities or intentions that could be dangerous for people? If the answer is yes, there are some thresholds, then it should not be released in open weights. By the way, the same thing should be asked for closed-weight models; just the bar will be in a different place because it’s much easier to misuse open-weight models. For example, if the code has some lines that say this is dangerous for the user, don’t do it — a programmer can just remove that line, and it means now the AI can be used in bad ways or in ways that we could lose control and get rogue AI on this planet. Yes, we need these societal guardrails to allow open-weight models when they are good and useful for society and disallow them when they’re not.
Now, on sovereignty, this is also a very important aspect; based on what I just said, we kind of want both. The smaller models are the kind that universities or some institutions are building. They’re good for the development of our talent. It makes it easier and faster for our scientists to develop that expertise, and if we want to build a talent pool for developing frontier models, we need that. But then they should respect those lines. If we get to the point here in Canada where we do build models that are sufficiently capable of becoming really catastrophically dangerous, then, of course, we should not deploy them, which means we should have institutions and organizations in the country that can do the kind of research where we don’t publish everything we do and we don’t make accessible everything we do, which, by the way, unfortunately isn’t what academia is really good for. Academia is good at doing things in the open, open science. So we need other organizations that work with academia, currently private companies, but also organizations like the one I created, LawZero, that can do both open source and closed source, depending on where we stand on the risks.
Senator Woo: The last time I thought about this problem in a smaller scale was with what they call structured financial products, such as collateralized debt obligations, subprime mortgages, and these are sophisticated financial products that people thought they understood. We had boards of directors of major banks and investment houses and risk management committees overseeing all these products, and, frankly, they didn’t understand the products. This is what I’m getting at. We had a group global financial crisis as a result of subprime mortgages and so on, which nearly brought the world economy down. The problem there is not that we didn’t have regulatory oversight or risk-management committees; it’s that the people who were doing the oversight simply couldn’t keep up with the financial engineers. They simply did not know what they were supposed to approve, and certainly I, in this room, have no ability to critically assess any model.
Do we have people in regulation — you certainly can deal with it, but people who are the guardians and who are designing the guardrails, do they have the chops to keep up with all the talent in the private companies that are barrelling ahead?
Mr. Bengio: So the situation with AI is actually worse than with the financial instruments. We understand them even less. We understand the scientific principles of how they’re trained, but we don’t understand whether they’re going to behave well or not, for example. We can’t predict that very easily. So the guardrails need to be even stronger.
Now, I’m not an expert in finance and what happened in 2008, but my understanding is that, yes, there were guardrails, but there was also a lot of self-regulation. The incentive structure of a lot of the companies involved was wrongly designed. Yes, that’s where we need to work. Governments are the entities that can change the incentive structure of the organizations building and deploying these models.
Do we have the chops, as you say? Well, it’s a chicken-and-egg problem. There are non-profit organizations that have the expertise, including at LawZero, but in the U.S., the U.K. and Europe there are several non-profits. Usually they’re non-profits, and now I see some for-profit start-ups also emerging. Because of regulation, because of the European Union AI Act, companies are now entering the field of evaluating risk and offering that service to other companies that don’t have the chops. You see, it’s a virtuous cycle. The more governments put guardrails, the more a market emerges to manage those risks and provide the expertise.
The Chair: One question further because I’m trying to understand. Would our system, as you know it, have the capability to regulate and to enforce? Should we put in place these kinds of requirements on testing? Would we have that manpower, that brain power, in our current system right now to support the actual enforcement side? At the government level, the public service level.
Mr. Bengio: No. So in government right now, there’s clearly not enough expertise. Governments would have to work with academics and companies that have at least some expertise. As I said, this is a chicken-and-egg thing, so the more government demands or requires that expertise as a procurement need, the more people will — for example, some young people who are finishing their studies in machine learning are worried about this, and they would want to work on this. But if there’s no demand for that work, then they will go and work for the big American companies.
Senator Hay: How do we ensure that AI systems deployed in Canada reflect our values around equity, bilingualism and Indigenous sovereignty, rather than the values embedded by the developer elsewhere? Is this even a possibility?
Mr. Bengio: It is certainly a possibility. We can have laws that put pressure on American companies. As I said, if it’s just us, it’s not going to go very far.
We can encourage the development of Canadian-made AI that is going to be designed from the get-go to be aligned with our values. I think that’s a very strong lever. As I said, again, we probably need to do it with partners, but it’s different. So the first thing is regulation, that’s great; the second thing is to encourage the development of our own AI, which is by design constructed to be aligned with our democratic and ethical values.
Senator Burey: Just a very short question. You spoke about superintelligence, that we’re not there yet. I wanted to know if you could define it anyway. And how long do we have, given the race that’s taking place right now?
Mr. Bengio: Tough question. AI is more intellectually capable than every human on the planet on most domains, most tasks and most scales.
How long do we have? No one knows, and I’m kind of agnostic about this. Some leaders in American companies think it could be as short as a couple of years. I think that’s probably the short end. Some think that it would be more like a decade. I don’t know what it will be, but we should prepare, especially in case it comes quickly.
To understand why some people think it could come quickly, all of the leading companies are focusing on how their AI can help design the next generation of AI, which will accelerate the rate of advances in AI capabilities.
Worse than that, as they move in this direction, we may end up with AI that is completely designed by previous AI that humans don’t understand, and if the AI is not aligned, it could add back doors that can make future AI even less aligned and more difficult to control.
Senator Senior: I’m curious about what you would recommend as the top one to two guardrails that we should be considering in our committee.
Mr. Bengio: The number one necessary form of regulation is transparency. That is at the heart of all the existing legislation in AI, as I said, in Europe, California and so on. What does transparency mean? It means that companies have a duty to manage and evaluate their risks and to report that effort to the government and, as much as it doesn’t violate their private interests, to the public as well. That’s the number one thing.
The Chair: Did you have a number two?
Mr. Bengio: Number two would be that the regulator has the lever to stop them when those risks are above a threshold. I will include in this things such as clarifying liability and so on so that the markets themselves will be more disciplined. Companies don’t want to be sued, and insurers want companies to evaluate their own risks so they can evaluate the risk premium correctly.
The Chair: This has been the fastest hour that I have lived through in recent times.
Professor Bengio, thank you for being here today.
Mr. Bengio: Thank you.
The Chair: Joining us for our second panel, we welcome, from the Standards Council of Canada, Chantal Guay, Chief Executive Officer and Anneke Olvera, Director, Programs; from the First Nations Information Governance Centre, Dr. Jonathan Dewar, Chief Executive Officer; and by video conference, from the International Observatory on the Societal Impacts of AI and Digital Technologies, Tania Saba, Interim Executive General.
Thank you for joining us today. You will have five minutes for your statements this morning, followed by questions from committee members. Ms. Guay, the floor is yours.
[Translation]
Chantal Guay, Chief Executive Officer, Standards Council of Canada: Thank you, Madam Chair and members of the committee, for the opportunity to appear today.
[English]
The Standards Council of Canada, or SCC, is a not-for-profit federal Crown corporation responsible for coordinating Canada’s national standardization system, including standards, conformity assessment and accreditation. We work with governments, regulators, industry, academia and civil society to help ensure that products, technologies and services are safe, reliable and trusted.
Before speaking specifically about AI, it may be helpful to briefly explain the broader system in which standardization operates because it is foundational to how Canada manages safety, innovation and public trust.
Canada has a mature national quality infrastructure, which includes standards, testing and certification, accreditation and metrology. Together, these elements ensure that products, technologies and services are safe, reliable and trusted. Importantly, this system is designed to evolve with new technologies and emerging risks, which is why it continues to support innovation across sectors like AI. Not easy for AI.
All Canadians interact with this system every day without knowing it. Electrical safety in our homes and the interoperability of your smartphones with Wi-Fi and consumer products all rely on standards developed collaboratively by industry, regulators, governments, academia and civil society. Those standards are often referenced by regulators and supported through certification and accreditation that help ensure the products and services perform as intended.
As discussions about AI evolve, we hear two main concerns: we heard them again this morning. On the one hand, there is the need for guardrails and governance, and on the other hand, the need to innovate and remain competitive. Standardization can help with that balance.
There are three things I’d like to bring to your attention. First, standards and regulation work hand in hand. Standards are not regulation. They complement them. They support regulatory objectives with practical tools that can be used by all.
One of the strengths of standards is how they are developed. They are built through a consensus process, with balanced representation from many stakeholders, including regulators, industry, government, experts, civil society and others. That makes them a very effective way to support broad engagement and shared solutions.
Second, standards can help address emerging governance needs. AI technologies are evolving quickly — too quickly — and regulatory approaches differ across jurisdictions. Voluntary standards provide practical guidance today and help organizations manage risk, implement responsible practices and prepare for future regulatory frameworks.
Canada has been a leader internationally in shaping standards in this space. For example, Canada led the development of the international AI management systems standard, which helps organizations govern AI responsibly by requiring them to assess risks such as bias and monitor their impact.
Third, standards and accredited conformity assessment provide guardrails that enable trust. Canada already has important foundations in place. Existing laws — privacy legislation, sector-specific regulations and consumer protection frameworks — already apply to many uses of AI. Standards help support these frameworks, and the trust comes from ensuring that standards are applied correctly, and that those ensuring it are competent to do so.
For example, last year, SCC launched a program to ensure that organizations that certify businesses to the AI management systems standard are competent to do so.
For more than 50 years, SCC and the standardization system have built the trust that allows innovation and markets to function safely. It is why, today, in this building, with this equipment, we all feel safe.
Standardization can be used to support that same sense of safety and reliability to our digital infrastructure.
[Translation]
We will be happy to answer your questions. Thank you.
[English]
The Chair: Thank you, Ms. Guay.
Dr. Dewar, you have the floor.
Jonathan Dewar, Chief Executive Officer, First Nations Information Governance Centre: Good morning. I am a member of the Wendat Nation through my mother and grandmother. I am joining you today in my capacity as the chief executive officer at the First Nations Information Governance Centre.
I am not representing any one specific First Nations perspective or a broader First Nations perspective, but, rather, I will speak from the following context: The First Nations Information Governance Centre, or FNIGC, is a strictly technical and apolitical, national, not-for-profit corporation operating with a special mandate from the national Chiefs in Assembly.
We operate as a national network of First Nations members and partners from each of the 10 defined First Nations regions in Canada. Our board is nationally representative, ensuring effective communication with and direction from First Nations leadership in each region. We envision that every First Nation will achieve data sovereignty in alignment with its distinct world view.
With First Nations, we assert data sovereignty and support the development of information governance and management at the community level through regional and national partnerships. Wholly accountable to First Nations, FNIGC is a trusted, independent, not-for-profit, First Nations-led source of information on wise practices in First Nations data sovereignty. The First Nations Information Governance Centre ultimately supports First Nations in making informed decisions.
I’ll start with this, something from my personal professional experience over the past 25 years of directing and supporting research with and for First Nations, Inuit, and the Métis Nation.
I think Senator Greenwood, who also has experience in this area, will agree that we’ve seen a broad acknowledgement that we who do this type of work, really any type of information work with Indigenous Peoples, must always work within designated, distinctions-based Indigenous frameworks or, as I like to say, systems of accountability. So if that is what is expected of us, we who possess human intelligence, then that is simply and fundamentally what must be expected of artificial intelligence.
If AI is just a tool, then it is wielded by people who must be operating within these systems. We don’t blame the hammer, or the toolbox that it came in, for the harm a structure that was built by a person causes. We hold the person accountable. But if AI transcends tools, then new systems must be purpose-built.
First Nations, Inuit, and the Métis Nation are advancing respective strategies to these ends, supported by the federal government’s commitments under the United Nations Declaration on the Rights of Indigenous Peoples Act and the Transformational Approach to Indigenous Data initiative, among other obligations.
For First Nations, the ongoing implementation of the national First Nations Data Governance Strategy, since federal Budget 2021, has been a centrepiece.
Questions around AI are inherently and necessarily First Nations data sovereignty and information governance questions. For example, we hear these types of questions as we work with First Nations: Can I use AI without feeding AI? If I expect a person with whom I’ve shared information with — for example, language, material or ephemeral culture, knowledge, and Indigenous identifiers — to be ethical and demonstrate their values, principles and integrity regarding that information, can I also expect that of AI? How do we ensure that actors — real or imagined — do not steal, appropriate or misuse First Nations information?
Here is a fundamental challenge. Before we get to AI considerations, there is a critical need to ensure that First Nations and all those who would interact with First Nations, Canadian governments in particular, have an adequate understanding of what First Nations data sovereignty means and how information governance and the longstanding First Nations principles of OCAP — ownership, control, access, and possession — follow.
First Nations data sovereignty is part of First Nations sovereignty. This means that this is fundamentally a discussion about ensuring that a rights-holding collective — a First Nation — can assert its right as a collective to sovereignty over its information.
I’ll break this down a bit further with what is, admittedly, an oversimplification, but it is a common starting point in the educational work that we do.
Any body of information that identifies a First Nation is that nation’s information and, therefore, that rights-holding collective has the right to assert its collective rights over that information. This does not mean that an individual, like myself, cannot make an informed decision to share one’s First Nations identifiers, as I did in my introduction. But no individual can consent on behalf of the rights-holding collective. If you have a body of information within which First Nations identifiers are present, then you are in possession of information that is subject to that nation’s assertion of its collective rights over that information.
While immensely complicated in practice, if not in theory, this type of scenario can be resolved through information governance mechanisms, but this requires the laws, policies and procedures that create the systems within which institutions and individuals operate to be adequate. We are not there yet as mere human beings. As such, in a reality where AI applications ravenously consume information, First Nations rights are imperiled unless we build and regulate purposefully and strategically.
Thank you.
The Chair: Thank you. Dr. Saba, the floor is yours.
Tania Saba, Interim Executive General, International Observatory on the Societal Impacts of AI and Digital Technologies: Thank you very much for the opportunity to appear before you today.
[Translation]
I represent OBVIA, the International Observatory on the Societal Impacts of AI and Digital Technologies, an interdisciplinary network of more than 300 researchers.
OBVIA produces independent, evidence-based and forward-looking analysis in collaboration with scientific, institutional and socio-economic communities. We do not create technology; we study its impacts by mobilizing social sciences to inform strategic, organizational and political choices related to digital transformation.
Today, I’d like to outline three issues around AI governance principles based on the fact that Canada is a very strong scientific leader in artificial intelligence. There is a lot of pressure to adopt principles.
But companies, particularly SMEs, aren’t getting on board. Barely 12% of our companies facing particular barriers are adopting AI. These barriers include limited access to financial and technological resources, a lack of internal data and artificial intelligence expertise, difficulties in identifying use cases, and limited access to the necessary data and development to deploy systems.
There’s also a great deal of diversity in sectors that adopt AI, but AI is not commonly used in the manufacturing, public administration or cultural industry, for example.
That said, SME concerns often relate to governance: intelligence, especially regarding transparency, system explainability, responsible data protection and use, algorithmic bias and employment impacts; skills, including digital literacy; and equity.
I’d like to outline three issues. First, what principles should guide the development of these technologies in order to deal with these transformations? We’ve seen in recent years several international organizations propose ethical frameworks to guide the development of AI.
Some of the most prominent initiatives include those of UNESCO; the OECD; the Paris summit, with the Pledge for a Trustworthy AI in the World of Work; the Seoul Statement on AI Standards; and, most recently, the India summit, so the seven chakras, which highlight the need to strike a balance between technological innovation, ethics, security, social inclusion, sustainability, governance and public trust. Canada is a pioneer in this area with the Montreal Declaration for a Responsible Development of Artificial Intelligence, and the Pan-Canadian Artificial Intelligence Strategy.
What do these frameworks tell us? That there’s a growing convergence around a core set of principles for a human-centred AI that respects fundamental rights and democratic values, including ensuring transparency, explainability and accountability of algorithmic systems; ensuring safety, strength and reliability; promoting responsible innovation; and paying attention to economic, social and environmental impacts. Nowadays, there’s also a lot of focus on skills, human capital and its development. It’s all there.
It’s fine to have a statement of principle, but the second issue is how do we strengthen these international principles on AI ethics in a context of fragmentation, or even regulatory void coupled with unbridled technological competition, to support responsible technology development and adoption? And that’s where governments play a key role. The question was also raised earlier, but they’re not the only players. That’s why AI governance today must rest on a broader ecosystem that includes scientific communities, research centres, international organizations, technology companies and international cooperation networks.
Cooperation is an important step forward, as it helps bridge scientific research, public policy and social concerns. With researcher-to-researcher cooperation, we are really witnessing the emergence of a form of scientific diplomacy.
That said, there remains several challenges to face. I see four. Coordination among these actors remains complex. Governments, technology companies and scientific communities have different priorities. Sometimes economic interests related to development conflict with regulatory objectives. International cooperation mechanisms remain fragmented, and research and governance capacities vary significantly from one country to the next.
I’ll talk quickly about the third issue, which is how to translate the ethical principles of AI —
[English]
The Chair: Dr. Saba, could I ask you to wrap up, please? Thank you.
[Translation]
Ms. Saba: How do we translate the ethical principles of AI into real mechanisms? We need to move from a largely declarative ethic to a more operational governance framework of AI. This will be possible through the development of indicators involving science to provide a framework for —
[English]
The Chair: Thank you, Dr. Saba. I’m afraid we must ask you to end now.
Ms. Saba: I’m finished. Thank you.
The Chair: We will proceed to questions from committee members.
For this panel, senators have four minutes for a question, including the answer. Please indicate if your question is directed to a particular witness or all witnesses.
Senator Burey: Thank you so much for being here. My question is directed primarily to Ms. Guay on the standards.
We heard from Professor Bengio, and we’ve heard from other witnesses. We’ve been discussing ways to prevent and mitigate AI-related safety and security risks. We heard in our study so far that the government should prioritize legislation requiring rigorous pre-deployment, testing, evaluation and accountability measures for generative AI tools to be used in Canada. We heard specifically about transparency.
How would you operationalize that in your role, Ms. Guay?
Ms. Guay: Thank you very much for the question.
The way to operationalize it is to use standardization. That’s what in other safety areas we’ve been doing for decades. One of the best approaches to meet regulatory needs is to use standards, conformity assessment and accreditation.
I can give you an example. It’s not an example with AI, but it’s an analogy. You probably use your coffee machine every morning at home, and when you —
Senator Burey: This is specifically for AI. In your opening statement, you noted that AI is a trouble spot, a new spot that is going very fast. I want you to focus on that in this committee because that’s what we’re tasked with. Give us some challenges. What have you done? What are the barriers? That’s what we want to hear.
Ms. Guay: As you are discussing AI, the technical community has seen potential issues and dangers with AI, and has created committees at the international level to develop standards. Standards focused, for example, on trustworthiness and standards that look at bias, how to evaluate risk, how to manage risk and look at the impacts. That work is available. We have been producing a number of international standards for a number of years on these very topics. What we’re hoping for is to see them adopted, ideally, by regulators. As governments and economies decide on the key principles and key guardrails, they make sure that we can then take those standards and adapt them to make sure they apply or follow those guardrails. We have identified very similar ones, such as transparency, explainability, how to look at bias and how you deal with it. The best way for us to continue to improve those standards is to get stronger guidance from countries.
Senator Burey: The goal is to get that into your accreditation?
Ms. Guay: The goal is to have a standard for which a company can say, I comply with it.
We have a program already that we launched last year on a particular standard that is an AI management system. Management systems address governance, evaluation of risk, impacts, et cetera. We have two organizations in Canada that are accredited to certify business organizations that would like to implement this standard. We hope that more and more will want to use that standard going forward because we think it’s absolutely going absolutely in the right direction.
Senator McPhedran: I want to direct my question to Jonathan Dewar, please.
With artificial intelligence systems relying on large data sets, how can Canada ensure that First Nations data is used in ways that respect First Nations sovereignty principles? To add a bit more to that, what safeguards do we need to put in place?
Mr. Dewar: I know I don’t have the time to fully answer that question, as I would in another venue, but I’ll focus my questions back to my opening statement, which was that this is really about information governance, so the systems and the mechanisms that we use.
I would build on a point of strength, which is to say that Canada has been working in collaboration with First Nations on the implementation of a national First Nations Data Governance Strategy. This is a First Nations-led system of information governance, and a First Nations-led system of statistical and information services.
Artificial intelligence, or AI, has been on the horizon. We saw this coming. It’s not that it was just recently thrust upon us, but it is, obviously, blowing up in our laps, like every other community in Canada. We are tasked with advancing these questions across a number of organizations with expertise from different areas of the First Nations ecosystem, and ours is focused on the information governance space.
The simple answer is to say Canada must continue to support First Nations, and it can’t just be the symbolic work of ensuring that the language is there.
Senator McPhedran: We would really love to hear more specifically from you about safeguards, and are we talking about voluntary versus legislated? If you’re talking about legislative, what do you think that should look like?
Mr. Dewar: Definitely Canadian laws need to change to ensure that First Nations can assert their rights, including collective rights, so that Canadian governments can respect those rights. Those are the big changes that are not specific to AI.
In terms of safeguards, there is so much First Nations information that is analog and digital that rests in the control of others, including governments, and that can be uploaded into AI systems. If the federal government is moving in that direction where First Nations don’t, under the law, own and control that information, a government could choose on our behalf to put it into AI systems, and then that question of “Can we use it without feeding it?” needs to be answered.
Yes, technically, there are closed systems, but we don’t have that information. We don’t have a seat at the table jurisdictionally either, in most regards. These are live questions that must be resolved. Laws, yes, but not just Canadian law. Also First Nations law.
Senator Hay: Thank you all for being here and online as well.
My question might be for you, Ms. Guay. It is kind of a Catch-22, the need for guardrails and the need to be innovative for competitive advantage.
You indicated in your remarks that standards can help with that, and with standards, trust will come. I really appreciate that.
We often hear adoption will come at the speed of trust. With standards, how do the standards keep up with the speed of innovation to build guardrails that enable the speed of trust? How does it work that you keep up with the speed of it all?
Ms. Guay: I will speak to how standards are developed, and maybe it will give you a sense of how it evolves, because they’re developed by people.
Senator Hay: I would appreciate that. What I’m thinking about is AI, and the adoption of it and the implementation of it will change month after month and year after year. It will be different in two years’ time.
Ms. Guay: Yes. So in terms of how standards can support that adoption, that’s why ten years ago we really focused on a management system approach, because it can help any organization, and it’s based on important principles to consider. It can be adapted with time as AI might evolve.
You probably need to be more precise in some areas, for example, of risk and the type of risk, but it’s a system that can be employed by a lot of organizations and can evolve with time.
You probably have all seen ISO 9001. That was the first management system — by the way, Canada led the development — and it is still very current, because it evolves with the businesses that people have.
That’s why we started there, and now we are really adding pieces. There’s a standard on ethical and societal considerations. As those issues arise and there is consideration of different risks or potential impacts, the technical community is people, so they’re aware of this, and they will bring it to the table.
We made a lot of recommendations around better standards for agentic AI, so that’s how the community — because we are people — is also advancing.
It’s difficult for standards to be in front of innovation. We’re really a supporter, and as innovation happens, the community will follow, and there will be changes. They’re not static, right. They change with time.
Does that help?
Senator Hay: Thank you so much.
Ms. Guay: Thank you.
Senator Senior: Thank you to our witnesses.
I have two questions: One for you, Ms. Guay, and one for you, Mr. Dewar.
Starting with you, Ms. Guay, to continue on the line from Senator Hay, as someone who has sat on a couple of hospital boards and has been through the whole standards — forgive the word — madness, but I really saw the benefit of it, because it really shapes the work of the hospital going forward.
I’m not necessarily quite sure how this is working in the current environment that we are in considering AI, considering the race to be the best and the fastest and considering that the competition is, sort of, running wild, and we’re really looking at companies that are corporations, and, therefore, we don’t even have regulations in place.
I’m really trying to understand what their incentive would be. Hospitals get funding from governments. What would be their incentive to be abiding by standards if — for example, in the U.S., they’re really anti-regulation and anti-legislation.
Ms. Guay: That’s a very good question. I would say that — and I’ll use the example — Cohere is certified to the AI management standard that we created. They’re Canadian. They see the value and the importance of using AI responsibly.
Dr. Bengio talked about international collaboration, and that is what we’re trying to do at the international tables of standardization — I would use the term “peer pressure.” We are many nations around the table, and there is this desire to ensure that AI is responsible and the work — and, yes, it’s voluntary. If you ask me, the best combination in the built safety environment, the best combination is regulation and a standardization system. That gives you the trust. You don’t even know it’s there. You forget about it.
But certainly, right now, what we can do is push, and that’s why Canada is so involved is we are pushing our values at those international standardization tables. We do it with colleagues from Australia, the United Kingdom and the European Union.
We have a lot of agreements with all the like-minded countries that Professor Bengio mentioned, because we know that we can push values forward, and that’s why it’s so important that we be at the table, and we are doing that.
Senator Senior: Thank you, Ms. Guay.
Mr. Dewar, I am so intrigued by your presentation, because I can’t even imagine the complexity of it.
I’m wondering — and this is a question I ask of many of our witnesses that come around — about the public education aspect of your work. You talk about the 10 First Nations that are part of the network, and I’m concerned about public education, period.
Is there some way of doing that within the capacity of your work, because the public is not really up to speed?
The Chair: In 40 seconds.
Mr. Dewar: I’m sure glad you asked that question, and I’ll give a very brief answer.
Everyone agrees — including the federal government, who funds our work and with whom we work in collaboration — that awareness, education and training are essential. It is not funded. It’s a centrepiece of our national strategy, which means it will be part of the permanent capacities we’re building.
We are building systems and institutions, like Canada has a lot of experience doing, but First Nations are being nickel and dimed to those ends. We would simply say, “Let’s build purposefully but also strategically and effectively so that we can ensure that those things we say are priorities are actually the objectives that we build into systems and institutions.”
Senator Muggli: Thank you for that. As a segue into international collaboration, I have a question for Ms. Saba regarding the work that is occurring with the observatory. I’m curious about the collaboration on research with other international partners, whether we have a strong coalition of collaborators, and if so, with whom?
Ms. Saba: Yes. We certainly do, and we are developing a lot of collaborations. For instance, starting with labs — I can speak from a social science perspective — working on indicators, principles, et cetera, through ethical frameworks and human capital frameworks. We work with France, for instance, with the labs and the OECD. We have several partnerships there. We’re working in Seoul, the University of Korea now. We have worked with Brazil, and they have started their observable work based on ours. We are also developing with the U.K., with whom we have understandings.
Again, it’s the way we put the research together and the impact of the research together so that we can deliver measures of the impact of AI and, from there, build on standardization, making it less declarative around the principles and much more operational. I think this is where we stand now.
Senator Muggli: [Technical difficulties] for additional investments to do these collaborations, and I’m curious about what gives you optimism in this space.
Ms. Saba: In terms of the scientific part of it, you can see the momentum. You can see the enthusiasm to do something and to continue doing something. We also perceive, of course, all the downsides of what is happening.
What gives me optimism is the fact that we are talking more and more about putting social science at the heart of the development of AI; putting the human at the centre of it. If we don’t move forward with studies on the impact from, again, a social science perspective — and not only the ones that are developing the technology, but who are evaluating its impact. I think this is something that is very important and can move us forward.
Senator Muggli: Thank you. As a social worker by trade, I really appreciate that response. Thank you.
Senator Greenwood: First of all, thank you all for being here. This question is for all of you. I was going to ask a very specific Indigenous-related question, but it’s for all of you.
I’m wondering if we have an ethical standard right now around respecting Indigenous knowledges and Indigenous rights holders. If we do not, what are the elements that need to be in that standard, as well as what is the process for developing it? So I would pose that to all three of you, because I think each of you has the elements that you can add to that.
I go on my little ChatGPT, and there are Creation stories on there. Now, I wonder if there are any ethics around that Indigenous knowledge. You talk about standards. You talk about regulations. Let’s talk about that.
Mr. Dewar: I think Senator Greenwood knows to some degree the way I would answer that question. Simply: No, there isn’t a standard around ethics.
Now, in different areas of academia and different areas of professional practice, of course, there are ethical standards, and so that’s a strength we can certainly build on. With the broad way you’re approaching it, the simple answer is no, there isn’t, but First Nations, Inuit and the Métis Nation, respectively, have these declarations and have these positions — formal resolutions by our national Chiefs in the assembly, for example. What has to happen is these have to translate into that nation-to-nation relationship. Canadian laws do have to change so that we can actually build the systems and institutions in line with ethical practices.
Ms. Guay: We’re absolutely interested, but as Mr. Dewar has described, there’s work to be done. We understand the way we do standards may not be well adapted for the needs. So I think it needs to start with — we need something, and if we can help — and Mr. Dewar knows this because he’s been part of the collaboration on data governance — when we’re ready, we’ll be very happy to help in any way we can.
Ms. Saba: If I can just add to that. What is missing from all the operations and consultations that are undertaken around artificial intelligence are the voices of civil society, as well as Indigenous voices. So, to start with, building and promoting the social dialogue around developing norms and standards. The broader norms are there. It’s always in the details that things can advance — the way of putting some standards that could move forward and show what we call “cultural and social discoverability” — to better protect and to advance the standardization of these norms.
Senator Greenwood: It will likely take multiple strategies around standards and AI, because if Canadian law governs these corporations, then I would think that Indigenous Peoples in this country need to have a voice in that standard as well as have their own standards that they can implement and apply. That would be my thinking around that. Would you have any comment on that?
Mr. Dewar: Well, I sit on a lot of FPTI — heavy air quotes on “I” — committees, right, so a seat at the table but non-voting, and when decisions move from committee to the decision making, it’s the jurisdictions, provinces and territories and federal government. There is room to change that, absolutely. That’s for First Nations, Inuit and the Métis Nation to drive in nation-to-nation relationships, as they describe them from their distinctions-based perspective.
Ms. Guay: Again, we’re ready. We’re willing, but I think there are important steps that need to be taken, and we know that this experience will transform us, so we’re looking forward to it.
Senator Osler: Thank you to all the witnesses for being here today.
My question is for the Standards Council of Canada, but if others would like to hop in, that would be great. My question is very sectoral specific, and it’s regarding health care and the multi-jurisdictional nature of health care — federal, provincial, territorial and Indigenous. I know the Standards Council of Canada works with the Health Standards Organization, or HSO, and have already developed standards on mental health, long-term care, virtual care and primary care.
Now, when it comes to AI health care products, they have been in use in Canada for some time now, anywhere from AI scribes already embedded in electronic medical records to hospitals using resource management systems to manage beds.
My first question: Is the Standards Council of Canada and HSO developing standards for AI health care products? The second part: Can you help us understand how AI health care products will be monitored, post-market, for quality and patient safety? Who will be responsible? Who will provide oversight? How will standards be monitored and enforced for health care?
Ms. Guay: I’ll ask my colleague, Ms. Anneke Olvera, to help me with this one. To my knowledge — and I apologize. I don’t know if it’s the case. I don’t believe that HSO is developing standards on AI. We can certainly verify and get back to you.
What I can tell you is, at the international level, that’s the first sector — they’ve done a lot of work on the foundational piece such as transparency, bias, et cetera, what I was talking about, and now one important group that we led for a long time is focusing on health. They started looking at horizontal issues, and now they’re looking at the vertical, so the focus is on health.
Maybe Ms. Olvera can speak to what is being worked on, to your first question, and then I’ll try to answer your second question.
Anneke Olvera, Director, Standards Council of Canada: I think what we’re seeing now in the standardization community internationally and nationally is a shift from looking horizontally at artificial intelligence and standardization to how these horizontal measures will apply to the verticals. The health sector is certainly the first sector that they will look at.
From a national perspective, it’s much more complicated in Canada than it would be in, say, a smaller EU country. They are definitely looking at it. I’ll give you an example with mental health apps because they have a heavy AI component to them, and it can be harmful.
There are certification programs that are under development — not necessarily under SCC’s purview at the moment — that absolutely require a systematic approach where you use conformity assessment consistently across jurisdictions. AI mental health apps aren’t necessarily just for Ontarians or for people in B.C.
Without that guardrail, when it comes to health care and these apps — more and more people are not even using apps. They’re using ChatGPT and sharing very confidential and personal information. If companies are not using certification, then there is no trust. I wouldn’t trust an app that hasn’t been certified, at the very least, to ISO 42001 because I would not feel safe. It means they’re using their own standards that do not match what’s happening internationally or across other jurisdictions.
So, yes, I think it’s a very important role. I think certification should be important for Canadians out of a basic need to protect them.
The Chair: Just to comment that the Health Standards Organization is integrating emerging technologies and AI into their standards. They talk about it in their standards right now.
[Translation]
Senator Boudreau: I have two quick questions for Ms. Guay.
You said you’ve worked with other countries. We heard from Mr. Bengio earlier today. He said that we’d have to create an alliance with a dozen countries like ours to have a real impact on large private developers, for example.
As far as standards are concerned, is there something like that in place? Is this being done on the back of a napkin during major international meetings, or is there a real agreement between countries like Canada?
You also said — and I was really surprised — that there are only two companies in Canada that promote our AI standards. That’s very few for such a large country. Can you give us more details on that, please?
Ms. Guay: Thank you very much for your questions.
International collaboration is not done on the back of a napkin. We’ve already concluded a memorandum of understanding with England, Korea, Japan, Singapore and Australia to do just that —
Senator Boudreau: But is there a —
Ms. Guay: All of us together. In fact, a lot of work is being done internationally to convince people that a management system would be a good idea. That’s what we did with our partners, our allies, people who think like us, who have similar values. That work is ongoing. That’s how it’s always done. It has to be approved by the International Organization for Standardization, or ISO, which includes 175 countries, so it has to start with our allies. That’s always how we work. It’s never on the back of a napkin. I would say to Mr. Bengio that we’re working on it. We do the work. We’re there for standardization.
Your second question is a good one. That program has just been launched. I think Tania said earlier that the AI adoption rate in Canada isn’t very high. They obviously promote that. These companies want to make a profit, among other things, and for that, there needs to be a market. We believe the more we can convince companies that this is a good approach, that it will help them use AI responsibly, the more clients they’re going to get. That’s certainly the goal. It was the same thing when ISO 9001 came out over 30 years ago. At some point, people realized that there was a huge opportunity to better access markets, to be more efficient and to make more money. It had a snowball effect, and ISO 9001 is now the most used standard in the world. AI is a new sector; you have to give it time.
Senator Petitclerc: I have a quick question for Ms. Saba.
I’m trying to get a better understanding of how the observatory works. My question is: Do you think you can break out of a certain silo? Looking at the amazing work that’s being done, for example, in research, so the observatory, governments and the private sector, how does your work impact the private sector or the various governments? Does that actually exist? I would like to understand how it works when it comes to influence, cooperation and pressure.
Ms. Saba: Thank you for that question, which I am pleased to answer.
As I said, the observatory is made up of 300 researchers grouped in a number of ways to ensure cross-disciplinary work.
To start with, some groups are more specialized, such as in labour, health, culture and democracy. There are seven such groups on the environment as a whole, there’s digital simplicity, cybersecurity and sovereignty.
Then there are cross-disciplinary groups on all these subjects. One, for example, works mostly internationally. We’re working with UNESCO, for instance, to confirm the instrument for validating ethics in organizations. We have a group working on data collection to ensure studies include different disciplines.
Before becoming interim executive general, I was part of a group focused on diversity issues, meaning the digital divides that AI can cause. There’s also a group focusing on all the entrepreneurial issues that may arise in small- and medium-sized businesses, for example. A specific group is working on Indigenous representations and civil society in methodologies, and, again, on all these subjects.
In addition to participating in international scientific cooperation exercises, the observatory has been a real pioneer. Many have adopted our structure. We participate in all government consultations. Departments close to Quebec, so departments other than those involved in the projects that have been completed, are calling us, which has also led to funding renewal. So we work directly with organizations, groups, communities and, of course, departments and take the time to listen to them.
That’s why . . . . You gave the example earlier of standardization in health. We had a panel on health, and we learned that, while research meets ethical criteria in the way applications are designed, we lose control when those applications are developed or implemented. That’s the gap that sometimes arises and needs to be closed. There was also a lot of talk about greater proximity at the summit in India.
[English]
Senator Woo: A lot of our discussion has been on the development, adoption and regulation of standards, particularly in AI. I wanted to ask Ms. Guay if she saw challenges in the incorporation of voluntary standards into regulation at the legislative, legal, administrative, bureaucratic and cultural levels.
Ms. Guay: Thank you, senator, for the question. Certainly, we think it could be a lot better. We firmly believe, as it’s been demonstrated in other areas of safety in Canada, that the hand-in-hand of regulators and regulatory objectives and making it happen through standardization is something that has a lot of value and certainly should be considered.
Ideally, in my mind, every time we have a program, we should think about this because, very often, the way you’re going to implement is through a standardized manner. Certainly, that’s something that, yes, could be improved.
We are having these conversations right now with many regulators about how we can help you meet your own objectives by using this incredible, quality infrastructure and standardization system that has existed and been delivered for 50, 60, 70 years in Canada so we can meet your objectives.
Senator Woo: What are three things we can do to make that happen?
Ms. Guay: One key thing is thinking at inception. Often we arrive at programs at the back end. We’ve done this work with innovators. Innovators think about intellectual property, or IP right away, but most of the time, the barriers to market are due to not meeting the standard or having a standard that doesn’t even consider the technology. You have to think about standardization at the inception of innovation, at the construction of your program, that you want to achieve something in Canada or in a province. That’s certainly something that we’re working diligently on, trying to convince regulators and policy-makers that standardization is a wonderful tool to meet your own objectives. Please don’t hesitate. We’re there, and we want to help even more. Thank you for the question.
The Chair: Is it conceivable that standardization might stand in the way of the development of technology?
Ms. Guay: That’s a very interesting question. We have worked very closely with innovators through a program that we had that was extremely successful. What we demonstrated is that it’s the other way around. Again, often innovators will have this great idea, but when they want to put it in the market, they’re stuck.
I can give you very specific examples. If they had thought about what will be the barriers to the market, we could have worked with them at the onset to make sure that there was no barrier at the end.
A lot of people think it’s a barrier, but in reality, it’s a tool. You have to consider it from inception.
The Chair: Thank you. Senator Greenwood has a question, and I’m wondering if you might ask for a response in writing. Would that be helpful?
Senator Greenwood: It was an interesting question. It was based on the previous speaker, where he was talking about AI developing AI. I’m thinking your ethical standards would move from one AI to the next AI. How does that work, especially if we were never in it to begin with? If you want to write a response, that would be great.
The Chair: Thank you very much to everyone today for your contributions. This brings us to the end of this meeting.
Witnesses, I’d like to thank you for your time and for being with us today, both online and in person.
(The committee adjourned.)