Investing in innovation in socially important areas with a market failure

a banner with the title "Investing in innovation in socially important areas with a market failure" with the headshot of an older white man


Yoshua Bengio

Universite de Montreal

Full Professor

Mila, Quebec AI Institute

Scientific director and founder


Scientific Director

Many experts, but not enough people, are well aware that markets can fail to achieve the goal of maximizing long-term wealth. Indeed, they fail in ways threatening us all in the years and decades to come (the most vulnerable or unlucky being likely to suffer more). For example, the health of our atmosphere, one of our shared resources, is going down the drain because of human-induced greenhouse gases. These gases cost nothing (or not enough) to put out in the air but enable a profit for market actors and a social benefit for those of us using fossil fuels for transportation or eating meat. So long as individuals and companies are not incentivized to behave differently or internalize the environmental costs of their activities, we will very likely continue to race towards a climate and biodiversity catastrophe. When each person or corporation maximizes their individual interests, it can easily end up with most of us losing, and this is at the heart of major market failures, like those involving pollution and climate change. 

For over a century, economists have studied and grasped how implementing a collective incentive system (i.e., via government policies), in principle, would offer up a solution to those market failures to maximize individual interests and better align them with the maximization of the common good. For instance, by taxing socially nefarious behavior (like polluting) or rewarding socially beneficial behavior (like carpooling, using public transportation, or biodiverse reforestation). An external objective can thus be added to the natural objective of market participants, making it possible for them to internalize something which would otherwise not affect their behavior. This piece argues that society should invest in innovations needed to tackle major challenges in areas where markets alone are insufficient to generate the necessary investments. This includes new technologies such as artificial intelligence (AI) applied to mitigating or adapting to climate change, fighting infectious diseases (in particular preventing future pandemics), or, more generally, substantially improving services typically provided by governments, such as education, health and social services. In those areas, there are many cases of market failures. I will focus on two examples: technologies to fight climate change, and technologies to develop new drugs to fight against antimicrobial resistance. In both cases, there is activity in academia and some but not enough in industry, and a great potential for governments to create new economic sectors.1,2

Fighting Climate Change

It would be possible to stimulate a much greater level of innovation in the case of climate change if governments were to drastically increase the effective price of carbon emissions, ideally to reach the true social cost of carbon. We do not yet know how to correctly assess these costs given the variables at play, such as unknown future events that can impose a great degree of risk (e.g., due to climate tipping points). For political reasons, such rapid increases are unlikely, but governments have recourse to other, more feasible options. One possibility is to introduce regulation that will force markets to better assess the social cost of carbon and mandatory insurance to cover that cost as perceived by future generations, as we recently proposed.3 Another is for governments to massively incentivize R&D in targeted relevant innovations. As we previously documented, AI has many applications helping to mitigate and adapt to climate change, from more efficient climate modeling, energy resources, and power management to new materials for batteries, electrolysis, or carbon capture.4 

Fighting Antimicrobial Resistance

Another public health issue plagued by a similar problem is that of antimicrobial resistance. From a business perspective, it is not lucrative for pharmaceutical companies to invest in discovering new antibiotics—as well as in drugs against diseases that overwhelmingly impact developing countries, for that matter—because it is not sufficiently profitable. One reason for the market failure is that to avoid the development of new resistant strains, the new antibiotics should only be distributed as a last resort tool where frontline antibiotics founder, meaning sales volumes of the new drugs would be low.  

If nobody invests in that research, we all lose (let alone all die if a really deadly and transmissible strain emerges). Healthcare systems everywhere—not just in developing countries—would likely crumble since antibiotics are crucial to our current medical infrastructure for any form of surgery and cancer therapies, for example. Antimicrobial resistance and climate change may be existential risks since we do not know how bad things could turn out if we don’t take a sharp turn, with our species’ survival potentially at stake. As we saw during the pandemic, the rich and poor will be affected, with the most vulnerable standing to lose the most. We now stand globally at over a million deaths per year caused by antimicrobial resistance, yielding already $2bn GDP reduction per year in Canada and growing fast. Without serious intervention strategies, deaths attributed to antimicrobial resistance could reach 10 million per year globally by 2050, according to projections.5 The economic costs will be over 100 trillion dollars by then,5 around $120bn just for Canada.6 

Current policies that incentivize R&D fall short of achieving the desired outcomes for several reasons. First, they tend to broadly cover all types of innovations (e.g., R&D tax credits). It would be strategically more optimal to stimulate innovation with greater intensity in areas of market failure and great importance to society, like those mentioned above, which would require mission-oriented funding. Second, they typically work through cost-sharing formulae (where for example 50% of the funding comes from the government, the rest from industry), which may not provide a sufficient incentive precisely because of the market failure. Finally, these incentives can either “push,” funding part of the R&D costs, or “pull,” providing an expectation of greater commercial value downstream, e.g., via taxation or procurement policies. These incentives could create1,2 new technological sectors (think about the impact of DARPA on the creation of the internet and the military industry in the US). These sectors could extend into the more traditional market-driven applications because methodologies like those developed for AI or biotechnology tend to be horizontal and applicable to both socially motivated and commercially motivated products and services. In the face of global challenges, governments should actively seek to join forces internationally to ensure their intellectual protection policies align for the greater good of all, even if they are at odds with the immediate interests of a company. For instance, they could provide incentives for open science, data sharing, and reduced access costs for low and middle-income countries,7 which would greatly increase the productivity of the invested funds thereby benefiting global society.


1 Mazzucato, Mariana. “The entrepreneurial state.” Soundings 49, no. 49 (2011): 131-142.

2 Mazzucato, Mariana. “Mission-oriented innovation policies: challenges and opportunities.” Industrial and Corporate Change 27, no. 5 (2018): 803-815.

3 Bengio, Yoshua, Prateek Gupta, Dylan Radovic, Maarten Scholl, Andrew Williams, Christian Schroeder de Witt, Tianyu Zhang, and Yang Zhang. “(Private)-Retroactive Carbon Pricing [(P) ReCaP]: A Market-based Approach for Climate Finance and Risk Assessment.” arXiv preprint arXiv:2205.00666 (2022).

4 Rolnick, D., Donti, P. L., Kaack, L. H., Kochanski, K., Lacoste, A., Sankaran, K., … & Bengio, Y. (2022). Tackling climate change with machine learning. ACM Computing Surveys (CSUR), 55(2), 1-96.

5 O’Neill UK government study: Review on Antimicrobial Resistance (London). Antimicrobial Resistance: Tackling a Crisis for the Health and Wealth of Nations: December 2014. Review on antimicrobial resistance, 2014.

6 McCubbin, K.D., Anholt, R.M., De Jong, E., Ida, J.A., Nóbrega, D.B., Kastelic, J.P., Conly, J.M., Götte, M., McAllister, T.A., Orsel, K. and Lewis, I., 2021. Knowledge gaps in the understanding of antimicrobial resistance in Canada. Frontiers in Public Health, p.1523.

7 Artificial Intelligence for Public Good Drug Discovery (2021), Global Partnership on AI report.