Machine learning reliability and costs: A European grant at SDA Bocconi with Alessandro Rudi

With the arrival of Alessandro Rudi from the École Normale Supérieure in Paris, SDA Bocconi School of Management is now hosting, for the first time, a project funded by the European Research Council (ERC).

At the core of many widely used artificial intelligence solutions is machine learning (ML) — the field that develops models and algorithms enabling systems to learn from past experiences and improve over time. While ML-based tools, including generative AI (GenAI) systems like ChatGPT, have attracted enormous media attention and massive investment, issues of reliability and cost-effectiveness remain.

Through the project REAL (Reliable and cost-effective large-scale machine learning), launched in 2021 and set to conclude in April 2027, Professor Rudi aims to tackle two major challenges: the often-uncertain accuracy of AI predictions and the high computational resources these systems require.

When it comes to reliability and accuracy, anyone who has used these tools has likely experienced hallucinations, outputs that have no basis in reality. However, other, less obvious issues are just as critical. “If I ask one of these systems to make a prediction,” Rudi explains, “it will return an answer but without any estimate of uncertainty or potential error. Ideally, a predictive system should provide at least a confidence interval, i.e. a numerical range within which the true result is likely to fall.”

For example, if a system predicts that a company's market capitalization will be $62 billion in one year, we would consider the prediction more reliable if it says there is a 95% chance the true value will fall between $59 billion and $65 billion (a confidence interval), rather than between $40 billion and $84 billion.

When the system's output is not numerical but verbal, it is possible to define (and mathematically model) a confidence set, indicating how much a sentence can vary while still remaining a valid answer to the original request.

“Intuitively,” says Rudi, “if we don’t know how reliable a person is, we try to form an opinion by interacting with them multiple times. Similarly, with the REAL project, we try to build confidence intervals or sets by questioning the system. In the case of the $62 billion forecast, I would ask if $60 billion is still acceptable, then $57 billion, $65 billion, and so on. For verbal responses, I would slightly modify the answers in many different ways and ask whether they still count as valid responses.”

The massive scale of ML-based systems poses both economic and political challenges. Developing and running such complex tools consumes enormous amounts of energy and money, and these costs will eventually be passed on to users. Moreover, only a handful of multinational corporations can now afford to develop such systems, raising serious competition concerns.

From a customer’s perspective, how can one know whether a given ML-based AI system is offered at a fair price? “Our work,” says Rudi, “seeks to determine the minimum development cost of a system that can guarantee a given level of uncertainty, providing a benchmark for what a fair price might be.”

Rudi’s return to Italy was supported by Fondazione Cariplo’s ERC Attrattività program, aimed at recruiting European Research Council grant winners to strengthen the local research ecosystem.

A computer engineer by training, Rudi earned a PhD with a strong mathematical focus at the Italian Institute of Technology (IIT). He has held postdoctoral positions at the Massachusetts Institute of Technology (MIT) in Boston and the Institut national de recherche en sciences et technologies du numérique (INRIA) in Paris.

SDA Bocconi School of Management

• What is the ERC?

  The European Research Council (ERC) is a European institution established in 2007 to fund and promote scientific excellence within the European Union. Its primary goal is to support researchers pursuing innovative, high-risk ideas with the potential to drive major scientific and technological breakthroughs. Grants are awarded on a competitive basis, with scientific excellence as the sole selection criterion