According to a new paper published by the Bank of Spain, the regulators must make “significant efforts” to evaluate machine learning models used to predict credit defaults.
In their working paper, Andrés Alonso and José Manuel Carbó find machine learning (ML) models “outperform” traditional models when predicting credit default, but only up to a certain point. New reports show that more advanced ML models may offer gains in classification power of up to 20%.
According to the paper, the financial sector is increasingly adopting machine learning tools to manage credit risk. In this environment, supervisors face the challenge of allowing credit institutions to benefit from technological progress and financial innovation, while at the same ensuring compatibility with regulatory requirements and that technological neutrality is observed.
Andrés Alonso and José Manuel Carbó propose a new framework for supervisors to measure the costs and benefits of evaluating ML models, aiming to shed more light on this technology’s alignment with the regulation.
They follow three steps:
First, they identify the benefits by reviewing the literature. They observe that ML delivers predictive gains of up to 20?% in default classification compared with traditional statistical models.
Second, they use the process for validating internal ratings-based (IRB) systems for regulatory capital to detect ML’s limitations in credit risk mangement. They identify up to 13 factors that might constitute a supervisory cost.
Finally, they propose a methodology for evaluating these costs. For illustrative purposes, they compute the benefits by estimating the predictive gains of six ML models using a public database on credit default. Then they calculate a supervisory cost function through a scorecard in which they assign weights to each factor for each ML model, based on how the model is used by the financial institution and the supervisor’s risk tolerance.
From a supervisory standpoint, having a structured methodology for assessing ML models could increase transparency and remove an obstacle to innovation in the financial industry.
__________________________________________________
Sources:
https://econpapers.repec.org/paper/bdewpaper/2032.htm
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3724374
https://www.centralbanking.com/fintech/7704801/paper-offers-method-for-evaluating-machine-learning-models
https://www.bde.es/f/webpi/SES/staff/alonsoandres/files/Risk_adjusted_performance_machine_learning_models_credit_default_predictionsuerf.pdf
https://www.eba.europa.eu/sites/default/files/document_library/About%20Us/EBA%20Research%20Workshops/2020/Presentations/936780/Session%202.2%20-%20presentation%20-%20Jose%20Manuel%20Carbo%20-%20%20Understanding%20the%20performance%20of%20machine%20learning%20models%20to%20predict%20credit%20default.pdf?retry=1
Président de l'association
