A Comparison of Classifiers Precision when Replicating Credit Ratings, and Their Applications

Christian HĂžjdevang & Jakob Radoor Frederiksen

Student thesis: Master thesis

Abstract

This thesis contributes to the study of the efficacy of using different models for replicating corporate credit ratings with only publicly available data. The purpose of this is to determine whether an autonomous model could become competitive with the well-established credit rating agencies.
In total, five different models are tested, the simplest of which are a linear regression and a logistic regression. The remaining three models are all based on machine learning theory, and as a result thereof, they are much more sophisticated and complex. The machine learning models are a k-Nearest Neighbor, a Decision Tree and a Support Vector Machine. These models have been carefully selected, as we assess them to be best fit for replicating credit ratings.
We have used the theory behind fundamental credit analysis as the framework for building our models and for collecting data. We however, have used an entirely different approach to fundamental credit analysis, compared to how it is depicted in the prevalent literature. This has led to a significantly different dataset compared to the ones used in similar studies, and we partly attribute our good results produced by our models to this.
The dataset collected consists of credit ratings issued between 1981 and 2017 to US and EU corporates. The dataset consists of 4,294 credit ratings and 832 different independent variables, which leads to a total dataset of nearly 3.6 million data points. The companies belong to four different industries, which has made the results more valid and useful, as it shows the usefulness of having a flexible model.
On a randomized dataset, all of the machine learning models achieve precisions above 50% correctly classified ratings thereby beating a model using the previous rating as the estimated rating. The best model is a Decision Tree, which classify 55.05% of the observations correctly. The above results continue to hold true when the models are tested on a time sorted dataset, where the machine learning models all correctly classify more than 60% of the observations. On this configuration of the dataset, the Decision Tree and Support Vector Machine models both achieve a 61.49%. These results are among the very best results in the current literature.
Practical applications of machine learning models in areas such as credit ratings are still relatively undeveloped and untested. Furthermore, other factors are protecting the credit rating agencies from competition including their considerable resources and a protective legislation. Based on all of the above, we expect it to be some time before machine learning models become a viable competitor to the established credit rating agencies, despite the promising results.

EducationsMSc in Finance and Accounting, (Graduate Programme) Final Thesis
LanguageDanish
Publication date2018
Number of pages96