In the last decade, machine learning models have started to be increasingly used in all areas of our lives. They are present in the form of recommendation systems, predictive models, as well as risk assessment systems or decision-making support systems.
Even in areas, where decisions are made by field experts (doctors in medicine or business analysts in economics) machine learning models are increasingly used.
In such areas, however, it is crucial that the expert or decision-maker has a good understanding of how the used model has calculated its prediction.
A big opportunity in these areas is presented by interpretable machine learning models. These models achieve lower predictive accuracy, but the users are able to identify which attributes were used and how they affect the model prediction. Because of that, additional explanation of the machine learning model is not required.
In this work, we evaluated the performance of the interpretable machine learning model called RiskSLIM. The result of its learning is a simple scorer where user can add points from each attribute and calculate assessment of the risk. We have explored how different features of datasets affect learning time and other performances of the machine learning model. Additionally, we have explored if RiskSLIM is capable of identifying complex relations between learning attributes and how it performs on non-monotone datasets. The algorithm is restricted to binary datasets, which are very rare in the real world. Since all numerical attributes have to be discretized and binarized to get binary attributes, we have researched multiple types of discretization and then implemented and compared their performances. To compare predictive accuracy of RiskSLIM we have used well known algorithm called XGBoost, which is currently one of the best methods for machine learning on tabular datasets. We have tried to expand the algorithm in a way that could also predict multiclass problems. Here again we found multiple types of expansions of an algorithm, implemented them, and then compared them between each other. We compared both predictive accuracy and their interpretability. In the last part, we tried to analyze and replace the CPLEX library that RiskSLIM uses to solve mixed integer nonlinear problem. Since developers have to pay for the licence of this library (except for academics purposes), we have tried to replace it with an open-source one. We have identified functions of CPLEX that are used in RiskSLIM, and replaced their interactions with the same functionalities of the open-source libraries.
We can conclude that RiskSLIM presents an interpretable alternative to more popular machine learning approaches. The offsets of more interpretable predictions are lower predictive accuracy, longer learning time (especially when dataset has more than 20 attributes) and a limitation to only be able to work with binary datasets. To bypass this limitation, we implemented several methods of discretization and tested several ways to expand algorithm usability on multiclass problems. We found out that the best discretization technique was presented by Fayyad and Irani. The best expansion of an algorithm was with Directed Decision Acyclic Graph. When trying to replace CPLEX we learned, that none of the alternatives supports all the required functionalities, which RiskSLIM uses. Because of that, its replacement is not possible without more complex interference in the code of RiskSLIM.
|
