In contrast to estimating accuracy of entire regression models, reliability estimates of regression predictions allow the user to decide how to handle individual predictions given by the regression model. The thesis describes the implementation of a package for estimating the reliability of individual regression predictions in the computer language R. The work describes four implemented methods: sensitivity analysis, bagging, local cross-validation and local modeling of prediction errors. A method for evaluation has also been implemented, which helps the user to choose the best reliability estimation method for their dataset. The reliability of each predicted example gives the user important information about its quality. We also present the architecture, parameters and process of execution from the functions’ call to its return values. The implemented library offers the possibility of parallel execution for faster calculation of reliability estimates. By computing the correlation with the actual prediction error the user can decide which method works best. Lastly, we evaluate the time and memory efficiency of our package and compared the efficiency of parallel and
single thread execution. We accomplished noticeable improvements in the time and memory complexity of the LCV and CNK methods.
|
