A very simple safe-Bayesian random forest

Quadrianto, Novi and Ghahramani, Zoubin (2015) A very simple safe-Bayesian random forest. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37 (6). pp. 1297-1303. ISSN 0162-8828

[img]
Preview
PDF - Accepted Version
Download (2MB) | Preview

Abstract

Random forests works by averaging several predictions of de-correlated trees. We show a conceptually radical approach to generate a random forest: random sampling of many trees from a prior distribution, and subsequently performing a weighted ensemble of predictive probabilities. Our approach uses priors that allow sampling of decision trees even before looking at the data, and a power likelihood that explores the space spanned by combination of decision trees. While each tree performs Bayesian inference to compute its predictions, our aggregation procedure uses the power likelihood rather than the likelihood and is therefore strictly speaking not Bayesian. Nonetheless, we refer to it as a Bayesian random forest but with a built-in safety. The safeness comes as it has good predictive performance even if the underlying probabilistic model is wrong. We demonstrate empirically that our Safe-Bayesian random forest outperforms MCMC or SMC based Bayesian decision trees in term of speed and accuracy, and achieves competitive performance to entropy or Gini optimised random forest, yet is very simple to construct.

Item Type: Article
Schools and Departments: School of Engineering and Informatics > Informatics
Subjects: Q Science > QA Mathematics > QA0273 Probabilities. Mathematical statistics
Depositing User: Novi Quadrianto
Date Deposited: 18 Jun 2015 19:57
Last Modified: 14 Mar 2017 06:07
URI: http://sro.sussex.ac.uk/id/eprint/54606

View download statistics for this item

📧 Request an update