QuaGha14.pdf (2.32 MB)
A very simple safe-Bayesian random forest
journal contribution
posted on 2023-06-08, 21:11 authored by Novi QuadriantoNovi Quadrianto, Zoubin GhahramaniRandom forests works by averaging several predictions of de-correlated trees. We show a conceptually radical approach to generate a random forest: random sampling of many trees from a prior distribution, and subsequently performing a weighted ensemble of predictive probabilities. Our approach uses priors that allow sampling of decision trees even before looking at the data, and a power likelihood that explores the space spanned by combination of decision trees. While each tree performs Bayesian inference to compute its predictions, our aggregation procedure uses the power likelihood rather than the likelihood and is therefore strictly speaking not Bayesian. Nonetheless, we refer to it as a Bayesian random forest but with a built-in safety. The safeness comes as it has good predictive performance even if the underlying probabilistic model is wrong. We demonstrate empirically that our Safe-Bayesian random forest outperforms MCMC or SMC based Bayesian decision trees in term of speed and accuracy, and achieves competitive performance to entropy or Gini optimised random forest, yet is very simple to construct.
History
Publication status
- Published
File Version
- Accepted version
Journal
IEEE Transactions on Pattern Analysis and Machine IntelligenceISSN
0162-8828Publisher
Institute of Electrical and Electronics Engineers (IEEE)External DOI
Issue
6Volume
37Page range
1297-1303Department affiliated with
- Informatics Publications
Full text available
- Yes
Peer reviewed?
- Yes
Legacy Posted Date
2015-06-18First Open Access (FOA) Date
2015-06-18First Compliant Deposit (FCD) Date
2015-06-18Usage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC