XGBoost: A Scalable Tree Boosting System
Tianqi Chen
University of Washington
tqchen@cs.washington.edu
Carlos Guestrin
University of Washington
guestrin@cs.washington.edu
ABSTRACT
Tree boosting is a highly effective and widely used machine
learning method. In this paper, we describe a scalable end-
to-end tree boosting system called XGBoost, which is used
widely by data scientists to achieve state-of-the-art results
on many machine learning challenges. We propose a novel
sparsity-aware algorithm for sparse data and weighted quan-
tile sketch for approximate tree learning. More importantly,
we provide insights on cache access patterns, data compres-
sion and sharding to build a scalable tree boosting system.
By combining these insights, XGBoost scales beyond billions
of examples using far fewer resources than existing systems.
Keywords
Large-scale Machine Learning
1. INTRODUCTION
Machine learning and data-driven approaches are becom-
ing very important in many are
data/tree/boosting/insig/hts/scalable/XG/Boost/widely/learning/
data/tree/boosting/insig/hts/scalable/XG/Boost/widely/learning/
-->