Prior Specification Constructors
dbartsPriors.RdA list of constructor functions building the prior specifications that the
split.probs (dart()) and leaf.prior arguments of
dbarts and the fitting functions accept. Bundling them keeps generic names like
normal out of the search path, where another package could mask them
or be masked depending on attach order.
The residual priors chisq and fixed are what a family
object's sigma setting takes - family = gaussian(sigma =
chisq(3, 0.9)) - which is the one place every entry point reaches the
residual prior; this vocabulary resolves by bare name inside that argument
too.
Format
A list of functions:
cgm(power = 2, base = 0.95, split.probs = NULL)The value of the retired
tree.priorargument ofdbartsanddbartsSpec, which is accepted with a warning until dbarts 1.1-0, whencgmis removed. Write the tree prior aspower,baseandsplit.probsinstead. ANULLpowerorbaseis the default.split.probsweights the choice of split variable:NULLis uniform, a named vector assigns by column or term name (with an optional".default"element), an unnamed vector assigns by position. Entries must be non-negative and finite, with at least one positive. Named or positional probabilities are matched to the data when a sampler is built. A column of probability zero is never split on, just as one a forest'svarsleaves out is never split on, on every path: where the columns of positive probability are used up along a path, the node cannot split. A sampler's$setModelrefuses a zero on a column its trees split on.dart(a = 0.5, b = 1, rho = NULL, alpha = 1, update.alpha = TRUE, update.delay = NULL)DART (Linero 2018), given as
split.probs = dart(): a Dirichlet prior over the split-variable probabilities inducing variable selection. The tree's depth is the fitting function'spowerandbase, as without DART.alphais the concentration, optionally sampled (update.alpha) on a grid with a Beta(a,b) prior onalpha / (alpha + rho);rhodefaults to the number of predictors.NAis a missing value, not a spelling of the default, and is refused forrhoandupdate.delayalike. Updates hold untilupdate.delayiterations have passed (default: half the control's burn-in), so the forest is likelihood-informed when counts first enter the Dirichlet.normal(k = NULL, sd = NULL)Normal prior on the leaf values, named one of two ways, never both. Either names the spread of the whole forest - the sum of trees - not of one tree.
kis relative to the scale the data fixes: a positive scalar, a hyperprior onkbuilt withchi, orNULLfor the default, 2 for continuous responses andchi(1.5, 2)for binary ones and counts (nbinom). The continuous default follows Chipman, George, and McCulloch's argument that with leaf standard deviationsigma_mu = 0.5 / (k * sqrt(m))formtrees,kprior standard deviations of \(f(x)\) span the whole response range regardless ofm; seedbarts's Details for the response-scaling caveat this relies on. A string such as"chi(1.5)"is kept for 0.9-x.k = Infis accepted and holds every tree at zero, so the fit is the offset, plus on a continuous response the center of the rangey - offsetspans.sdis the standard deviation of the normal prior on the leaf model's own parameter, for the forest total, on the scale the family's forest fits: a positive number, or a hyperprior on the sd itself built withinvchi. It takes no string form. The scale and thek.scalekis relative to, per family:family sdis stated ink.scaleatk = 1gaussian, student response units half the training response range aft log survival time half the observed log-time range probit, ordinal probit latent 3 logistic log-odds latent \(\pi\sqrt{3}\) nbinom log mean 3 hazard its link's latent 3 or \(\pi\sqrt{3}\) so a fixed
kis thesdk.scale/k. For the constant leafsdis exactly the prior standard deviation of \(f(x)\) at every \(x\); for the other leaf models it is the sd of their parameter, and the prior spread of \(f(x)\) it implies is a consequence stated under each. Under amonotoneconstraint (seedbarts)sdis the prior sd of a leaf value that no ordered neighbor bounds; a leaf a neighbor bounds is drawn from a normal \(\sqrt{\pi / (\pi - 1)}\) times as wide, truncated to the ordering, which matches its marginal prior variance tosd^2, andsdis a lower bound on the prior sd of \(f(x)\) in the interior. A namedsdis absolute, and is another way of writingk: the sampler runssd = sask = k.scale / sagainst the table'sk.scale, andsd = invchi(df, s)ask = chi(df, k.scale / s)started atk.scale / s, and restates either whenever the response transform is re-anchored (seedbartsSampler-class), so the sd, or the prior on it, keeps its meaning where akmoves with the data. A drawn sd's current value lags at a re-anchor until its next draw, as a drawnkdoes. Thek.scalea sampler or a fit reports is the table's, however the prior was written. It is refused on multinomial and multi-forest models (whose spreads come from their calibration maps, the latter stated throughforest(sd = ), a different quantity), and on a hurdle fit, whose two parts are on different scales.NULLleaves either unnamed;NAis a missing value and is refused, here and inlinearandgp. A barenormal()given as a forest'samplitudeis something else, a normal prior on that forest's drawn coefficient whose median size is 1 (on a factor, the difference between two levels' coefficients); seeforest.linear(columns, k = NULL, sd = NULL)Each leaf fits an intercept plus a linear term in the designated continuous predictor columns instead of a constant, so the forest models smoothly-varying coefficients.
columnsnames model matrix columns (character) or indexes them (numeric) and is matched to the data when a sampler is built; unordered factor columns cannot be designated. The covariates are standardized internally and every coefficient shares thenormal(k)prior. Reported leaf values keep the intercept;getTreesadds onebeta.<column>column per covariate.xbartaccepts the same specification through its ownleaf.priorargument.kandsdname the spread as they do fornormal,sdper standardized covariate: every coefficient, the intercept included, has prior sdsdfor the forest total, so the prior sd of \(f(x)\) issd * sqrt(1 + ||z||^2)at standardized covariates \(z\), andsdis its lower bound, attained at the covariate means.gp(columns, k = NULL, lengthscale = NULL, max.leaf.size = 256L, sd = NULL)Each leaf fits a smooth Gaussian-process function of the designated continuous predictor columns, drawn under a squared-exponential kernel whose prior scale ties to
kexactly as the other leaf priors;columnsresolves as forlinear, andkmay be fixed or given thechihyperprior, sampled from the drawn functions' standardized magnitudes.lengthscalefixes the kernel lengthscales on the standardized covariate scale, one value per column or one recycled scalar;NULLuses the median pairwise-distance heuristic per column. Leaves holding more thanmax.leaf.sizeobservations fall back to constant fits, confining the cubic kernel cost. Fewer trees are needed than for constant leaves: 10 to 25 is a reasonable range, since each leaf already fits a smooth function, while past about 50 the per-tree shrinkage leaves each function too small to improve on constant fits and the cost, linear in the number of trees and cubic in the leaf size, keeps rising. The two settings interact: a tree typically holds only two or three leaves, so a leaf carries a sizable fraction of the sample andmax.leaf.sizemust be comparable to that fraction for the kernel to be used at all; loweringn.treesgrows deeper trees and is the effective way to bring leaves under the cap, whereas loweringmax.leaf.sizeonly sends more leaves to the constant fallback. Because that fallback is silent - the fit stays coherent, it is simply not a Gaussian process over the leaves that took it - every leaf evaluation is counted. The counts ride the fit asgp.fallback(a named pair,evaluationsandfallbacks), and a run in which more than a quarter of evaluations fell back warns once (classdbartsGPFallbackWarning), giving the share: at the default cap on a design of any size that is the normal outcome, not an error, and the warning is the notice that the cap rather than the kernel is doing the modeling. The remedies are the two the paragraph above names, in the order it names them. Function-valued fits ride prediction only:getTreesreportsNAleaf values, andkeepTreesstorage grows with the leaf sizes.xbartaccepts the same specification through its ownleaf.priorargument.kandsdname the spread as they do fornormal,sdas the amplitude of the forest total's Gaussian process: the prior sd of \(f(x)\) issdat a leaf's own rows and decays away from them, sosdis its upper bound.predictat a training row re-krigs the jitter-free posterior mean from the cached kernel and drawn training values, while the fit recorded during sampling includes a small conditioning nugget on the kernel diagonal; the two differ by roughly 2e-3 to 3e-3 at training locations (never during MCMC, which reads the recorded fit directly - onlypredictand test-data prediction re-krig).chi(df = 1.5, scale = 2)Chi hyperprior over
k, sampled along with the rest of the model:kis given a chi distribution withdfdegrees of freedom and scalescale. The default,chi(1.5, 2), centers the sampledknear the field-standard fixed value of 2 (prior median 1.9) while letting it adapt to the data. It is the same prior assd = invchi(df, k.scale / scale).scale = Infremains accepted, but the posterior it giveskis improper: with little signal, few trees or many degrees of freedom,kcan drift to infinity, where every leaf is zero and the trees add nothing to the fit. The first argument wasdegreesOfFreedomin 0.9-x; that name is accepted, with a warning, until dbarts 1.1-0.invchi(df = 1.5, scale)Scaled inverse chi hyperprior over a leaf prior's
sd, on the sd's own scale: \(sd = scale / \chi_{df}\), equivalently \(sd^2\) is scaled inverse chi-square withdfdegrees of freedom.scalehas no default, since a spread on the family's scale has no data-free one;scale = 0is the improper \(sd^{-(df + 1)}\) limit. It is the same prior ask = chi(df, k.scale / scale), andchi(df, Inf)isinvchi(df, 0); the binary defaults areinvchi(1.5, 1.5)on probit andinvchi(1.5, pi * sqrt(3) / 2)on logistic. In Stan's terms it isscaled_inv_chi_square(df, scale / sqrt(df))on the variance, and an inverse gamma with shapedf / 2and scalescale^2 / 2. No other distribution on the sd is offered: this is the one conjugate to the normal leaves, which the sampler draws exactly; a half-t or half-Cauchy would need a non-conjugate update. Refused under a monotone constraint, as achionkis.chisq(df = 3, quant = 0.9)Chi-squared prior on the residual variance. Its scale is set from
sigest, the residual-scale estimate supplied (or derived) at creation:quantis the prior probability that the residual standard deviation is below that estimate, sosigest = 1gives the prior under which \(\sigma\) is below 1 with probabilityquant.dfis a single positive finite number andquanta single number between 0 and 1: an infinitedf, the limit that holds \(\sigma\) at a point, is refused, andfixed()states a held value.fixed(value = 1)Fixed residual variance, in squared response units. It is the residual scale rather than a distribution over one, so it ignores
sigest- which is refused beside it unless it equals the square root ofvalue, when it is accepted with a message and is an error from dbarts 1.1-0 - and suppresses the sampler's own \(\sigma\) draw.valueis a single positive finite number.student(df = 1)A Student-t prior with
dfdegrees of freedom, finite and at least 1, on a forest's drawn coefficient, given as a forest'samplitude; seeforest. A barestudent()meansdf = 1here, the half-Cauchy, while the response family of the same name,dbartsFamilies$student, estimates its df when none is given; each is read by where it is written.
Details
Inside the prior arguments of the fitting functions - and inside a family's
sigma - the same constructors are available by bare name, so dbarts(..., leaf.prior =
normal(chi(1.5))) works regardless of what packages are attached: those
arguments are evaluated with this vocabulary layered over the calling
environment (for an argument a wrapper forwards through its ...,
the one it was written in), along with num.vars, the number of predictor columns.
A prior object built ahead of time with dbartsPriors$... can be
passed to the same arguments. A wrapper's named formal that forces a bare
constructor call before passing it on reaches R's own could-not-find-function
error instead, extended with a hint naming dbartsPriors$normal(...) and
its siblings.
References
Chipman, H., George, E., and McCulloch, R. (2010) BART: Bayesian additive regression trees. The Annals of Applied Statistics, 4(1), 266–298. doi:10.1214/09-AOAS285 .
Linero, A.R. (2018) Bayesian regression trees for high-dimensional prediction and variable selection. Journal of the American Statistical Association, 113(522), 626–636.
Gramacy, R.B. and Lee, H.K.H. (2008) Bayesian Treed Gaussian Process Models With an Application to Computer Modeling. Journal of the American Statistical Association, 103(483), 1119–1130.
Examples
prior <- dbartsPriors$normal(dbartsPriors$chi(1.5))
x <- matrix(runif(200), ncol = 2)
y <- x[, 1] + rnorm(100, 0, 0.5)
sampler <- dbarts(y ~ x, leaf.prior = prior, power = 1.5)
## DART: a Dirichlet prior over split-variable probabilities, useful
## when only a handful of predictors out of many are relevant
set.seed(0)
n <- 60L
x.dart <- matrix(runif(n * 8), n)
y.dart <- 4 * x.dart[, 1] + rnorm(n, 0, 0.2)
fit.dart <- dbarts(y.dart ~ x.dart, split.probs = dart(),
control = dbartsControl(n.chains = 1L,
n.threads = 1L), n.trees = 10L)
samples.dart <- fit.dart$run(20L, 20L)
## linear leaves: each leaf fits an intercept plus a slope in x1,
## useful for a function that is smooth and roughly linear in a subset
## of the predictors
set.seed(1)
x1 <- runif(n)
x2 <- runif(n)
y.lin <- 3 * x1 + rnorm(n, 0, 0.2)
df.lin <- data.frame(x1, x2, y.lin)
fit.lin <- dbarts(y.lin ~ x1 + x2, df.lin, leaf.prior = linear("x1"),
control = dbartsControl(n.chains = 1L,
n.threads = 1L), n.trees = 10L)
samples.lin <- fit.lin$run(20L, 20L)
## gp leaves: each leaf fits a smooth Gaussian-process function of x1,
## useful for a smoothly-varying, nonlinear function
set.seed(2)
y.gp <- sin(2 * pi * x1) + rnorm(n, 0, 0.2)
df.gp <- data.frame(x1, x2, y.gp)
fit.gp <- dbarts(y.gp ~ x1 + x2, df.gp,
leaf.prior = gp("x1", max.leaf.size = 52L),
control = dbartsControl(n.chains = 1L,
n.threads = 1L), n.trees = 10L)
samples.gp <- fit.gp$run(20L, 20L)
## the cap of 52 observations per leaf sends some leaves to the constant
## fallback (about a tenth of the evaluations here), counted in gp.fallback;
## a warning comes only when more than a quarter fall back
print(unlist(attr(samples.gp, "gp.fallback")))
#> evaluations fallbacks
#> 2158 354