Package {pboost}

pboost: Profile Boosting Framework for Parametric Models

Description

A profile boosting framework for feature selection in parametric models. It offers a unified interface pboost() and several wrapped models, including linear model, generalized linear models, quantile regression, Cox proportional hazards model, beta regression, spatial auto-regressive models.

Author(s)

Maintainer: Zengchao Xu zengc.xu@aliyun.com [copyright holder]

Authors:

• Zengchao Xu zengc.xu@aliyun.com [copyright holder]

See Also

Useful links:

• https://github.com/paradoxical-rhapsody/pboost

• Report bugs at https://github.com/paradoxical-rhapsody/pboost/issues

Extended Bayesian Information Criterion

Description

The Extended BIC possesses the selection consistency in high-dimensional model.

It can be called by the fitted model that has standard logLik method to access the attributes nobs and df, such as lm, glm.

Usage

EBIC(object, p, p.keep, ...)

Arguments

Details

The extended BIC (EBIC) is defined as

EBIC(obj) = BIC(obj) + 2 * r * log(choose(p - |p.keep|, df - |p.keep|)).

Value

A function to obtain the EBIC value of a fitted object.

References

• Jiahua Chen and Zehua Chen (2008). Extended Bayesian information criteria for model selection with large model spaces. Biometrika, 95(3):759–771. doi:10.1093/biomet/asn034

• Jiahua Chen and Zehua Chen (2012). Extended BIC for small-n-large-p sparse GLM. Statistical Sinica, 22(2):555–574. doi:10.5705/ss.2010.216

Extract Coefficients from Lasso Penalized Models

Description

Extract Coefficients from Lasso Penalized Models

Usage

## S3 method for class 'penaltyglmnet'
coef(object, ...)

Arguments

Value

Named vector of non-zero coefficients of under cross-validation error.

Extract Coefficients from Non-Convex Penalized Models

Description

Extract Coefficients from Non-Convex Penalized Models

Usage

## S3 method for class 'penaltyncvreg'
coef(object, ...)

Arguments

Value

Named vector of non-zero coefficients of under cross-validation error.

Extract Coefficients from Non-Convex Penalized Models

Description

Extract Coefficients from Non-Convex Penalized Models

Usage

## S3 method for class 'penaltyrq'
coef(object, ...)

Arguments

Value

A named vector of coefficients.

Forward Regression Selection for Beta Regression Models

Description

fbetareg() inherits the usage of the function betareg::betareg, and performs forward regression selection for beta regression models.

Usage

fbetareg(
  formula,
  data,
  subset,
  na.action,
  weights,
  offset,
  link = c("logit", "probit", "cloglog", "cauchit", "log", "loglog"),
  link.phi = NULL,
  type = c("ML", "BC", "BR"),
  dist = NULL,
  nu = NULL,
  control = betareg.control(...),
  model = TRUE,
  y = TRUE,
  x = FALSE,
  ...,
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A betareg model object fitted on the selected features.

Forward Regression Selection for Cox proportional hazards Model

Description

Forward regression selection for Cox model.

Usage

fcoxph(
  formula,
  data,
  weights,
  subset,
  na.action,
  init,
  control,
  ties = c("efron", "breslow", "exact"),
  singular.ok = TRUE,
  robust,
  model = FALSE,
  x = FALSE,
  y = TRUE,
  tt,
  method = ties,
  id,
  cluster,
  istate,
  statedata,
  nocenter = c(-1, 0, 1),
  ...,
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A coxph model object fitted on the selected features.

Forward Regression Selection for Generalized Linear Models

Description

fglm() inherits the usage of glm, and performs forward regression selection for generalized linear models.

Usage

fglm(
  formula,
  family = gaussian,
  data,
  weights,
  subset,
  na.action,
  start = NULL,
  etastart,
  mustart,
  offset,
  control = list(...),
  model = TRUE,
  method = "glm.fit",
  x = FALSE,
  y = TRUE,
  singular.ok = TRUE,
  contrasts = NULL,
  ...,
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

fglm.fit(
  x,
  y,
  weights = rep.int(1, NROW(y)),
  start = NULL,
  etastart = NULL,
  mustart = NULL,
  offset = rep.int(0, NROW(y)),
  family = gaussian(),
  control = list(),
  intercept = TRUE,
  singular.ok = TRUE,
  selectFun = "logLik",
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A glm model object fitted on the selected features.

Forward Stepwise Spatial Auto-Regressive Model

Description

Forward stepwise strategy for spatial auto-regressive model.

Usage

flagsarlm(
  formula,
  data = list(),
  listw,
  na.action,
  Durbin,
  type,
  method = "eigen",
  quiet = NULL,
  zero.policy = NULL,
  interval = NULL,
  tol.solve = .Machine$double.eps,
  trs = NULL,
  control = list(),
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

fsar.fit(
  x,
  y,
  w,
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

Model object fitted on the selected features.

Forward Regression Selection for Linear Models.

Description

flm() inherits the usage of the function lm, and performs forward regression selection for linear models.

Usage

flm(
  formula,
  data,
  subset,
  weights,
  na.action,
  method = "qr",
  model = TRUE,
  x = FALSE,
  y = FALSE,
  qr = TRUE,
  singular.ok = TRUE,
  contrasts = NULL,
  offset,
  ...,
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

flm.fit(
  x,
  y,
  offset = NULL,
  method = "qr",
  tol = 1e-07,
  singular.ok = TRUE,
  ...,
  selectFun = "logLik",
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A lm model object fitted on the selected features.

Forward Regression Selection for Quantile Regression Models

Description

frq() inherits the usage of the function quantreg::rq, and performs forward regression selection for quantile regression models.

Usage

frq(
  formula,
  tau = 0.5,
  data,
  subset,
  weights,
  na.action,
  method = "br",
  model = TRUE,
  contrasts = NULL,
  ...,
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

frq.fit(
  x,
  y,
  tau = 0.5,
  method = "br",
  ...,
  selectFun = "logLik",
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A rq model object fitted on the selected features.

NULL

Forward Regression Selection Framework

Description

frs() is a generic workhorse function of forward regression selection for parametric regression.

Usage

frs(
  xmat,
  yvec,
  fitFun,
  ...,
  use.formula = TRUE,
  use.intercept = TRUE,
  selectFun = logLik,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

Model object fitted on the selected features.

See Also

fbetareg, fcoxph, fglm, flm, frq, fsar.

Profile Boosting for Beta Regression

Description

pbetareg inherits the usage of betareg::betareg.

Usage

pbetareg(
  formula,
  data,
  subset,
  na.action,
  weights,
  offset,
  link = c("logit", "probit", "cloglog", "cauchit", "log", "loglog"),
  link.phi = NULL,
  type = c("ML", "BC", "BR"),
  dist = NULL,
  nu = NULL,
  control = betareg.control(...),
  model = TRUE,
  y = TRUE,
  x = FALSE,
  ...,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A betareg model object fitted on the selected features.

Examples

## Not run: 
set.seed(2026)
n <- 300
p <- 20
x <- matrix(runif(n*p), n)
mu <- runif(n)
phi <- 1.0

shape1 <- mu * phi
shape2 <- (1-mu) * phi
y <- rbeta(n, shape1, shape2)
DF <- data.frame(y, x)

pbetareg(y ~ ., DF, verbose=TRUE)
fbetareg(y ~ ., DF, verbose=TRUE)

## End(Not run)

Profile Boosting Framework

Description

pboost() is the generic workhorse function of profile boosting framework for parametric regression.

Usage

pboost(
  xmat,
  yvec,
  fitFun,
  scoreFun,
  stopFun = "EBIC",
  ...,
  use.formula = TRUE,
  use.intercept = TRUE,
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

Model object fitted on the selected features.

See Also

pbetareg, pcoxph, pglm, plm, prq, psar.

Profile Boosting for Cox proportional hazards Model

Description

Profile boosting for Cox model.

Usage

pcoxph(
  formula,
  data,
  weights,
  subset,
  na.action,
  init,
  control,
  ties = c("efron", "breslow", "exact"),
  singular.ok = TRUE,
  robust,
  model = FALSE,
  x = FALSE,
  y = TRUE,
  tt,
  method = ties,
  id,
  cluster,
  istate,
  statedata,
  nocenter = c(-1, 0, 1),
  ...,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A coxph model object fitted on the selected features.

Examples

library(survival)
set.seed(2026)
n <- 300
p <- 200

DF <- data.frame(
    time = rpois(n, 5),
    status = rbinom(n, 1, 0.3),
    matrix(rnorm(n*p), n)
)

pcoxph(Surv(time, status) ~ ., DF, verbose=TRUE)
fcoxph(Surv(time, status) ~ ., DF, verbose=TRUE)

Penalized Methods for Feature Selection

Description

Penalized methods for feature selection. These functions are only designed for comparison of numerical simulations.

• pen_glmnet(x, y, family): LASSO penalized models.

• pen_ncvreg(x, y, family, penalty): Non-Convex penalized models.

• pen_rq(x, y, tau, penalty): penalized quantile regression models.

• pen_sar(x, y, rho, w, penalty): penalized spatial auto-regressive models.

Usage

pen_glmnet(x, y, family)

pen_ncvreg(x, y, family, penalty)

pen_rq(x, y, tau, penalty)

pen_sar(x, y, rho, w, penalty)

Arguments

Value

Set of selected features.

Examples

library(survival)
set.seed(2026)
n <- 10
p <- 20

x <- replicate(p, rnorm(n))
b0 <- runif(3, 1.5, 2.0)
eta <- drop(x[, 1:3] %*% b0)

## ---------- linear ----------
y <- rnorm(n, eta)

pen_glmnet(x=x, y=y, family="gaussian") |> coef()
pen_ncvreg(x=x, y=y, family="gaussian", penalty="MCP") |> coef()
pen_ncvreg(x=x, y=y, family="gaussian", penalty="SCAD") |> coef()


## ---------- logistic ----------
y <- rbinom(n, 1, 1.0 / (1.0 + exp(-eta)))

pen_glmnet(x=x, y=y, family="binomial") |> coef()
pen_ncvreg(x=x, y=y, family="binomial", penalty="MCP") |> coef()
pen_ncvreg(x=x, y=y, family="binomial", penalty="SCAD") |> coef()


## ---------- cox ----------
h0 <- 0.01
censoringRate <- 0.3
survivalTime <- -log(runif(n)) / (h0 * exp(eta))
censoringTime <- rexp(n, rate = -log(1 - censoringRate)/median(survivalTime))
y <- cbind(
    time = pmin(survivalTime, censoringTime),
    status = as.numeric(survivalTime <= censoringTime)
)

pen_glmnet(x=x, y=y, family="cox") |> coef()
pen_ncvreg(x=x, y=y, family="cox", penalty="MCP") |> coef()
pen_ncvreg(x=x, y=y, family="cox", penalty="SCAD") |> coef()


## ---------- quantile ----------
y <- eta + rt(n, 2)

pen_rq(x=x, y=y, tau=0.5, penalty="lasso") |> coef()
pen_rq(x=x, y=y, tau=0.5, penalty="MCP") |> coef()
pen_rq(x=x, y=y, tau=0.5, penalty="SCAD") |> coef()


## ---------- sar ----------
w0 <- set_rook_matrix(5, n/5)
rho0 <- 0.5
y <- solve(diag(n) - rho0 * w0, rnorm(n, eta))

pen_sar(x=x, y=y, rho=rho0, w=w0, penalty="lasso") |> coef()
pen_sar(x=x, y=y, rho=rho0, w=w0, penalty="MCP") |> coef()
pen_sar(x=x, y=y, rho=rho0, w=w0, penalty="SCAD") |> coef()

Profile Boosting for Gaussian Graphical Model

Description

Profile boosting for Gaussian graphical model.

Usage

pggm(
  S,
  nObs,
  maxK = floor(min(nObs - 1, NROW(S) - 1, 50)),
  digits = 8,
  verbose = FALSE
)

Arguments

Value

Index set of identified features.

Examples

library(MASS)
library(Matrix)

set.seed(2025)
n <- 1000
p <- 10

Omega <- Diagonal(p)
diag(Omega[1:4, 2:5]) <- diag(Omega[2:5, 1:4]) <- 0.5
Sigma <- chol2inv(chol(Omega))
X <- mvrnorm(n, rep(0, p), Sigma, empirical=TRUE)
S <- cov(X)
system.time( egg <- pggm(S, n) )

Profile Boosting for Generalized Linear Models.

Description

pglm inherits the usage of the built-in function glm.

Usage

pglm(
  formula,
  family = gaussian,
  data,
  weights,
  subset,
  na.action,
  start = NULL,
  etastart,
  mustart,
  offset,
  control = list(...),
  model = TRUE,
  method = "glm.fit",
  x = FALSE,
  y = TRUE,
  singular.ok = TRUE,
  contrasts = NULL,
  ...,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

pglm.fit(
  x,
  y,
  weights = rep.int(1, NROW(y)),
  start = NULL,
  etastart = NULL,
  mustart = NULL,
  offset = rep.int(0, NROW(y)),
  family = gaussian(),
  control = list(),
  intercept = TRUE,
  singular.ok = TRUE,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Value

A glm model object fitted on the selected features.

References

Zengchao Xu, Shan Luo and Zehua Chen (2022). Partial profile score feature selection in high-dimensional generalized linear interaction models. Statistics and Its Interface. doi:10.4310/21-SII706

Examples

set.seed(2026)
n <- 200
p <- 100
x <- matrix(rnorm(n*p), n)

eta <- drop( x[, 1:3] %*% runif(3, 1.0, 1.5) )
y <- rbinom(n, 1, 1/(1+exp(-eta)))
DF <- data.frame(y, x)

## ---------- pboost ----------
pglm(y ~ ., "binomial", DF, verbose=TRUE)
pglm(y ~ ., "binomial", DF, stopFun=BIC, verbose=TRUE)

scoreLogistic <- function(object) {
   eta.hat <- object[["linear.predictors"]]
   return(object[["y"]] - 1/(1+exp(-eta.hat)))
}
(result <- pboost(x, y, glm, scoreLogistic, family="binomial", verbose=TRUE))
all.vars(formula(result)[[3]])

## ---------- frs ----------
fglm(y ~ ., "binomial", DF, verbose=TRUE)
fglm(y ~ ., "binomial", DF, stopFun=BIC, verbose=TRUE)

frs(x, y, glm, family="binomial", verbose=TRUE)

Profile Boosting for Spatial Auto-regressive Model

Description

Profile boosting variable selection for spatial auto-regressive (SAR) model

\bm{y} = \rho W \bm{y} + X \beta + \bm{\varepsilon},

where \bm{y}, \bm{\varepsilon} \in \mathbb{R}^n, X \in \mathbb{R}^{n \times p}, \rho \in (-1, 1).

Usage

plagsarlm(
  formula,
  data = list(),
  listw,
  na.action,
  Durbin,
  type,
  method = "eigen",
  quiet = NULL,
  zero.policy = NULL,
  interval = NULL,
  tol.solve = .Machine$double.eps,
  trs = NULL,
  control = list(),
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

psar.fit(x, y, w, stopFun = "EBIC", keep = NULL, maxK = NULL, verbose = FALSE)

Arguments

Value

Model object fitted on the selected features.

Examples

set.seed(2026)

w <- set_rook_matrix(15, 15)

n <- NROW(w)
p <- 30
x <- matrix(rnorm(n*p), n) %*% chol(0.7^abs(outer(1:p, 1:p, "-")))
eta <- drop(x[, 1:3] %*% runif(3, 1.5, 2.0))

rho0 <- 0.2
sig0 <- 1.0
y <- solve(diag(n) - rho0 * w, rnorm(n, eta, sd=sig0)) |> drop()

## ---------- pboost ----------
system.time( egg <- psar.fit(x, y, w, verbose=TRUE) )
y.tilde <- (diag(NROW(x)) - egg[["rho"]] * w) %*% y

beta.hat <- egg[["beta"]]
idx <- as.integer(sub("[[:alpha:]]", "", names(beta.hat)))
sig2.hat <- mean( (y.tilde - drop(x[, idx, drop=FALSE] %*% beta.hat))^2 )
print( egg[["sig2"]] - sig2.hat )

## Not run: 
system.time(
plagsarlm(y ~ ., data=data.frame(y, x), listw=spdep::mat2listw(w, style="W"), verbose=TRUE)
)

## End(Not run)

## ---------- frs ----------
fsar.fit(x, y, w, verbose=TRUE)

## Not run: 
system.time(
flagsarlm(y ~ ., data=data.frame(y, x), listw=spdep::mat2listw(w, style="W"), verbose=TRUE)
)

## End(Not run)

Profile Boosting for Linear Models.

Description

plm inherits the usage of the built-in function lm.

Usage

plm(
  formula,
  data,
  subset,
  weights,
  na.action,
  method = "qr",
  model = TRUE,
  x = FALSE,
  y = FALSE,
  qr = TRUE,
  singular.ok = TRUE,
  contrasts = NULL,
  offset,
  ...,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

plm.fit(
  x,
  y,
  offset = NULL,
  method = "qr",
  tol = 1e-07,
  singular.ok = TRUE,
  ...,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

Arguments

Details

plm is an equivalent implementation to the sequential lasso method proposed by Luo and Chen(2014, doi:10.1080/01621459.2013.877275).

Value

A lm model object fitted on the selected features.

References

• Zengchao Xu, Shan Luo and Zehua Chen (2022). Partial profile score feature selection in high-dimensional generalized linear interaction models. Statistics and Its Interface. doi:10.4310/21-SII706

• Shan Luo and Zehua Chen (2014). A Sequential Lasso Method for Feature Selection with Ultra-High Dimensional Feature Space. Journal of the American Statistical Association, 109(507):223–232. doi:10.1080/01621459.2013.877275

Examples

set.seed(2026)
n <- 300
p <- 200
x <- matrix(rnorm(n*p), n)

eta <- drop( x[, 1:3] %*% runif(3, 1.0, 1.5) )
y <- rnorm(n, eta, sd=sd(eta))
DF <- data.frame(y, x)

plm(y ~ ., DF, verbose=TRUE)
plm(y ~ ., DF, stopFun=BIC, verbose=TRUE)
pboost(x, y, lm, residuals, verbose=TRUE)

flm(y ~ ., DF, verbose=TRUE)
flm(y ~ ., DF, stopFun=BIC, verbose=TRUE)
frs(x, y, lm, verbose=TRUE)

Profile Boosting for Quantile Regression Models

Description

prq inherits the usage of the function quantreg::rq.

Usage

prq(
  formula,
  tau = 0.5,
  data,
  subset,
  weights,
  na.action,
  method = "br",
  model = TRUE,
  contrasts = NULL,
  ...,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = FALSE
)

prq.fit(
  x,
  y,
  tau = 0.5,
  method = "br",
  ...,
  stopFun = "EBIC",
  keep = NULL,
  maxK = NULL,
  verbose = TRUE
)

Arguments

Value

A rq model object fitted on the selected features.

Examples

library(quantreg)
set.seed(2026)
n <- 300
p <- 20
x <- matrix(rnorm(n*p), n)

eta <- drop( x[, 1:3] %*% runif(3, 1.0, 1.5) )
y <- eta + (1.0 + x[, 3]) * rnorm(n)
DF <- data.frame(y, x)

tau <- 0.5
prq(y ~ ., tau, DF, verbose=TRUE)

BIC <- function(obj) AIC(obj, k=-1)
prq(y ~ ., tau, DF, stopFun=BIC, verbose=TRUE)
frq(y ~ ., tau, DF, stopFun=BIC, verbose=TRUE)

scoreFun <- function(object)
   return(ifelse(object[["y"]] < fitted(object), tau - 1, tau))

pboost(x, y, rq, scoreFun, BIC, tau=tau, verbose=TRUE)

prq.fit(x, y, verbose=TRUE)
frq.fit(x, y, verbose=TRUE)

Functions for Spatial Auto-regressive Model

Description

• get_rho(): MLE of \rho in SAR model. See psar.

• sar.fit(): Fit SAR model with given \rho.

• set_rook_matrix(): Construct Rook weight adjacency matrix.

• logLik(), BIC(), residuals(), coef().

Usage

get_rho(x, y, w)

sar.fit(x, y, w)

## S3 method for class 'sarpboost'
residuals(object, ...)

## S3 method for class 'sarpboost'
logLik(object, ...)

## S3 method for class 'sarpboost'
BIC(object, ...)

## S3 method for class 'sarpboost'
coef(object, ...)

set_rook_matrix(rNum, cNum)

Arguments

Value

Value or list.