Solves a system of nonlinear equations

This function solves a system of equations, whose solution characterizes the asymptotic bias and variance of the M-estimator (in case of the MLE, it is the negative log-likelihood).

Usage

find_param(
  rho_prime = rho_prime_logistic,
  f_prime1 = f_prime1_logistic,
  f_prime0 = f_prime0_logistic,
  kappa,
  gamma,
  beta0 = 0,
  intercept = TRUE,
  verbose = FALSE,
  x_init = NULL
)

Arguments

rho_prime

A function that computes the success probability \(\rho'(t) = \mathrm{P}(Y=1 | X^\top \beta = t)\), here \(\beta\) is the coefficient. The default is logistic model.

f_prime1

A function. Derivative of the loss function when \(Y = 1\). The default is the derivative of the negative log-likelihood of logistic regression when \(Y = 1\).

f_prime0

A function. Derivative of the loss function when \(Y = -1\). The default is the derivative of the negative log-likelihood of logistic regression when \(Y = -1\).

kappa

Numeric. The problem dimension \(\kappa = p/n\).

gamma

Numeric. Signal strength \(\gamma = \sqrt{\mathrm{Var}(X^\top \beta)}\).

beta0

Numeric. Intercept.

intercept

If TRUE, the glm contains an intercept term. intercept = TRUE by default.

verbose

If TRUE, print progress at each step.

x_init

Initial values for the parameters.

Value

A vector solution to the system. When gamma != 0 and b !=0, returns \((\alpha_\star, \lambda_\star, \sqrt{\kappa}\sigma_\star, b_\star)\). When signal strength is zero (gamma = 0), returns the solution to the system with three equations \((\alpha_\star = 0, \lambda_\star, \sqrt{\kappa}\sigma_\star, b_\star)\). When gamma = 0 and

b = 0, returns \((\alpha_\star = 0, \lambda_\star, \sqrt{\kappa}\sigma_\star)\).

References

The Impact of Regularization on High-dimensional Logistic Regression, Fariborz Salehi, Ehsan Abbasi and Babak Hassibi, Proceedings of NeurIPS 2019.

Examples

# Compute parameters for a logistic model
param <- find_param(kappa = 0.1, gamma = sqrt(5))
# Asymptotic bias
param[1]
#> [1] 1.169051
# Standard deviation
param[3] / sqrt(0.1)
#> [1] 3.349089
# Another example
param <- find_param(kappa = 0.1, gamma = 0, intercept = FALSE)
# Asymptotic standard deviation
param[2] / sqrt(0.1)
#> [1] 1.565013