solve_kappa computes the problem dimension \(\kappa\) where
the phase transition occurs in binary regression, given \(\beta\) and \(\gamma_0\). solve_beta and solve_gamma
computes \(\beta_0\) and \(\gamma_0\) on the phase transition curve
given the other one and \(\kappa\).
Usage
solve_kappa(rho_prime, beta0, gamma0)
solve_beta(rho_prime, kappa, gamma0, verbose = FALSE)
solve_gamma(rho_prime, kappa, beta0, verbose = FALSE)Arguments
- rho_prime
Function. Success probability \(\rho(t) = \mathrm{P}(Y=1\,|\, X^\top \beta = t)\)
- beta0
Numeric. Intercept value.
- gamma0
Numeric. Signal strength.
- kappa
Numeric. Problem dimension on the phase transition curve.
- verbose
Print progress if
TRUE.
Value
Numeric. Problem dimension \(\kappa\) (\(\beta\) or \(\gamma\)) on the phase transition curve.
Details
When covariates are multivariate Gaussian, the phase transition dimension can be characterized as following. $$ \kappa > h_{\mathrm{MLE}}(\beta_0, \gamma_0) \implies \lim_{n,p\to\infty} \mathrm{P}(\text{MLE exists}) = 0 $$ $$ \kappa < h_{\mathrm{MLE}}(\beta_0, \gamma_0) \implies \lim_{n,p\to\infty} \mathrm{P}(\text{MLE exists}) = 1. $$ The function \(h\) is defined to be $$ h_{\mathrm{MLE}}(\beta_0, \gamma_0) = \min_{t_0, t_1 \in \mathbb{R}} \mathbb{E}\left[(t_0 Y + t_1 V - Z)_+^2 \right], $$ where \(X\sim\mathcal{N}(0,1)\) and \(\mathrm{P}(Y=1|X) = 1- \mathrm{P}(Y=-1|X) = \rho'(\beta_0 + \gamma_0 X) \). \(Z\sim\mathcal{N}(0,1)\) and is independent of \(X,Y\). The phase transition curve is thus \(\kappa(\beta_0, \gamma_0)\). It also depends on the success probability \(\rho'\).
References
The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression Emmanuel J. Candes and Pragya Sur, Ann. Statist., Volume 48, Number 1 (2020), 27-42.
Examples
if (FALSE) {
# when Y is independent of X, should return 0.5 for logistic model
# should return 0.5
rho_prime_logistic <- function(t) 1 / (1 + exp(-t))
solve_kappa(rho_prime_logistic, 0, 0)
}