gyp900818
Show that the ridge regression estimate is the mean (and mode)
of the posterior distribution, under a Gaussian prior β ∼ N(0, τI), and
Gaussian sampling model y ∼ N(Xβ, σ2I).
hyjouc
[未知用户]
\(Y|X\thicksim N(X\beta,\sigma^2I)\),and \(\beta\thicksim N(0,\tau I)\),
then using the Beyes formula we can get \(N(X\beta,\sigma^2I)*N(0,\tau I)\),
Simplifying the above formula, \(\beta|x\thicksim C*e^{-\frac{\|Y-X\beta\|_2+\frac{\sigma^2}{\tau}\|\beta\|_2}{2\sigma^2}}\),maximizing the posterior,then you can get the ridge estimator.
Similarly you can get lasso estimator by assuming the \(\beta\) prior as double exponential distribution.