Variance Regularization#
- pylit.methods.var_reg.var_reg(omegas, E, lambd)#
This is the variance regularization method. The interface is described in Methods.
The objective function
\[\]is implemented as
\[f(\boldsymbol{\alpha}) = \frac{1}{2} \| \boldsymbol{R} \boldsymbol{\alpha} - \boldsymbol{F} \|^2_2 + \frac{n}{2} \lambda \, \mathrm{Var}_{\boldsymbol{p}}[\boldsymbol{\omega}], \quad \boldsymbol{p} = E \boldsymbol{\alpha},\]with the gradient
\[\nabla_{\boldsymbol{\alpha}} f(\boldsymbol{\alpha}) = \boldsymbol{R}^\top (\boldsymbol{R} \boldsymbol{\alpha} - \boldsymbol{F}) + \lambda \, (\mathbb{E}_{\boldsymbol{p}}[\boldsymbol{\omega}] - \bar{\omega}) \, E^\top \boldsymbol{\omega},\]the learning rate
\[\eta = \frac{1}{\| \boldsymbol{R}^\top \boldsymbol{R} \| + (\lambda / k^2) \, \| \boldsymbol{E} \boldsymbol{\omega} \| \|\boldsymbol{E}\| \|\boldsymbol{\omega}\|}\]and the solution
\[\textit{Solution not available.}\]where
\(\boldsymbol{R}\): Regression matrix,
\(\boldsymbol{F}\): Target vector,
\(\boldsymbol{\omega}\): discrete frequency vector,
\(k\): length of the discrete frequency vector.
\(\bar{\omega}\): Mean of the discrete frequency vector \(\boldsymbol{\omega}\),
\(E\): Evaluation matrix,
\(\boldsymbol{\alpha}\): Coefficient vector,
\(\lambda\): Regularization parameter,
\(n\): Number of samples.
- Return type: