Adaptive#

pylit.optimizer.adaptive.adaptiveRF(R, F, x0, steps, optim_RFx0, residuum_mode=False)#

This is a wrapper for optimization methods.

Description#

Solves the optimization problem (1) using an adaptive incremental wrapper around a given optimization method. This method partitions the parameter space into blocks of size steps and applies the provided optimizer optim_RFx0 sequentially to increasingly larger subsets of features. After each block, the solution is updated based on either the residuum or the objective error (epsilon). This ensures that the optimization focuses on the most relevant features first and improves convergence second.

Algorithm#

Let \(n\) be the number of columns of \(R\) and steps the block size. The algorithm proceeds as follows:

A. Initialization

  • Verify that steps divides the number of columns of \(R\).

  • Initialize empty feature set and solution placeholders.

B. Iterative Block Optimization
  • For each block of steps features:

    • Append the new block to the current feature set.

    • Run the optimizer optim_RFx0 on the reduced regression problem.

    • Evaluate the solution using either the residuum or epsilon.

    • If the new solution improves the selected metric, update the current solution and feature set.

C. Final Assembly

  • Construct the final solution vector by inserting optimized values at their corresponding indices.

The procedure terminates when all feature blocks have been processed. The final solution corresponds to the block configuration yielding the minimal residuum or epsilon, depending on residuum_mode.

type R:

ndarray

param R:

Regression matrix of shape (m, n).

type F:

ndarray

param F:

Target vector of shape (m,).

type x0:

ndarray

param x0:

Initial guess for the solution, shape (n,).

type steps:

int64

param steps:

Block size for adaptive optimization. Must be positive and divide n.

type optim_RFx0:

callable

param optim_RFx0:

Optimization function with signature optim_RFx0(R, F, x0) returning a Solution object.

type residuum_mode:

bool

param residuum_mode:

If True, the selection of the best solution is based on residuum. Otherwise, the epsilon value is used. Default is False.

rtype:

Solution

returns:

A Solution object containing the final iterate, epsilon, and residuum.

Notes

  • This adaptive wrapper is particularly useful for high-dimensional regression problems where a full optimization may be inefficient.

  • The truncation of x0 as initial guess for blocks may not be optimal.