toolsrtm.inversion

Trait-inversion tools: CARS-PLS and VIF predictor selection, LUT nearest- neighbour (“merit function”) matching, and a 12-algorithm ML dispatcher (PLSR/SVM/RF/GB/NN/Bayesian/AdaBag/BRNN/xGB/RVM/qLASSO/Ensemble) built on scikit-learn/xgboost. Direct port of ToolsRTM::carspls/get.cars.pls, getVIF, get.inversionOpt, get.inversion, hybrid_inversion/ hybrid_inversionE.

Note

Needs the optional ml extra: pip install "toolsrtm[ml]". Nothing in this module is imported by toolsrtm/__init__.py’s own import chain – a plain import toolsrtm never requires scikit-learn/xgboost.

Note

R’s get.inversion/hybrid_inversion dispatch to specific caret methods (bartMachine, rqlasso, rvmLinear, AdaBag, brnn, …). See ALGORITHMS for exactly which scikit-learn/xgboost estimator each algorithm name maps to, and, where there’s no direct equivalent, what was substituted and why.

Trait-inversion tools: CARS-PLS feature selection, VIF-based collinearity pruning, LUT nearest-neighbour (“merit function”) inversion, and a multi-algorithm ML dispatcher.

Python port of ToolsRTM/R/carspls.R / get.cars.pls.R, getVIF.R, get.inversion.R, hybrid_inversion.R / hybrid_inversionE.R, and get.inversionOpt.R. Needs the optional ml extra (pip install toolsrtm[ml]: scikit-learn, xgboost) – none of these functions are imported by toolsrtm/__init__.py at import time, and each imports its own ML dependencies lazily so a plain import toolsrtm never requires scikit-learn/xgboost to be installed.

R’s get.inversion/hybrid_inversion dispatch by name to specific caret methods (bartMachine, rqlasso, rvmLinear, AdaBag, brnn, …). Several of those have no direct scikit-learn/xgboost equivalent; where that’s the case the docstring of the relevant function says exactly which estimator was substituted and why. Unlike the pure radiative-transfer math ported elsewhere in this package, none of this module is verified to floating-point precision against R – caret’s own cross-validated tuning is stochastic, so a Python port using different (but comparable) estimators and search grids will not reproduce R’s numbers bit-for-bit even in principle. What’s verified instead: each algorithm runs end-to-end on held-out data and produces sane, comparable-magnitude accuracy metrics (see tests/test_inversion.py).

class toolsrtm.inversion.CarsPlsResult(coef, n_var, rmsecv, num_lv, optimal_iteration, min_error, selected_variables)[source]

Bases: object

Result of carspls(). Mirrors the R CARS list 1:1 except selected_variables is 0-indexed (Python) instead of 1-indexed (R).

Parameters:
coef: ndarray

(n_vars, iteration) coefficient path

n_var: ndarray

(iteration,) number of retained variables per iteration

rmsecv: ndarray

(iteration,) cross-validated RMSE per iteration

num_lv: ndarray

(iteration,) best number of latent variables per iteration

optimal_iteration: int

1-indexed iteration with the lowest RMSECV (matches R)

min_error: float
selected_variables: ndarray

0-indexed column positions into the original X

toolsrtm.inversion.carspls(X, y, n_lv=2, fold=10, scale_pretreat=True, iteration=50, partition_type='interleaved', verbose=False)[source]

Competitive Adaptive Reweighted Sampling for PLS variable selection.

Python port of carspls/get.cars.pls (R, original algorithm by Yizeng Liang & Hongdong Li, MATLAB->R port by Hongdong Li 2009). At each of iteration rounds: fits a PLS model on the currently-retained variables, cross-validates it to get an RMSECV curve over 1..n_lv components, records the coefficient-magnitude-ranked variable importance, and forcibly eliminates the lowest-ranked variables via an exponentially decreasing retention schedule (Monte-Carlo/EDF sampling). The iteration with the lowest RMSECV gives the final selected variable set.

Parameters:

  • X (ndarray) – (n_samples, n_vars) predictor matrix.

  • y (ndarray) – (n_samples,) response vector.

  • n_lv (int) – number of PLS latent variables (components) to fit/tune over.

  • fold (int) – number of cross-validation segments.

  • scale_pretreat (bool) – if True, scale (not just center) each predictor.

  • iteration (int) – number of CARS-PLS elimination rounds.

  • partition_type (Literal['interleaved', 'consecutive', 'random']) – cross-validation fold assignment: "interleaved" (round-robin), "consecutive" (contiguous blocks), or "random".

  • verbose (bool) – print progress per iteration (matches R’s own screen output).

Returns:

CarsPlsResult.

Return type:

CarsPlsResult

toolsrtm.inversion.get_vif(frame, columns=None, thresh=10.0, trace=True)[source]

Backward-elimination variable selection by Variance Inflation Factor.

Python port of getVIF (R, VIF function originally from https://beckmw.wordpress.com/2013/02/05/collinearity-and-stepwise-vif-selection/). Iteratively regresses each remaining variable on all others; drops the variable with the highest VIF (1 / (1 - R^2)) as long as any VIF exceeds thresh.

Parameters:

  • frame (ndarray) – (n_samples, n_vars) array, or a pandas.DataFrame.

  • columns (Sequence[str] | None) – variable names, required if frame is a bare array; ignored (and taken from frame.columns) if frame is a DataFrame.

  • thresh (float) – VIF threshold above which a variable is flagged as collinear.

  • trace (bool) – print each elimination step (matches R’s own console output).

Returns:

names (or 0-indexed positions, if columns is None and frame is a bare array) of the retained variables.

Return type:

list[str] | list[int]

class toolsrtm.inversion.InversionOptResult(rfl_best: 'np.ndarray', lut_best: "'pd.DataFrame'")[source]

Bases: object

Parameters:

  • rfl_best (np.ndarray)

  • lut_best (pd.DataFrame)

rfl_best: np.ndarray

(n_obs, n_wave) best-matching (n_opt-averaged) simulated spectra

lut_best: pd.DataFrame

(n_obs, n_lut_columns) n_opt-averaged LUT parameters per observation

toolsrtm.inversion.get_inversion_opt(rfl_sensor, rfl_rtm, lut, wave=None, method='merit-RMSE', n_opt=1, custom_stat=None)[source]

LUT (look-up table) inversion by nearest-neighbour spectral matching.

Python port of get.inversionOpt (R). For each observed spectrum in rfl_sensor, ranks every simulated spectrum in rfl_rtm by a merit (error) function and averages the n_opt best matches’ LUT parameters and reflectance. Fully vectorized (broadcasts each observation against the whole LUT at once) rather than R’s nested per-row loop – same algorithm, no numerical differences expected for the built-in merit functions (verified against a hand-computed reference below).

Parameters:

  • rfl_sensor (ndarray) – (n_obs, n_wave) observed/sensor reflectance.

  • rfl_rtm (ndarray) – (n_lut, n_wave) simulated reflectance from the LUT.

  • lut – (n_lut, n_params) pandas.DataFrame of the LUT’s input parameters.

  • wave (Sequence[float] | None) – wavelengths corresponding to columns of rfl_sensor/rfl_rtm (only used to name the returned reflectance columns; optional).

  • method (str) – one of "merit-RMSE", "merit-NRMSE", "merit-MAE", "merit-NMB", "merit-FGE", or "merit-custom.metric" (requires custom_stat).

  • n_opt (int) – number of best-matching LUT rows to average per observation.

  • custom_stat (Callable[[ndarray, ndarray], ndarray] | None) – optional f(sim, obs) -> error merit function, broadcast over the last axis exactly like the built-in ones; overrides method.

Returns:

InversionOptResult.

Return type:

InversionOptResult

toolsrtm.inversion.ALGORITHMS = {'AdaBag': "sklearn AdaBoostRegressor over shallow DecisionTreeRegressor stumps (matches caret method 'AdaBag')", 'BRNN': "sklearn MLPRegressor with strong L2 (alpha) regularization -- substitute: approximates 'Bayesian regularization' via explicit weight decay, not R's brnn Gauss-Newton/Levenberg-Marquardt fit", 'Bayesian': "sklearn BayesianRidge -- substitute: no BART implementation in sklearn/xgboost; BayesianRidge is a Bayesian *linear* model, not R's bartMachine (Bayesian additive trees)", 'Ensemble': "sklearn StackingRegressor(GB + SVR + MLP, final_estimator=LinearRegression) (matches caretEnsemble::caretStack(..., method='glm'))", 'GB': "sklearn GradientBoostingRegressor (matches caret method 'gbm')", 'NN': "sklearn MLPRegressor, single hidden layer (matches caret method 'nnet')", 'PLSR': "sklearn PLSRegression, n_components tuned by 5-fold CV (matches caret method 'pls')", 'RF': "sklearn RandomForestRegressor, max_features tuned (matches caret method 'rf')", 'RVM': "sklearn BayesianRidge -- substitute: no Relevance Vector Machine in sklearn/xgboost; BayesianRidge shares RVM's sparsity-favouring linear-Bayesian character", 'SVM': "sklearn SVR(kernel='rbf'), gamma/C tuned by grid search (matches caret method 'svmRadial' via e1071::tune.svm)", 'qLASSO': "sklearn QuantileRegressor(quantile=0.5, solver='highs'), L1-penalized (matches caret method 'rqlasso')", 'xGB': "xgboost XGBRegressor(booster='gblinear') (matches caret method 'xgbLinear')"}

R’s caret-method name each algorithm dispatches to, and, where scikit-learn/ xgboost has no direct equivalent, the substitution actually used here.

class toolsrtm.inversion.InversionResult(model_label: 'str', model: 'object', predictions: 'dict', statistics: 'dict', importance: 'dict | None')[source]

Bases: object

Parameters:

  • model_label (str)

  • model (object)

  • predictions (dict)

  • statistics (dict)

  • importance (dict | None)

model_label: str
model: object
predictions: dict

np.ndarray, “test”: np.ndarray}

Type:

{“train”

statistics: dict

{“r2”:.., “rmse”:.., “mae”:..}, “test”: {…}}

Type:

{“train”

importance: dict | None

importance}, or None if not available for this algorithm

Type:

{input_name

toolsrtm.inversion.get_inversion(data, dep_var, inputs, algorithm='PLSR', seed=123, n_samples=500, test_size=0.3)[source]

Fit and evaluate a plant-trait inversion model with one of 12 algorithms, on a held-out train/test split.

Python port of get.inversion (R). See ALGORITHMS for exactly which scikit-learn/xgboost estimator each algorithm name dispatches to, and, for the 4 algorithms with no direct equivalent (Bayesian, BRNN, RVM – and note AdaBag/xGB/qLASSO do have close matches), what was substituted and why. Unlike R’s version, tuning here is a single small grid search per algorithm (not caret’s full repeated-CV search), to keep this runnable as a demo rather than a multi-hour job – the same design choice already used by Scripts/Python/*/2_inversion_ml.py.

Parameters:

  • data – pandas.DataFrame containing dep_var and all inputs columns.

  • dep_var (str) – name of the response column to predict.

  • inputs (Sequence[str]) – names of the predictor columns.

  • algorithm (str) – one of the keys of ALGORITHMS.

  • seed (int) – random seed for the train/test split and any stochastic estimator.

  • n_samples (int | None) – if given and less than len(data), randomly subsample this many rows before splitting (matches R’s own tuning-sample-size argument).

  • test_size (float) – fraction of (sub-sampled) data held out for testing.

Returns:

InversionResult.

Return type:

InversionResult

class toolsrtm.inversion.HybridInversionResult(model: 'object', keep_variables: 'list[str]', statistics: "'pd.DataFrame'", predictions: 'dict')[source]

Bases: object

Parameters:

  • model (object)

  • keep_variables (list[str])

  • statistics (pd.DataFrame)

  • predictions (dict)

model: object
keep_variables: list[str]

predictor columns actually used, after pattern/collinearity selection

statistics: pd.DataFrame

rows “train”/”test” (+ “field” if field_data given), columns r2/rmse/mae

predictions: dict

np.ndarray, “test”: np.ndarray[, “field”: np.ndarray]}, original (untransformed) scale

Type:

{“train”

toolsrtm.inversion.hybrid_inversion(lut, input, split=0.8, seed=None, method=None, collinearity=None, pattern=None, trans=True, field_data=None, acron=None)[source]

Fit a single-algorithm trait-inversion model with optional predictor selection (by name pattern, then optionally VIF or CARS-PLS pruning) and an optional log-transform of the response.

Python port of hybrid_inversion (R). method dispatches to the same 5 estimators as get_inversion()’s SVM/RF/GB/NN/ Ensemble ("nnet" here maps to "NN" there, matching R’s own caret method name) – see ALGORITHMS for what each one is. Note: R’s train/test split uses caret::createDataPartition (percentile- stratified on the response); this port uses a plain random split via scikit-learn, which is not percentile-stratified – a documented approximation, not expected to change results materially for the LUT-sized (typically hundreds of rows) datasets this is meant for.

Parameters:

  • lut – pandas.DataFrame with the response column input and candidate predictor columns.

  • input (str) – name of the response column to predict.

  • split (float) – train-fraction of the train/test split (R’s own convention; note this is the train fraction, unlike get_inversion()’s test_size).

  • seed (int | None) – random seed.

  • method (str | None) – one of "SVM", "RF", "GB", "nnet", "Ensemble". Defaults to "SVM" (matches R’s own default).

  • collinearity (Literal['VIF', 'CARS'] | None) – None (use every pattern-matched column), "VIF" (prune via get_vif()), or "CARS" (select via carspls()).

  • pattern (str | None) – substring that predictor column names must contain (e.g. "B" for reflectance-band columns named B1, B2, …); if None, every column except input is a candidate.

  • trans (bool) – log-transform input before fitting (matches R’s own default); predictions/statistics are reported back on the original scale.

  • field_data – optional pandas.DataFrame of field observations to validate against, in addition to the LUT’s own test split.

  • acron (str | None) – suffix appended to input to find the observed column in field_data (e.g. acron="_obsv" looks for f"{input}_obsv"). Required if field_data is given.

Returns:

HybridInversionResult.

Return type:

HybridInversionResult

toolsrtm.inversion.hybrid_inversion_ensemble(lut, input, split=0.8, seed=None, collinearity=None, pattern=None, field_data=None, acron=None)[source]

hybrid_inversion() with method="Ensemble" fixed.

Python port of hybrid_inversionE (R) – the R function is hybrid_inversion with the model choice hardcoded to the 3-model (GB + SVM + neural net) stacking ensemble and no trans/log-transform option, which this wrapper matches (trans=False).

Parameters:

  • input (str)

  • split (float)

  • seed (int | None)

  • collinearity (Literal['VIF', 'CARS'] | None)

  • pattern (str | None)

  • acron (str | None)

Return type:

HybridInversionResult