Custom LightGBM objective: MSE with a squared covariance penalty

Build a custom LightGBM objective callback that minimizes a standard squared-error loss plus a soft penalty on the covariance between the per-sample residual r = y_pred - y_true and the (centered) labels y_true. The penalty pushes the model toward "vertical equity" by discouraging residuals that systematically scale with y.

This is an R port of the LGBCovPenalty objective from an active collaboration (see the soft-vertical-equity-constrained-mass-appraissal repository). It is intended to be used when the model is trained in log-space (so the "diff" residual is equivalent to a log-ratio).

Penalty (using mean-centered labels y_centered = y_true - mean(y_true)): $$\mathrm{cov} = \frac{1}{n} \sum_{i=1}^{n} r_i \tilde{y}_i$$ $$\mathrm{penalty} = \frac{\rho}{2} \, n \, \mathrm{cov}^2$$ where \(\tilde{y}\) is the vector of mean-centered labels. A diagonal Hessian approximation is used (matches the reference Python implementation).

Usage

make_objective_mse_cov(rho, y_mean)

Arguments

rho

Numeric. Non-negative penalty weight. rho = 0 recovers plain MSE.

y_mean

Numeric. Mean of the training labels. Should be computed once from the training set and captured here so the centering is stable across iterations.

Value

A function with signature function(preds, dtrain) suitable for passing as the obj argument of lightgbm::lgb.train.