questions about AutoSmoothingResult

[Bhare8972]

To whom it may concern,

Sorry I'm using a ticket to ask a question, but I couldn't see any other way to contact y'all.

This package looks VERY interesting to me. I've been looking for a python implementation of DeBoor smoothing splines, and was afraid I'd have to implement it myself. However, I have a few questions.

First, how does AutoSmoothingResult work? What criterion does it use to find a smoothing parameter? (may I suggest the answer be put in the documentation?)

Second, scipy univariate smoothing splines have an option where essentially you set the desired chi-squared fit and the algorithm automatically adjusts the number of knots until that condition is met. Does csaps have a similar option? (except instead of adjusting knots, the smoothing paramter is adjusted ofc), that is, something like the second equation under "de Boors approach" on the page https://en.wikipedia.org/wiki/Smoothing_spline.

Thanks!

[espdev]

Hi,

The smoothing parameter is computed just like in MATLAB implementation. See: https://csaps.readthedocs.io/en/latest/formulation.html#definition

The calculation of the smoothing spline requires the solution of a linear system whose coefficient matrix has the form , with the matrices and depending on the data sites . The automatically computed smoothing parameter makes p*trace(A) equal (1 - p)*trace(B).

csaps/csaps/_sspumv.py

Lines 233 to 244 in b7f7cdc

	def _compute_smooth(a, b):
	"""
	The calculation of the smoothing spline requires the solution of a
	linear system whose coefficient matrix has the form p*A + (1-p)*B, with
	the matrices A and B depending on the data sites x. The default value
	of p makes p*trace(A) equal (1 - p)*trace(B).
	"""
	def trace(m: sp.dia_matrix):
	return m.diagonal().sum()
	return 1. / (1. + trace(a) / (6. * trace(b)))

"de Boors approach" and iterative adjusting knots algorithm are not used.

Also we can use the smoothing parameter normalization for adjusting to X scale:

csaps/csaps/_sspumv.py

Lines 247 to 261 in b7f7cdc

	def _normalize_smooth(x: np.ndarray, w: np.ndarray, smooth: Optional[float]):
	"""
	See the explanation here: https://github.com/espdev/csaps/pull/47
	"""
	span = np.ptp(x)
	eff_x = 1 + (span ** 2) / np.sum(np.diff(x) ** 2)
	eff_w = np.sum(w) ** 2 / np.sum(w ** 2)
	k = 80 * (span ** 3) * (x.size ** -2) * (eff_x ** -0.5) * (eff_w ** -0.5)
	s = 0.5 if smooth is None else smooth
	p = s / (s + (1 - s) * k)
	return p

If you want, you can suggest PR with an another implementation of auto-smoothing algorithm, and we will add it to the package as an option.

[Bhare8972]

Thank you for the reply.

Is there any material on how to get some intuitive understanding of what autosmooth is doing? Perhaps a research paper citation?

Someday I'll try to implement the smoothing option, and make a PR once I do. But it will be awhile.

[espdev]

Only the following description from Matlab documentation without any links and citations, unfortunately.

Matlab implementation also has roughness weights through smoothing parameter vector. Currently, this is not implemented in csaps Python package.

[Bhare8972]

Thank you for your help.

Am thinking about how to implement this feature I want. It requires explicitly calculating the error measure.

Is there an easy way, inside the method _make_spline of the CubicSmoothingSpline class (at

csaps/csaps/_sspumv.py

Line 264 in b7f7cdc

def _make_spline(x, y, w, smooth, shape, normalizedsmooth):

)

to calculate the error measure? (I can't tell as I don't know what all the variables in _make_spline measure) Or does the spline need to be evaluated at x-locations, after creation, to calculate the error measure?