Stein-type Estimator Assisted by Dynamics

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Estimating the equilibrium parameters of environmental systems is a fundamental task, yet it is often hampered by sparse and noisy sensor data. While the standard Maximum Likelihood Estimator (MLE) is intuitive, Stein’s paradox famously shows that it is statistically inefficient in high dimensions. To address this, we introduce STEADY, an estimator that generalises Stein’s paradox by integrating physical knowledge. We derive our estimator from a principled empirical Bayes model where the prior distribution over the equilibria is a direct consequence of the stationary properties of the system’s governing differential equations. This leads to a novel adaptive shrinkage mechanism, where the amount of shrinkage applied to each observation is naturally modulated by the physical stability of the measured system. We provide a rigorous frequentist analysis of our estimator, proving that STEADY not only dominates the MLE but is also minimax under certain conditions, offering the strongest possible guarantee of robustness. We validate our claims on synthetic data and demonstrate STEADY’s utility on the global Argo ocean float dataset, showing that it effectively filters noise to reveal the “North Atlantic Warming Hole”
Original languageEnglish
Title of host publicationProceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1 (KDD ’26)
Place of PublicationNew York, US
PublisherACM
Number of pages12
Volume1
ISBN (Electronic)9798400722585
DOIs
Publication statusAccepted/In press - 23 Nov 2025
EventKDD 2026: 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining - International Convention Center Jeju (ICC Jeju), Jeju, Korea, Republic of
Duration: 9 Aug 202613 Aug 2026
https://kdd2026.kdd.org/

Conference

ConferenceKDD 2026
Country/TerritoryKorea, Republic of
CityJeju
Period9/08/2613/08/26
Internet address

Keywords

  • Parameter Estimation
  • Shrinkage Estimators
  • Stein’s Paradox
  • Dynamical Systems
  • Environmental Modelling

Fingerprint

Dive into the research topics of 'Stein-type Estimator Assisted by Dynamics'. Together they form a unique fingerprint.

Cite this