High Dimensional Ensemble Kalman Filter

The ensemble Kalman Filter (EnKF), as a fundamental data assimilation approach, has been widely used in many fields of earth science, engineering and beyond. However, several theoretical aspects of the EnKF remain unknown, especially when the state variable is of high dimensional accompanied with high resolution observation and physical models. This paper first proposes several high dimensional EnKF (HD-EnKF) methods that provide consistent estimators for the important forecast error covariance and the Kalman gain matrix. It then studies the theoretical properties of the EnKF under both the fixed and high dimensional state variables, which provides the mean square errors of the analysis states to the underlying oracle states offered by the Kalman filter and gives the much needed insight into the roles played by the forecast error covariance on the accuracy of the EnKF. The accuracy of the data assimilation under the misspecified physical model is also considered. Numerical studies on the Lorenz-96 and the Shallow Water Equation models illustrate that the proposed HD-EnKF algorithms perform better than the standard EnKF methods as they provide more robust and accurate assimilated results. The HD-EnKF was applied to assimilate sea temperature in the Northwest Pacific, which showed more accurate out-sample performances than the existing methods.