Covariance estimation via fiducial inference

ISSN 2475-4269

CN 31-2182/O1

Jan Hannig ,

Department of Statistics & Operations Research, University of North Carolina, Chapel Hill, NC, USA

Randy C. S. Lai ,

Department of Statistics, University of California, Davis, CA, USA

Thomas C. M. Lee

Department of Statistics, University of California, Davis, CA, USA

Pages 316-331 | Received 23 May. 2020, Accepted 10 Jan. 2021, Published online: 15 Nov. 2021,

Abstract
Full Article
References
Citations

As a classical problem, covariance estimation has drawn much attention from the statistical community for decades. Much work has been done under the frequentist and Bayesian frameworks. Aiming to quantify the uncertainty of the estimators without having to choose a prior, we have developed a fiducial approach to the estimation of covariance matrix. Built upon the Fiducial Berstein–von Mises Theorem, we show that the fiducial distribution of the covariate matrix is consistent under our framework. Consequently, the samples generated from this fiducial distribution are good estimators to the true covariance matrix, which enable us to define a meaningful confidence region for the covariance matrix. Lastly, we also show that the fiducial approach can be a powerful tool for identifying clique structures in covariance matrices.

References

Abramowitz, M., & Stegun, I. A. (1964). Handbook of mathematical functions: with formula, graphs and mathematical tables. Courier Corporation. [Google Scholar]
Avella-Medina, M., Battey, H. S., Fan, J., & Li, Q. (2018). Robust estimation of high-dimensional covariance and precision matrices. Biometrika, 105, 271–284. https://doi.org/10.1093/biomet/asy011 [Crossref], [Web of Science ®], [Google Scholar]
Bickel, P. J., & Levina, E. (2008a). Covariance regularization by thresholding. The Annals of Statistics, 36, 2577–2604. https://doi.org/10.1214/08-AOS600 [Crossref], [Web of Science ®], [Google Scholar]
Bickel, P. J., & Levina, E. (2008b). Regularized estimation of large covariance matrices. The Annals of Statistics, 36, 199–227. https://doi.org/10.1214/009053607000000758 [Crossref], [Web of Science ®], [Google Scholar]
Brooks, S. P., Giudici, P., & Roberts, G. O. (2003). Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions. Journal of the Royal Statistical Society, 65(1), 3–39. https://doi.org/10.1111/rssb.2003.65.issue-1 [Crossref], [Google Scholar]
Cai, T., & Liu, W. (2011). Adaptive thresholding for sparse covariance matrix estimation. Journal of the American Statistical Association, 106, 672–684. https://doi.org/10.1198/jasa.2011.tm10560 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
Cisewski, J., & Hannig, J. (2012). Generalized fiducial inference for normal linear mixed models. The Annals of Statistics, 40(4), 2102–2127. https://doi.org/10.1214/12-AOS1030 [Crossref], [Web of Science ®], [Google Scholar]
Cui, Y., & Hannig, J. (2019). Nonparametric generalized fiducial inference for survival functions under censoring, with discussion and rejoinder by the author. Biometrika, 106, 501–518. https://doi.org/10.1093/biomet/asz016 [Crossref], [Web of Science ®], [Google Scholar]
Fischer, E. (1908). Uber den hadamardschen determinantensatz. Archiv D Math U Phys, 13, 32–40. [Google Scholar]
Fisher, R. A. (1922). On the mathematical foundations of theoretical statistics. Philosophical Transactions of the Royal Society of London. Series A, 222, 309–368. https://doi.org/10.1098/rsta.1922.0009 [Crossref], [Google Scholar]
Fisher, R. A. (1930). Inverse probability. Proceedings of the Cambridge Philosophical Society, 26, 528–535. https://doi.org/10.1017/S0305004100016297 [Crossref], [Google Scholar]
Fisher, R. A. (1933). The concepts of inverse probability and fiducial probability referring to unknown parameters. Proceedings of the Royal Society of London Series A, 139, 343–348. [Crossref], [Google Scholar]
Fisher, R. A. (1935). The fiducial argument in statistical inference. The Annals of Eugenics, 6, 91–98. [Crossref], [Google Scholar]
Förstner, W., & Moonen, B. (1999). A metric for covariance matrices. In Quo vadis geodesia? Festschrift for Erik W. Grafarend on the occasion of his 60th birthday, Schriftenreihe der Institute des Studiengangs Geodäsie und Geoinformatik (pp. 113–128). IAGB, Stuttgart. [Google Scholar]
Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics (Oxford, England), 9(3), 432–441. https://doi.org/10.1093/biostatistics/kxm045 [Crossref], [Web of Science ®], [Google Scholar]
Friedman, J., Hastie, T., & Tibshirani, R. (2010). Applications of lasso and grouped lass to estimation of sparse graphical models. Technical Report, Standford University. [Google Scholar]
Furrer, R., & Bengtsson, T. (2007). Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. Journal of Multivariate Analysis, 98, 227–255. https://doi.org/10.1016/j.jmva.2006.08.003 [Crossref], [Web of Science ®], [Google Scholar]
Ghosh, J. K., & Ramamoorthi, R. V. (2003). Bayesian nonparametrics. Springer Series in Statistiscs. Springer-Verlag. [Google Scholar]
Glagovskiy, Y. S. (2006). Construction of fiducial confidence intervals for the mixture of cauchy and normal distributions(Master's thesis). Colorado State University. [Google Scholar]
Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82(4), 711–732. https://doi.org/10.1093/biomet/82.4.711 [Crossref], [Web of Science ®], [Google Scholar]
Hannig, J., Feng, Q., Iyer, H., Wang, C., & Liu, X. (2018). Fusion learning for inter-laboratory comparisons. Journal of Statistical Planning and Inference, 195, 64–79. https://doi.org/10.1016/j.jspi.2017.09.011 [Crossref], [Web of Science ®], [Google Scholar]
Hannig, J., Iyer, H. K., Lai, R. C. S., & Lee, T. C. M. (2016). Generalized fiducial inference: a review and new results. Journal of the American Statistical Association, 111, 1346–1361. https://doi.org/10.1080/01621459.2016.1165102 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
Hannig, J., Iyer, H. K., & Wang, J. C. M. (2007). Fiducial approach to uncertainty assessment: account for error due to instrument resolution. Metrologia, 44, 476–483. https://doi.org/10.1088/0026-1394/44/6/006 [Crossref], [Web of Science ®], [Google Scholar]
Hannig, J., & Lee, T. C. M. (2009). Generalized fiducial inference for wavelet regression. Biometrika, 96, 847–860. https://doi.org/10.1093/biomet/asp050 [Crossref], [Web of Science ®], [Google Scholar]
Hannig, J., Lidong, E., Abdel-Karim, A., & Iyer, H. K. (2006). Simultaneous fiducial generalized confidence intervals for ratios of means of lognormal distributions. Austrian Journal of Statistics, 35, 261–269. https://doi.org/10.17713/ajs.v35i2&3.372 [Google Scholar]
Hannig, J., Wang, J. C. M., & Iyer, H. K. (2003). Uncertainty calculation for the ratio of dependent measurements. Metrologia, 4, 177–186. https://doi.org/10.1088/0026-1394/40/4/306 [Crossref], [Web of Science ®], [Google Scholar]
Huang, H.-C., & Lee, T. C. M. (2016). High-dimensional covariance estimation under the presence of outliers. Statistics and Its Interface, 9, 461–468. https://doi.org/10.4310/SII.2016.v9.n4.a6 [Crossref], [Web of Science ®], [Google Scholar]
Huang, J. Z., Liu, N., Pourahmadi, M., & Liu, L. (2006). Covariance matrix selection and estimation via penalised normal likelihood. Biometrika, 93, 85–98. https://doi.org/10.1093/biomet/93.1.85 [Crossref], [Web of Science ®], [Google Scholar]
Ipsen, I. C., & Lee, D. J. (2011). Determinant approximations. arXiv preprint arXiv:1105.0437. [Google Scholar]
Iyer, H. K., Wang, C. M. J., & Mathew, T. (2004). Models and confidence intervals for true values in interlaboratory trials. Journal of the American Statistical Association, 99, 1060–1071. https://doi.org/10.1198/016214504000001682 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
Jameson, G. (2013). Inequalities for gamma function ratios. The American Mathematical Monthly, 120(10), 936–940. https://doi.org/10.4169/amer.math.monthly.120.10.936 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
Lai, R. C. S., Hannig, J., & Lee, T. C. M. (2015). Generalized fiducial inference for ultrahigh dimensional regression. Journal of American Statistical Association, 110, 760–772. https://doi.org/10.1080/01621459.2014.931237 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
Lam, C., & Fan, J. (2009). Sparsistency and rates of convergence in large covariance matrix estimation. The Annals of Statistics, 37, 42–54. https://doi.org/10.1214/09-AOS720 [Crossref], [Web of Science ®], [Google Scholar]
Levina, E., Rothman, A. J., & Zhu, J. (2008). Sparse estimation of large covariance matrices via a nested lasso penalty. The Annals of Applied Statistics, 2, 245–263. https://doi.org/10.1214/07-AOAS139 [Crossref], [Web of Science ®], [Google Scholar]
Li, X., Su, H., & Liang, H. (2018). Fiducial generalized p-values for testing zero-variance components in linear mixed-effects models. Science China Mathematics, 61(7), 1303–1318. https://doi.org/10.1007/s11425-016-9068-8 [Crossref], [Web of Science ®], [Google Scholar]
Lidong, E., Hannig, J., & Iyer, H. K. (2008). Fiducial intervals for variance components in an unbalanced two-component normal mixed linear model. Journal of the American Statistical Association, 103, 854–865. https://doi.org/10.1198/016214508000000229 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
Liu, Y., & Hannig, J. (2016). Generalized fiducial inference for binary logistic item response models. Psychometrica, 81, 290–324. https://doi.org/10.1007/s11336-015-9492-7 [Crossref], [Web of Science ®], [Google Scholar]
Liu, Y., & Hannig, J. (2017). Generalized fiducial inference for logistic graded response models. Psychometrica, 82, 1097–1125. https://doi.org/10.1007/s11336-017-9554-0 [Crossref], [Web of Science ®], [Google Scholar]
Martin, R., & Liu, C. (2015). Inferential models: reasoning with uncertainty. CRC Press. [Crossref], [Google Scholar]
Pourahmadi, M. (2011). Covariance estimation: the GLM and regularization perspectives. Statistical Science, 26(3), 369–387. https://doi.org/10.1214/11-STS358 [Crossref], [Web of Science ®], [Google Scholar]
Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14(5), 465–658. https://doi.org/10.1016/0005-1098(78)90005-5 [Crossref], [Web of Science ®], [Google Scholar]
Rothman, A. (2012). Positive definite estimators of large covariance matrices. Biometrika, 99(3), 733–740. https://doi.org/10.1093/biomet/ass025 [Crossref], [Web of Science ®], [Google Scholar]
Rothman, A. J., Levina, E., & Zhu, J. (2009). Generalized thresholding of large covariance matrices. Journal of the American Statistical Association, 104, 177–186. https://doi.org/10.1198/jasa.2009.0101 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
Rothman, A. J., Levina, E., & Zhu, J. (2010). A new approach to Cholesky-based covariance regularization in high dimensions. Biometrika, 97, 539–550. https://doi.org/10.1093/biomet/asq022 [Crossref], [Web of Science ®], [Google Scholar]
Schweder, T., & Hjort, N. L. (2016). Confidence, likelihood, probability. Cambridge University Press. [Crossref], [Google Scholar]
Shi, W. J. (2015). Bayesian modeling for viral sequencing and covariance estimation via fiducial inference (PhD thesis). University of North Carolina at Chapel Hill. [Google Scholar]
Sonderegger, D., & Hannig, J. (2012). Bernstein–von Mises theorem for generalized fiducial distributions with application to free knot splines. Preprint. [Google Scholar]
van der Vaart, A. W. (1998). Asymptotic statistics, volume 3 of Cambridge series in Statistical and Probabilistic Mathematics. Cambridge University Press. [Google Scholar]
Wandler, D. V., & Hannig, J. (2011). Fiducial inference on maximum mean of a multivariate normal distribution. Journal of Multivariate Analysis, 102, 87–104. https://doi.org/10.1016/j.jmva.2010.08.003 [Crossref], [Web of Science ®], [Google Scholar]
Wandler, D. V., & Hannig, J. (2012). A fiducial approach to multiple comparisons. Journal of Statistical Planning and Inference, 142, 878–895. https://doi.org/10.1016/j.jspi.2011.10.011 [Crossref], [Web of Science ®], [Google Scholar]
Wandler, D. V., & Hannig, J. (2012). Generalized fiducial confidence intervals for extremes. Extremes, 15, 67–87. https://doi.org/10.1007/s10687-011-0127-9 [Crossref], [Web of Science ®], [Google Scholar]
Wang, J. C. M., Hannig, J., & Iyer, H. K. (2012). Pivotal methods in the propagation of distributions. Metrologia, 49, 382–389. https://doi.org/10.1088/0026-1394/49/3/382 [Crossref], [Web of Science ®], [Google Scholar]
Wang, J. C. M., & Iyer, H. K. (2005). Propagation of uncertainties in measurements using generalized inference. Metrologia, 42, 145–153. https://doi.org/10.1088/0026-1394/42/2/010 [Crossref], [Web of Science ®], [Google Scholar]
Wang, J. C. M., & Iyer, H. K. (2006a). A generalized confidence interval for a measurand in the presence of type-a and type-b uncertainties. Measurement, 39, 856–863. https://doi.org/10.1016/j.measurement.2006.04.011 [Crossref], [Web of Science ®], [Google Scholar]
Wang, J. C. M., & Iyer, H. K. (2006b). Uncertainty of analysis of vector measurands using fiducial inference. Metrologia, 43, 486–494. https://doi.org/10.1088/0026-1394/43/6/002 [Crossref], [Web of Science ®], [Google Scholar]
Williams, J., & Hannig, J. (2018). Non-penalized variable selection in high-dimensional linear model settings via generalized fiducial inference. Annals of Statistics, 47(3), 1723–1753. https://doi.org/10.1214/18-AOS1733 [Crossref], [Web of Science ®], [Google Scholar]
Wu, W. B., & Pourahmadi, M. (2003). Nonparametric estimation of large covariance matrices of longitudinal data. Biometrika, 90, 831–844. https://doi.org/10.1093/biomet/90.4.831 [Crossref], [Web of Science ®], [Google Scholar]
Xie, M., & Singh, K. (2013). Confidence distribution, the frequentist distribution estimator of a parameter: a review. International Statistical Review, 81, 3–39. https://doi.org/10.1111/insr.2013.81.issue-1 [Crossref], [Web of Science ®], [Google Scholar]

To cite this article: W. Jenny Shi, Jan Hannig, Randy C. S. Lai & Thomas C. M. Lee (2021)
Covariance estimation via fiducial inference, Statistical Theory and Related Fields, 5:4, 316-331,
DOI: 10.1080/24754269.2021.1877950
To link to this article: https://doi.org/10.1080/24754269.2021.1877950

Archives

References

Authors

About the Journal

Links

Search

Archives