Abstract

With increasing appearances of high-dimensional data over the past two decades, variable selections through frequentist likelihood penalisation approaches and their Bayesian counterparts becomes a popular yet challenging research area in statistics. Under a normal linear model with shrinkage priors, we propose a benchmark variable approach for Bayesian variable selection. The benchmark variable serves as a standard and helps us to assess and rank the importance of each covariate based on the posterior distribution of the corresponding regression coefficient. For a sparse Bayesian analysis, we use the benchmark in conjunction with a modified BIC. We also develop our benchmark approach to accommodate models with covariates exhibiting group structures. Two simulation studies are carried out to assess and compare the performances among the proposed approach and other methods. Three real datasets are also analysed by using these methods for illustration.

References

1. Andrews, D. F., & Mallows, C. L. (1974). Scale mixtures of normal distributions. Journal of the Royal Statistical Society, Series B, 36, 99–102. [Google Scholar]
2. Bayarri, M. J., Berger, J. O., Jang, W., Ray, S., Pericchi, L. R., & Visser, L. (2019). Prior-based Bayesian information criterion (PBIC). Statistical Theory and Related Fields, 3(1), 2–13. https://doi.org/10.1080/24754269.2019.1582126 [Taylor & Francis Online], [Google Scholar]
3. Bradic, J., Fan, J., & Wang, W. (2009). Penalized composite quasi-Likelihood for ultrahigh-Dimensional variable selection. Journal of the Royal Statistical Society, Series B, 73(3), 325–349. https://doi.org/10.1111/rssb.2011.73.issue-3 [Crossref], [Google Scholar]
4. Brown, P. J., Vannucci, M., & Fearn, T. (1998). Multivariate Bayesian variable selection and prediction. Journal of the Royal Statistical Society, Series B, 60(3), 627–641. https://doi.org/10.1111/rssb.1998.60.issue-3 [Crossref], [Google Scholar]
5. Campbell, A., Converse, P., & Rodgers, W. (2008). The quality of American life: Perceptions, evaluations, and satisfactions. Russell Sage Foundation. [Google Scholar]
6. Chen, J., & Chen, Z. (2008). Extended Bayesian information criterion for model selection with large model spaces. Biometrika, 95, 759–771. https://doi.org/10.1093/biomet/asn034 [Crossref], [Web of Science ®], [Google Scholar]
7. Cohen, S., Gottlieb, B., & Underwood, L. (2000). Social relationships and health. In S. Cohen, L. Underwood, & B. Gottlieb (Eds.), Social support measurement and intervention. Oxford University Press. [Crossref], [Google Scholar]
8. Cohen, S., & Syme, L. (1985). Social support and health (Tech. Rep.). Academic. [Google Scholar]
9. Dellaportas, P., Forster, J. J., & Ntzoufras, I. (1997). On Bayesian model and variable selection using MCMC (Tech. Rep.). Department of Statistics, Athens University of Economics and Business. [Google Scholar]
10. Deutsch, S., Lyle, R., Dermitzakis, E., Attar, H., Subrahmanyan, L., Gehri, C., Parand, L., Gagnebin, M., Rougemont, J., Jongeneel, C., & Antonarakis, S. (2005). Gene expression variation and expression quantitative trait mapping of human chromosome 21 genes. Human Molecular Genetics, 14(23), 3741–3749. https://doi.org/10.1093/hmg/ddi404 [Crossref], [Web of Science ®], [Google Scholar]
11. Dicker, L., Huang, B., & Lin, X. (2011). Variable selection and estimation with the seamless-l0 penalty. Statistica Sinica, [Web of Science ®], [Google Scholar]
12. Fan, J., Han, X., & Gu, W. (2012). Estimating false discovery proportion under arbitrary covariance dependence. Journal of the American Statistical Association, 107(499), 1019–1035. https://doi.org/10.1080/01621459.2012.720478 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
13. Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360. https://doi.org/10.1198/016214501753382273 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
14. Frank-Stromborg, M., & Olsen, S. (2003). Instruments For Clinical Health-Care Research (Jones and Bartlett Series in Oncology, 3rd edition) (Tech. Rep.). Jones & Bartlett Learning. [Google Scholar]
15. Gall, T., Charbonneau, C., Clarke, N., Grant, K., Joseph, A., & Shouldice, L. (2005). Understanding the nature and role of spirituality in relation to coping and health: A conceptual framework. Canadian Psychology, 46(2), 88–104. https://doi.org/10.1037/h0087008 [Crossref], [Web of Science ®], [Google Scholar]
16. George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association, 85, 398–409. [Google Scholar]
17. Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika, 82, 711–732. https://doi.org/10.1093/biomet/82.4.711 [Crossref], [Web of Science ®], [Google Scholar]
18. Hoerl, A., & Kennard, R. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12(1), 55–67. https://doi.org/10.1080/00401706.1970.10488634 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
19. Hoti, F., & Sillanpää, M. J. (2006). Bayesian mapping of genotype x expression interactions in quantitative and qualitative traits. Heredity, 97(1), 4–18. https://doi.org/10.1038/sj.hdy.6800817 [Crossref], [Web of Science ®], [Google Scholar]
20. House, J., & Khan, R. (1985). Measures and concepts of social support. In S. Cohen & S. L. Syme (Eds.), Social support and health (Tech. Rep.). [Google Scholar]
21. Huang, J., Ma, S., Xie, H., & Zhang, C. (2009). A group bridge approach for variable selection. Biometrika, 96, 339–355. https://doi.org/10.1093/biomet/asp020 [Crossref], [Web of Science ®], [Google Scholar]
22. Knobf, M. (2007). Psychological responses in brest cancer survivors. Seminars in Oncology Nursing, 23(1), 71–83. https://doi.org/10.1016/j.soncn.2006.11.009 [Crossref], [Google Scholar]
23. Kuo, L., & Mallick, B. (1998). Variable selection for regression models. Sankhya Series B, 60, 65–81. [Google Scholar]
24. Kyung, M., Gilly, J., Ghosh, M., & Casella, G. (2010). Penalized regression, standard errors, and Bayesian lassos. Bayesian Analysis, 5(2), 369–411. https://doi.org/10.1214/10-BA607 [Crossref], [Web of Science ®], [Google Scholar]
25. Lin, Y., & Zhang, H. (2006). Component selection and smoothing in smoothing spline analysis of variance models. The Annals of Statistics, 34(5), 2272–2297. https://doi.org/10.1214/009053606000000722 [Crossref], [Web of Science ®], [Google Scholar]
26. Lv, J., & Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. The Annals of Statistics, 37(6A), 3498–3528. https://doi.org/10.1214/09-AOS683 [Crossref], [Web of Science ®], [Google Scholar]
27. O'Hara, R. B., & Sillanpää, M. J. (2009). Review of Bayesian variable selection methods: What, how and which. Bayesian Analysis, 4(1), 85–117. https://doi.org/10.1214/09-BA403 [Crossref], [Web of Science ®], [Google Scholar]
28. Park, T., & Casella, G. (2008). The Bayesian Lasso. Journal of the American Statistical Association, 103(482), 681–686. https://doi.org/10.1198/016214508000000337 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
29. Purnell, J., & Andersen, B. (2009). Religious practice and spirituality in the psychological adjustment of survivors of breast cancer. Counseling and Values, 53(3), 165–182. https://doi.org/10.1002/(ISSN)2161-007X [Crossref], [Google Scholar]
30. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. https://doi.org/10.1214/aos/1176344136 [Crossref], [Web of Science ®], [Google Scholar]
31. Shen, X., Pan, W., & Zhu, Y. (2011). Likelihood-based selection and sharp parameter estimation. Journal of the American Statistical Association. [Web of Science ®], [Google Scholar]
32. Silliman, R., Dukes, K., Sullivan, L., & Kaplan, S. (1998). Breast cancer care in older women: Sources of information, social support, and emotional health outcomes. Cancer, 83, 706–711. https://doi.org/10.1002/(ISSN)1097-0142 [Crossref], [Web of Science ®], [Google Scholar]
33. Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2012). A sparse-group lasso. Journal of Computational and Graphical Statistics. [Web of Science ®], [Google Scholar]
34. Stamey, T., Kabalin, J., McNeal, J., Johnstone, I., Freiha, F., Redwine, E., & Yang, N. (1989). Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate.II. radical prostatectomy treated patients. Journal of Urology, 141(5), 1076–1083. https://doi.org/10.1016/S0022-5347(17)41175-X [Crossref], [Web of Science ®], [Google Scholar]
35. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267–288. [Crossref], [Google Scholar]
36. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., & Knight, K. (2005). Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(1), 91–108. https://doi.org/10.1111/rssb.2005.67.issue-1 [Crossref], [Web of Science ®], [Google Scholar]
37. Tipping, M. (2001). Sparse Bayesian learning and the relevance vector machine. Journal OfMachine Learning, 1, 211–244. [Crossref], [Web of Science ®], [Google Scholar]
38. Yoo, G., Levine, E., Aviv, C., Ewing, C., & Au, A. (2010). Older women, breast cancer, and social support. Supportive Care in Cancer, 18, 121521–1530. https://doi.org/10.1007/s00520-009-0774-4 [Crossref], [Web of Science ®], [Google Scholar]
39. Yuan, M., & Lin, Y. (2005). Efficient empirical bayes variable selection and esimation in linear models. Journal of the American Statistical Association, 100(472), 1215–1225. https://doi.org/10.1198/016214505000000367 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
40. Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1), 49–67. https://doi.org/10.1111/rssb.2006.68.issue-1 [Crossref], [Web of Science ®], [Google Scholar]
41. Zhang, C. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2), 894–942. https://doi.org/10.1214/09-AOS729 [Crossref], [Web of Science ®], [Google Scholar]
42. Zhou, N., & Zhu, J. (2010). Group variable selection via a hierarchical lasso and its oracle property. Statistics and Its Inference, 3, 557–574. [Crossref], [Web of Science ®], [Google Scholar]
43. Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476), 1418–1429. https://doi.org/10.1198/016214506000000735 [Taylor & Francis Online], [Web of Science ®], [Google Scholar]
44. Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, Series B, 67(2), 301–320. https://doi.org/10.1111/rssb.2005.67.issue-2 [Crossref], [Google Scholar]

Ting Ye, Yanyao Yi. (2021) Comment: inference after covariate-adaptive randomisation: aspects of methodology and theory. Statistical Theory and Related Fields 0:0, pages 1-2.