Review Articles

Bayesian variable selection via a benchmark in normal linear models

Jun Shao ,

a KLATASDS-MOE, School of Statistics, East China Normal University Shanghai, People's Republic of China;b Department of Statistics, University of Wisconsin-Madison, Madison, WI, USA

Kam-Wah Tsui ,

b Department of Statistics, University of Wisconsin-Madison, Madison, WI, USA

Sheng Zhang

b Department of Statistics, University of Wisconsin-Madison, Madison, WI, USA

Pages 70-81 | Received 13 Aug. 2019, Accepted 15 Mar. 2020, Published online: 27 Mar. 2020,
  • Abstract
  • Full Article
  • References
  • Citations


With increasing appearances of high-dimensional data over the past two decades, variable selections through frequentist likelihood penalisation approaches and their Bayesian counterparts becomes a popular yet challenging research area in statistics. Under a normal linear model with shrinkage priors, we propose a benchmark variable approach for Bayesian variable selection. The benchmark variable serves as a standard and helps us to assess and rank the importance of each covariate based on the posterior distribution of the corresponding regression coefficient. For a sparse Bayesian analysis, we use the benchmark in conjunction with a modified BIC. We also develop our benchmark approach to accommodate models with covariates exhibiting group structures. Two simulation studies are carried out to assess and compare the performances among the proposed approach and other methods. Three real datasets are also analysed by using these methods for illustration.


  1. Andrews, D. F., & Mallows, C. L. (1974). Scale mixtures of normal distributions. Journal of the Royal Statistical Society, Series B36, 99–102. [Google Scholar]
  2. Bayarri, M. J., Berger, J. O., Jang, W., Ray, S., Pericchi, L. R., & Visser, L. (2019). Prior-based Bayesian information criterion (PBIC). Statistical Theory and Related Fields3(1), 2–13. [Taylor & Francis Online], [Google Scholar]
  3. Bradic, J., Fan, J., & Wang, W. (2009). Penalized composite quasi-Likelihood for ultrahigh-Dimensional variable selection. Journal of the Royal Statistical Society, Series B73(3), 325–349. [Crossref], [Google Scholar]
  4. Brown, P. J., Vannucci, M., & Fearn, T. (1998). Multivariate Bayesian variable selection and prediction. Journal of the Royal Statistical Society, Series B60(3), 627–641. [Crossref], [Google Scholar]
  5. Campbell, A., Converse, P., & Rodgers, W. (2008). The quality of American life: Perceptions, evaluations, and satisfactions. Russell Sage Foundation. [Google Scholar]
  6. Chen, J., & Chen, Z. (2008). Extended Bayesian information criterion for model selection with large model spaces. Biometrika95, 759–771. [Crossref][Web of Science ®], [Google Scholar]
  7. Cohen, S., Gottlieb, B., & Underwood, L. (2000). Social relationships and health. In S. Cohen, L. Underwood, & B. Gottlieb (Eds.), Social support measurement and intervention. Oxford University Press. [Crossref], [Google Scholar]
  8. Cohen, S., & Syme, L. (1985). Social support and health (Tech. Rep.). Academic. [Google Scholar]
  9. Dellaportas, P., Forster, J. J., & Ntzoufras, I. (1997). On Bayesian model and variable selection using MCMC (Tech. Rep.). Department of Statistics, Athens University of Economics and Business. [Google Scholar]
  10. Deutsch, S., Lyle, R., Dermitzakis, E., Attar, H., Subrahmanyan, L., Gehri, C., Parand, L., Gagnebin, M., Rougemont, J., Jongeneel, C., & Antonarakis, S. (2005). Gene expression variation and expression quantitative trait mapping of human chromosome 21 genes. Human Molecular Genetics14(23), 3741–3749. [Crossref][Web of Science ®], [Google Scholar]
  11. Dicker, L., Huang, B., & Lin, X. (2011). Variable selection and estimation with the seamless-l0 penalty. Statistica Sinica[Web of Science ®], [Google Scholar]
  12. Fan, J., Han, X., & Gu, W. (2012). Estimating false discovery proportion under arbitrary covariance dependence. Journal of the American Statistical Association107(499), 1019–1035. [Taylor & Francis Online][Web of Science ®], [Google Scholar]
  13. Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association96(456), 1348–1360. [Taylor & Francis Online][Web of Science ®], [Google Scholar]
  14. Frank-Stromborg, M., & Olsen, S. (2003). Instruments For Clinical Health-Care Research (Jones and Bartlett Series in Oncology, 3rd edition) (Tech. Rep.). Jones & Bartlett Learning. [Google Scholar]
  15. Gall, T., Charbonneau, C., Clarke, N., Grant, K., Joseph, A., & Shouldice, L. (2005). Understanding the nature and role of spirituality in relation to coping and health: A conceptual framework. Canadian Psychology46(2), 88–104. [Crossref][Web of Science ®], [Google Scholar]
  16. George, E. I., & McCulloch, R. E. (1993). Variable selection via Gibbs sampling. Journal of the American Statistical Association85, 398–409. [Google Scholar]
  17. Green, P. J. (1995). Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika82, 711–732. [Crossref][Web of Science ®], [Google Scholar]
  18. Hoerl, A., & Kennard, R. (1970). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics12(1), 55–67. [Taylor & Francis Online][Web of Science ®], [Google Scholar]
  19. Hoti, F., & Sillanpää, M. J. (2006). Bayesian mapping of genotype x expression interactions in quantitative and qualitative traits. Heredity97(1), 4–18. [Crossref][Web of Science ®], [Google Scholar]
  20. House, J., & Khan, R. (1985). Measures and concepts of social support. In S. Cohen & S. L. Syme (Eds.), Social support and health (Tech. Rep.). [Google Scholar]
  21. Huang, J., Ma, S., Xie, H., & Zhang, C. (2009). A group bridge approach for variable selection. Biometrika96, 339–355. [Crossref][Web of Science ®], [Google Scholar]
  22. Knobf, M. (2007). Psychological responses in brest cancer survivors. Seminars in Oncology Nursing23(1), 71–83. [Crossref], [Google Scholar]
  23. Kuo, L., & Mallick, B. (1998). Variable selection for regression models. Sankhya Series B60, 65–81. [Google Scholar]
  24. Kyung, M., Gilly, J., Ghosh, M., & Casella, G. (2010). Penalized regression, standard errors, and Bayesian lassos. Bayesian Analysis5(2), 369–411. [Crossref][Web of Science ®], [Google Scholar]
  25. Lin, Y., & Zhang, H. (2006). Component selection and smoothing in smoothing spline analysis of variance models. The Annals of Statistics34(5), 2272–2297. [Crossref][Web of Science ®], [Google Scholar]
  26. Lv, J., & Fan, Y. (2009). A unified approach to model selection and sparse recovery using regularized least squares. The Annals of Statistics37(6A), 3498–3528. [Crossref][Web of Science ®], [Google Scholar]
  27. O'Hara, R. B., & Sillanpää, M. J. (2009). Review of Bayesian variable selection methods: What, how and which. Bayesian Analysis4(1), 85–117. [Crossref][Web of Science ®], [Google Scholar]
  28. Park, T., & Casella, G. (2008). The Bayesian Lasso. Journal of the American Statistical Association103(482), 681–686. [Taylor & Francis Online][Web of Science ®], [Google Scholar]
  29. Purnell, J., & Andersen, B. (2009). Religious practice and spirituality in the psychological adjustment of survivors of breast cancer. Counseling and Values53(3), 165–182. [Crossref], [Google Scholar]
  30. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics6(2), 461–464. [Crossref][Web of Science ®], [Google Scholar]
  31. Shen, X., Pan, W., & Zhu, Y. (2011). Likelihood-based selection and sharp parameter estimation. Journal of the American Statistical Association[Web of Science ®], [Google Scholar]
  32. Silliman, R., Dukes, K., Sullivan, L., & Kaplan, S. (1998). Breast cancer care in older women: Sources of information, social support, and emotional health outcomes. Cancer83, 706–711. [Crossref][Web of Science ®], [Google Scholar]
  33. Simon, N., Friedman, J., Hastie, T., & Tibshirani, R. (2012). A sparse-group lasso. Journal of Computational and Graphical Statistics[Web of Science ®], [Google Scholar]
  34. Stamey, T., Kabalin, J., McNeal, J., Johnstone, I., Freiha, F., Redwine, E., & Yang, N. (1989). Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate.II. radical prostatectomy treated patients. Journal of Urology141(5), 1076–1083. [Crossref][Web of Science ®], [Google Scholar]
  35. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B58, 267–288. [Crossref], [Google Scholar]
  36. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., & Knight, K. (2005). Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology)67(1), 91–108. [Crossref][Web of Science ®], [Google Scholar]
  37. Tipping, M. (2001). Sparse Bayesian learning and the relevance vector machine. Journal OfMachine Learning1, 211–244. [Crossref][Web of Science ®], [Google Scholar]
  38. Yoo, G., Levine, E., Aviv, C., Ewing, C., & Au, A. (2010). Older women, breast cancer, and social support. Supportive Care in Cancer18, 121521–1530. [Crossref][Web of Science ®], [Google Scholar]
  39. Yuan, M., & Lin, Y. (2005). Efficient empirical bayes variable selection and esimation in linear models. Journal of the American Statistical Association100(472), 1215–1225. [Taylor & Francis Online][Web of Science ®], [Google Scholar]
  40. Yuan, M., & Lin, Y. (2006). Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology)68(1), 49–67. [Crossref][Web of Science ®], [Google Scholar]
  41. Zhang, C. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics38(2), 894–942. [Crossref][Web of Science ®], [Google Scholar]
  42. Zhou, N., & Zhu, J. (2010). Group variable selection via a hierarchical lasso and its oracle property. Statistics and Its Inference3, 557–574. [Crossref][Web of Science ®], [Google Scholar]
  43. Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association101(476), 1418–1429. [Taylor & Francis Online][Web of Science ®], [Google Scholar]
  44. Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society, Series B67(2), 301–320. [Crossref], [Google Scholar]

Ting Ye, Yanyao Yi. (2021) Comment: inference after covariate-adaptive randomisation: aspects of methodology and theoryStatistical Theory and Related Fields 0:0, pages 1-2.