In this paper, we study high-dimensional sparse multiplicative models for positive response data and propose a variable sorted active set (VSAS) algorithm for finding the L0 regularized least product relative error (LPRE) estimator. The VSAS algorithm is derived from the local quadratic approximation based on the Karush-Kuhn-Tucker (KKT) conditions of L0-penalized LPRE objective function. Under the condition of restricted invertibility, we establish an explicit L∞ upper bound for the sequence of solutions generated by the VSAS algorithm. We further obtain an optimal convergence rate for the proposed estimator with high probability in finite iterations. In addition, our estimator enjoys the oracle property with high probability if the target signal exceeds the detectable level. Finally, extensive simulations and two real-world applications are conducted to illustrate the effectiveness of the proposal.

References

• Bühlmann, P., Kalisch, M., & Meier, L. (2014). High-dimensional statistics with a view towards applications in biology. Annual Review of Statistics and Its Applications, 1(1), 255–278.
• Cao, Y., Kang, L., Li, X., Liu, Y., Luo, Y., & Yang, Q. (2023). Newton-Raphson meets sparsity: Sparse learning via a novel penalty and a fast solver. IEEE Transactions on Neural Networks and Learning Systems, 35(9), 11057–12067.
• Chen, Z., Fan, J., & Li, R. (2018). Error variance estimation in ultrahigh dimensional additive models. Journal of the American Statistical Association, 113(521), 315–324.
• Chen, X., Ge, D., Wang, Z., & Ye, Y. (2014). Complexity of unconstrained L2-Lp minimization. Mathematical Programming, 143(1-2), 371–383.
• Chen, K., Guo, S., Lin, Y., & Ying, Z. (2010). Least absolute relative error estimation. Journal of the American Statal Association, 105(491), 1104–1112.
• Chen, K., Lin, Y., Wang, Z., & Ying, Z. (2016). Least product relative error estimation. Journal of Multivariate Analysis, 144, 91–98.
• Chen, Y., Liu, H., & Ma, J. (2022). Local least product relative error estimation for single-index varying-coefficient multiplicative model with positive responses. Journal of Computational and Applied Mathematics, 415, 114478.
• Chen, Y., Ming, H., & Yang, H. (2024). Efficient variable selection for high-dimensional multiplicative models: A novel LPRE-based approach. Statistical Papers, 65(6), 3713–3737.
• Cheng, C., Feng, X., Huang, J., Jiao, Y., & Zhang, S. (2022). L0-regularized high-dimensional accelerated failure time model. Computational Statistics and Data Analysis, 170, 107430.
• Do, H., Cheon, M., & Kim, S. (2020). Graph structured sparse subset selection. Information Sciences, 518, 71–94.
• Fan, Q., Jiao, Y., & Lu, X. (2014). A primal dual active set algorithm with continuation for compressed sensing. IEEE Transactions on Signal Processing, 62(23), 6276–6285.
• Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.
• Fan, J., & Lv, J. (2008). Sure independence screening for ultra-high dimensional feature space. Journal of the Royal Statistical Society Series B, 70(5), 849–911.
• Hao, M., Lin, Y., & Zhao, X. (2016). A relative error-based approach for variable selection. Computational Statistics and Data Analysis, 103, 250–262.
• Hu, D. (2019). Local least product relative error estimation for varying coefficient multiplicative regression model. Acta Mathematicae Applicatae Sinica, English Series, 35(2), 274–286.
• Huang, J., Jiao, Y., Jin, B., Liu, J., Liu, Y., & Yang, C. (2021). A unified primal dual active set algorithm for nonconvex sparse recovery. Statistical Science, 36(2), 215–238.
• Huang, J., Jiao, Y., Kang, L., & Liu, Y. (2021). Fitting sparse linear models under the sufficient and necessary condition for model identification. Statistics and Probability Letters, 168, 108925.
• Huang, J., Jiao, Y., Kang, L., Liu, J., Liu, Y., & Lu, X. (2022). GSDAR: A fast Newton algorithm for l0 regularized generalized linear models with statistical guarantee. Computational Statistics, 37(1), 507–533.
• Huang, J., Jiao, Y., Liu, Y., & Lu, X. (2018). A constructive approach to L0 penalized regression. Journal of Machine Learning Research, 19(10), 1–37.
• Huang, J., Jiao, Y., Lu, X., Shi, Y., Yang, Q., & Yang, Y. (2022). PSNA: A pathwise semismooth Newton algorithm for sparse recovery with optimal local convergence and oracle properties. Signal Processing, 194, 108432.
• Javanmard, A., & Montanari, A. (2014). Confidence intervals and hypothesis testing for high-dimensional regression. Journal of Machine Learning Research, 15(82), 2869–2909.
• Li, P., Jiao, Y., Lu, X., & Kang, L. (2022). A data-driven line search rule for support recovery in high-dimensional data analysis. Computational Statistics and Data Analysis, 174, 107524.
• Li, X., Shi, Z., & Leung, C. (2022). Sparse index tracking with K-sparsity or ϵ-deviation constraint via ℓ0-norm minimization. IEEE Transactions on Neural Networks and Learning Systems, 34(12), 10930–10943.
• Liu, W., Ke, Y., Liu, J., & Li, R. (2022). Model-free feature screening and FDR control with knockoff features. Journal of the American Statistical Association, 117(537), 428–443.
• Liu, H., & Xia, X. (2018). Estimation and empirical likelihood for single-index multiplicative models. Journal of Statistical Planning and Inference, 193, 70–88.
• Ming, H., Liu, H., & Yang, H. (2022). Least product relative error estimation for identification in multiplicative additive models. Journal of Computational and Applied Mathematics, 404, 113886.
• Ming, H., & Yang, H. (2024a). A fast robust best subset regression. Knowledge-Based Systems, 284, 111309.
• Ming, H., & Yang, H. (2024b). L0 regularized regularized logistic regression for large-scale data. Pattern Recognition, 146, 110024.
• Natarajan, B. (1995). Sparse approximate solutions to linear systems. SIAM Journal on Computing, 24(2), 227–234.
• Shi, Y., Huang, J., Jiao, Y., & Yang, Q. (2020). A semismooth Newton algorithm for high-dimensional nonconvex sparse learning. IEEE Transactions on Neural Networks and Learning Systems, 31(8), 2993–3006.
• Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B, 58(1), 267–288.
• Vershynin, R. (2018). High-dimensional Probability: An Introduction with Applications in Data Science. Cambridge University Press.
• Wang, H. (2009). Forward regression for ultra-high dimensional variable screening. Journal of the American Statistical Association, 104(488), 1512–1524.
• Wang, L., Kim, Y., & Li, R. (2013). Calibrating non-convex penalized regression in ultra-high dimension. The Annals of Statistics, 41(5), 2505–2536.
• Wang, Z., Liu, W., & Lin, Y. (2015). A change-point problem in relative error-based regression. Test, 24(4), 835–856.
• Wen, C., Li, Z., Dong, R., Ni, Y., & Pan, W. (2023). Simultaneous dimension reduction and variable selection for multinomial logistic regression. INFORMS Journal on Computing, 35(5), 1044–1060.
• Wen, C., Wang, X., & Zhang, A. (2023). 0 trend filtering. INFORMS Journal on Computing, 35(6), 1491–1510.
• Wen, C., Zhang, A., Quan, S., & Wang, X. (2020). BeSS: An R package for best subset selection in linear, logistic and cox proportional hazards models. Journal of Statistical Software, 94(4), 1–24.
• Zhang, C. (2010). Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2), 894–942.
• Zhang, J., Feng, Z., & Peng, H. (2018). Estimation and hypothesis test for partial linear multiplicative models. Computational Statistics and Data Analysis, 128, 87–103.
• Zhang, J., Lin, B., & Yang, Y. (2022). Maximum nonparametric kernel likelihood estimation for multiplicative linear regression model. Statistical Papers, 63(3), 885–918.
• Zhang, J., Zhu, J., & Feng, Z. (2019). Estimation and hypothesis test for single-index multiplicative models. Test, 28(1), 242–268.
• Zhang, Y., Zhu, J., Zhu, J., & Wang, X. (2023). A splicing approach to best subset of groups selection. INFORMS Journal on Computing, 35(1), 104–119.
• Zhao, P., Yang, Y., & He, Q. (2022). High-dimensional linear regression via implicit regularization. Biometrika, 109(4), 1033–1046.
• Zheng, Z., Zhang, J., & Li, Y. (2022). L0-regularized learning for high-dimensional additive hazards regression. INFORMS Journal on Computing, 34(5), 2762–2775.
• Zhou, S., Pan, L., & Xiu, N. (2021). Newton method for L0-regularized optimization. Numerical Algorithms, 88(4), 1541–1570.
• Zhu, J., Wen, C., Zhu, J., & Wang, X. (2020). A polynomial algorithm for best-subset selection problem. Proceedings of the National Academy of Sciences of the United States of America, 117(52), 33117–33123.

To cite this article: Hao Ming, Hu Yang & Xiaochao Xia (16 Feb 2025): L0-regularized highdimensional sparse multiplicative models, Statistical Theory and Related Fields, DOI: 10.1080/24754269.2025.2460148

To link to this article: https://doi.org/10.1080/24754269.2025.2460148