Bayesian Lasso probit Model: A simulation Study

Anwer Fawzi  Ali; Zahraa Khalid  Gaafar

Anwer Fawzi Ali Al-Qadisiyah Governorate Education Directorate
Zahraa Khalid Gaafar Al-Qadisiyah Governorate Education Directorate

Keywords: Bayesian Model, Lasso, Probit Model, Classification, Simulation

Abstract

This paper investigates the probit regression model from the Bayesian variable selection perspective. We employ lasso method to select the relevant predictor variable that produced a more interpretable probit regression model with high predictive ability ridge parameter has been added to variance – covariance matrix to overcome the problem of multicollinearity. Also, we developed the hierarchical model based on the proposed variable selection methodology, and new MCMC Gibbs sampling algorithm have expanded to generate the samples from the proposed posterior distributions. Simulation scenarios have studied to evaluate the performance of proposed model and compare the results with some exists methods. The results demonstrate that the proposed model have a competitive ability with the other models.

References

[1] Ai-Jun, Y. and Xin-Yuan, S. (2010). Bayesian variable selection for disease classification using gene expression data. Bioinformatics, 26(2), 215-222.

[2] Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88(422), 669{679.

[3] Alnasser, H . (2014). On Regression and least Absolute shrinkage and selection operator. M.SC , University of Victoria , department Mathematical and Statistics.

[4] Bae, K. and Mallick, B. K. (2004). Gene selection using a two-level hierarchical Bayesian model. Bioinformatics, 20(18), 3423-3430.

[5] Barber, D. (2012). Bayesian reasoning and machine learning. Cambridge University Press.

[6] Benoit, D. F., & Van den Poel, D. (2012). Binary quantile regression: a Bayesian approach based on the asymmetric Laplace distribution. Journal of Applied Econometrics, 27 (7), 1174-1188.

[7] Benoit, D. F., Alhamzawi, R., & Yu, K. (2013). Bayesian lasso binary quantile regression. Computational Statistics, 28(6), 2861-2873.

[8] Chib, S. and Greenberg, E. (1998). Analysis of multivariate probit models. Biometrika, 85(2), 347-361.

[9] Hilali, H.K.A., and Alhamzawi, R. (2019). Bayesian Adaptive Lasso binary regression with ridge parameter. IOP Conf. Series: Journal of Physics: Conf. Series 1294.

[10] Imai, K. and van Dyk, D. A. (2005). A Bayesian analysis of the multinomial probit model using marginal data augmentation. Journal of econometrics, 124(2), 311-334.

[11] Jiao, X. and van Dyk, D. A. (2015). A corrected and more e_cient suite of MCMC samplers for the multinomal probit model. arXiv preprint arXiv:1504.07823.

[12] Kotti, E., Manolopoulou, I., and Fearn, T. (2015). Bayesian variable selection in the probit model with mixture of nominal and ordinal responses. In Workshop Autonomous Citizens: Algorithms for Tomorrow's Society.

[13] Lamnisos, D., Gri_n, J. E., and Steel, M. F. (2012). Cross-validation prior choice in Bayesian probit regression with many covariates. Statistics and Computing, 22(2), 359-373.

[14] Maddala, G. S. (1987). Limited dependent variable models using panel data. Journal of Human resources, 307-338.
‏
[15] Mallick, H., and Yi, N. (2014). A new Bayesian LASSO. Statistics and its interface, 7(4), 571-582.
‏
[16 ] Martin, Andrew D., Quinn, Kevin M., and Park, Jong Hee. (2011). MCMC pack: Markov Chain Monte Carlo in R. Journal of Statistical Software. 42(9): 1-21.

[17] Park, T., and Casella, G. (2008). The bayesian lasso. Journal of the American Statistical Association, 103(482), 681-686.

[18] Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1), 267-288.

[19] Zhou, X., Wang, X., and Dougherty, E. (2006). Multi-class cancer classification using multinomial probit regression with Bayesian gene selection. IEEE Proceedings Systems Biology, 153(2), 70-78.