基于Ramp损失函数的原空间支持向量回归机

doi:2016.02.003

华东师范大学学报(自然科学版) ›› 2016, Vol. 2016 ›› Issue (2): 20-29.doi: 2016.02.003

基于Ramp损失函数的原空间支持向量回归机

袁玉萍^[1] , 安增龙^[2]

1. 黑龙江八一农垦大学~~信息与计算科学系, 黑龙江~~大庆~163319 2.黑龙江八一农垦大学~~经济管理系，黑龙江~~大庆~163319

收稿日期:2015-03-04 出版日期:2016-03-25 发布日期:2016-07-25
通讯作者: 安增龙, 男, 教授, 研究方向为人力资源管理. E-mail:anzl@2001@163.com
作者简介:袁玉萍, 女, 博士, 研究方向为运筹学与优化. E-mail: byndyyps@sina.com
基金资助:
高等学校博士学科点专项科研基金(20112305110002)；
黑龙江八一农垦大学博士\\\rule{18mm}{0mm}科研启动基金(XDB2015-23);
黑龙江农垦总局科研资助项目(HNK11A-14-07)

Support vector machine in the primal space based on\\ the ramp loss function\\[6mm

YUAN Yu-Ping ^[1] , AN Zeng-Long^[2]

Received:2015-03-04 Online:2016-03-25 Published:2016-07-25

摘要/Abstract

摘要： 针对传统支持向量机对噪声敏感的问题，给出一种基于不对称形式的二次不敏感控制型\,ramp\,损失函数的支持向量回归机，采用凹凸过程优化和光滑技术算法，将非凸优化问题转化为连续且二次可微的凸优化问题，利用有限步终止的\,Amijo-Newton\,优化算法，求解所建立的优化模型，并分析了算法的收敛性.该算法不仅可以保持支持向量的稀疏性，而且还可以控制训练样本中的异常值.实验结果表明，该模型保持了很好的泛化能力，无论对模拟数据还是标准数据都具有一定的拟合精度，与标准支持向量机模型相比，不仅能够降低噪声和孤立点的影响，而且也具有较强的鲁棒性.

关键词: 支持向量回归机, 异常值, 损失函数, 凹凸过程

Abstract: Aiming at the problem of standard support vector machine being sensitive to the noise, a new method of support vector regression (SVR) machine based on dissymmetry quadratic and controlled-insensitive loss function is proposed. Using the concave and convex process optimization and the smooth technology algorithm, the problem of non-convex optimization is transformed into the problem of the continuous and twice differentiable convex optimization. Using the Amijo-Newton optimized algorithm of finite iteration termination, the established optimization model is solved, and the convergence of the algorithm is analyzed. The algorithm can not only keep the sparse nature of support vector, but also control the abnormal values of the training sample. The results of theexperiment showed that the support vector regression machine model proposed kept good generalization ability, and the model could fit better both the simulated data and the standard data. Compared with the standard support vector machine (SVM) model, the proposed model not only can reduce the effects of noise and outliers, but also has stronger robustness.

Key words: support vector regression; , outliers; , loss function; , concave-convex procedure

中图分类号:

TP181
TP391

袁玉萍, 安增龙. 基于Ramp损失函数的原空间支持向量回归机[J]. 华东师范大学学报(自然科学版), 2016, 2016(2): 20-29.

YUAN Yu-Ping, AN Zeng-Long. Support vector machine in the primal space based on\\ the ramp loss function\\[6mm[J]. Journal of East China Normal University(Natural Sc, 2016, 2016(2): 20-29.

参考文献

[1]XIU F J, ZHANG Y, JIAN C L. Fuzzy SVM with a new fuzzy membership function [J]. Neural Computing and Application, 2006 (15): 268-276.
[2]LIU Y H, CHEN Y T. Face recongnition using total margin-based adaptive fuzzy support vector machines [J]. IEEE Trans on Neural Networks, 2007, 18(1): 178-192.
[3]YU S, YANG X W, HAO Z F, et al．An adaptive support vector machinelearning algorithm for large classification problem [J]. Lecture Notes in Computer Science, 2006, 3971: 981-990.
[4]LIN C F, WANG S D. Fuzzy support vector machines [J]. IEEE Transactions on Neural Networks, 2002, 3(2): 464-471.
[5]JIN B, ZHANG Y Q．Classifying very large data sets with minimum enclosing ban based support vector machine [C]//Proceedings of the 2006 IEEE International Conference on Fuzzy Systems. Vancouver BC, 2006: 364-368.
[6]BO L, WANG L, JIAO L. Recursive finite Newton algorithm for support vector regression in the primal [J]. Neural Computation, 2007,19(4): 1082-1096.
[7]CHEN X B, YANG J, LIANG J, et al. Recursive robust least squares support vector regression based on maximum correntropy criterion [J]. Neurocomputing, 2012, 97: 63-73.
[8]杨俊燕, 张优云, 朱永生. 不敏感损失函数支持向量机分类性能研究~[J].西安交通大学学报, 2007, 4l(11): 1315-1320.
[9]HUANG H, LIU Y. Fuzzy support vector machines for pattern recognition and data mining [J]. International Journal of Fuzzy Systems, 2002, 4(3): 3-12.
[10]WANG L, JIA H, LI J. Training robust support vector machine with smooth ramp loss in the primal space [J] Neurocomputing, 2008, 71(13/14/15): 3020-3025.
[11]ZHAO Y, SUN J. Robust support vector regression in the primal [J]. Neural Networks, 2008, 21(10): 1548-1555.
[12]YANG S H, HU B G. A stagewise least square loss function for classfication [C]//Proceedings of the SIAM International Conference on Data Mining. 2008, 120-131.
[13]KIMELDORF G S, WAHBA G. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines [J]. Annals of Mathematical Statistics, 1970, 41(2): 495-502.
[14]YUILLE A L, RANGARAJAN A. The concave-convex procedure (CCCP) [J]. Neural Computation, 2003, 15(4): 915-936.
[14] FUNG G, MANGASARIAN O L. Finite Newton method for Lagrangiansupport vector machine classification [J]. Neurocomputing, 2003,55(1/2): 39-55.

基于Ramp损失函数的原空间支持向量回归机

Support vector machine in the primal space based on\\ the ramp loss function\\[6mm

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 4

编辑推荐

Metrics

本文评价

[1]	曹坤, 刘素霞. 上海市水资源生态足迹分析与SVR预测[J]. 华东师范大学学报(自然科学版), 2016, 2016(4): 139-149.
[2]	黄维忠. 平衡损失函数下风险相依回归信度模型[J]. 华东师范大学学报(自然科学版), 2013, 2013(1): 30-40.
[3]	艾摩尔. 熵和对称损失函数下MINQUE和简单估计的比较(英)[J]. 华东师范大学学报(自然科学版), 2004, 2004(2): 31-36,5.
[4]	艾摩尔. 协方差矩阵的MINQUE和简单估计的比较[J]. 华东师范大学学报(自然科学版), 2004, 2004(1): 40-44.