Convex losses and their applications to SVM, SVR, and Shallow Neural Networks

Researchers proposed new convex losses for Support Vector Machines (SVM) and shallow neural networks, applied to binary classification, and solved the primal SVM problem using Particle Swarm Optimization. Experiments on small datasets with Nested Cross-Validation showed that generalization measures were the same with or without the new losses, despite theoretical potential for enhancement.

arXiv:2608.14288v1 Announce Type: new Abstract: We propose multiple new convex losses for SVM and Neural Networks, applied to binary classification tasks. While there are practical limitations in exploiting them with the dual SVM models, we are able to use them with SVM primal formulation and Neural Networks. In detail, the primal SVM problem with the modified losses has been solved with the Particle Swarm Optimization algorithm. We prove that the proposed losses are a generalization of the standard loss, and we experiment them with several small data-sets. This preliminary study shows that using pattern correlations inside the loss function could in theory enhance the generalization performances on some data-sets. To evaluate the performance of each loss, we adopt a Nested Cross-Validation procedure. Results show that generalization measures are the same with or without the new losses.