Détail du document
Identifiant
oai:arXiv.org:2410.03030
Sujet
Computer Science - Computer Vision... Computer Science - Artificial Inte...Auteur
Wu, Boqian Xiao, Qiao Wang, Shunxin Strisciuglio, Nicola Pechenizkiy, Mykola van Keulen, Maurice Mocanu, Decebal Constantin Mocanu, ElenaCatégorie
Computer Science
Année
2024
Date de référencement
12/03/2025
Mots clés
artificial dense robustness computer training dynamic sparseRésumé
It is generally perceived that Dynamic Sparse Training opens the door to a new era of scalability and efficiency for artificial neural networks at, perhaps, some costs in accuracy performance for the classification task.
At the same time, Dense Training is widely accepted as being the "de facto" approach to train artificial neural networks if one would like to maximize their robustness against image corruption.
In this paper, we question this general practice.
Consequently, we claim that, contrary to what is commonly thought, the Dynamic Sparse Training methods can consistently outperform Dense Training in terms of robustness accuracy, particularly if the efficiency aspect is not considered as a main objective (i.e., sparsity levels between 10% and up to 50%), without adding (or even reducing) resource cost.
We validate our claim on two types of data, images and videos, using several traditional and modern deep learning architectures for computer vision and three widely studied Dynamic Sparse Training algorithms.
Our findings reveal a new yet-unknown benefit of Dynamic Sparse Training and open new possibilities in improving deep learning robustness beyond the current state of the art.
;Comment: Accepted at ICLR 2025
Wu, Boqian,Xiao, Qiao,Wang, Shunxin,Strisciuglio, Nicola,Pechenizkiy, Mykola,van Keulen, Maurice,Mocanu, Decebal Constantin,Mocanu, Elena, 2024, Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness
Systematic Feature Design for Cycle Life Prediction of Lithium-Ion Batteries During Formation
cycle prediction framework feature design features life formation
