Dokumentdetails
ID
oai:arXiv.org:2405.17181
Thema
Computer Science - Machine Learnin... Computer Science - Computer Vision...Autor
Yang, Sheng Zavatone-Veth, Jacob A. Pehlevan, CengizKategorie
Computer Science
Jahr
2024
Auflistungsdatum
29.05.2024
Schlüsselwörter
classification layers computer regularization robustness adversarial learningZusammenfassung
The vulnerability of neural network classifiers to adversarial attacks is a major obstacle to their deployment in safety-critical applications.
Regularization of network parameters during training can be used to improve adversarial robustness and generalization performance.
Usually, the network is regularized end-to-end, with parameters at all layers affected by regularization.
However, in settings where learning representations is key, such as self-supervised learning (SSL), layers after the feature representation will be discarded when performing inference.
For these models, regularizing up to the feature space is more suitable.
To this end, we propose a new spectral regularizer for representation learning that encourages black-box adversarial robustness in downstream classification tasks.
In supervised classification settings, we show empirically that this method is more effective in boosting test accuracy and robustness than previously-proposed methods that regularize all layers of the network.
We then show that this method improves the adversarial robustness of classifiers using representations learned with self-supervised training or transferred from another classification task.
In all, our work begins to unveil how representational structure affects adversarial robustness.
;Comment: 15 + 15 pages, 8 + 11 figures
Yang, Sheng,Zavatone-Veth, Jacob A.,Pehlevan, Cengiz, 2024, Spectral regularization for adversarially-robust representation learning
SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models
methods salience efficiency quantization language slim-llm llms
Prediction Models for Suicide Attempts among Adolescents Using Machine Learning Techniques
using data ml prediction 0 suicide adolescents
