Written by Emma Thompson for The Future of EdTech
Original photo: Courtesy of The Future of EdTech
Birmingham City University (BCU), based in the United Kingdom, has published new research in collaboration with UK tech company Covatic on improving AI system defenses against black-box attacks.
The study was released in the journal Expert Systems with Applications.
AI models used in sectors such as healthcare and autonomous vehicles are often targeted through subtle changes to input data. In black-box attacks, adversaries repeatedly test a model to reverse-engineer how it works, allowing them to feed altered inputs that can lead to incorrect outputs, such as misidentifying a stop sign or misinterpreting medical images.
The research proposes a method that applies basic image alterations, rotations or resizing, before data enters the model. This pre-processing step is designed to disrupt adversarial inputs and improve resistance to such attacks.
Testing showed a 21% increase in performance compared to standard models without defenses, and a 2.3% to 4.6% improvement over alternative strategies such as random noise defense, depending on the system used.
Original photo: Courtesy of The Future of EdTech
Birmingham City University (BCU), based in the United Kingdom, has published new research in collaboration with UK tech company Covatic on improving AI system defenses against black-box attacks.
The study was released in the journal Expert Systems with Applications.
AI models used in sectors such as healthcare and autonomous vehicles are often targeted through subtle changes to input data. In black-box attacks, adversaries repeatedly test a model to reverse-engineer how it works, allowing them to feed altered inputs that can lead to incorrect outputs, such as misidentifying a stop sign or misinterpreting medical images.
The research proposes a method that applies basic image alterations, rotations or resizing, before data enters the model. This pre-processing step is designed to disrupt adversarial inputs and improve resistance to such attacks.
Testing showed a 21% increase in performance compared to standard models without defenses, and a 2.3% to 4.6% improvement over alternative strategies such as random noise defense, depending on the system used.