A new attack on artificial intelligence called Data Poisoning recently surfaced in the cyberspace, affecting the generative AI Midjourney or Dall-E, resulting in unexpected and unpredictable outcomes.
Data poisoning occurs when algorithms are trained on datasets containing manipulated images. The NightShade tool, used by researchers and artists, subtly alters image pixels, which may go unnoticed to the human eye but can be detrimental to computer vision.
This phenomenon is particularly significant for neural networks that learn from large amounts of data, often of dubious origin, including copyrighted images. Data poisoning can cause artificial intelligence models to revert to previous errors, such as incorrectly displaying hands or distorting objects, like dogs with six paws.
The developers of Nightshade hope that their tool will compel major tech companies to respect copyright. However, there is a risk that users may intentionally employ “poisoned” images to disrupt image generators. As a consequence, a balloon can appear as an egg, and an image in Monet’s style can transform into Picasso’s style.
In response to this, several technological and human solutions have been proposed. These include meticulous data selection for training and the utilization of diverse models to detect anomalies. One technological correction is the use of “ensemble learning” (Ensemble Modeling), where models are trained on multiple distinct subsets of data and compared to identify specific issues. This approach can be employed not only for training but also to identify and eliminate suspicious “poisoned” images.
Auditing is another viable option. One approach to auditing involves developing a small, carefully chosen, and well-labeled dataset using “held-out” data that is never utilized for training. This dataset can then be used to verify the accuracy of the model.
Nevertheless, this issue also raises broader concerns regarding technological governance and the rights of artists. Perhaps, instead of viewing data poisoning as a problem, it is essential to perceive it as an innovative solution aimed at safeguarding fundamental moral rights.