Researchers have discovered potential vulnerability in the software of artificial intelligence (AI) NVIDIA, which can lead to safety disorders and disclosure of confidential information.
NVIDIA has developed NEMO Framework – a system designed to work with large language models, which are the basis for AI products such as chatbots. This system is used in the business to combine the company’s own data with language models in order to provide answers to questions, thus replacing the work of representatives of the support service or providing simple healthcare tips.
However, researchers from Robust Intelligence, companies with headquarters in San Francisco, found that security restrictions installed for safe use of AI can be easily circumvented. They found this out by spent several hours on data analysis using the NVIDIA system.
During one of the experiments, the researchers were able to force the NVIDIA system to replace the letter “I” with “j”, which led to the disclosure of personal identified information from the database. Researchers found that they can circumvent security restrictions in other ways, for example, to force the model to deviate from what it is not supposed to. By repeating the example of NVIDIA about the narrow discussion of the employment report, they were able to translate the model on topics, such as the state of health of the Hollywood movie star and the Franco -Prussian war – despite the prohibitions designed to prevent artificial intelligence beyond specific topics
The ease with which researchers have circumvented security measures emphasizes the problems that companies of artificial intelligence faces when trying to commercialize one of the most promising technologies that have appeared in the Silicon Valley in recent years.
Researchers advised their customers to avoid the use of NVIDIA product. NVIDIA said that she corrected one of the main reasons for the problem indicated by analysts.