Experts of the Israeli company Offensive Ai Lab released a technique that allows you to decipher texts from intercepted messages of chat bots. Kaspersky laboratory told in detail the details of the study.
The new technique is an attack on third-party channels (Side-channel Attack), based on the analysis of tokens lengths in encrypted messages. Since chat bots using large language models (LLM) do not transmit information not by words or symbols, but tokens (symbols found in the text of the text), the study of tokens lengths allows you to guess the contents of the messages. On the Openai website there is “tokenizer”, which allows you to understand how it works.
Tocenization of messages by the GPT-3.5 and GPT-4
models
The main vulnerability is that chat bots send tokens sequentially, without using compression or coding methods, which facilitates the task with the attackers. Some chat bots (for example, Google Gemini) are protected from this kind of attacks, but most others were vulnerable.
To restore the text, researchers used two LLM models, one of which specialized in restoring standard entrance messages, the second – in the further text of communication. The effectiveness of the restoration of the text was about 29%, and the guessing of the general topic of the conversation was about 55%.
attack scheme
A feature of this attack is its dependence on the language of communication: it is most effective for the English language due to characteristic long tokens, while for other languages, including Russian, the effectiveness of the attack is noticeably lower.
Even languages close to English from German and Romanesque groups have tokens, the length of which is on average 1.5-2 times less. In the Russian language, the average token is even shorter – usually it is only a couple of characters, which significantly reduces the potential efficiency of attack.
It is worth emphasizing that the use of such a method is unlikely to reliably identify specific details, such as names, numerical values, dates, addresses and other critical data.
In response to the publication of this technique, the developers of chat bots, including Cloudflare and Openai, began to introduce the method of adding “garbage” data (Padding), which reduces the likelihood of a successful attack. Probably, the rest of the developers of chat bots will also introduce protection so that communication with chatbots becomes safer.