Meta Releases Code Llama: A Generation AI Model
Meta continues actively developing in the field of generative artificial intelligence. The latest innovation is the introduction of the open system “Code Llama”, which has the ability to generate and explain code in natural language, specifically English.
“Code Llama” supports several programming languages including Python, C++, Java, PHP, Typescript, C# and Bash, making it comparable to popular tools like Github Copilot and Amazon Codewhisper.
According to Meta’s blog, coding models like Code Llama are available for evaluation of their capabilities, identifying issues, and addressing vulnerabilities.
The underlying technology behind Code Llama is the LLAMA 2 text generation model, which Meta previously made available to the public. The same dataset used for training Llama 2, with an emphasis on code, was utilized to train Code Llama.
Contextually, parameters refer to the model’s components studied using historical training data, while tokens represent raw text. For instance, the word “fantastic” will be divided into tokens: “fan,” “tas,” and “tic”.
An interesting feature is that certain Code Llama models can incorporate code into existing code, with the ability to process up to 100,000 code tokens as input data.
However, code generation tools are not without risks. Studies have indicated that engineers using AI tools are more susceptible to creating vulnerabilities in their applications.
In their blog post, Meta acknowledged that “Code Llama” may generate “inaccurate” or “contemporary” responses to requests. Despite these risks, the company imposes minimal restrictions on the use of Code Llama, requiring users to agree not to employ the model for malicious purposes.