presented Open tools is distributed under the Apache 2.0 license. The project includes a finished model, code for training your model, utilities for testing the results of the model, tools for supplementing the model with context from external index and adapting the base model to solving your own problems.
The base model of machine learning is proposed as the basis of the bot ( gpt-neoxt-base-20b ), built using a language model covering about 20 billion parameters and optimized for conversational communication.
To teach the model, data obtained from the collections of projects laion , together and ontoocord.ai .
To expand the existing knowledge base, a system is proposed that can extract additional information from external repositories, API and other sources. For example, it is possible to update information using data from Wikipedia and news tapes. Additionally, a moderation model is trained using 6 billion parameters based on the model href=”https://hugingFace.co/togethercomputer/gpt-jt-6b-v1″> gpt-jt and designed for filtering of unacceptable questions or restrictions on discussions within certain topics.
Separately, the project can be noted Chatllama , offering the library for creating intellectual assistants, Similar to ChatGPT. The project is developing with an eye on the possibility of implementing on their own equipment and creating personalized solutions, designed for the coverage of narrow areas of knowledge (for example, medicine, jurisprudence, games, scientific research, etc.). Chatllama code supplied under the license gplv3.
The project supports the use of models based on the architecture of Llama (Large Language Model Meta AI), proposed META. The full LLAMA model covers 65 billion parameters, but for
Chatllama is recommended to use options with 7 and 13 billion parameters or GPTJ (6 billion), GPTNEOX (1.3 billion), 20BOPT (13 billion), Bloom (7.1 billion) and Galactica (6.7 billion). Initially, LLAMA models are supplied only to researchers according to the special prepared a script , which allows you to download the model to anyone.