A group of researchers from Tel Aviv University opened The source texts associated with the machine learning system mdm (Motion Diffusion Model), which allows to generate realistic human movements. The code is written in Python using a framework
Pytorch and spreads under the MIT license. To conduct experiments, you can use as finished models , and conduct models for models independently using the proposed scripts, for example, using a collection of three -dimensional images of a person humanml3d . The system requires a GPU with CUDA support.
The application of traditional capabilities for the anim of human movements is difficult due to complications associated with a wide variety of possible movements and the difficulty of their formal description, as well as because of the great sensitivity of human perception to unnatural movements. Earlier attempts to use generative machine learning models had problems with quality and limited expressiveness.
The proposed system attempted to use diffusion models , which are inherently suitable for simulating human movements. But they are not devoid of shortcomings, such as high requirements for computing resources and the complexity of management. To minimize the disadvantages of diffusion models in MDM, a neural network with architecture “ Transformer ” and predicting the sample (sample) instead of noise prediction at each stage, which simplifies the prevention of anomalies, such as loss of surface contact from foot.
To control the generation, it is possible to use a textual description of the action in a natural language (for example, “a person goes forward and leans to raise something from the ground”) or the use of typical actions, such as “running” and “jumping”. The system can also be used to edit movements and replenish the lost parts. Researchers were tested, the participants of which were proposed to choose a better result from several options – in 42% of cases, people preferred synthesized movements, and not real.