Google’s Lumiere Makes Significant Jump in Generative Video AI
The new generative model of Google, Lumiere, has made a significant advancement in the field of creating artificial intelligence. This model has the ability to generate high-quality, realistic, or surreal video clips that can last up to five seconds. Lumiere can also analyze static images or specific parts of those images, and respond flexibly to user’s textual requests. The project can be accessed via Lumiere’s website.
One of the unique features of Lumiere is its ability to copy image styles and create a series of videos based on different topics, all while maintaining a consistent visual style. This particular function can be leveraged to create branded content, among other applications.
Lumiere is also capable of transforming the styles of original videos by converting objects into different forms such as Lego, Origami, or flowers, depending on the parameters specified by the user.
An especially notable feature of Lumiere is the “InpAnding” function, which allows users to remove undesirable elements from a video. By simply smearing the unwanted part of the image, Lumiere can automatically fill the space seamlessly, making the changes virtually invisible to the viewer.
The creators of Lumiere claim that the “Space-Time U-Net” architecture enables the model to build a video in one pass, unlike previous models that required the generation of initial and final frames, and then attempted to reconstruct the intermediate frames.
Currently, Lumiere is a research project, which liberates Google from the need to impose constraints based on copyright policies, disinformation, security, hatred, nudity, confidentiality, etc. Hypothetically, the model can be utilized to create high-quality videos on a wide range of topics.