Meta, the parent company of Facebook and Instagram, introduced two new AI video editing tools. Emu Video and Emu Edit are designed to enhance the video editing experience for users on both Instagram and Facebook.
What are Emu Video & Emu Edit?
Emu Video generates brief four-second videos accompanied by a caption, photo, or image along with a corresponding description. Emu Edit, in turn, simplifies the process of modifying and editing videos with text prompts.
With Emu Video, Meta introduces a simple approach to text-to-video generation based on diffusion models, making video generation tasks respond to various inputs, including text only, image only, and both text and image.
How does it work? First, the tool generates images based on a text prompt, and then generates video based on both the text and the generated image. Meta notes that this "factorized" or split approach to video generation enables efficient training of video generation models.
Emu Edit is a novel approach designed to streamline various image editing tasks, offering enhanced capabilities and precision to image editing.
The tool allows for free-form editing through instructions, covering tasks such as local and global editing, background removal and addition, color and geometry transformations, detection and segmentation, and more.
Unlike many current generative AI models, Emu Edit strictly follows instructions, ensuring that pixels unrelated to the given instructions in the input image remain unchanged, Meta claims.
"In order to train the model, we’ve developed a dataset that contains 10 million synthesized samples, each including an input image, a description of the task to be performed, and the targeted output image. We believe it’s the largest dataset of its kind to date.", Meta says in a press release.
Meta Platforms has been rapidly advancing in the field of artificial intelligence. In addition to image generative AI, Meta has recently launched text-to-music tools: AudioCraft by Meta, claimed to be the best AI music generator presently existing on the market and is taken as a benchmark by other developers, and AudioCraft that has multiple models which are trained and grow as we speak, namely AudioGen and MusicGen.
