Adobe to Introduce Video Generation in Firefly by Year-End

Users will have their first opportunity to experiment with Adobe’s AI-powered video generation tools in just a few months. Adobe plans to integrate features from its Firefly Video model into the Premiere Pro beta app and make them accessible through a dedicated free website by the end of 2024.

The three key features set to debut are Generative Extend, Text to Video, and Image to Video. Currently, these features are available in a private beta but are expected to become public soon. Generative Extend allows users to extend any video by an additional two seconds, seamlessly blending the extension with the existing footage. This feature will be incorporated into the Premiere Pro beta app. Meanwhile, Firefly’s Text to Video and Image to Video models, which generate up to five seconds of video from text prompts or images, will be accessible via the Firefly website. Adobe indicated that the time limit for these generated videos might be extended in the future.

Adobe, known for its creative software, is venturing into the realm of generative AI with Firefly to address the evolving demands for AI-driven content creation. The advent of generative AI tools, such as OpenAI’s Sora and Runway’s Gen-3 Alpha, has revolutionized video creation by producing clips quickly, though these early models often face issues with consistency and reliability in professional settings.

Adobe’s approach aims to distinguish itself through controllability and integration with existing workflows. Ely Greenfield, Adobe’s CTO of digital media, highlighted the strong demand for Firefly’s tools to enhance and expedite creative processes. He noted that Firefly’s generative fill feature in Adobe Photoshop has become one of the most popular additions in recent years.

While Adobe has not yet disclosed pricing details for these new AI video features, its other Firefly tools use a credit-based system for Creative Cloud subscribers, where each credit typically corresponds to one generation result.

In a demonstration, Greenfield showcased the capabilities of Generative Extend. The feature extends a video by predicting and generating the next few seconds based on the last frames, including background noise but excluding voices and music due to licensing constraints. An example included an astronaut video, where the extension was noticeable but seamlessly integrated with the scene’s visual continuity.

The Text to Video and Image to Video features, which will be accessible on [firefly.adobe.com](http://firefly.adobe.com), allow users to create short video clips by providing text or image inputs. Adobe highlights that these tools are particularly adept at generating accurate text within the videos, addressing a common challenge faced by AI models in this domain. This enhancement aims to improve the clarity and usability of generated content, making these tools more practical and effective for users creating video content from textual or visual prompts.

Adobe is adopting a cautious strategy with robust safeguards to prevent the creation of inappropriate or harmful content using its Firefly AI models. These safeguards include built-in blocks that prevent the generation of videos containing nudity, drugs, and alcohol. Furthermore, Firefly's models are explicitly designed to avoid generating content involving public figures, such as politicians and celebrities, unlike some competitors who may include such data in their training sets. This approach aims to ensure responsible use and mitigate potential misuse of the technology.