OpenAI Launches Sora, a Video-Generating Tool, but Holds Back on Key Feature
On Monday, OpenAI launched its video-generating tool, Sora, which allows users to create videos using text prompts or images. However, the company has decided not to release a key feature for most users pending further testing.
The Held-Back Feature: Using Real People as References
The feature in question generates a video using an uploaded photo or footage of a real person as a reference. OpenAI says that it will give a "subset" of Sora users access to this feature, but it won’t roll out the capability broadly until it has had a chance to fine-tune its approach to safety.
Potential for Misuse
OpenAI is taking a cautious approach to this feature because of the potential for misuse. The company wrote in a blog post that "the ability to generate a video using an uploaded photo or video of a real person as the ‘seed’ is a vector of potential misuse that we are taking a particularly incremental approach toward to learn from early patterns of use."
Balancing Creativity and Safety
Rohan Sahai, Sora’s product lead, stated during a livestream presentation that "we obviously have a big target on our back as OpenAI, so we want to prevent illegal activity with Sora, but we also want to balance that with creative expression." He acknowledged that this would be an ongoing challenge and encouraged users to provide feedback if moderation doesn’t quite get it right.
The Controversy Surrounding Generative Video
Generative video is a powerful tool, but it’s also a controversial one. The use of deepfakes and misinformation has become a significant concern in recent years. According to data from ID verification service Sumsub, deepfake fraud worldwide increased by more than 10 times from 2022 to 2023.
OpenAI’s Measures to Prevent Misuse
To prevent misuse, OpenAI is taking several measures:
- A filter detects whether a generated video depicts someone under the age of 18 and applies a "stricter threshold" for moderation related to sexual, violent, or self-harm content.
- All Sora-generated videos contain metadata to show their provenance, specifically metadata that abides by the C2PA technical standard. This metadata can be removed, but it’s intended as a way for platforms to verify the authenticity of the video.
The Future of Generative Video
OpenAI’s decision to hold back on this feature highlights the challenges and complexities surrounding generative video. As technology continues to evolve, it’s essential to balance creativity with safety and ensure that these tools are used responsibly.
What’s Next for Sora?
As OpenAI continues to refine its approach to safety, it will be interesting to see how Sora develops and evolves. With the launch of this tool, OpenAI is opening up new possibilities for creators and users alike. However, it’s crucial that these innovations are accompanied by robust safeguards to prevent misuse.
Conclusion
The launch of Sora marks an exciting step forward in the world of generative video. While there are challenges to be addressed, OpenAI’s commitment to safety and creativity is a promising sign for the future of this technology. As users begin to explore the capabilities of Sora, it will be essential to monitor its development and provide feedback on how to ensure responsible use.
Related News