OpenAI announced its text-to-video model, Sora, which can create realistic and imaginative scenes from text instructions.
Initially, Sora will be available to red teams to assess potential damages or risks in critical areas, which will not only improve the safety and security features of the model, but also allow OpenAI to incorporate the perspectives and expertise of cybersecurity experts.
Access will also be extended to visual artists, designers and filmmakers. This diverse group of creative professionals was invited to test and provide feedback on Sora, to refine the model to better serve the creative industry. Their insights are expected to guide the development of features and tools that will benefit artists and designers in their work, OpenAI said in blog post which contains additional information.
Sora is a sophisticated AI model capable of creating intricate visual scenes that contain numerous characters, different types of movement, and detailed representations of both subjects and their backgrounds.
Its advanced understanding goes beyond simply tracking user queries; Sora interprets and applies knowledge of how these elements naturally occur and interact in the real world. This capability enables the generation of highly realistic and contextually accurate images, demonstrating the deep integration of artificial intelligence with an understanding of the dynamics of the physical world.
“We are working with red teams — domain experts in areas such as misinformation, hateful content and bias — who will challenge the model. We’re also building tools to help detect deceptive content, such as a detection classifier that can detect when Sora generated the video. We plan to include C2PA metadata in the future if we implement the model into an OpenAI product,” OpenAI said in the post. “In addition to developing new techniques to prepare for deployment, we are leveraging existing security methods we’ve built for our products that use DALL·E 3, which also apply to Sora.”
OpenAI has implemented strict content moderation mechanisms within its products to maintain compliance with usage rules and ethical standards. Its text classifier can scrutinize and reject any text input requests that require content that violates these rules, such as extreme violence, sexual content, hateful images, celebrity likenesses, or intellectual property violations.
Similarly, advanced image classifiers are used to review each frame of generated videos, ensuring they conform to set usage policies before they are shown to users. These measures are part of OpenAI’s commitment to the responsible implementation of artificial intelligence, with the aim of preventing misuse and ensuring that generated content complies with ethical guidelines.