OpenAI's new text-to-video AI called Sora made big waves this week with its ability to generate photorealistic video clips. The samples showed impressive scenes like people walking through snow and camera shots following a vintage SUV.
This appears to be a major advance for generative AI, with potential far beyond just video. OpenAI refers to Sora as a "world simulator" that can understand important 3D aspects of the physical world. It can output CGI-like digital landscapes or videos of people in real settings.
As OpenAI states, "Our results suggest scaling video generation models is a promising path towards building general purpose simulators of the physical world."
Sora emerges from evolving diffusion transformer models previously used for high-res image generation. It was trained on large amounts of captioned video to link footage with text prompts.
Beyond creating new videos, Sora can extend existing clips or make video from AI images. During development, researchers saw "emergent capabilities" like simulating people, animals, and environments.
The generated videos have remarkably smooth camera moves in 3D space, suggesting strong understanding. OpenAI even hints Sora could grow into a platform for gaming and "highly-capable simulators."
However, Sora still has limitations, like not fully capturing cause-and-effect. It also lacks safeguards against misuse. So OpenAI is slowly rolling it out to assess risks and harms. Despite limitations, Sora foreshadows a future where AI-generated video becomes indistinguishable from real footage.