![\"A](\"https:\/\/reshebniki-online.com\/wp-content\/uploads\/2024\/02\/GettyImages_1249183770.0.jpg\")
OpenAI\u2019s Sora is designed to be a \u201cworld simulator.\u201d Right now it\u2019s having trouble breaking a glass.<\/p>\n
A tiny fluffy monster kneels in wonder beside a lit candle. Two small pirate ships battle inside a churning cup of coffee. An octopus crawls along the sandy floor of the ocean. A Dalmatian puppy leaps from one windowsill to another. These are among a series of demo videos of OpenAI\u2019s Sora, revealed last week, which can turn a short text prompt into up to a minute of video. <\/p>\n
The artificial intelligence<\/a> model is not yet open to the public, but OpenAI has released the videos, along with the prompts that generated them. This was quickly followed by headlines calling Sora \u201ceye-popping\u201d <\/a>and \u201cterrifying<\/a>\u201d and \u201cjaw-dropping<\/a>.\u201d <\/p>\n OpenAI researchers Tim Brooks and Bill Peebles told the New York Times that they picked \u201csora,\u201d Japanese for \u201csky,\u201d to emphasize the \u201cidea of limitless creative potential.\u201d There is another term, though, that OpenAI uses to describe Sora: a potential \u201cworld simulator,<\/a>\u201d one that, over time, could create \u201chighly-capable simulators of the physical and digital world, and the objects, animals and people that live within them.\u201d <\/p>\n It\u2019s not there yet. While the available demo videos of Sora at work can feel uncanny and realistic, OpenAI\u2019s technical paper on the model notes its many \u201climitations.\u201d While Sora can sometimes accurately represent the changes on a canvas when a paint-laden brush sweeps across it or create bite marks in a sandwich after showing a man taking a bite, Sora \u201cdoes not accurately model the physics of many basic interactions,\u201d such as a glass breaking. People and objects can spontaneously appear and disappear, and like many AI models, Sora can \u201challucinate.\u201d <\/p>\n Some AI experts, like Gary Marcus, have raised doubts<\/a> about whether a model like Sora could ever learn to faithfully represent the laws of physics. But just as DALL-E and ChatGPT improved over time, so could Sora. And if its goal is to become a \u201cworld simulator,\u201d it\u2019s worth asking: What is the world that Sora thinks it\u2019s simulating? <\/p>\n OpenAI has made that question kind of tough to answer, as the company has not disclosed much<\/a> about what data was used to train Sora. But there are a couple of things we can infer. First, though, let\u2019s look at how Sora works. <\/p>\n Sora is a \u201cdiffusion transformer,\u201d which is a fancy way of saying that it combines a couple different AI methods in order to work. Like many AI image generators (think DALL-E or Midjourney), Sora creates order from chaos based on the text prompt it receives, gradually learning how to turn a bunch of visual noise into an image that represents that prompt. That\u2019s diffusion. The transformer bit has to do with how those still images relate to each other, creating the moving video. And Sora, OpenAI says, is designed to be a video-generating generalist. <\/p>\n In order to do this, Sora would need a lot of data to learn from, reflecting a wide variety of styles, topics, duration, quality, and aspect ratios. OpenAI said in its technical paper that its development \u201ctakes inspiration from large language models which acquire generalist capabilities by training on internet-scale data.\u201d While not directly saying this, it\u2019s probably safe to guess that Sora, too, learned from some training data that was taken from the internet. <\/p>\n It\u2019s also possible, argued Nvidia AI researcher Jim Fan, that Sora was trained on a data set that incorporates a large amount of \u201csynthetic\u201d<\/a> data from the latest version of Unreal Engine, a 3D graphics creation tool that is best known for powering the visuals in video games. OpenAI also has some agreements with companies that could provide large amounts of data for training purposes, like Shutterstock<\/a>. As for the data that OpenAI did not, in the past, use with the agreement of its creator or publisher, well, there are some pending copyright lawsuits<\/a>. <\/p>\n AI bias is not new, and as Vox has explained before,<\/a> it can be tough to combat. It creeps into training data and algorithms that power AI models in a lot of different ways. Since we don\u2019t know what data Sora was trained on, and the tool is not available for the public to test, it\u2019s hard to speak in much detail about how biases might be reflected in the videos it creates. <\/p>\n Sam Altman, OpenAI\u2019s CEO, has said that he believes AI will eventually learn to rid itself of bias.<\/p>\n \u201cI\u2019m optimistic that we will get to a world where these models can be a force to reduce bias in society, not reinforce it,\u201d he said to Rest of World<\/a> last year. \u201cEven though the early systems before people figured out these techniques certainly reinforced bias, I think we can now explain that we want a model to be unbiased, and it\u2019s pretty good at that.\u201d <\/p>\n AI bias and ethics experts like Timnit Gebru have argued that this is exactly what people should not trust AI companies to do, telling the Guardian<\/a> last year that we shouldn\u2019t simply trust AI systems, or the people behind them, to self-regulate harms and bias. <\/p>\n A lot of the praise for Sora\u2019s demo videos stems from their realism. And that\u2019s exactly why disinformation experts are concerned here. <\/p>\n A new study indicates that AI-generated propaganda <\/a>created by GPT-3 (i.e., not even the newest GPT model powering the current generation of AI tools) can be just as persuasive as human-written content and takes a lot less effort to produce. Now apply that to video. Even without being able to faithfully replicate Earth physics, there are plenty of ways that a tool like Sora could be used, right now, to hurt and mislead people. <\/p>\n \u201cThis is definitely slick, but I see two main uses: 1) to sell people more stuff (via ads) 2) to make non-consensual\/misleading content to manipulate or harass people online,\u201d wrote Sasha Luccioni, an AI research scientist at HuggingFace, on X<\/a>. \u201cGenuine question – why is everyone so excited?\u201d <\/p>\n OpenAI announced Sora a couple weeks after a wave of explicit, nonconsensual deepfakes of Taylor Swift circulated on social media. The images, as 404 media reported<\/a>, were created with AI by exploiting loopholes in the systems that are designed to prevent exactly this from happening. <\/p>\n To address potential biases and misuses of Sora, OpenAI is allowing only a small group of testers to evaluate its safety risks: \u201cWe are working with red teamers \u2014 domain experts in areas like misinformation, hateful content, and bias \u2014 who are adversarially testing the model,\u201d the company said in a statement on X<\/a>. <\/p>\n Underneath all this are concerns about what Sora and other tools like it will do to the livelihoods of creative professionals, whose work has been used \u2014 often without payment \u2014 to train AI tools in order to approximate their jobs.<\/p>\n Altman, on X, was taking follower suggestions for new Sora videos in order to show off glimpses of our glorious future, which will evidently be these AI-generated podcasting dogs: <\/p>\nUnknown worlds<\/h3>\n
Biased worlds <\/h3>\n
Made-up worlds <\/strong>
\n<\/h3>\nA world with podcasting AI dogs, I guess <\/strong>
\n<\/h3>\n