How OpenAI's text-to-video tool Sora could change science – and society – Nature.com
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
You can also search for this author in PubMed Google Scholar
You have full access to this article via your institution.
Sora is one of several AI tools that generates video from text promptsCredit: OpenAI
The release of OpenAI’s Sora text-to-video AI tool last month was met with a mix of trepidation and excitement from researchers who are concerned about misuse of the technology. The California-based company showcased Sora’s ability to create photorealistic videos from a few short text prompts, with examples including clips of a woman walking down a neon-lit street in Tokyo and a dog jumping between two windowsills.
Tracy Harwood, a digital-culture specialist at De Montfort University in Leicester, UK, says she is “shocked” by the speed at which text-to-video artificial intelligence (AI) has developed. A year ago, people were laughing at an AI-produced video of the US actor Will Smith eating spaghetti. Now some researchers are worried that the technology could upend global politics in 2024.
OpenAI, which also developed ChatGPT and the text-to-image technology DALL·E, debuted Sora on 15 February, announcing that it was making the technology “available to red teamers to assess critical areas for harms or risks”. ‘Red teaming’ refers to the process of conducting simulated attacks or exploitation of a technology to see how it would cope with nefarious activity, such as the creation of misinformation and hateful content, in the real world.
Sora isn’t the first example of text-to-video technology; others include Gen-2, produced by Runway in New York City and released last year, and the Google-led Lumiere, announced in January. Harwood says she has been “underwhelmed” by some of these other offerings. “They are becoming more and more vanilla in what they present to you,” she says, adding that the programs require very specific prompts to get them to produce compelling content.
Misinformation is a major challenge for these text-to-video technologies, Harwood adds. “We’re going to very quickly reach a point in which we are swamped with a barrage of really compelling-looking information. That’s really worrying.”
That poses particular problems with upcoming elections, including the US presidential election in November and an impending general election in the United Kingdom. “There will be colossal numbers of fake videos and fake audio circulating,” says Dominic Lees, who researches generative AI and filmmaking at the University of Reading, UK. Fake audio of the leader of the UK Labour Party, Keir Starmer, was released in October 2023, and fake audio of US President Joe Biden encouraging Democrats not to vote circulated in January.
One solution might be to require text-to-video AI to use watermarks, either in the form of a visible mark on the video, labelling it as AI, or as a telltale artificial signature in the video’s metadata, but Lees isn’t sure this will be successful. “At the moment watermarks can be removed,” he says, and the inclusion of a watermark in a video’s metadata relies on people actively researching whether a video they’ve watched is real or not. “I don’t think we can honestly ask audiences across the world to do that on every video they’re looking at,” says Lees.
There are potential benefits to the technology, too. Harwood suggests it could be used to present difficult text, such as an academic paper, in a format that is easier to understand. “One of the biggest things it could be used for is to communicate findings to a lay audience,” she says. “It can visualize pretty complex concepts.”
Another potential use might be in health care, with text-to-video AI able to talk to patients in place of a human doctor. “Some people might find it disconcerting,” says Claire Malone, a consultant science communicator in the United Kingdom. “Others might find it extremely convenient if they want to ask a medical professional questions multiple times a day.”
Text-to-video AI tools such as Sora could help researchers to wade through huge data sets, such as those produced by the European particle-physics laboratory CERN near Geneva in Switzerland and other large scientific projects, says Malone. Generative AI could “sift out code and do the mundane tasks of research”, she adds, but also do “much more sophisticated work [such as] giving it data and asking it to make predictions”.
Concerns have also been raised by people working in creative industries. The US actor Tom Hanks suggested last year that AI could enable him to continue appearing in films “from now until kingdom come” after his death. “If you were a young ambitious actor thinking about their future, and you were told ‘I’m sorry, Tom Hanks is always going to play the leading roles’, would you plan a future in that?” says Lees.
Text-to-video AI will throw up broad issues for society to face. “We’re going to have to learn to evaluate the content we see in ways we haven’t in the past,” says Harwood. “These tools put the opportunity to be a media content creator in the hands of everybody,” she says. “We’re going to be dealing with the consequences of that. It’s a fundamental shift in the way material will be consumed.”
Nature 627, 475-476 (2024)
doi: https://doi.org/10.1038/d41586-024-00661-0
Reprints and permissions
Meaningfulness in a scientific career is about more than tangible outputs
Correspondence
Four years on: the career costs for scientists battling long COVID
Career Feature
Chatbot AI makes racist judgements on the basis of dialect
News
Giant plume of plasma on the Sun’s surface and more — February’s best science images
News
Passion, curiosity and perseverance: my mission to capture women in science on camera
Career Q&A
Mysterious exploding star and more — January’s best science images
News
So … you’ve been hacked
Technology Feature
A year in the life: what I learnt from using a time-tracking spreadsheet
Career Column
How AI is being used to accelerate clinical trials
Nature Index
The Marine Science Program in the Biological and Environmental Science and Engineering (BESE) Division at King Abdullah University of Science and …
Saudi Arabia (SA)
King Abdullah University of Science and Technology
The UBC Department of Physics and Astronomy, invites applications for a three-year lecturer position in physics at the UBC campus in Vancouver.
Greater Vancouver, British Columbia (CA)
Department of Physics and Astronomy – University of British Columbia
UAB Postdoc- Department of Genetics, Dr. Lizhong Wang's Lab The positions of Postdoc are available in Dr. Lizhong Wang’s laboratory within the De…
Birmingham, Alabama (US)
UAB
Memphis, Tennessee
The University of Tennessee Health Science Center (UTHSC)
Memphis, Tennessee
The University of Tennessee Health Science Center (UTHSC)
You have full access to this article via your institution.
An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.
© 2024 Springer Nature Limited
This article was autogenerated from a news feed from CDO TIMES selected high quality news and research sources. There was no editorial review conducted beyond that by CDO TIMES staff. Need help with any of the topics in our articles? Schedule your free CDO TIMES Tech Navigator call today to stay ahead of the curve and gain insider advantages to propel your business!
Advertisement
You can also search for this author in PubMed Google Scholar
You have full access to this article via your institution.
Sora is one of several AI tools that generates video from text promptsCredit: OpenAI
The release of OpenAI’s Sora text-to-video AI tool last month was met with a mix of trepidation and excitement from researchers who are concerned about misuse of the technology. The California-based company showcased Sora’s ability to create photorealistic videos from a few short text prompts, with examples including clips of a woman walking down a neon-lit street in Tokyo and a dog jumping between two windowsills.
Tracy Harwood, a digital-culture specialist at De Montfort University in Leicester, UK, says she is “shocked” by the speed at which text-to-video artificial intelligence (AI) has developed. A year ago, people were laughing at an AI-produced video of the US actor Will Smith eating spaghetti. Now some researchers are worried that the technology could upend global politics in 2024.
OpenAI, which also developed ChatGPT and the text-to-image technology DALL·E, debuted Sora on 15 February, announcing that it was making the technology “available to red teamers to assess critical areas for harms or risks”. ‘Red teaming’ refers to the process of conducting simulated attacks or exploitation of a technology to see how it would cope with nefarious activity, such as the creation of misinformation and hateful content, in the real world.
Sora isn’t the first example of text-to-video technology; others include Gen-2, produced by Runway in New York City and released last year, and the Google-led Lumiere, announced in January. Harwood says she has been “underwhelmed” by some of these other offerings. “They are becoming more and more vanilla in what they present to you,” she says, adding that the programs require very specific prompts to get them to produce compelling content.
Misinformation is a major challenge for these text-to-video technologies, Harwood adds. “We’re going to very quickly reach a point in which we are swamped with a barrage of really compelling-looking information. That’s really worrying.”
That poses particular problems with upcoming elections, including the US presidential election in November and an impending general election in the United Kingdom. “There will be colossal numbers of fake videos and fake audio circulating,” says Dominic Lees, who researches generative AI and filmmaking at the University of Reading, UK. Fake audio of the leader of the UK Labour Party, Keir Starmer, was released in October 2023, and fake audio of US President Joe Biden encouraging Democrats not to vote circulated in January.
One solution might be to require text-to-video AI to use watermarks, either in the form of a visible mark on the video, labelling it as AI, or as a telltale artificial signature in the video’s metadata, but Lees isn’t sure this will be successful. “At the moment watermarks can be removed,” he says, and the inclusion of a watermark in a video’s metadata relies on people actively researching whether a video they’ve watched is real or not. “I don’t think we can honestly ask audiences across the world to do that on every video they’re looking at,” says Lees.
There are potential benefits to the technology, too. Harwood suggests it could be used to present difficult text, such as an academic paper, in a format that is easier to understand. “One of the biggest things it could be used for is to communicate findings to a lay audience,” she says. “It can visualize pretty complex concepts.”
Another potential use might be in health care, with text-to-video AI able to talk to patients in place of a human doctor. “Some people might find it disconcerting,” says Claire Malone, a consultant science communicator in the United Kingdom. “Others might find it extremely convenient if they want to ask a medical professional questions multiple times a day.”
Text-to-video AI tools such as Sora could help researchers to wade through huge data sets, such as those produced by the European particle-physics laboratory CERN near Geneva in Switzerland and other large scientific projects, says Malone. Generative AI could “sift out code and do the mundane tasks of research”, she adds, but also do “much more sophisticated work [such as] giving it data and asking it to make predictions”.
Concerns have also been raised by people working in creative industries. The US actor Tom Hanks suggested last year that AI could enable him to continue appearing in films “from now until kingdom come” after his death. “If you were a young ambitious actor thinking about their future, and you were told ‘I’m sorry, Tom Hanks is always going to play the leading roles’, would you plan a future in that?” says Lees.
Text-to-video AI will throw up broad issues for society to face. “We’re going to have to learn to evaluate the content we see in ways we haven’t in the past,” says Harwood. “These tools put the opportunity to be a media content creator in the hands of everybody,” she says. “We’re going to be dealing with the consequences of that. It’s a fundamental shift in the way material will be consumed.”
Nature 627, 475-476 (2024)
doi: https://doi.org/10.1038/d41586-024-00661-0
Reprints and permissions
Meaningfulness in a scientific career is about more than tangible outputs
Correspondence
Four years on: the career costs for scientists battling long COVID
Career Feature
Chatbot AI makes racist judgements on the basis of dialect
News
Giant plume of plasma on the Sun’s surface and more — February’s best science images
News
Passion, curiosity and perseverance: my mission to capture women in science on camera
Career Q&A
Mysterious exploding star and more — January’s best science images
News
So … you’ve been hacked
Technology Feature
A year in the life: what I learnt from using a time-tracking spreadsheet
Career Column
How AI is being used to accelerate clinical trials
Nature Index
The Marine Science Program in the Biological and Environmental Science and Engineering (BESE) Division at King Abdullah University of Science and …
Saudi Arabia (SA)
King Abdullah University of Science and Technology
The UBC Department of Physics and Astronomy, invites applications for a three-year lecturer position in physics at the UBC campus in Vancouver.
Greater Vancouver, British Columbia (CA)
Department of Physics and Astronomy – University of British Columbia
UAB Postdoc- Department of Genetics, Dr. Lizhong Wang's Lab The positions of Postdoc are available in Dr. Lizhong Wang’s laboratory within the De…
Birmingham, Alabama (US)
UAB
Memphis, Tennessee
The University of Tennessee Health Science Center (UTHSC)
Memphis, Tennessee
The University of Tennessee Health Science Center (UTHSC)
You have full access to this article via your institution.
An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday.
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.
© 2024 Springer Nature Limited
This article was autogenerated from a news feed from CDO TIMES selected high quality news and research sources. There was no editorial review conducted beyond that by CDO TIMES staff. Need help with any of the topics in our articles? Schedule your free CDO TIMES Tech Navigator call today to stay ahead of the curve and gain insider advantages to propel your business!
