The creators of a short film based on Sora told the truth about the work of the AI tool

Sora from OpenAI amazed the public with the smoothness and realism of the generated clips. However, what was missing from the debut was that the creators put a huge amount of effort into getting the video to its final form. Shy Kids, the creators of the short film Air head , shared that they used professional productions, complete with robust storyboarding, editing and color grading.

Tech Apr 30, 2024 0 170 Add to Reading List

The creators of a short film based on Sora told the truth about the work of the AI tool

The digital production team behind the short film Air head, Shy Kids, have revealed the truth behind OpenAI's AI tool Sora .

Post-production artist Patrick Cederberg shared that the film was created using professional staging, complemented by robust storyboarding, editing, color grading and post-production work such as rotoscoping and VFX.

Disadvantages of Sora

The team produced a behind-the-scenes video explaining the challenges they encountered while working with OpenAI's Sora.

Shy Kids clarifies that the user interface provided the ability to enter text prompts, which ChatGPT OpenAI then converted into longer sentences that triggered the generation of clips. There are currently no other input options, so the results were always different, even when using the same prompt over and over again.

Generated frame of the short film Air head in original form and quality

To create the scenes in Air Head, the team created several clips based on a common script. However, there was no way for the yellow ball head to remain the same from scene to scene. In some cases, the balloon turned out to be red. In other cases, a face was drawn on the ball.

Generated frame of the short film Air head in original form and quality

The Air Head character known as Sonny often had a string attached to the balloon. Because Sora automatically generated it, the team had to edit the video manually.

Generated frame of the short film Air head in original form and quality

Shy Kids only used generated material from Sora, but much of it had to be sorted, processed, stabilized, and scaled up. The clips were low resolution, so the team used additional AI tools to process them.

The team ended up creating hundreds of generations, each lasting between 10 and 20 seconds, and ended up using only a small portion of them. Patrick Cederberg estimated the ratio of unused to used frames to be 300 to 1.

Many videos were made to appear as if they were shot in slow motion, although this was not indicated in the promo. For this reason, the clips had to be regenerated to create the illusion of being filmed in real time.

The music in the short film was composed by the Shy Kids themselves, and the main character was voiced by Cederberg.

Shy Kids notes that Sora is a new and basic framework, but doubts that the tool in its current form will take the film business by storm and be released in the near future. However, artificial intelligence continues to develop, and Sora regularly has new analogues .