When AI starts to understand the language of shots

AltNext 2025 Featured Talk

The Age of AI Directors: When Large Models Master the Grammar of Montage

As large models begin to understand the logic of camera movement and the grammar of montage, is visual storytelling entering a completely new stage? Will directors be replaced or redefined?

At AltNext 2025, Shengshu Technology CEO Luo Yihang and director Cheng Liang held an in-depth conversation on the core topic of the “AI Director Era.” Their discussion ranged from whether AI can truly “understand” cinematic language, to how AI-generated shot lists and storyboards are reshaping creative workflows, to the boundaries of human–machine collaboration, and whether upgraded tools imply deeper shifts in creative power and narrative methods.

Before the sharing began, AltNext organized a special live AI visual creation showcase.

On the day before the event, director Liang Cheng provided an experimental short film prompt. Using the latest Vidu model, the Shengshu Technology team completed the entire production within a limited timeframe, from character design and shot generation to final video output, and premiered the result live at the conference.

This real-time generated short film not only offered a direct demonstration of the latest progress in generative video for visual storytelling and camera orchestration, but also brought the question “Is AI entering professional visual production workflows?” into concrete work that could be viewed and discussed. Building on this showcase, director Liang Cheng and Shengshu Technology CEO continued with an in-depth conversation on AI directors, cinematic language, human–AI collaboration, and the future forms of image creation.

‍

If large models master camera movement and montage grammar, will directors be “replaced”?

Liang Cheng

Theoretically speaking, that makes you our gravediggers. It feels like the technological force we fear the most, right?

From what I observe so far: for general commercial advertising content, AI is already good enough. But for very deep long-form like narratives films and series, AI is still not enough for now.

Yihang Luo

I don’t define it as an “AI director,” but rather as an AI partner to directors. It’s more like an executive collaborator, like an AI executor or AI renderer.

I see creative work as having several dimensions: first, creativity and story; second, visual presentation; third, production efficiency; fourth, production cost.

Creativity and story must still come from directors and human creators, which AI cannot replace. AI generates and imitates based on learning, but it does not truly understand the artistic meaning behind the cinematic language.

However, in terms of visual quality, efficiency, and cost, AI can be greatly improved. A production cycle that once took a year might be shortened to six months or even one month, and costs could drop to a fraction of what they were. AI’s key value lies in these dimensions.

In my definition, AI is a partner and co-worker. Humans still lead the creative process, while AI helps translate imagination into visible form more effectively. Just like the evolution of film technology from shooting techniques to virtual production to today’s generative AI, this is essentially a tool upgrade, not a replacement of creators.

What stage has AI reached in understanding cinematic language?

Liang Cheng

I’m actually quite surprised. AI can now translate our shot breakdowns very accurately and show a strong understanding of camera movement.

It took me over twenty years in the industry to gradually understand how and why a camera should move from one position to another, but your systems have quickly grasped many of these patterns. Of course, these are mostly the more commercial and formulaic parts of visual language, like advertising and Hollywood style cinematography. But even styles like Cannes new wave, French handheld aesthetics, breathing-style camera movement, and more raw editing rhythms can now be imitated.

Yihang Luo

Now we mainly focus on commercial-grade output. I divide AI video generation into three levels: entertainment-grade, commercial-grade, and professional-grade. Entertainment-grade supports mass expression. Commercial-grade includes advertising, animation, and brand content. Professional-grade refers to theatrical films.

The technology already supports entertainment-grade use and is moving toward commercial-grade. But strong work is not only about camera movement. It also involves character emotion, eye expression, and subtle performance details. In these areas, machines still cannot compare with real actors.

How will AI-generated shot lists and storyboards change the director’s workflow?

Yihang Luo

Previously, between script and shooting, there were written scripts and hand-drawn storyboards. Now AI video generation can directly produce storyboard previews. You can input photos of the main character or scenes, combine them with shot descriptions as prompts, and generate dozens of shot variations within minutes.

Directors and actors can quickly build shared understanding through these video references. They are not necessarily the final output, but serve as a discussion medium and transitional layer that helps teams align their imagination faster.

Liang Cheng

Commercial storyboards tend to follow patterns, and we used to rely on storyboard artists to draw them. Sometimes that felt painful to me, because I understood the script deeply but had to depend on others to visualize it. If AI can pre-visualize storyboards, it’s a major help for some directors.

Why create an animated version instead of a live-action AI version?

Yihang Luo

If you feed a full script directly into a large model to generate video, character consistency across different shots is hard to maintain, the protagonist may look different in every shot. That’s a probabilistic issue. Commercial and professional productions require high consistency, so we chose an animated version this time it’s more controllable and better suited to the current stage of the technology.

Liang Cheng

That’s a smart approach. It also makes me think about why audiences are increasingly accepting virtual characters. Real human faces are specific and can limit imagination, while animated characters leave more room for projection. Everyone can form their own interpretation. Virtual characters make emotional projection easier and allow for greater tolerance.

Will AI-generated films replace live-action cinema, or will both coexist long-term?

Liang Cheng

I believe fully AI-generated feature films and series will definitely appear in the future, but not necessarily as replacements for live-action. A director’s real value lies in capturing human emotion and expressing humanity which is very hard for machines to fully replace.

Yihang Luo

Breakthroughs in creative boundaries always come from humans. Human imagination is limitless — something AI cannot replace. AI’s greater importance is in assisting people by taking over tedious execution work, allowing creators to focus on creativity itself. The key is human–machine collaboration, not replacement.

From storyboard to video production, from execution to collaboration, AI is rapidly entering the visual production pipeline. Yet the core of AI-driven image creation still lies in the understanding of character, emotion, and humanity.

Technology can accelerate expression, but it cannot replace the creator’s judgment of meaning.

The “AI Director Era” is not about directors being replaced, but about creative methods being expanded by technology. As machines take on more execution work, humans can devote more energy to imagination.