Mark Horrobin on Grading and AI

Let’s talk about AI…some more!

The recent explosion of AI-related content, particularly pieces outlining specific workflows and techniques, is truly remarkable. It's mind-boggling to think that just 18 months ago, AI was largely theoretical, a distant promise on the horizon. Now, it's an undeniable force, rapidly integrating into creative industries and redefining how we approach our craft.

In any discussion about the headwinds facing our industry, AI stands out as the dominant force. Conversations swing wildly, from legitimate concerns about job losses and widespread copyright infringement to the exhilarating possibilities of new creative avenues and unprecedented cost efficiencies. Beyond the practicalities, there's also the profound, existential question: can ?

It’s clear that every discipline within the creative landscape is being affected to some degree, and colour grading is certainly no exception. We’ve reached a pivotal point where AI-generated content is increasingly making its way to the grading desk, like a snowball gaining momentum as it rolls downhill.

The speed at which AI is developing means that any observations made today could be redundant tomorrow. However, based on recent projects, some current insights might still prove useful.

Let’s start with the tools themselves, which I view as a net positive. AI and machine learning are significantly augmenting the colourist’s ability to achieve their desired creative objectives. Software developers have wholeheartedly embraced this technology, incorporating AI applications into their products. These integrations demonstrably improve efficiency for the colourist, streamlining once-arduous tasks and allowing for greater time to focus on nuanced detail and creative sculpting during a session.

Consider the ability to quickly roto a shot – a complete impossibility in the not-so-distant past, now a remarkable reality in the grading session. Generated sky replacements are another recent, powerful addition to the grading suite. Furthermore, third-party developers are creating open FX plugins for a multitude of time-consuming tasks, such as shot matching. While there’s still room for refinement with all these new additions, cumulatively, these developments free up the colourist to truly sculpt the image and realise its full creative potential, rather than being bogged down by technical grunt work.

However, working with AI-generated footage presents its own unique set of challenges. The source material is generally supplied as compressed MP4 files, which can lead to artifacts and noise. That said, it’s surprisingly encouraging how much information can be recovered from these files, allowing for truly creative and stylized results.

In fact, AI footage often requires as much, if not more, work for colourists compared to traditional live-action shots. The work becomes incredibly granular, demanding deep use of texturing and depth-of-field tools to seamlessly integrate the two formats. Combatting the often-criticised 'plastic' look, correcting inconsistent lighting and colours, and recovering or reducing details become the primary focus as the colourist strives to elevate the AI-generated elements to a more natural and coherent state. Our recent project for MG, which was a mix of live-action and AI-generated shots, perfectly illustrated this. We ended up spending considerably more time bedding in the AI footage than grading the live-action segments. The final results were excellent, but they were achieved through much hard work, meticulous attention to detail, and a somewhat different thought process compared to established grading practices.

As mentioned earlier, the astonishing pace of technological advancement means these observations may have a very short shelf life. Future improvements in AI footage, such as the adoption of log formats and better codecs, will undoubtedly further enhance creative possibilities. And if there’s one thing we can be absolutely sure of, it’s that there will be future improvements. The landscape is constantly shifting, and staying abreast of these changes is paramount for anyone working in this dynamic field.