Beyond the first experiment
I’ve already written about my first forays into AI-powered filmmaking, and how I’ve found the experience unsettling, exciting, and validating of my own creativity.
But I haven’t stopped experimenting. After Radical Speed Month, I took advantage of an AI training workshop to explore how I could use these tools in the brand storytelling context.
During the workshop, I created a short ad for WordPress.com. In parallel, I also developed a set of reusable skills that guide me through the production process, and kept a production diary so I could continue checking in with my feelings on this whole creative-versus-robot situation.
The ad will never see the light of day, but that’s OK! It was just a stepping stone on my journey to hone in on the best parts of AI brand storytelling.
Soon afterward, I had the chance to try out my new suite of skills. I brainstormed an animation idea with a colleague that featured the WordPress mascot, Wapuu, as a lovable character getting into different situations.
This pivot to animation turned my creativity on its head, yet again. I no longer have to struggle with the uncanny valley of humanoid generations that distracted from the storytelling. Instead, I can create an entirely new world, and tap into the emotional pain that our products solve.
The latest video
Here’s my latest video:
Many writers want to get “back to the cursor” — share their thoughts on their own corner of the internet instead of being worried about SEO, analytics, and social media. The WordPress.com Write editor is a distraction-free writing experience that lets writers escape the overwhelm that can accompany writing online.
With this video, I wanted to tap into that feeling of sitting down to write, then being paralysed by the wave of distractions that can blow in. The Write editor clears away all of these distractions, giving a calm place to just … write.
My process
The Write video marks a shift in my AI filmmaking, from solo experimentation to a real project with stakeholders.
Ideating and writing
I started the process with Claude, developing and honing a plugin through each of my experiments. The plugin contains five skills that work through ideation, writing, prompting, and editing.
I use Claude as a sounding board. I chat with it about my ideas, and it can research anything I need it to, whether that’s on the open web or in our internal documentation. It’s more like a writers’ room than a vending machine: more useful than talking to a rubber duck, but falling short of a human.
Image generation
When it came to image generation, I experimented with a few different models. As of the time of writing, ChatGPT still does the best job generating according to my specifications, with legible text and an accurate finger count.
I’ve learned along the way that a character and location reference sheet is critical to consistent generation. I generated a lot of images, created a storyboard, generated even more images, swapped things out, and finessed the story structure.
Video generation
Moving into video, I experimented with platforms like Google Flow, Runway, and Seedance. Through reading reviews, I discovered Higgsfield, a platform that allows switching between models. This is very useful for model comparisons, as well as a seamless way to switch to a new model when it inevitably eclipses the abilities of the last. Using an all-in-one platform like this means I can stay inside the same user account and billing structure, which saves time and admin. Although I still like Kling’s model, I landed on Seedance as it had a slight edge (at the time of writing … always at the time of writing!).
Editing
Once I had all of my clips, I imported them into Final Cut Pro for the edit. I still prefer doing the edit without AI, because I need a tool that gives me more control. Editing in Final Cut Pro allowed me the freedom to change timing of clips, add freeze frames where action needed to pause, and of course titles and sound (the first video’s music track was generated in Suno, and the second was in Remixy).
Another benefit of using non-AI tools in the process is that I don’t need to spend credits for every step. In particular, editing requires a lot of tweaking, and I can do as much as I want without forking over each time.
During the edit, I sometimes realised that something wasn’t working, and headed back to Higgsfield (or even a step further back, to ChatGPT) to fix it. Occasionally, I found myself needing to do some image edits in Photoshop so they would animate in a cleaner way. Sometimes I ditched whole beats of the story when they weren’t working. The editing process is truly where it all comes together.
Approval
And then, I presented the video to the stakeholders — marketers, product leads, designers, and storytellers. I got invaluable feedback that pushed the project into a better, more refined, more polished version.
The next video
Right after I wrapped the Write video, I created another one to promote WordPress.com’s site editing tools. Editing website templates, headers, and footers used to be challenging, especially for non-technical users. So I wanted to highlight the delightful way our tools make site editing easy.
What’s next
I still have mixed feelings about using AI in creative work. But as my experiments continue, I’ve also been learning how these tools fit into my creative practice, and developing a better sense of where my boundaries are. Surprisingly, I actually use analog media for artmaking more often now, than before I started using AI! Maybe it’s something about touching grass …
Since finishing these two videos, I’ve dabbled in using Seedance to animate my own illustrations (you might see those in an upcoming YouTube video). This is super interesting to me, because it allows me to keep the truly enjoyable part (drawing), but have AI handle the tedious and difficult part (animating). Which each project, I feel like I’m honing in on the best way to use the tools as an extension of my creativity.
Before AI, I wouldn’t have had the budget to make these kinds of videos. Now, I can experiment, tweak, rework, and ship for a fraction of the time and the cost. The idea -> reality pipeline has been dramatically shortened.
But there are still problems. AI continues to fall into (at best) clichéd and (at worst) stereotyped patterns for storytelling. It needs a creative with enough storytelling experience to catch these tropes and steer them into less charted territory.
AI still resists divergent thinking, and therefore creates hollow output without a lot of guidance. I never lose sight of the fact that these are still a set of tools to achieve a creative vision, and are not capable of having a creative vision of their own.
I continue to interrogate my own experience of using these tools, and how they interact with my favourite parts of the creative process. Categorising a tool as a loom, slide rule, or crane depends on the creative’s relationship to the task.
I’d rather create my own images than generate them, and that’s why I’m now moving into animating my illustrations. But animation is really hard, time-consuming, and expensive. I would prefer not to spend my time tweaking every frame. I love editing the videos myself, because it gives me such granular control over the finished product.
At each stage, I stay in touch with my creativity by asking: “does this bring me joy?”
Leave a Reply