AI Transparency

Once we know roughly what kind of creature might belong to a song, we have a new problem: knowing that it is a frog, an octopus or a camel tells us surprisingly little about who that artist actually is.

This is usually where visual generative AI enters the process. We give an image model the first description of the character, its musical world, attitude, appearance and whatever else we already know. The result gives us something to react to. Sometimes it is remarkably close. Sometimes it is a frog wearing sunglasses and absolutely nothing more.

And that is where the actual character development begins...

Shaping the DNA

We iterate. We change proportions, expressions, clothes, colours and materials. We inpaint details with other tools, replace things that do not belong, add accessories and gradually establish the visual DNA that makes the character recognisable across completely different situations.

Because asking an image model for “a cool frog musician” can certainly produce a cool frog musician. It just won’t produce Rana Mala.

The same goes for personality. What would this artist wear on stage? How do they move? Are they elegant, ridiculous, intimidating, hopelessly vain or completely unaware of how cool they are? What happens when the cute little octopus from the first character sketch gets bracelets, heart-shaped sunglasses, a stage full of lights and discovers that she rather enjoys twerking?

At some point we are no longer designing an animal. We are designing an artist.

Past & Curious

Every artist needs a past: Where did they grow up? What music influenced them? What happened before ZooTube found them? What are they proud of? What went horribly wrong? Which rumours are true, which ones are nonsense, and which ones should probably not be discussed while the lawyers are present?

Language models are excellent creative sparring partners here. We provide the character, direction and context, then use AI to explore possibilities, develop stories and occasionally throw something wonderfully strange into the room that none of us had considered.

That is one of the things we like most about generative AI. Used well, it does not merely execute instructions. It can react to an idea and send something unexpected back. We still decide what belongs in the Zoo. But we do not necessarily want to know every answer before we ask the question.

Eventually the artists also begin to exist in relation to one another. Who gets along? Who secretly hates whose music? Who would share a stage? What does the cockroach think of all this? Those relationships feed back into biographies, visuals, videos and sometimes even the music itself.

Lyrics

Lyrics usually begin much like the artist does: with us. The basic idea, subject, attitude, story, hook or wonderfully questionable sentence comes first. From there, language models can help turn that raw material into something that actually wants to be sung.

That can mean working on rhyme schemes, internal rhymes, meter, syllable count, phrasing, repetition, lyrical density or simply finding a cleaner way to say something without murdering the original idea in the process.

Could we write lyrics without AI? Of course. Frogfather is a musician and has been writing songs since long before anyone thought a chatbot needed a record deal — from songs written with a guitar to complete studio productions.

So why use AI? Because that is rather the point of this entire project.

Music Generation

Once the lyrics and musical direction are ready, generative music tools enter the room. They do not simply receive a request for “a good song”. We work with ideas for genre, instrumentation, arrangement, vocal character, dynamics, intros, verses, choruses, bridges, breaks, drops, solos and all the little decisions that make one version feel right and another one feel like it should quietly leave the building.

Then the iterative part begins.

Sometimes we get surprisingly close within a handful of generations. Sometimes it takes dozens. Sometimes we are still changing individual sections, instruments or performances after a hundred attempts. There is no magic number because the goal is not to generate something. The goal is to find the version that belongs to the song.

In that sense, the process is not entirely alien to traditional music production. Give a session guitarist an idea for a riff and a good musician will not simply reproduce the notes living inside the producer’s head. They interpret them. They bring their own instincts, experience and accidents into the room. Sometimes the result is wrong. Sometimes it is better than the idea you started with.

Generative AI can behave in a surprisingly similar way. We give direction, listen to what comes back, keep what works, reject what does not and adjust the instructions. The difference is that our session musician can apparently play every instrument, never sleeps and occasionally develops a complete misunderstanding of what a bridge is.

Eventually we have something that sounds like a finished song.

Which is usually when we take it apart again.

Eventually, we have a generated version that contains the song we were looking for. This is where generation ends and a rather different kind of production begins.

The first thing we usually do with our apparently finished song is take it apart.

Vocals, drums, bass, guitars, keyboards and whatever else has survived the journey are separated into stems. “Stems” deserves slightly generous quotation marks here: these are not clean multitracks recorded separately in a studio. They are individual parts extracted from an already mixed piece of audio, complete with bleed, artefacts and occasional evidence that the machine had ideas of its own.

From dusk till DAW

Those parts move into the DAW, where we can finally work on the track at a much more detailed level. Sections get edited or rearranged. Levels and dynamics change. Parts disappear. Transitions are rebuilt. A chorus may need more weight, a verse less clutter, or that guitar we loved in the second half may suddenly reveal that it should have been there twenty seconds earlier.

When something is missing, we make it.

That might mean returning to a generative music tool and creating new variations of a particular instrument or section. It might mean using another specialised AI tool. Or it might mean staying completely offline and reaching for synthesizers, samplers, software instruments, recorded instruments or one of the professional sound libraries whose invoices have caused the Executive Producer considerable emotional distress.

Sometimes the changes are small. Sometimes the arrangement moves far enough away from the generated version that we end up with several distinct productions: a radio edit, an extended version, a live interpretation or something else the song decided it wanted to become.

Fix the Mix!

Then comes the particularly entertaining problem of mixing it.

Generated audio arrives with very different baggage from traditionally recorded multitracks. It may already be heavily compressed. Frequency information can be limited. Instruments can bleed into one another. Separation can introduce artefacts. A vocal might sound fantastic until you discover that a small piece of the snare drum has apparently moved into the singer.

So the usual studio toolbox still applies — EQ, compression, saturation, automation, spatial processing, restoration, layering and plenty of listening — but the way we use it changes. Sometimes we are polishing a sound. Sometimes we are repairing it. Quite often we are doing both.

The final mix is then mastered like any other ZooTube release: balancing tone, dynamics, impact, translation and loudness until the track works outside the studio as well as it does inside it.

This is also where the human ears become rather difficult to replace. AI can generate endless possibilities. It cannot decide which version belongs on the record. That decision — including the deeply scientific process of listening again tomorrow because everything sounded better at 2:17 a.m. — remains ours.

Only then do we have a master.

A finished song creates another problem: sooner or later, somebody has to make the animals move.

Our videos use generative AI extensively, but the process starts long before anything moves. We develop the concept, scenes, locations, performances, visual references and camera ideas first. What is the artist doing? Where are they doing it? What does the scene need to communicate? And, occasionally, why did somebody decide that an octopus should perform this particular dance move?

From there, the video gradually gets built shot by shot rather than generated as one large magical object.

Setting the Stage

Most shots begin as still images. We place the artist in the right environment, develop wardrobe, lighting, pose, expression and composition, and keep refining until the frame belongs to the same visual world as everything around it.

That often involves several different generative and editing tools. Parts are regenerated, inpainted, extended or rebuilt. Backgrounds change. Hands get repaired. Accessories disappear and return. A character who looked exactly right five minutes ago suddenly develops a completely different face because apparently consistency is a luxury item.

The important part is that our artists already have an established visual DNA. We are not asking a model to invent Rana Mala again for every shot. We are trying to make the Rana Mala we already know exist inside a new scene.

Making It Move

Once a frame works, we can start thinking about motion. That means performance, camera movement, timing, environmental animation and all the small details that stop a five-second clip from looking like a photograph having a nervous episode.

We give video models specific direction: how the artist should move, what the camera should do, what should remain still, how quickly an action should happen and which parts of the frame are absolutely not invited to mutate.

Sometimes the first generation works. More often, one thing works while another does not. The camera move is perfect but the character changes. The performance is great but the stage rearranges itself. The lights behave beautifully while a guitar quietly becomes a different guitar halfway through the shot.

So we generate again. Adjust. Rebuild the start frame. Create an end frame. Change the motion. Try another model. Keep the useful accident. Throw away the expensive disaster.

A few seconds of finished footage can therefore represent dozens of generations and a surprising amount of very human decision-making.

Cut. Grade. Repeat.

Generated clips are still only source material. The actual video comes together in post-production.

Shots are selected, trimmed and edited to the music. Timing gets tightened. Continuity problems are hidden or repaired. Clips may be stabilised, retimed, reframed or composited. Colour grading brings material from different models and generations into the same visual world.

Titles, graphics, subtitles, lyric timing and final mastering happen here too, using the same conventional editing and motion-design tools we would use on any other production.

And just like with the music, AI can provide an effectively endless supply of possibilities. The useful skill is not producing more of them. It is knowing which five seconds deserve to survive the edit.

Only then do we have a video.

We hope you enjoyed seeing the Zoo from the less polished side for a change.

AI can be used to make disposable tracks, rushed visuals and suspiciously enthusiastic bot armies at remarkable speed. It can also be used very differently.

For us, that is where things get interesting: new tools, new possibilities and fewer barriers between a strange idea and actually making the thing. More room to experiment, fail spectacularly, try again and create things that might otherwise never exist.

There is another pleasant side effect too. Character design, animation, elaborate video worlds and productions like these can easily require budgets reserved for the biggest players in the music industry.

We would rather invest those millions in food for the Zoo.