This is a post explaining the Canon XF folder structure that’s used by Canon professional video cameras, principally the XF305 and C300 (there may be others).
Some of this relates to Premiere, and how it is vital to import Canon XF clips properly using Media Browser (never using File > Import or drag and drop).
But mostly it is about the way the Canon folder structure and file naming works, which is something I learnt a lot about years ago onsite at the BBC, and have been meaning to share ever since. I’m planning to post about the Sony and Panasonic and Red structures, too.
To be honest, I was always hoping someone else would do it – but nobody ever has. So either nobody else understands it, or the few of us who do have just been hoping someone else would shoot first.
So, here’s the thing:
Professional cameras record their data onto cards in complex folder structures and files.
From our phones and pocket cameras, we’re used to video clips being recorded as single .MOV or .MP4 files which you can copy off and view and edit just by themselves.
This is not the case with video clips recorded by professional Canon, Sony, Panasonic and Red cameras.
This is for various reasons to do with large data rates, card file systems and stupid design by engineers who don’t have to use what they make.
Worse still, there’s a different complex folder and file structure for every camera manufacturer. Multiple different proprietary standards.
You don’t get this kind of bullshit with a consumer product. Consumers wouldn’t put up with it. They shoot a clip, they want a single file for that clip copied to their computer, end of story.
But in this niche, we have to use what we’re given, and there aren’t enough of us to complain about it; still fewer who understand it enough to point out authoritatively how ridiculous it is.
So it continues, and people like us have to suck it up and make it work, even though it costs the production industry millions of hours and dollars in postproduction wastage every year.
And as I start to write this, I’m realising that – because of this insanity – I now have to actually describe what the word “Clip” is going to mean in this post, to avoid confusion.
When I’m talking about a Clip here, I’m talking about the bit of video and audio that the cameraperson recorded between the moment they pressed Record and the moment they pressed stop.
You’d think that this was a perfectly obvious statement – but unfortunately, over in the professional camera dimension, “Clip” is not a single file.
Yes, I guess probably the most important thing to know about a professional camera card format is:
There can be many files and folders that make up a single Clip.
In Canon XF cameras, a clip is not just made up of a video/audio file, it’s also made up of a stack of metadata files, in a complex nested folder structure that must be retained intact for editing & media applications to be able to read it properly.
And worse, if a single clip is recorded for longer than 5 minutes 15 seconds, even just the video/audio part of it will not be made up of one file – it will start to be made up of several separate consecutive video/audio files.
And the problem is that if you don’t have all the right elements in exactly the right place in relation to each other, the card isn’t readable/mountable by any media software, including Premiere. Your cards will no longer work the way they’re supposed to. You won’t be able to view or import your clips.
So a 64GB card can’t be broken up into its constituent elements. It has to remain a 64GB card.
As I said, this madness is not just Canon’s. All the others have their own version of it.
In Panasonic P2 cards, they separate video and audio into separate folders; and then for good measure, they create separate audio files for every channel!
In Red, they split up a single clip into several consecutive video files, with a metadata file alongside.
In Sony, there’s a single video/audio file for each clip (hurray!), but it’s hidden among a forest of metadata files on several levels (boo!).
I’ve been having to explain all this recently to a MAM (Media Asset Management) system manufacturer, whom one of our clients wanted to be able to ingest professional video camera card folder structures, and it reminded me that this information is still just NOT OUT THERE.
Even Canon’s own Canon Professional Network site doesn’t contain it in their special education section – just info about their other less complex formats. I raised it with a few people on their side years ago and tried to get more details direct from engineers, but with no luck. In the end, I could figure most of it out myself, guessing at the purpose of one or two obscurely coded metadata files.
I had to do this because I was supporting the ingest of thousands of hours worth of footage, and was trying to build a script that would let us separate out individual clips, rename them, archive them individually, and then recombine them back into a single card. It was important to try and save money and time in the archiving and restoring of petabytes of data. (See bottom)
After figuring out how all the formats work, I made my script work. But of course, we then couldn’t roll it out as a supported application at the large media companies we support, in case I got hit by a car and nobody else knew how to support it in future. It was not fundable, and too complicated. If you’re interested in it, let me know.
But anyway, for those of you who are in the same position as I was, trying to figure out how a Canon XF card works and how to reconstruct it in order to support post production or fix a problem, this is how it works:
(I’d go and get a cup of tea, if I were you – we’re going to be here a while, and it’s not going to be much fun.)