Cessen's Ramblings

2008 - 04 - 01

A Complete Audio/Video Pipeline On Linux

I accomplished something exciting tonight. At least, exciting for me. You see, one of the very last things that was keeping Windows on my computer was the lack of good audio and video tools for Linux. Being a film maker at heart, there's no way I could abandon the tools that let me make movies.

Tonight, I finally patched together an entire audio/video pipeline that takes me all the way from video capture and audio recording, to video editing and audio mastering. Not that I did this all in one night. I've been looking for these tools for a long time, and slowly collecting software and taking note of what works and what doesn't. But tonight the last piece of the puzzle fell in place.

The tools I've assembled are as follows:

I don't want to bore you too much with the details, but here's a quick run-down of what role each of these play:

Kino is for video capturing. (In theory it's also supposed to do video editing, but I've never found it useful for that.) Audacity is for audio recording and sound cleanup (noise removal etc.) Blender is for video editing (I find it amusing that the best open-source video editor is primarily a 3D animation program...). Ardour and xjadeo are for audio- sequencing/mixing/mastering/etc. Ardour does the audio, and xjadeo displays video in sync with Ardour. JACK is a little difficult to explain, but it basically links audio software together in cool ways. For example, it's the intermediary that Ardour and xjadeo use to stay in sync. But it can do much much more. ffmpeg is for converting video and audio files to different formats, and for doing final muxing and encoding.

Now it's important to note that some of these programs are not for the faint of heart, and many of them require command-line knowledge to use effectively. For me that's perfectly fine, but for less technically inclined people that's a serious problem. But this pipeline is for me. So... wootz.

For those who are curious about a few of the details, read on.

Ardour, JACK, and xjadeo were the final pieces that I finally put together tonight. I've known about Ardour and JACK for a long time, and have tried to get them up and running countless times. Every time ended in me giving up, frustrated. But tonight, on a whim, I decided to try again. And I did so with a different tactic: a command line tactic. Turns out it's simpler (at least for me) to get JACK up and running from the command line. The GUI for it was cluttered and extremely confusing/frustrating, but the command-line was dead simple. I think a combination of that and progress in Linux audio since the last time I tried contributed to my success this time around. I had minimal trouble getting it up and running, and now that I've done it once it will be a snap in the future. Ardour is a fairly full-featured piece of DAW (Digital Audio Workstation) software, and will more than fill my needs. I'm especially happy with how flexible it is about buses and routing audio signals. It basically lets you plug anything into anything (wootz!). It also has support for LADSPA audio plugins (and VST plugins if you're willing to compile it from source -- which I am). Its UI is going to take some getting used to, but already after a couple of hours playing around with it I'm starting to feel reasonably comfortable with it.

Ardour alone isn't enough for doing audio work for video, however. You also need to be able to edit the audio to video -- synchronized. So after getting Ardour up and running, I immediately started searching for a way to display video synchronized with it. It wasn't long before I stumbled upon xjadeo. Xjadeo is a super-simple video display application, designed for the sole purpose of syncing up with audio applications via JACK. In fact, it is so specific to that purpose that it doesn't even have a play button. It literally does nothing except display video synced up with other audio applications. And it's not specific to Ardour, either. It will sync up with any JACK-compatible audio application that can emit SMPTE timing data. So if I find other audio apps to add to my pipeline, it's not unlikely that I may be able to use them synced with video as well. Cool, eh?

Blender may seem like an odd choice for video editing, but it really is the best open-source video editor I've found. And especially after the improvements from Project Peach, it really 'aint too shabby. It has full multi-track support, can use almost any video/audio format thanks to its ffmpeg backend, and just generally has all the necessary features for "basic" advanced video editing. It even has a nifty "metastrip" feature that I'm to lazy to explain, but which is really awesome for keeping large, complex edits organized and sane. All this is not to say that it isn't lacking in some ways, but all the things that I feel I strictly need are there. Plus, I'm already familiar with its UI.

The only thing really lacking in my pipeline now is a convenient way to do color grading. For now I can use Blender for that (via its node-based compositing system), but at some point I may want a more convenient tool for the job. Perhaps I should write some powerful-ish color-grading tools for its sequence editor...

(And of course that last bit is the beauty of open source for me: I can write my own tools if I need to.)

Anyhoo, if anyone is curious about the details of how I got everything running (it's really not hard for a geek), let me know and I'll make another blog post.