Draft:Elixir/Bite-sized video transcoder

Background

Video files take up a surprising amount of disk space, so they're almost always compressed in storage, and while being streamed over the Internet. In many cases you'll need to change the type of compression, generally known as transcoding, for example because video is stored in a high-efficiency, modern format but you want to provide it in a less efficient, more compatible, older format. This process of changing compression takes even more disk space and loads of processing time.

Choosing the compression parameters requires several advanced graduate degrees or you'll end up with a huge and blurry mess.

A single use-case

Narrowing down to look at just one use for transcoding, Wikimedia Commons asks that uploaders use open and royalty-free formats, which is great unless your video is already stored in a patent-encumbered format like .mp4 or .mov file, the sad story for most of us.

There are options!

Problem statement

Once a long time ago, I used the "easy" method, which involves uploading your video to the Video2commons website. It's an amazing service to provide when it works... Like I was saying, I uploaded one hour of video, waited a full day, and found that the job had failed. I emailed some people, felt sad, and tried the upload again. It worked, a day later.

The more exciting method is to download a transcoder such as Handbrake, empty your trash can, and put some tea next to your laptop fan to warm up.

Why did it fail?

My anecdote is a decade old, and the emails were sent from a company account which I can no longer access, so rather than piecing together my faulty memories I'll just make wild guesses. Maybe a disk filled up, or my upload was truncated, or some safety rule kicked my job for taking too long. Either way, this might be a programming thing that could be improved. Halfway-processed jobs should be restartable.

Does it scale?

The cloud solution is a bit scary: if my transcoding took a day, what about all the other people trying to convert their videos? There has to be some limit, beyond which we're either waiting or rejected. It's also possible that the limit is set so high that the mob of people trying to upload videos to Wikimedia Commons can all be helped.

System load graphs for the live Video2common service can be viewed publicly, but they're a bit of an enigma. It looks like it receives lots of small jobs, and is only occasionally used for longer videos. It's rare to see two long jobs overlapping. The usual pattern is that one user uploads many videos in a batch, and then hours might pass until the next user jumps on.

Reality check: don't use the cloud

I arrogantly imagined that I would write a "better" cloud thing which wouldn't fail so surprisingly, but I forgot to question the cloud concept itself. Before considering the questions, or actually programming or reading more about the existing tools, I called my friend T——, who knows video. She explained that normal people just use Handbrake, and more importantly, it's irresponsibly inefficient to upload video and download it again, especially if you're starting with a larger, raw format.

I was happy to end the exploration there, for a few months at least...

Hybrid client-server

What if the same or similar code could be run on a remote server, or locally on the user's computer? Then scaling might be accomplished by using the local machine when possible.

The cloud's strengths of ease-of-use would combine with the efficiency advantages of running locally.

TBD: explore.