The Codec Wars – Moving Forward with HEVC

Editor’s Note: When our CTO, Mahmoud Al-Daccak, submitted his predictions for 2019, he gave a very in-depth piece. It explored the benefits of improving HEVC as well as the importance of focusing on the purpose of technological innovations. A shortened version was included in our 2019 video streaming technology predictions blog post. However, our editorial team felt our readers would benefit from the full piece – please enjoy.

My colleagues asked me for my New Year’s predictions. I procrastinated as much as I could hoping that they would go away, but the pressure kept mounting and I panicked. You see, asking me to predict the future is like asking me what my favorite color is – it gives me a headache! So instead, I would like to focus on making the future. It certainly sounds like more fun to me, or at least, in my case, more relaxing.

2018 saw a lot of discussion on a number of topics, notably, the so-called “codec wars.”

And frankly, these codec wars have caused far more confusion than progress. By jumping into the war with a competing codec – I’m looking at you, AV1 – the natural progression of HEVC was held back. While H.264 had the chance to evolve and become refined over time – HEVC was practically put on “pause.” The effort to improving it received far less attention than jumping to define and develop AV1, VP9, VVC, and who knows how many other parallel efforts.

So, we’ve gotten ourselves into a bit of a tricky situation. How can we change course and ensure that our efforts are cumulative rather than parallel? How can we shape the future of codecs in 2019?

What are our Options?

Overall, AV1 seems better than HEVC, but it comes with a cost – specifically, a 2000% increase in complexity and required computing power. And frankly, the jury is still out on whether the video quality merits such an increase. AV1 may be an appealing codec for the likes of Netflix who can afford the massive increase on their electricity bill in exchange for the 10%-20% savings on bandwidth. For any other user, I would advise focusing on your use case and the value which you bring to your customers.

VP9 is also attractive, although with less net benefits than HEVC. There is no need to compound the war of the codecs with the browser wars (which are coming to another close,) especially if you don’t have a dog in the fight. Without getting into geeky debates, if you are not dealing with video delivery to end viewers, then don’t torment yourself with the details of VP9. It’s not the codec for you.

If your business is to bring value to broadcast contribution and distribution customers with a focus on live video workflows, then HEVC is the codec for you. Toss aside the nerdy debates for a moment – you have customers to serve. HEVC is baked into billions of chipsets from encoders, to TVs, STBs, and mobile devices. You should invest your energy into making this codec better. With focus, and with some advances in VVC in 2019, HEVC can be enhanced to deliver on its promised value, compared to H.264.

Wait… what is VVC? Is this another codec?!

What is VVC, and moreover, what is the value in waiting for it?

VVC promises a few evolutionary improvements beyond the brute-force rehashing and complexification of decades-old encoding techniques at the expense of a massive increase in computing consumption.

There are a few advanced techniques that are very promising – refined block partitioning, affine motion transforms, decoder-side search, and a few other possibilities.

These evolutionary steps will allow for a substantial increase in encoding efficiency. Importantly, they will only incur a modest increase in complexity, when compared to HEVC. (In rough estimates, 400% for encoding and 120% for decoding)

I’m no codec guru. How can I contribute to HEVC?

You can definitely help improve HEVC (or any other codec, if you so choose.) There are many techniques to choose from. These techniques include perceptual filtering and deblocking methods to overcome bit allocation between solid backgrounds and complex foregrounds, and bit allocation to objects of interest.

Advancements in AI and the lightweight analysis algorithms that leverage machine learning models, have opened a world of possibilities to identify objects of interest, and enhance how they are perceived or signaled. Content and network adaptive control, (adjusting the encoding parameters to the viewing conditions,) also shows a lot of promise for increasing perceptual encoding efficiency. Finally, applying visual enhancement techniques at the decoder end, derived from encoding parameters and conditions, shows great promise in improving the overall quality of experience.

So, here you have it! For 2019, let’s take control of our destiny and help shape the future by focusing more on adding value to our customers rather than engaging in ideological wrangling and propellerhead debates. Let’s understand our customers’ use cases and apply the tools we have today to address their needs.

If you are working with HEVC, and are looking for the next steps to improve your workflow, please consider looking at our white paper: What to Consider Before Buying an HEVC Encoder. In it, you will learn which workflows most benefit from HEVC and how to choose the best encoder for your use case.