August 29, 2016 by Romain Bouqueau

A view on VP9 and AV1 part 2: important considerations

Introduction

The success of a video coding standard depends on many factors. Many articles try to benchmark the performance of codec implementations or make comments about the ecosystem of codecs. But I have not seen any article about the standardization process or the bitstream format. While the video quality is key, I believe the bitstream format is important too, but is probably less accessible or easy to communicate about.

This is a two-part analysis. Part 1 compares the HEVC and VP9/AV1 codecs in the light of their bitstream formats. Part 2 (this article) will look at some other important considerations.

Video codecs: factors of success

The success of a codec is made of several components:

  • Compression: the video quality at given bitrate constraints.
  • Performance: how it compares to other codec implementations (in term of CPU versus quality/bitrate balance).
  • A non-ambiguous specification. This requires many people to read it.
  • Some tools to help editing better standards.
  • A clear patent and royalty policy: of course free is better but innovation always comes at a cost.
  • Simplicity to implement: too much complexity leads to incompatibilities and delays in implementations, and requires more hardware surface (hence increases the power consumption).
  • Software implementations for both encoders and decoders.
  • Room for hardware acceleration.
  • Ability to embed in popular containers.
  • Analyzers (the Google VP9 decoder comes with a nice ‘–enable-coeff-range-checking’ configure option).
  • Reference vectors (it seems that fuzzing with AFL leads to more interesting vectors than what codec vendors provide).
  • Competition with other codecs: having competitors helps get better.

Standardization and patents considerations

VP9 and AV1 bitstream formats are far simpler than AVC or HEVC but are also less mature. That's a consequence of the development process: VP9 was done by one company, HEVC was standardized by a consortium via a lengthy process.

In this process, many MPEG members were able to insert their patented feature. In that regards, the AV1 codec is important for the video ecosystem. The industry is poisoned by the current patent system which produces closed ecosystems, important fees and needlessly complex standards. AV1 is being backed by huge companies and patent holders. But AV1 probably also makes use of important external patents while being royalty-free. So it will be interesting to see if there will be lawsuits ; the results of these lawsuits may change the behavior of the industry.

Deployment consideration

For most managers, the video problem is considered solved and their solution is to take open-source projects such as FFmpeg, GPAC, MediaInfo and VLC to build a complete workflow. This analysis is premature in my opinion, even if many managers commonly share it. As a consequence, possible issues in VP9/AV1 are not taken seriously enough. If they were fixed, we would not be far from a convincing alternative to HEVC.

Conclusion

We like VP9 and AV1 for their simplicity, openness, and patent-free approach. In the Part 1 of this article, we focused on the VP9/AV1 bitstream format. We concluded that AV1 had the potential to compete with HEVC from a bitstream format point of view but still contained a few serious issues.

In this Part 2, we focused on several considerations other than quality and performance. Codecs should be evaluated with regards to their ecosystems and a strategic view: the failure of VP9 or AV1 is possible but would it be important for the AV1 supporting entities? Is this initiative a way to achieve other goals?

Links

A view on VP9 and AV1 part 1...

previous post

<

Summary of IBC 2016 from the GPAC Licensing...

next post

>
  • Jamaika - September 01, 2016
    "We like VP9 and AV1 for their simplicity, openness, and patent-free approach. In the Part 1 of this article, we focused on the VP9/AV1 bitstream format. We concluded that AV1 had the potential to compete with HEVC from a bitstream format point of view but still contained a few serious issues." In my opinion, the conclusions too vague. What does it mean? In forum doom9 Netflix presented the results of Their large-scale study, Comparing X265 is x264 and VP9 (libvpx) ... It is appropriate to eliminate libvpx. https://www.youtube.com/watch?v=wi1B...utu.be&t=1h25s "Analyzers (the Google VP9 decoder comes with a nice ‘–enable-coeff-range-checking’ configure option)." Nothing mentioned about the problem of decoding sRGB mode. There was also described the "test-16bit-internal". What is it for?
  • Romain Bouqueau - September 01, 2016
    The doom9 thread is detailed but it only covers the compression side. But if you look at what happens with HEVC, the issue for large-scale deployments is not related to compression. There are many aspects that are just not documented and I wanted to list them. I also wanted to highlight VP9/AV1 codecs and use HEVC (which is well more known) as a point of comparison. Our articles are definitely not a quality comparison of codec implementations. For the conclusion, please read Part 1 that describes the bitstream format and let me know if I can clarify on some points. I made a full article on it because it is important for conformance and transport which are subjects where the GPAC team shines. I don't understand your question about sRGB, could you rephrase?
  • Romain Bouqueau - September 01, 2016
    The doom9 thread is detailed but it only covers the compression side. But if you look at what happens with HEVC, the issue for large-scale deployments is not related to compression. There are many aspects that are just not documented and I wanted to list them. I also wanted to highlight VP9/AV1 codecs and use HEVC (which is well more known) as a point of comparison. Our articles are definitely not a quality comparison of codec implementations. For the conclusion, please read Part 1 that describes the bitstream format and let me know if I can clarify on some points. I made a full article on it because it is important for conformance and transport which are subjects where the GPAC team shines. I don't understand your question about sRGB, could you rephrase?
  • Jamaika - September 01, 2016
    He treats it as a loose conversation. "Our articles are definitely not a quality comparison of codec implementations." I understand. Plans GPAC to implement codecs VPX / AV1? I don't know what the creators intended in codec libvpx. What is the function and how do I use the color space sRGB?
  • indolering - September 06, 2016
    You state that "AV1 probably also makes use of important external patents." This assertion has been made about the VPx lineage going back to VP3 (which was the basis for Theora). Despite MPEG-LA throwing everything they could at VP8, the most they got was a cross licensing deal from Google. AV1 will receive even greater scrutiny and have access to more patents than any VPx codec. Given VPx's 15-year track record, it is more likely that AV1 won't step on any external patents.
  • Romain Bouqueau - September 07, 2016
    Thanks for your comment. The patent concern would deserve a few articles to be covered. However the current HEVC patent issues (with patents being hold outside of MPEG-LA or any running patent pool) is a real issue in my opinion.
  • Zach Lym - September 09, 2016
    the current HEVC patent issues (with patents being hold outside of MPEG-LA or any running patent pool) is a real issue in my opinion. Note that the VP8 cross-licensing deal doesn't cover new encoding techniques used in VP9, so why isn't MPEG-LA and HEVC going after Google? I can see trolls saving their frivolous lawsuits to try and mar AV1's reputation, but hopefully Google will either forge another cross-licensing deal or beat them in court. Even if they lost, wouldn't AV1 convert into a royalty bearing codec ... just like H.26x? Seriously, I'm asking! With any luck, the technique would only be used in a higher-level profile that can be disabled. If it's a serious mess, then AOM will rip out the offending technique and we will move onto AV1.1. Even if AV1 fails hard, AV2 will benefit from the legal vetting of AV1 and will avoid the DCT patent thicket via lapped-transforms. So yes, I agree with you that patent trolling is a serious problem. But your assertion that AV1 *probably* steps on external patents doesn't square with history and seems like a very lazy argument (friendly poke : ).
  • Romain Bouqueau - September 13, 2016
    AV1’s reputation, but hopefully Google That's not only about Google and it makes things significantly more complex. will either forge another cross-licensing deal or beat them in court. I think that's what AOM is looking for. That's even more important than the codec quality (so we may have to wait for AV2 or AV3 as you suggest, even if AOM obviously markets otherwise). Even if they lost, wouldn’t AV1 convert into a royalty bearing codec … just like H.26x? Seriously, I’m asking! If they fail to get an agreement for HEVC, how could they succeed for AV1? If it’s a serious mess, then AOM will rip out the offending technique and we will move onto AV1.1 You cannot do that due to conformance issues. That would make a new codec and chip vendors would lose millions.
  • Zach Lym - September 14, 2016
    That’s not only about Google and it makes things significantly more complex.
    I would think that the pooled IP of the dozen companies and their web of influence over purchasing decisions would give AOM much greater bargaining power than Google had when they forced MPEG-LA into backing down on their VP8 claims.
    I think that’s what AOM is looking for.
    Yes, and their experience with VP3-VP9 indicate that they will succeed. Don't forget that Google has invested a quarter of a billion dollars in codec development.
    If they fail to get an agreement for HEVC, how could they succeed for AV1?
    Well, no-one needed to for VP9 : ) But let's say a court finds that AV1 steps on a single patent, is there any legal mechanism limiting the licensing fees? Can they charge more than what they are asking for another codec?
    You cannot do that due to conformance issues. That would make a new codec and chip vendors would lose millions.
    Depends on the patent and the implementation. But even on fixed-function hardware (if the cost of supporting 1.0 and 1.1 is minimal) streaming services could at least reduce their per-unit licensing fees on nth generation units. Sorry to drag this out, I am grateful for your replies. I just don't think it's fair to characterize AV1's chances of stepping on a patent as "probable" when VPx avoided it, especially with the increased legal scrutiny and expanded the patent pool. I would love to see an article with more in-depth discussion!
  • johnny423 - June 17, 2017
    Interesting discussion (400+ comments) regarding Google ANS patent application for AV1: https://www.reddit.com/r/programming/comments/6h08z5/google_is_currently_trying_to_patent_video/?sort=confidence
  • Romain Bouqueau - June 18, 2017
    Thanks for sharing.
  • - Comments are closed -