Real-World Use of AI for Better Video Compression


Sunday, April 24 | 9:15 AM - 9:35 AM


Info Alert

Create or Log in to My Show Planner to see Videos and Resources.




Log in to your planner to join the zoom meeting!


Info Alert

This Session Has Not Started Yet

Be sure to come back after the session starts to have access to session resources.

This session will describe how Artificial Intelligence (AI) can be applied to improve the balance between video compression efficiency and CPU loading. This will target the real-world application of existing compression standards, notably AVC and HEVC, using AI to enhance the compression performance of encoders. Two specific areas are described, relating to specific high-level and low-level problems, where AI is a good solution to a complex problem with a large data set.

The session will introduce the basic concepts of neural networks, then show how these can be applied to the optimizations, including the results in the form of achieved improvements to bitrate vs quality. As part of this, the processing costs of the neural networks themselves will be included as part of the overall CPU budget, explaining where and why AI can be a good solution. The results show meaningful reductions in bitrate (and/or CPU required) are possible when AI is applied to these specific areas.

AI is often considered to be either a panacea that can solve all problems or an abstract concept that is too difficult and costly to use. This session focuses on two specific applications of AI for video compression, one operating at the high (picture sequence) level, and the other operating at a very low decision-making level. These are two separate problems to solve, and the AI-based solutions can both be applied.

The session will introduce the basic concepts of AI processing via neural networks and their training, then for each of the target uses, highlight the nature of the problem to be solved, why AI is a good candidate, the cost/benefit analysis, some insight into the training process and the results achieved.

The neural networks are applied for the following cases:

a) making decisions at the high level about how to make best use of a finite CPU budget but using neural networks to learn the relationship between content characteristics and the effectiveness of sets of compression tools. In doing so, it means that they can be applied in real time, dynamically adapting to the nature of the content as it arrives.

b) making decisions at the low level for HEVC when deciding how to perform Coding Tree Unit (CTU) splitting, dynamically based on detailed content characteristics.

In both cases, the aim is to process content in real time, using characteristics from the pre-processing stages, and training neural networks to learn to perform the decision making in real time. This makes it possible to make better use of the available CPU resources, thus either improving the bitrate-quality achieved or by reducing the CPU cost and environmental impact whilst achieving equivalent performance.

The results show that meaningful improvements are achieved by applying the appropriate type of AI to suitable problems when suitable data is available for that processing.

A Sub-Session Under:

Artificial Intelligence Applications for Broadcast


Tony Jones
Principal Technologist

Session Type:
Paper Presentation