Best practices for building energy-efficient client-side media-oriented web applications

Last updated on December 8, 2022 - Back to blog home page

Tweet

Just over a year ago we decided to join the Greening of Streaming industry forum. This really was a blessing, and I can only be thankful for having met so many talented and responsible people since then. We have conducted research on energy consumption on client devices and share our findings with our peers. We are now ready to tell the world what we have learnt about building energy-efficient client-side media-oriented web applications. Our goal here is to help web developers across the world to build low-carbon-footprint web applications and provide real-world solutions that can be implemented as of late 2022.

As part of our effort to share our findings I would encourage our readers to review the slides we presented during Greening of Streaming Green Stream Episode 2.

• Do not use autoplay: unless you have a business requirement to use it - do not go for autoplay - autoplayed video can cause poor viewing experience and is not environmental-friendly. Media content should only start upon explicit user interaction.
• Do not use media content preloading: fetching content from the server when we do not know if the viewer will watch content is unnecessary, especially if that viewer is on cellular network.
• Make your Adaptive Bitrate Streaming (ABR) logic more environment-friendly
  • Do not account for device pixel ratio in your ABR logic: some mobile devices can have a high device pixel ratio (3 or more). This means that in a 640x360 CSS pixels sized player, a classical ABR algorithm could deliver up to a 1920x1080 resolution in a small, windowed player. This is unwanted behaviour, even if premium mobile devices can render the crispy details of a HD or UHD resolution, the physical size of the screen will prevent extra value to be added to the viewing experience. Those high definition resolutions should only be delivered for a fullscreen experience on larger screens. Dismissing the device pixel ratio information (or capping it) in your ABR algorithm can lead to significant energy savings and even improve viewer experience in lower-end devices or degraded network conditions.
  • Cap resolution to current player size in your ABR logic: as mentioned before, there is very little value to throw HD or UHD resolutions to a 640x360 CSS pixels sized player, even when device pixel ratio is equal (or limited) to 1. Your ABR algorithm needs to account for player size and only deliver resolutions that fits the computed viewing area. Also note that player size may vary during playback: viewers can go fullscreen, change orientation, resize browser window - your ABR logic should dynamically adapt to changes in player sizes.
  • Compute initial bandwidth with efficiency: thanks to API like the Network Information API it is possible to get accurate value for client-side initial available bandwidth. No need to test anymore with downloading a random fragment in your ABR renditions.
  • Make your ABR logic less aggressive: be more conservative when it comes to switching up and more liberal when it comes to switching down while remaining in acceptable waters for the overall viewing experience - every byte counts.
• Use Media Capabilities API: when delivering content with HLS or MPEG-DASH you should consider using a combination of codecs to optimise viewing experience. HLS or MPEG-DASH manifests can list renditions encoded with various codecs, therefore giving the option to client-side applications to opt for the best available codec based on Media Capabilities API results. Typically, Media Capabilities API can tell if a specific combo of resolution/framerate/bitrate encoded with a specific codec can give the viewer a smooth and energy-efficient playback experience. Our research shows that energy-efficient rendering with smooth playback happens when hardware decoding is available on the targeted device. Software decoding is never energy-efficient with the exception of lower resolutions encoded with newer codecs. It should be noted that when new codecs are released on the market, software decoding may become available before hardware decoding, so making assumption about hardware decoding support for a specific codec on a specific device should not happen - this is why testing before using is important as the best match is not always the good match. AVC/H.264 is the default standard for legacy support so you will need to include this codec in your list of codecs. You should also have a more modern codec like AV1 or HEVC/H.265 listed to take advantage of the better coding efficiency available with those new codecs. We recommend using a combination of H.265/H.264 to our customers but a AV1/H.264 combination could also work depending on your targeted environment:
  • H.265/H.264: Apple devices and Smart TVs
  • AV1/H.264: Emerging support but better performance than H.265
  In theory you could use a combination of AV1/H.265/H.264 but the carbon cost to having all those 3 codecs listed in your HLS or MPEG-DASH manifest could be significant so we recommend using 2 codecs until we can more accurately assess the benefits/cost of having more codecs listed in a HLS or MPEG-DASH manifest.
• Leave the viewer with a choice: while it is possible to evaluate client-side device capabilities in terms of network, CPU or codecs support, we may not always be in a position to completely understand the actual viewing experience that an energy-efficient configuration provides. A minority of viewers may experience discomfort because of energy-efficient configurations - remember this is a trade-off: low-energy-consumption over quality in a good enough context. While we do feel that energy-efficient configuration should be the default for media-oriented web applications built in 2022+, always present the viewer with an opt-out of energy-saving option so that your viewers can retain control of their devices.

We will continue learning from building energy-efficient client-side web applications in the coming years and we will continue sharing our findings. The above guidelines are a first step towards more advanced considerations to further reduce the carbon footprint of the streaming media chain, not just the client-side aspect of it.