Radiant Media Player

Preview thumbnails



Documentation sections


Introduction

Radiant Media Player supports the loading of preview thumbnails (snapshot from a scene at a given timestamp) for video content. These thumbnails are displayed in a tooltip that is shown when a user hovers the seek bar. This is particularly helpful for better seeking into long-form content.

The thumbnails must be assembled into a mosaic (multiple thumbnails stitched together into a single image a.k.a thumbnail sprites). This is for better performance and a smoother user experience while seeking. The mosaic of thumbnails can be in JPEG, PNG or GIF format though we generally recommend using JPEG. Each individual image from the mosaic of thumbnails must be 128px in width. The thumbnail height should be calculated based on the aspect ratio of the video. Below are examples of how to generate a properly formatted mosaic of thumbnails with ffmpeg that will work with Radiant Media Player.

Radiant Media Player uses WebVTT files to load an parse thumbnails. The WebVTT file consists of a list of cues with the following data:

  • The time-range the current thumbnail represents. Note that the range needs to be in (HH:)MM:SS.MMM format. Only this exact notation will be parsed.
  • The URI to the thumbnail for this time-range. The URI is relative to the WebVTT file location (not to the page). Each individual image from the mosaic of thumbnails can be located by appending its coordinates using spatial media fragment (xywh format).

Preview thumbnails are supported for on-demand video only. They will not show on mobile device where the seek-bar cannot be hovered.


Player code & WebVTT file examples

WebVTT file

WEBVTT

00:00.000 --> 00:05.000
assets/bbb-sprite.jpg#xywh=0,0,128,72

00:05.000 --> 00:10.000
assets/bbb-sprite.jpg#xywh=128,0,128,72

00:10.000 --> 00:15.000
assets/bbb-sprite.jpg#xywh=256,0,128,72

00:15.000 --> 00:20.000
assets/bbb-sprite.jpg#xywh=384,0,128,72

A complete example can be viewed here.

WebVTT files should be saved in UTF-8 encoding to prevent character issues.

Since Radiant Media Player 5 we also support referencing multiple sprites within the same VTT file:

WEBVTT

00:00.000 --> 00:05.000
assets/bbb-sprite-1.jpg#xywh=0,0,128,72

00:05.000 --> 00:10.000
assets/bbb-sprite-1.jpg#xywh=128,0,128,72

00:10.000 --> 00:15.000
assets/bbb-sprite-2.jpg#xywh=256,0,128,72

00:15.000 --> 00:20.000
assets/bbb-sprite-2.jpg#xywh=384,0,128,72

Player code example

Preview thumbnails are passed to the player with the seekBarThumbnailsLoc setting (default value for seekBarThumbnailsLoc is an empty string). This setting must be a string pointing to a URI representing the location of the WebVTT file holding the preview thumbnails. Note that the WebVTT file is loaded with XMLHttpRequest and thus it must be delivered with proper CORS setting for cross-site requests.

<script src="https://cdn.radiantmediatechs.com/rmp/5.0.8/js/rmp.min.js"></script>
<div id="rmpPlayer"></div>
<script>
var src = {
  hls: 'https://your-hls-url.m3u8'
};
var settings = {
  licenseKey: 'your-license-key',
  src: src,
  width: 640,
  height: 360,
  // here we tell the player where to look for the WebVTT preview thumbnails file
  seekBarThumbnailsLoc: 'https://www.radiantmediaplayer.com/media/thumbnails/bbb-thumbnails.vtt',
  poster: 'https://your-poster-url.jpg'
};
var elementID = 'rmpPlayer';
var rmp = new RadiantMP(elementID);
rmp.init(settings);
</script>

See it at work


Preview thumbnails API

getThumbnails()
rmp.getThumbnails();

This method returns a String URI for the current preview thumbnails VTT file loaded in the player.

setThumbnails(newVTTThumbnailsURI)
rmp.setThumbnails('https://your-server-url/new-thumbnails.vtt');

This method sets the preview thumbnails VTT file to be loaded by the player. In case player fails to load the new VTT file a 1008 warning event will fire.


Generating a mosaic of thumbnails with FFmpeg

In order to generate a mosaic of thumbnails that can be referenced in the WebVTT file we will use FFmpeg. FFmpeg is a popular command line tool to process media content. Refer to the FFmpeg documentation for installing and using FFmpeg.

Our mosaic of thumbnails will be generated from the video content itself, typically a MP4 file. Example of command line:

ffmpeg -i input.mp4 -filter_complex "select='not(mod(n,120))',scale=128:72,tile=11x11" -frames:v 1 -qscale:v 3 -an mosaic.jpg

A couple of things to notice:

  • select='not(mod(n,120))': this is where we tell FFmpeg to only extract 1 frame every 120 frames (one frame every 5 seconds for a input.mp4 which runs at 24 FPS).
  • scale=128:72,tile=11x11: here we tell FFmpeg that the width of each item within the mosaic image should be 128px width and 72px height. In the above example of content has a 16:9 aspect ratio. The width of each item within the mosaic must be 128px. The height should be adjusted based on your content aspect ratio. We then need to tell FFmpeg what tile size should be used. In this case we need 11x11 because 596 (duration) / 5 (gap between each thumbnail frame) ~ 120 frames. So we need a mosaic that can hold at least 120 frames - the closest being 11x11 = 121 frames.
  • qscale:v you can adjust this value to reduce or augment the size/quality of your thumbnails (the lower the qscale:v value the better quality).

Radiant Media Player scope of support does not cover using or installing FFmpeg. It is up to you to produce a compatible mosaic of thumbnails with FFmpeg or another tool.


Content duration considerations

If you are not using multiple sprites, you may need to adjust the frequency at which you pick thumbnails from a video stream based on content duration. For short-form content, it is best to pick them at shorter intervals (like extracting one frame every 2 seconds) while for long-form content the gap between frame extraction should be higher. This is because the seek-bar will always be limited in size on your viewer device and providing too much frames for your mosaic can have adverse effects like confusing navigation or loading a mosaic image that is unnecessary large for the viewer use-case. Here are some suggestions for gaps in frame extraction based on content duration:

  • duration <= 2 minutes: one frame every 2 seconds
  • 2 minutes < duration <= 10 minutes: one frame every 5 seconds
  • 10 minutes < duration <= 30 minutes: one frame every 10 seconds
  • 30 minutes < duration <= 60 minutes: one frame every 20 seconds
  • 60 minutes > duration: one frame every 30 seconds

Generating a WebVTT file

To make your day easier we have released a node.js based VTT file generation script for our preview thumbnails feature. First you will need to create your mosaic image file as described above. Once this is done you can head to GitHub for installing and using our helper script.

Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 3.0 License.