Radiant Media Player

Preview Thumbnails



Documentation sections


Introduction

Radiant Media Player supports the loading of preview thumbnails (snapshot from a scene at a given timestamp) for a video. These thumbnails are displayed in a tooltip when a user hovers the seek bar. This is particularly helpful for better seeking into long-form content.

The thumbnails must be assembled into a mosaic (multiple thumbnails stitched together into a single image a.k.a thumbnail sprites). This is for better performance and a smoother user experience while seeking. The mosaic of thumbnails can be in JPEG, PNG or GIF format though we generally recommend using JPEG. Each individual image from the mosaic of thumbnails must be 128px in width. The height should be calculated based on the aspect ratio of the video. Below are examples of how to generate a properly formatted mosaic of thumbnails with ffmpeg that will work with Radiant Media Player.

Radiant Media Player uses WebVTT files to get preview thumbnails. The WebVTT file consists of a list of cues with the following data:

  • The time-range the tooltip thumbnail represents. Note the range needs to be in (HH:)MM:SS.MMM format. Only this exact notation will be parsed.
  • The URL to the tooltip thumbnail for this time-range. The URL is relative to the WebVTT file location (not to the page). Each individual image from the mosaic of thumbnails can be located by appending its coordinates using spatial media fragment (xywh format).

But let us walk through a complete example this will be easier to understand :)

Preview thumbnails are supported for on-demand video only with HLS, DASH or progressive download.


Player code & WebVTT file examples

WebVTT file

WEBVTT

00:00.000 --> 00:05.000
assets/bbb-sprite.jpg#xywh=0,0,128,72

00:05.000 --> 00:10.000
assets/bbb-sprite.jpg#xywh=128,0,128,72

00:10.000 --> 00:15.000
assets/bbb-sprite.jpg#xywh=256,0,128,72

00:15.000 --> 00:20.000
assets/bbb-sprite.jpg#xywh=384,0,128,72

A complete example can be viewed here.

Player code example

Preview thumbnails are passed through the player with the seekBarThumbnailsLoc setting (default value for seekBarThumbnailsLoc is an empty string). This setting should be a string pointing to a fully qualified URL (e.g. not a relative or absolute path) representing the location of the WebVTT file holding the preview thumbnails. Proper CORS settings are expected by the player to retrieve this WebVTT file as it is ajax-loaded.

<script src="https://cdn.radiantmediatechs.com/rmp/v4/latest/js/rmp.min.js"></script>
<div id="rmpPlayer"></div>
<script>
var bitrates = {
  hls: 'https://dqwp3xzzbfhtw.cloudfront.net/vod/smil:bbb.smil/playlist.m3u8'
};
var settings = {
  licenseKey: 'your-license-key',
  bitrates: bitrates,
  width: 640,
  height: 360,
  // here we tell the player where to look for the WebVTT preview thumbnails file
  seekBarThumbnailsLoc: 'https://www.radiantmediaplayer.com/media/thumbnails/bbb-thumbnails.vtt',
  poster: 'https://www.radiantmediaplayer.com/images/poster-rmp-showcase.jpg'
};
var elementID = 'rmpPlayer';
var rmp = new RadiantMP(elementID);
rmp.init(settings);
</script>

See it at work


Generating a mosaic of thumbnails with FFmpeg

In order to generate a mosaic of thumbnails that can be referenced in the WebVTT file we will use FFmpeg. FFmpeg is a popular command line tool to process media content. Refer to the FFmpeg documentation for installing and using FFmpeg.

Our mosaic of thumbnails will be generated from the video content itself, typically a MP4 file. Example of command line for short-form content:

ffmpeg -i bbb-input.mp4 -filter_complex "select='not(mod(n,120))',scale=128:72,tile=6x6" -frames:v 1 -qscale:v 3 -an bbb-mosaic.jpg

A couple of things to notice:

  • select='not(mod(n,120))': this is where we tell FFmpeg to only extract a frame every 120 frames. In the above example the bbb-input.mp4 file has a 99 seconds duration and a 24 fps frame rate. 120 represents one frame every 5 seconds which should be a decent average value to extract thumbnails for a content with a 1-2 minutes duration.
  • scale=128:72,tile=6x6: here we tell FFmpeg that the width of each item within the mosaic image should be 128px width and 72px height. In the above example of content has a 16:9 aspect ratio. The size of each item within the mosaic should be adjusted based on your content aspect ratio. We then need to tell FFmpeg what tile format should be used. Better results are obtained with multiple of 2 (e.g. 4x4, 6x6, 8x8, 16x16 ...). In this case we need 6x6 because 99 / 5 ~ 20 so we need a mosaic with at least 20 items - the closest being 6x6.

Long-form content considerations: while the above command line will work great for short-form content, long-form content needs updated command line settings. Let us take an example of a 1h16m16s content with 16:9 aspect ratio at 24 fps.

  • We are not going to take one frame every 5 seconds - this would produce a huge JPEG mosaic which would be painful to load and won't bring any added value to the viewer. Instead we are going to take one frame every minute.
  • The content is 4576 seconds long. If we take a frame every minute we will have a total of about 76 frames. Now a 8x8 tile is not going to be enough because it only has 64 frames. So we will opt for a 10x10 tile system which can hold up to 100 frames.

Our resulting command line will look like:

ffmpeg -i ed-long.mp4 -filter_complex "select='not(mod(n,1440))',scale=128:72,tile=10x10" -frames:v 1 -qscale:v 3 -an ed-long-mosaic.jpg

An example of mosaic of thumbnails generated with the above command line can be viewed here. The resulting WebVTT file can be viewed here.

You will need to experiment a bit in order to find the best command line for your content type. Based on your settings you can also adjust qscale:v from your FFmpeg command line to reduce or augment the size/quality of your thumbnails (the lower the qscale:v value the better quality).

Radiant Media Player scope of support does not cover using or installing FFmpeg. It is up to you to produce a compatible mosaic of thumbnails with FFmpeg or another tool.