Telnet Networks - Managing Network Performance

Shenick diversifEye OTT Video & Adaptive Stream Performance Testing

OTT Video & Adaptive Stream Performance Testing

3 screens of OTT Video

Video on 3 screens is rapidly coming of age, the success of adaptive streaming protocols results in more video being delivered to a greater number of screen sizes e.g. Smart Phone, Tablet, PC or living room TV. Quality and reliability of delivery is still a key focus on all screen sizes. 

Adaptive streaming delivers a number of content sources with various screen resolutions, frame sizes and frame rates. Testing performance on an adaptive streaming delivery solution is a complex challenge, as the content delivery mechanism can be dynamically altered dependent on a number of possible scenarios which will include device type and network bandwidth performance.

Why use Per flow stateful emulation to test OTT adaptive streaming performance?

On reading of the associated manifest file of any adaptive stream it's possible to see a number of unique content sources exist at a given time, to serve the greatest number of varying end-user devices. A basic test strategy is to ensure each content source is accessible. The test would require a number of consumer devices of varying screen sizes. However, as the delivery platform is likely to be used by several thousand end-users a more cost effective means of testing is to use emulation. Emulation is an accurate representation of consumer devices such as smart-phones, tablet PCs, TVs, etc. 

As each device is independent and interacts in a dynamic manner with the OTT serving platform, a stateful test solution is required where each emulated device can dynamically alter its request to suit the network characteristics. The stateful approach ensures the greatest number of real-life network conditions is tested. 

A significant reason to test OTT video with a per flow strategy, is to measure performance of the incoming content on a per request basis. Defining Quality of Experience for OTT adaptive video streams is the inclusion of a variety of performance measurements which include server timing responses, accurate playback of the correct video source and download speeds.

Why use diversifEye?

Shenick's diversifEye is the only stateful per flow solution available that enables users emulate stateful OTT streaming clients along with providing per flow performance measurements on a per emulated device, per request basis. 
As diversifEye delivers stateful emulation of clients, it can connect to 3rd party servers. diversifEye provides performance measurements for leading OTT implementations such as MS Smooth and other popular streaming mechanisms. 
An example of diversifEye functionality is to emulate Smooth VOD and/or Smooth Live clients. diversifEye emulated clients are used to emulate Adaptive Rate Changes:

• Generate user profiles with specified steps and intervals
• Measure per request Get Time responses
• Assess bandwidth and download bitrates

Alternatively, diversifEye is used to emulate stateful fixed rate client scenarios requesting Smooth VOD / Live Streams at different fixed rates for the same or different video sources. An example is to emulate TV, Tablet, PC, and Smart Phone all viewing the same video encoded at different video quality levels. 

diversifEye stateful client emulation results in each emulated client reacting accordingly to the response codes coming from the serving platforms. An example of the actions taken in response to the received response codes include:

•  HTTP 404 Page Not Found > diversifEye client downshifts to a lower rate and requests the same time stamp.
• HTTP 412 Precondition > diversifEye client backs off for a specified period of time and attempts the connection again.

In addition to per flow emulation diversifEye provides Event Threshold and Notification (Optional Package). Apply Threshold events to any of the thousands of statistics captured in diversifEye. Capture and react in real time to important response codes (4xx).

The application note Testing Adaptive streaming with diversifEye provides more information and sample test scenarios.