Quality of Experience Engineering for Customer Added Value Services

From Evaluation to Monitoring

Abdelhamid Mellouk, Antonio Cuadra-Sanchez, Abdelhamid Mellouk, Antonio Cuadra-Sanchez

• Publisher: Wiley-ISTE
• Year: 2014
• Print ISBN: 9781848216723
• eBook ISBN: 9781118984338
• Edition: 1
• Topic: Technology & Engineering
• Subtopic: Electrical Engineering & Telecommunications
• Index: Technology & Engineering

The main objective of the book is to present state-of-the-art research results and experience reports in the area of quality monitoring for customer experience management, addressing topics which are currently important, such as service-aware future Internet architecture for Quality of Experience (QoE) management on multimedia applications.

In recent years, multimedia applications and services have experienced a sudden growth. Today, video display is not limited to the traditional areas of movies and television on TV sets, but these applications are accessed in different environments, with different devices and under different conditions. In addition, the continuous emergence of new services, along with increasing competition, is forcing network operators and service providers to focus all their efforts on customer satisfaction, although determining the QoE is not a trivial task. This book addresses the QoE for improving customer perception when using added value services offered by service providers, from evaluation to monitoring and other management processes.

1

Challenges for Quality of Experience Engineering for Added Value Services

Abdelhamid MELLOUK and Antonio CUADRA-SANCHEZ.

1.1. Introduction and challenges

In recent years, multimedia applications and services have experienced a sudden growth. Today, the video display is not limited to the traditional areas of movies and television on TV sets, but accesses these applications in different environments, devices and under different conditions.

In addition, the continuous emergence of new services along with increasing competition is forcing network operators and service providers to focus all their effort onto customer satisfaction, although determining the Quality of Experience (QoE) is not a trivial task.

Due to the emergence of different kinds of communication and networking technologies (core networks with intra-domain and inter-domain challenges, access networks, aggregation networks, spontaneous networks, Internet of Things, etc.) and the current and envisaged proliferation of different and specific types of services supported by these technologies (real-time services, IPTV, VoD, social networking, E-Health, multimedia, gaming, smart cities, etc.), traditional network control strategies are not sufficient to handle the complexity and diversity of information in use that we see today. There is a strong need to develop a new paradigm to obtain the continuity of network services based on the new concept of smart communications and user interaction. The user’s perception, such as emotions, intensity or satisfaction, is a natural way to interact with the real world, and will serve as a powerful metaphor to interact with online services, and performs affective computing paradigm. On the other hand, the use of artificial intelligence tools together with biologically inspired techniques is needed to control network behavior in real-time so as to provide users with the quality of service that they request, and to improve network robustness and resilience based on continuous user feedback.

The key idea of this book is to present a new paradigm driven by user perception and based on control theory and machine learning techniques in order to support smart communications to avoid any complete interruption in the whole chain of service treatment. The main goal is to present state-of-the-art research results and experience reports in the area of Quality Monitoring for customer experience management, addressing, amongst others, currently important topics such as Service-aware Future Internet architecture for Quality of Experience (QoE) management on multimedia applications with respect to the following steps:

– to develop, at the theoretical level, an appropriate unified formal model based on bio-inspired modeling and knowledge distribution in order to construct a scalable and robust environment with low complexity for large-scale dynamic networks;

– to study, at the empirical level, why and how user perception can be quantified, analyzed and influenced;

– to create, at the conceptual level, a general framework for protocol stacks dedicated to smart communications featuring multiple technologies, devices and users,

– finally, to build, at the engineering level, the appropriate model of programming abstractions and software architecture to develop a full-scale framework.

This book addresses the QoE for improving customer perception when using Added Value Services offered by service providers, from evaluation to monitoring and other management processes.

1.2. Contents

This section summarizes the content of the chapters that are gathered in this book.

1.2.1 Chapter 2: An ecosystem for customer experience management

In this chapter, the authors describe an ecosystem that allows us to manage customer experience in order to guarantee the quality levels delivered to end-users, which has been defined into the Eureka Celtic internet protocol network for quality of service intelligent support (IPNQSIS) project and is being adapted for over-the- top (OTT) services inside the Eureka Celtic next generation over-the-top multimedia service (NOTTS) project. The QoE ecosystem lies on a customer experience architecture formed by data acquisition level, monitoring level and control level. The work proposed in this chapter will settle the basis of next generation Customer Experience Management Systems (CEMS).

The authors present an overview of the CEMS under development within the IPNQSIS and NOTTS projects. On the one hand, a generic overall CEMS architecture is introduced and, on the other hand, it has been specialized for the IPNQSIS and NOTTS scopes, reinforcing specific areas such as network monitoring, as well as having IP television (IPTV) as a main application use case.

The IPNQSIS project, in which 18 companies and institutions from Spain, France, Sweden and Finland have collaborated, developed next generation management architectures to improve the QoE-driven network management. This project ended in April 2013 and its main objectives were accomplished, from the definition of a general Customer Experience Management (CEM) architecture to IPNQSIS prototypes focused on IPTV multimedia services. The results of the project consisted of Quality of Service (QoS) measuring tools, mechanisms to quantify the QoE, its correlation with the QoS parameters, and their influence on QoE. The outcome of the analysis can be applied to the integrated management of network resources to improve the user’s experience. This technology is also going to make it possible to develop tools to enable greater correlation between the quality of the service and the actual experience of the user, thereby ensuring greater customer satisfaction.

Future research in this area will extend its scope to next generation services such as OTT services, specifically the NOTTS project, which is continuing the activities regarding QoE management on a task dedicated to this purpose.

1.2.2 Chapter 3: Measuring MPEG frame loss rate to evaluate the QoE in IPTV services

The chapter describes a model to predict the QoE that is a function of the loss of the different types of moving pictures experts group (MPEG) frames, providing a mean opinion score of the delivered service. The authors have implemented this model in a network monitoring tool, which has been validated in both Intel and ARM platforms.

An empirical evaluation of the computational cost of both MPEG frame loss and packet loss ratio (PLR) measurement algorithms has been done with a desktop personal computer (PC) and a low-cost device, providing interesting results to decide which one is better to be used as a network probe. The system running on a PC can be used to measure at the core or access network, whereas the low-cost device can be used at the user’s premises.

The final results show that this model is able to better predict the QoE of such video services than just using the packet loss rate. Based on these results, they have defined a method to measure the QoE by capturing the live video channels, inspecting the packets to detect losses and applying the measured parameters with the obtained model. The authors have implemented a prototype and have also measured its performance, testing its feasibility in both personal computer (PC) and low-cost probes.

As future work to improve the QoE estimation model, the authors will investigate how the amount of movement also influences the perceived QoE. Also, it is interesting to consider how well the obtained model fits with an experiment using a panel of users to get a subjective evaluation of the videos watched.

1.2.3 Chapter 4: Estimating the effect of context on the QoE of audiovisual services

In order to estimate the effect of context on QoE of audiovisual services, in this work, the authors compared the results of formal subjective audiovisual assessment with more informal assessments performed in actual usage contexts (in this case two public exhibition halls). They observed significant differences in the results, both in terms of the mean opinion score (MOS) values and on the impact of the different quality-affecting factors. Interestingly the results show that the subjects in public places were less tolerant to the quality degradations than the subjects in the laboratory. Specifically, tests separating the effects of contextual factors on the basis of 1) voting behavior and 2) actual experience should be conducted.

In this way, in this chapter, the authors compared the results of a laboratory- based audiovisual assessment campaign with that of two separate (and smaller scale) campaigns carried out in public places, in a completely different context. Besides the explicit goal of comparing the results of subjective assessments in a lab versus non-lab environment, this work provides a first step into developing context-specific bias functions to easily and cheaply adapt quality models, typically trained on laboratory-based data, to new contexts of use. These experiments are the first in a series of experiments with the purpose of understanding the effect the context of use has on QoE.

The authors also demonstrated the viability and limitations of an audiovisual model trained on the laboratory-obtained data, when used in a different context, namely in crowded public places. The performance of the model in the exhibition context was inferior to the performance in the laboratory context. However, the estimations could still provide usable values for quality monitoring purposes, e.g. in public displays.

In addition, the authors are currently working on a model calibration method that uses information derived from lightweight user tests performed in specific context. The idea is to test and model the effects of the dominating influence factors in order to formulate a context specific correction function. To this end, and in order to understand different contexts of use and devices generally, user tests outside the laboratory shall be continued.

1.2.4 Chapter 5: IPTV multiservice QoE management system

In this chapter, the authors justify the claim that suitable solutions to determine the quality of a video sequence based on multiservice probes are needed. This chapter deals with an IPTV multiservice QoE management system developed as part of the SAVAGE project.

The SAVAGE project aims to design and develop an advanced system for IPTV multiservice quality management. This system consists of a multiservice advanced probe embedded in a monitoring platform that can operate automatically and remotely, allowing information on the QoE perceived by the end users. The SAVAGE project started in July 2011 and ended in December 2013; they are now implementing the integration of QoE algorithms inside the multiservice probes.

This chapter describes the concepts and proposals related to the objective assessment of the quality of audiovisual services, which consist of the automatic estimation of the QoE perceived by users. First, the state-of-the-art in relation to multimedia quality metrics and, second, the authors describe the multiservice IPTV probe to be developed during the project. In addition, they present the global QoE management system.

The proposed system is intended to deal with the most common problems on these networks, which are, especially, packet losses and network delays that can cause important degradations of video quality, such as blocking effects (i.e. macroblocking), freezing the video and audio losses. In addition, other impairments could affect the quality perceived by the end users, as coding artifacts or capture distortions. However, in real video delivery systems these degradations are less dramatic since an acceptable quality of service under normal conditions should usually be guaranteed.

Furthermore, the assessment of the video quality would facilitate the work of planning and designing distribution networks, and could allow video distributors to implement user fees based on the final quality that they can enjoy in their homes. In addition, an interesting application is real-time monitoring of the video quality perceived by the end user of delivery networks.

Further activities of this project will both consolidate the management architecture and implement the multiservice probe in terms of a prototype to measure the QoS in IPTV platforms.

1.2.5 Chapter 6: High speed multimedia flow classification

This chapter presents a system that unifies the entire process involved in flow classification at high speed. It captures the traffic, builds flows from the received packets and, finally, classifies them inside a graphics processing unit (GPU), at 10 Gbps using commodity hardware.

The authors propose a technique to speed up Deep Packet Inspection (DPI) processing for multimedia protocols using GPUs, and the methodology to integrate it inside a network probe. This shows that DPI with deterministic finite automata (DFA) can be used at very high speed with practically no system overhead. However, other problems apart from high speed traffic classification arise: it is difficult to obtain real high-speed traffic (10 Gbps and over) and build the flows on the fly, which in other context (e.g. below 1 Gbps) could be seen as an obvious thing. The GPU modules process up to 29.7 Gbps, which means about 14.5 mega flows per second.

The tests show how important signatures are when using DPI for flow classification. Signatures define the accuracy of protocol classification. The accuracy of each signature and how it influences false positives and false negatives should be studied during the process of signature creation. An example is the Real-time Transport Protocol (RTP).

Finally, the authors point out that the proposed system has a wide variety of possibilities and configurations, allowing its use in other types of classification, such as hyper text transfer protocol (HTTP), peer-to-peer (P2P) or other non-multimedia protocols. In addition, the high configurability allows us to vary the latency and throughput according to the needs of a given network.

The results show that the achieved performance is very much influenced by the number of protocols to find, and it is limited by the number of network flows. In any case, the system reaches up to 29.7 Gbps (about 14.5 mega flows per second).

1.2.6 Chapter 7: User driven server selection algorithm for CDN architecture

This chapter presents a new routing algorithm based on QoE for Content Distribution Network (CDN) architecture. Theoretically, CDN architecture has two main layers: the routing layer and the metarouting layer. The latter is composed of several modules such as server placement, cache organization and server selection. The first two modules belong to the preparation related phase. More precisely, the server placement module tries to place the replica servers in an optimized way to minimize the delivery delay and the bandwidth consumption. Providers use the cache organization module to organize the content stored in replica servers in order to guarantee the availability, freshness and reliability of content.

Besides these two preparation-related modules, authors focus on the server selection module, which plays an important role in launching the operation phase of a CDN. The fundamental objective of the server’s selection is obviously to offer better performance than the origin server. Another added value of this selection process is lowering the costs of network resources. It is not easy to choose an appropriate server to provide service to users. The appropriate server may be neither the closest one in terms of hop-count or end-to-end delay, nor the least loaded server. The best server is the one that makes end user satisfied when using the provided service. So, the server selection process plays a key role in the decisive success of a CDN.

The chapter includes some related research on server selection methods in the context of CDN. Subsequently, the authors explain their motivation to develop a server selection scheme based on machine learning approaches called multi-armed bandits.

1.2.7 Chapter 8: QoE approaches for adaptive transport of video streaming media

In this chapter, authors discuss the different transport approaches for adaptive video streaming media, and how they influence the QoE. These approaches are solely based on the HTTP protocol, and are specially designed for video transportation over the Internet to support the wide range of devices and maximize end user’s perceived quality. The leading groups and companies, e.g. Microsoft, Apple, Adobe and MPEG/3GPP, have introduced their own standard approaches to facilitate the on demand or live adaptive video streaming transport over HTTP.

The main goals of adaptive video streaming are to improve and optimize user’s QoE by changing the video quality according to network parameters, end user’s device properties and other characteristics. There are five main quality metrics of video streaming that affect the user’s engagement during video watching, and influence user’s QoE. The adaptive video streaming approaches use transmission control protocol (TCP) as a transport protocol. Based on network conditions, TCP parameters provide the client with vital information, and streaming is managed by the end user.

1.2.8 Chapter 9: QoS and QoE effects of packet losses in multimedia video streaming

This chapter analyzes the effect in a multimedia video streaming, i.e. peer-to- peer Television (P2PTV), of a common traffic metric, i.e. packet losses, on the quality parameters, i.e. QoS and QoE. Traditionally, QoS has been used to asses and guarantee the compliance of the deployed Service Level Agreements (SLAs). However, most of the network performance metrics used to estimate the QoS, are only limited to certain aspects of traffic without considering the end user’s subjective perception.

In this context, with the increasing presence of multimedia traffic, the user’s perception (QoE) of networked (multimedia) services has become a major concern for content providers and network operators. While a plethora of works propose solutions for QoS and QoE, authors put the focus, in this chapter, on the relationship between a usual traffic metric and the QoS and QoE assessment.

8 Quality of Experience Engineering for Customer Added Value Services

1.2.9 Chapter 10: A model for QoE estimation based on QoS parameter monitoring for multimedia convergent services (3-play)

In this chapter, the authors present a model for the estimation of user perceived quality, or QoE, from network and/or service performance and/or quality parameters (QoS) in multimedia services, and specifically in triple-play (3P) services: television (TV), telephony and data services, managed and offered by a single operator as a single package. In particular, it focuses on 3P convergent services (deployed over a common, IP-based transport network), and the relationship between the quality perceived by the users of such services and the performance parameters of the underlying network. Specifically, it contributes to the on-line estimation of such quality (i.e. during service delivery, in real or near-real time).

This way the chapter presents a model for the estimation of quality as perceived by the users (i.e. the user QoE) in 3P services. The model is based on a matrix framework defined in terms of user types, service components and user perceptions on the user side, and agents, agent capabilities, and performance indicators on the network side. A quality evaluation process, based on several layers of evaluation functions, has been described, that allows us to estimate the overall quality of a set of convergent services, as perceived by the users, from a set of performance and/or QoS parameters of the convergent IP transport network.

The full sets of services, user perceptions, valuation factors, agents, agent capabilities and performance indicators, have been provided. The full matrix of matching points between agent capabilities and user perceptions has been developed for the particular case of residential (domestic) users with a specific information flow (contents server external to the Internet service provider (ISP), no contents caching outside the contents provider). Valuation and parameterization functions for all services have been provided. For global service quality evaluation, weights for the final services, derived from service usage statistics, have been provided, as well as an example of the use of the analytic hierarchy process (AHP) method for deriving the weights of the elementary services of a final service (Internet access) as well as the weights of the perceptions of an elementary service (digital video broadcast in IPTV). Statistical results for the quality model of a representative service (video quality in IPTV) are presented.

In the summary, the chapter shows the applicability of the proposed model to the estimation of perceived quality (QoE) in convergent 3P services.

1.2.10 Chapter 11: Quality of experience estimators in networks

The Celtic Plus Quality of Experience Estimators in Networks (QuEEN) project was conceived to create a suitable conceptual framework for QoE, and make it operational by means of a suitable software architecture and quality models for different services, covering the full stack from the infrastructure on which a service runs, to the user who experiences it. In this chapter, we will present some of the conceptual results produced so far within QuEEN (and other related activities, such as COST Action IC1003 Qualinet), and the proposed mechanisms from making these concepts operational; that is, a way to theoretically model QoE for any type of (online) service, and a way to go from these theoretical models to concrete implementations. Furthermore, we will introduce some applications of this approach and of QoE in general, such as SLA management, and QoE-driven network management. The rest of the chapter is organized as follows. In section 11.2, the authors present an overview of the main concepts related to QoE (in particular state- of-the art concepts such) that have been produced within QuEEN. Section 11.3 presents the Application-Resource-Context-User (ARCU) model which provides the theoretical framework we propose for developing QoE models. section 11.4 details the proposed mechanism for making these models operational, known as the QuEEN layered model. In section 11.5, we introduce applications of QoE, in particular as envisioned within QuEEN. Finally, we conclude the chapter in section 11.6, where we also discuss possible lines for future research in this domain.

In this chapter, the authors have provided an overview of the QuEEN project’s approach to estimating QoE for generic services, and exploiting these estimates in various ways. They propose a conceptual framework for understanding QoE, for different service...

Table of contents

• Cover Page
• Contents
• Title Page
• Copyright
• Preface Abdelhamid Mellouk
• 1 Challenges For Quality Of Experience Engineering For Added Value Services Abdelhamid Mellouk and Antonio Cuadra-Sanchez
• 2 An Ecosystem For Customer Experience Management Antonio Cuadra-Sanchez, Mar Cutanda-Rodriguez, Andreas AURELIUS, Kjell BrunnstrÖM, Jorge E. LóPEZ De Vergara, Martin Varela, Jukka-Pekka Laulajainen, Anderson Morais, Ana Cavalli, Abdelhamid Mellouk, Brice Augustin and Ismael Perez-Mateos
• 3 Measuring Mpeg Frame Loss Rate To Evaluate The Quality Of Experience In Iptv Services Diego Hernando-Loeda, Jorge E. LóPEZ DE VERGARA, Javier Aracil, David Madrigal and Felipe Mata
• 4 Estimating The Effect Of Context On The Qoe Of Audiovisual Services Toni Máki, Martín Varela and Jukka-Pekka Laulajainen
• 5 Iptv Multiservice Qoe Management System Antonio Cuadra-Sanchez, Mar Cutanda-Rodriguez, Alfredo Pérez-Montero, Eugenio Rogles, Jesús Gutiérrez and Fernando Jaureguizar
• 6 High Speed Multimedia Flow Classification Rafael Leira Osuna, Pedro Gómez Nieto, Ivan González and Jorge E. López De Vergara
• 7 User Driven Server Selection Algorithm For Cdn Architecture Said Hoceini, Hai Anh Tran and Abdelhamid Mellouk
• 8 QoE Approaches For Adaptive Transport Of Video Streaming Media M. Sajid Mushtaq, Brice Augustin and Abdelhamid Mellouk
• 9 QoS And Qoe Effects Of Packet Losses In Multimedia Video Streaming René Serral-Gracià, Eva MARíN-TORDERA, Xavier Masip-Bruin and Marcelo Yannuzzi
• 10 A Model For Quality Of Experience Estimation Based On Quality Of Service Parameter Monitoring For Multimedia Convergent Services (3-Play) Pedro Dela Cruz Ramos, Raquel Pérez Leal, Francisco GonzáLez Vidal and Luis Bellido Triana
• 11 Quality Of Experience Estimators In Networks Frédéric Guyard, Martin Varela, Lea Skorin-Kapov, Antonio Cuadra-Sanchez and Pedro J. Sevilla-Ramos.
• 12 QoE-Based Network Selection In Heterogeneous Environments Kandaraj Piamrat, Hyunhee Park, Kamal Deep Singh and César Viho
• List of Authors
• Index