An experimental evaluation and characterization of VoIP over an LTE-A network

Nowadays, the high demand for low-cost, voice-based communication services has a great impact on the growth of the mobile VoIP (Voice over IP) market. According to a recent Ericsson research [1], VoLTE (Voice over LTE) subscriptions will reach 5.5 billion by the end of 2023. This unprecedented growth is sustained by an ever decreasing cost of internet-based services, including VoIP, and flat-rate packages. However, by contrast to conventional IP services, which operate on a best-effort model, VoIP sessions pose real-time constraints on the network Quality of Service (QoS) metrics, aiming at acceptable levels of user Quality of Experience (QoE). VoIP sessions typically require a one-way latency between the callers of no more than 150 milliseconds, average jitter (that is the delay variation) under 30 msec, and packet loss rates not exceeding 1% [2].

However, the relationship between such QoS parameters and the achievable VoIP service QoE is highly non linear, and several compensation mechanisms exist in the network and terminals to address jitter or packet loss. When the audio codecs work in tandem with packet buffering and just-on-time re-transmission, it is possible to tackle to a certain extent the variability of mobile network conditions (e.g. in poor reception areas of during handover). Yet, determining how individual QoS parameters affect QoE under a broad range of conditions and codecs is a hard problem. With the present study, we aim to better evaluate the actual performance of mobile VoIP across a real-world LTE-Advanced (LTE-A) domain, under a range of operative conditions. We embark on an empirical analysis which allows us to capture the actual end-to-end VoIP quality achieved in a pilot study, for different codecs and network conditions. Two original contributions emerge in this work.

The first one, is an experimental performance assessment based on a real-world LTE measurement campaign. The end-to-end VoIP traffic has been collected in both mobile-to-mobile and mobile-to-fixed settings. Several voice flows are included, considering a range of audio codecs. In addition to the network parameters, we consider diverse metrics that are not typically taken into account in the literature, including the R-factor, the voice signal levels, the call setup delay, and the session disconnect delay.

The second contribution pertains the statistical modeling of the two most critical parameters in VoIP traffic: jitter and RTT. These metrics are treated as random variables and characterized by means of Generalized Extreme Value (GEV) distribution, along the estimation of their parameters (shape, scale, location) via the Maximum Likelihood (ML) criterion. This analysis reveals that jitter and RTT exhibit short and long-tail behaviors, respectively. It provides numerical results that play a crucial role in creating close-to-reality models. With this work, we make a step forward in the quest to better understand how the mismatch between real-time VoIP services (QoE) and non-real-time (best-effort) networks (QoS) affects end-to-end quality in mobile cellular networks.

The paper is organized as follows: Section II presents a review of the most significant literature in relation to this article. In Section III, we provide a detailed description of the experimental setting used to carry out the measurement campaign. Section IV provides performance evaluation results, with the dual purpose of providing scientific insights, but also serving as a benchmark for an operator to improve the efficiency of VoIP services. In Section V, we propose a statistical modeling of jitter and RTT, where some basic concepts about extreme value distributions are recalled. Finally, Section VI draws conclusions and explores possible research directions.

The characterization of mobile voice traffic continues to be an important research theme, since the network infrastructures are continuously evolving. It is particularly useful to evaluate the suitability of mobile networks as these get ready to support truly advanced services such as HD Video, VoLTE, HD Voice, to mention but a few. These have a great impact both on QoE (a subjective measure of perceived quality) [18, 4, 5, 7], and on network QoS metrics. QoS can benefit from the new high-speed networks but is negatively affected by uncontrolled phenomena that typically arise in a radio environment (weather conditions, obstacles, time-varying load of network nodes).

Most of works proposed in the literature are focused on characterizing such QoS metrics by performing simulations in “artificial" environments where some parameters (e.g. the packet loss) can be tuned. A simulated scenario obtained by means of the Vienna LTE-A downlink simulator has been considered in [8], where authors evaluate some throughput-based performance metrics, considering various spatial deployments. A simulator that accounts for the physical layer of the LTE infrastructure is instead proposed in [9], where the radio channel is reproduced by combining deterministic wave propagation structures with stochastic channel models. Interesting also the RTP performance study over a simulated LTE environment presented in [10], where a mobility scenario characterized by movement at 3 km/hour between two radio cells has been taken into account.

Many other works rely on the popular NS-2/NS-3 simulator equipped with LTE modules. Among such works we cite: [11] that proposes a performance evaluation of WebRTC traffic by using a mixed simulated/emulated environment with an LTE customized module; [12] where delay and throughput measures are derived for TCP traffic in an LTE setting; [13] where authors analyze the effect of mobility at high speeds with and without handover procedures; [14] that presents a performance evaluation of a streaming video service based on the Real-Time Messaging Protocol across an LTE domain, and where different QoS parameters have been considered.

Other works, focused on characterizing the performance of data traffic in a real LTE environment, have some limitations. A semi-realistic testbed is proposed in [15], where a limited set of performance figures are derived for 3G and 4G networks using Jperf, a software used to analyze and measure network traffic. In [16] a more realistic, yet still limited, campaign has been carried out to evaluate some traffic metrics in an LTE environment but no codecs are considered to vary across traffic sessions. Again, a throughput analysis has been assessed in [17] in a live LTE setting, where some key indicators (jitter, delay, R-Factor) are not be considered.

Inspired by this literature, we propose a detailed performance assessment of real mobile VoIP traffic across an LTE-A environment with the double aim of: i) going beyond the obvious limitations encountered when dealing with simulation tools, which typically cannot carry out an end-to-end quality analysis; and ii) offering a richer characterization of mobile VoIP traffic by considering also metrics that are typically not available in other studies.

Results of our study can also be applied to the 5G domain, whose architecture is quite close to LTE-A. In this sense, interesting findings appear in [18] where, by means of an extensive simulation campaign carried out through the OMNet++ network simulator, the performance of an algorithm called TYDER has been evaluated in terms of metrics such as throughput and delay. Furthermore, results presented in our paper might also be conveyed in 5G-compliant procedures such as VLM+ proposed in [19], where novel QoS-based concepts (e.g. Virtual link end-to-end QoS, profit driven QoS class prioritization) are taken into account. Finally, some attempts in characterizing QoE-related metrics for 5G scenarios have been performed in [20]. Although carried out in a controlled environment, where no external phenomena are present, authors evaluate metrics related to the subjective users’ perception of streamed videos.

This section is devoted at describing the scenario and the setting that have been considered during the experimental campaign (performed across the city of Salerno, Italy) pictorially illustrated in Fig. 1. All the described trials have been carried out by exploiting a real LTE network that, typically, is based on the following key nodes: the eNB (enhanced Node B) designated to supervise the overall radio communication, the MME (Mobility Management Entity) in charge of managing the VoIP sessions and the authentication procedures together with HSS (Home Subscriber Server), the SGW (Serving Gateway) having routing functions, the PGW (Packet Data Gateway) playing as a proxy server for external networks or domains (e.g. IP Multimedia Subsystem), the PCRF (Policy and Charging Rules Function) devoted at managing billing and charging procedures.

Two endpoints (say, $A$ and $B$) have been considered in the experiment. The first one ($A$) is an iOS-based smartphone equipped with Linphone, a popular open-source VoIP software that allows to make/receive phone calls by means of two protocols: the Session Initiation Protocol (SIP) in charge of managing the signalling part of the communication; and the Real-time Transport Protocol (RTP) in charge of carrying the call content. Linphone natively includes some audio codecs, even if some extra licensed codecs have been purchased to enrich the experiment (see Table I for a complete list of used codecs along with their main features). The smartphone is equipped with a 4G-Plus Sim Card that allows to exploit the LTE-A services offered by the telecom operator (Vodafone) during voice calls. The second endpoint ($B$) is a personal computer equipped with the same VoIP application and connected to a network sniffer used to capture the entire VoIP session during the call. The two endpoints are synchronized by means of the Network Time Protocol (NTP), and are both attached to the NTP primary server of the Italian Institute on Metrologic Research (thus, the NTP inaccuracy is negligible). All measurements are performed at the second endpoint where, through the network sniffer, we can access packets sent and received from both ends ($A\rightarrow B$, $B\rightarrow A$) providing consistent information, thanks to the common clock reference.

Aimed at providing a detailed characterization and a useful performance assessment, in this section we consider a set of experiments and measurements that include: i) a set of bivariate analyses useful to catch the mutual influences between some key factors (consumed bandwidth, RTT, R-Factor) and the jitter variation within the mobile scenario for various codecs; ii) a set of CDF-based comparative performance analyses amenable to characterize the behavior of some VoIP parameters when acting in the mobile and fixed scenario, and for various codecs; iii) a set of measurements aimed at characterizing some high-level parameters that intervene during the signalling phase of a VoIP session.

In this section, we analyze two QoS metrics that have a significant impact on communication, that are jitter and RTT [42, 43]. These are particularly important due to the best-effort nature of IP networks, whereby packet delivery time and order are not guaranteed. Hereafter, we introduce a statistical characterization of jitter and RTT (we just consider the case of mobile setting) based on the measurement campaign described in the previous sections.

In the scientific literature, several work have been devoted at modeling these metrics, although most studies are based on simulations and have lead to approximate results. It is commonly accepted that many QoS-related metrics follow heavy-tailed distributions, as confirmed by relevant work reported in [44], [45], [46], which specifically analyze RTT distributions.

Even more sophisticated statistical analyses have been devoted to jitter, leading to similar considerations. For instance, some analytical approximations are provided in [47], which characterize jitter as a function of background traffic in ATM networks. In [48], the authors propose an analytic model for jitter, by simulating incoming traffic as an interrupted Poisson process. In [49], a mathematical model of jitter in a queuing system supported by a simulated environment has been proposed. Again, in [50] the authors propose a model through a generalized Pareto distribution. Whereas in [51], jitter has been modeled by invoking self-similar structures and by evaluating the Hurst parameter to confirm its memoryless property.

By contrast to previous literature, our analysis is devoted to characterize RTT and jitter in a real scenario, with emphasis on a variety of VoIP flows and considering 8 different codecs. Among candidate heavy-tailed distributions, we choose the Generalized Extreme Value (GEV) distribution which offers the flexibility of embodying more distributions. Such a choice is confirmed by our empirical analysis where we perform selective goodness of fit tests (among a set of 10 distributions) based on the Schwarz criterion [52].

In this way, we can find the optimal distribution among 10 different candidate ones, representing jitter and RTT as Generalized Extreme Value (GEV) distributions.

The GEV distribution was originally introduced by Jenkinson in [53, 54], and has a CDF defined as:

where $\xi$, $\sigma$, and $\mu$ are shape, scale, and location parameters, respectively. Among these, shape is the most critical parameter since its value sign determines the type of distribution [55]. In particular, for $\xi>0$ the distribution is called Fréchet-type distribution; for $\xi<0$ the distribution becomes a Weibull-type distribution; and the case $\xi=0$ corresponds to the Gumbel distribution. Location and scale parameters govern the shift of distribution and the shift deviation, respectively. The distributions of RTT and jitter concerning the mobile traffic have been completely characterized by estimating GEV parameters through the ML criterion [56].

We should note that, although the maximum likelihood estimate (MLE) has been successfully applied, when dealing with extreme distributions a possible issue could arise. The regularity conditions required for the asymptotic properties of MLE could be violated in some cases. Smith [57] faced in detail this issue by proving that: for $\xi>-0.5$, ML estimators exhibit the standard asymptotic properties; for $-1<\xi<-0.5$, ML estimators are generally achievable even if they do not have asymptotic properties; and for $\xi<-1$, it is unlikely to obtain ML estimators. In our regimes we always fall in the first two cases, so no critical issues occur.

Let us now consider the PDF associated to (1):

Let us neglect the particular case $\xi=0$, which implies some different considerations concerning the Gumbel limit of GEV distribution, and consider the case $\xi\neq 0$. We will show in the forthcoming analysis that this case is in accordance with estimated shape parameter.

Let $\theta=(\xi,\sigma,\mu)$ be the triplet of unknown distribution parameters to be estimated; and assume that the empirical data (representing RTT and jitter realizations) $Z_{1},\dots,Z_{n}$ form a sequence of i.i.d. random variables. We can express the likelihood function as follows:

for $1+\xi\frac{z_{i}-\mu}{\sigma}>0$ and for all $z_{i}$.

The corresponding log-likelihood has the following form:

where we set $\omega_{i}=\xi^{-1}\ln\left(1+\xi\frac{z_{i}-\mu}{\sigma}\right)$. By definition, the MLE $\hat{\theta}=(\hat{\xi},\hat{\sigma},\hat{\mu})$ is defined as:

being $\Theta$ the set of variable bounds. The solution of likelihood system is obtained by differentiating (4), which does not however admit a closed form, and is solved with the Newton-Raphson’s method.

More precisely, in [58], the author proposes a modified version of Newton-Raphson’s iteration method designed to improve the speed and the probability of convergence. Such a $4$-step procedure (called MLEGEV) allows to guarantee the convergence by the introduction of some ad-hoc constraints.

In short, the cited routine solves the likelihood equation $\partial l/\partial\theta$ through the following iteration:

where the derivatives $v$ and $H$ follow by numerical calculation as analyzed in [59].

Figures 12 and 13 summarize the obtained results for jitter and RTT, respectively. More precisely, each subfigure consists of two panels: the upper one shows the real data distribution for each codec (red curve) and the GEV fitting distribution (blue curve), expressed through CDFs. The lower panel shows the absolute error obtained by considering the punctual difference between observed and fitted data.

Tables III and IV report the estimated values $\hat{\xi}$, $\hat{\sigma}$ and $\hat{\mu}$ for GEV distributions, modeling the jitter and RTT, respectively. The last column ($E_{max}$) indicates the maximum error between real data and fitted distributions, expressed in terms of Kolmogorov-Smirnov distance, namely $E_{max}=\sup_{x}|D_{n}(x)-D(x)|$, where $D_{n}(x)$ and $D(x)$ are the empirical and hypothesized distributions, respectively.

An interesting consideration may be drawn from the results, concerning the shape parameter $\xi$ that is strictly related to the tail behavior of distribution [56]. Considering jitter, $\xi$ has always a negative value, indicating a short-tailed behavior. In other words, the jitter probability distribution decays to zero very fast. This result is perfectly reasonable with the strict constraints (often indicated by Service Level Agreements - SLAs) that telco operators must obey to, since jitter is considered to be the primary source of quality degradation during a VoIP session. Thus, a de-jittering buffer on receiver side is often needed to mitigate this negative effect.

On the contrary, in case of RTT, $\xi$ always exhibits positive values, indicating a long-tailed behavior. This suggests that extreme values of RTT can occur more frequently than the case with jitter. This can be justified by considering that in complex (real-world) mobile scenarios, RTT is subjected to unpredictable variations due to phenomena that negatively interfere over the packet flows (e.g. multi-path fading). Thus, these more dispersed values slow down the tail’s decay.

These findings have interesting implications in terms of improving existing models and simulations of jitter and RTT, particularly in mobile communication scenarios. Furthermore, these results suggest that, regardless of the codec type, the probabilistic behavior of jitter and RTT may be well represented by a set of GEV distributions, which can capture both short- and long-tail structures at the same time.

A plethora of network tools available today aim at simulating VoIP traffic, with great interest arising around LTE-based environments. Yet, at the current state of play, it is not yet possible to realistically deal with the complexity of end-to-end mobile VoIP across an LTE infrastructure. This is because it is so hard to reproduce in simulation the phenomena that affect radio communications, such as time-variant interference, effect of weather, and state of load on nodes, to mention but a few. Due to these limitations, it is hard to accurately characterize QoS metrics.

In this article, we address this challenge, designing a detailed measurement campaign of voice traffic across an urban LTE-A domain. Our trials take into account two operative settings: a first scenario involving a set of LTE-A voice sessions between two fixed users; and a second scenario that accounts for LTE-A voice sessions between a mobile user (a driver) and a fixed one. The field trials have been carried out by considering families of $8$ different codecs, and a total of around 750,000 voice RTP packets. In this way, we carry out performance tests aimed at evaluating the impact that external environmental factors have on voice communication. We also capture the non-linear influence that different types of voice codecs have on QoS.

Based on the collected data, we then propose a statistical characterization of jitter and RTT, the two most critical quality metrics in VoIP. We derive a Generalized Extreme Value distribution for both metrics, estimating the pertinent parameters (shape, scale, and location) by means of Maximum Likelihood estimators. The resulting model gives a contrasting behaviour for jitter and RTT, which exhibit a short- and long-tail, respectively.

Our work offers a novel viewpoint on voice traffic characterization, based on metrics that are typically neglected in classical studies, particularly the R-Factor and voice signal level. Experimental results show how environmental phenomena (such as interference) can generate unpredictable behaviour that may result in counter-intuitive results. On occasions, we noticed that some mobility settings were exhibiting more stability than fixed ones. Statistical characterization helps to better understanding non-linear behaviour and appreciating end-to-end effects on quality. It is also of paramount importance in the improvement of simulation modeling.

Interestingly, our findings could be applied also to 5G networks, which have many aspects in common with the LTE-A architecture. This is the case of high-level metrics such as SDD/CSD that, derived in terms of SIP protocol performance, allow to easily extend the analysis to 5G enabling architectures such as the IP Multimedia Subsystem.

Finally, our study may be further extended, particularly in the scale of the measurement campaign and considering a broader range of parameters (e.g. user’s speed and environmental conditions). It will be beneficial to gain greater insights on the specific effects of handover on QoS. Vertical and horizontal handover are expected to create different effects, which will become increasingly important in future hybrid networks.