Opale Technical Blog
Deep insights, practical solutions, and expert analysis for tech enthusiasts.

POLQA vs PESQ – “Why does PESQ give a lower score?”

Author icon Opale Systems   Calendar icon Thursday, 23rd June 2022

Do I really need POLQA?

The question is understandable, as the cost of POLQA can be a bit higher than the cost of PESQ.MultiDSLAcustomers recognize great value and sometimes want to pursue it to the limit! Well, predictably, the answer is "It depends..." but to be more helpful, Opale has published a video describing specific cases where POLQA is the best solution and also when the PESQ metric - about 10 years older than POLQA - might be perfectly good enough. The video is available here , in case you missed it.

From time to time, customer questions bring to mind the practical reality of the PESQ vs. POLQA debate, and that's exactly what happened recently, when we got this question:

"Why do I only get a PESQ score of 1.90, when POLQA gives 3.86?"

 

How do you answer a question like that?

"Why do I only get a PESQ score of 1.90, when POLQA gives 3.86?"

 

MultiDSLA Analysis

But getting back to that question, here's how the two sets of results - POLQA first, then PESQ - came about:


POLQA vs PESQ
POLQA vs PESQ

So how does the survey work?

 

Step 1: Listen

Listening to the degraded signal alone doesn't reveal much. In fact, it sounds reasonably good, not excessively distorted, and there are no "gaps" in the speech. But then listening to the reference signal a few times reveals something interesting - the speech in the degraded signal actually comes out faster during the first of the two sentences!

 

Step 2: Look

Next, there's a visual clue, clear enough in the screenshots above. The weakest of the three traces - the "error surface" in the POLQA analysis - shows small, but fairly consistent errors. The same trace in the PESQ analysis (highlighted) shows larger errors throughout the first sentence and right at the beginning of the second. What is going on?

 

Step 3: Going Further

It's time to go through some of the other POLQA and PESQ graphs provided by MultiDSLA to see what we can discover. When we get to the Sample (Frame) Time Offset view, this is what we see, again POLQA first:


POLQA vs PESQ
POLQA vs PESQ

Well, they both tell the same story, sort of...

 

Latency/Delay Changes

POLQA tells us that during the first of the two sentences, the one-way latency (delay) decreases by more than 200 ms over a period of about 2.1 s. POLQA indicates that this is an approximately linear fit, corresponding to a speech "speed-up" of about 9.5%. Interestingly, speech pitch appears normal while this is occurring. One would expect that "speeding up" speech by nearly 10% would raise the pitch, but that is not what we hear.

What does PESQ say? Well, PESQ shows a similar overall effect but interprets the latency reduction as a series of discrete steps rather than a continuous process. So what's going on?

 

Codec behavior discovered

We find the explanation in the way some codecs handle adverse network conditions through a process called "time warping." Codecs with this capability belong to a class called relaxed code excited linear prediction (RCELP) that can vary the read rate of a jitter buffer while approximating the original signal pitch. Reading a signal faster than normal allows the codec to regain the time it "lost" after the jitter suppression buffer was increased to accommodate a delayed packet arrival time.

So what? And how does this explain the low PESQ score and the high POLQA score?

 

Mobile Codecs - State of the Art

The example in this blog is from a Skype Internet call, and the effect is pronounced and easy to see. But the same technique works on a smaller scale in codecs used in mobile networks, and that's where the limitations of PESQ became clear over a decade ago. This prompted interested parties to publish technical reports suggesting that inappropriate use of PESQ measurements could result in unfair, or even invalid, characterization of codec performance. You can see an example of this here.

 

And now there is POLQA

The partial obsolescence of PESQ was also one of the driving forces behind the development of a new objective voice quality analysis technique, which eventually led to the first commercial version of POLQA in 2011.

So, our client's simple question about PESQ vs. POLQA took us a long way, but we got to the explanation and discovered - guess what? That you have to use the right tools for the job!

 

Contact Opale Systems or your distributor for more information.

Open cookie management panel
Close panel
This site uses cookies to ensure its proper functioning. It also uses cookies from third party services to provide advanced functionality. At any time, you can choose which services you wish to activate or decide to withdraw your consent.
 
Customise accepted services
You are free to choose which services you wish to enable. By authorising these third party services, you agree to the deposit and reading of cookies and the use of tracking technologies necessary for their proper functioning. By withdrawing your consent for some of these services, some website features may no longer function.
Website navigation  Read more
The site writes a session cookie to enable it to function properly and to help with navigation. It cannot be deactivated.
Usage: 1 cookie, records the session identifier.
Time to live: The cookie is present during the entire session on the site. It becomes obsolete after 24 minutes of inactivity.
Mandatory
Media Popup
Display videos from Youtube or Dailymotion.
Google Analytics  Read more
Records website statistics.
 
Accept all Refuse all Manage
Follow us on LinkedIn Follow us on Youtube