# LS on Subjective Quality Evaluation of AI-based Ultra-low Bitrate Voice Codecs

## Document Information

**Source:** ITU-T Study Group 12  
**Reference:** SG12-LS18 (SG12-TD298R1)  
**Date:** Geneva, 9-18 September 2025  
**For Information to:** 3GPP SA4  
**Deadline:** June 2026

## Background and Context

ITU-T SG12 reviewed multiple contributions during its September 2025 meeting addressing ML- and AI-based speech processing features and their evaluation using subjective tests and objective models. A key contribution (SG12-C40) from Rohde & Schwarz SwissQual AG presented P.800 listening test results that included several AI-based ultra-low bitrate codecs.

## Main Technical Findings

### Performance of AI-based Ultra-low Bitrate Codecs

The P.800 listening tests demonstrated that several AI-based ultra-low bitrate codecs scored better than traditional low-bitrate codecs, proving their usefulness in the telecommunication context.

### Limitations of Current Test Methodologies

While the P.800 overall MOS rating procedure appears generally appropriate for evaluating these codecs, SG12 identified potential limitations:

- **Timbre shifts** may not be adequately reflected in traditional MOS ratings
- **Small drops in information** that could be critical in mission-critical calls might not be captured
- New test methods that better reflect these perceptual effects may be necessary

### E-model Integration Concerns

SG12 concluded that it would be **premature** to incorporate these AI-based codecs into the E-model (by deriving equipment impairment factors) due to:

- Subjective effects not yet fully covered by available test methods
- Rapid evolution and changes in the respective codec technologies

## Request for Feedback

ITU-T SG12 is seeking feedback from 3GPP SA4 based on their experience before deciding to:

- Start development of new appropriate subjective test methodologies
- Develop potential corresponding prediction models

The liaison includes attachment SG12-C40 describing a subjective ACR LOT testing of AI-based and ultra-low bitrate codecs in a full-scale real field context.