# Summary of S4-260272: ULBC Codec Testing in Background Noise

## Document Overview

This contribution proposes a testing framework for the Ultra-Low Bitrate Codec (ULBC) in noisy conditions, drawing from EVS codec testing methodologies. The document is a revision of S4-251786 from SA4#134 and proposes updates to TR 26.940 Clause 9.

## Background and Motivation

### Noise Suppression Considerations

The document argues against mandating NS algorithms within the codec specification based on several key considerations:

- **Device-Specific Optimization**: NS algorithms are typically optimized for specific device microphone array configurations. A generic NS algorithm applied uniformly could result in suboptimal performance across different device types.

- **Codec Robustness vs. NS Artifacts**: Testing ULBC with clean, noisy, and optionally NS-processed speech provides better understanding of the codec's inherent robustness. NS algorithms may introduce speech distortions that could bias codec testing results.

- **Emergency Call Requirements**: For emergency calls, preserving background noise is critical as it may contain important contextual information (alarms, traffic, voices) that helps identify the caller's environment or ongoing danger.

- **Complexity and Latency Concerns**: ML-based NS algorithms can be computationally complex, increasing power consumption and end-to-end latency. Mandating complex NS could burden some devices inefficiently.

The document advocates for flexibility in NS implementation to enable manufacturers to develop device-specific solutions.

## Proposed Testing Framework

### Core Testing Scenarios (Table 9.1.4.1)

Following EVS codec testing principles (TR 26.952), the proposal includes:

| Source Material | Noise Type | SNR | Test Methodology |
|----------------|------------|-----|------------------|
| Clean speech | - | - | ITU-T P.800 ACR and/or DCR |
| Speech + Noise | Stationary (car, etc.) | 15 dB | ITU-T P.800 DCR |
| Speech + Noise | Non-stationary (street, babble, etc.) | 20-25 dB | ITU-T P.800 DCR |

This framework aligns with EVS testing which used:
- Car noise at 15 dB
- Street noise at 20 dB  
- Office/babble noise at 20 dB
- ITU-T P.800 DCR methodology ("Degradation of Speech in Noise" DMOS test)

### Optional Extended Testing for Low SNR (Table 9.1.4.2)

To characterize ULBC robustness in challenging low SNR conditions:

| Source Material | Noise Type | SNR | Test Methodology |
|----------------|------------|-----|------------------|
| Speech + Noise | Stationary (car, etc.) | 5-10 dB | ITU-T P.800 DCR |
| Speech + Noise | Non-stationary (street, babble, etc.) | 10-15 dB | ITU-T P.800 DCR |
| NS processed speech + Noise | Stationary (car, etc.) | 5-10 dB | ITU-T P.800 DCR |
| NS processed speech + Noise | Non-stationary (street, babble, etc.) | 10-15 dB | ITU-T P.800 DCR |

**Key Notes:**
- To avoid bias, a **common NS processing tool** should be used for generating NS-processed speech
- Selection of specific noise types and the NS processing tool is FFS
- Reference is made to TR 26.989 v19.0.0 (MCPTT work) where EVS was evaluated in siren noise at 5 dB SNR

## Proposed Specification Changes

The document proposes adding new Clause 9.1.4 to TR 26.940 with two subclauses:

- **9.1.4.1 Background**: Captures the rationale for flexible NS implementation
- **9.1.4.2 Recommendations for ULBC Codec Testing**: Defines the testing framework with Tables 9.1.4.1 and 9.1.4.2

## Action Requested

The document seeks **Discussion and Agreement** on:
1. The proposed testing framework for ULBC in noisy conditions
2. Updates to TR 26.940 Clause 9 as specified in the text proposal