# Summary of S4-260156: Non-BE Conformance of Metadata Output in IVAS

## 1. Introduction and Background

The document addresses conformance testing issues for IVAS codec as defined in TS 26.252. Current conformance requirements include:
- Bit-exact conformance for both fixed-point and floating-point implementations (audio signal and metadata output)
- Non-bit-exact conformance with tolerances for floating-point implementations
- **Current limitation**: Metadata output deviation is set to zero (no deviation allowed)

## 2. Observed Issues

### Platform-Specific Non-Bit-Exact Behavior

Post-Release 19 experiments revealed that **metadata output does not always remain bit-exact** compared to conformance references across different platforms.

**Specific example identified:**
- **Platform**: Apple MacBook Pro with M4 ARM processor, Mac OS 15.7.3, Clang 16
- **Issue**: OMASA decoder conformance test vector produces minor numeric differences in output MASA metadata
- **Specific differences**: 
  - Direct-to-total and diffuse-to-total ratios affected
  - 9 frames contain maximum absolute difference of 1/255 each
  - Subjective impact is minimal

### Broader Implications

- Similar differences expected on other platforms
- Encoder-side implementation differences may create **larger differences in spatial direction metadata** due to minor metadata value changes affecting encoding decisions

### Analysis Tool Development

A **MASA metadata difference tool** has been developed as part of IVAS public collaboration to analyze metadata differences:
- Provides overall and frame-by-frame statistics
- Reports maximum and mean absolute difference values for spatial metadata parameters
- Includes difference reporting for descriptive metadata parameters
- Details provided in Annex A

## 3. Conclusions and Recommendations

The source proposes that **non-bit-exact tolerance for metadata in Annex A.3.1.2 of TS 26.252 should be adjusted**.

**Next steps requested:**
- Additional experiments needed to establish suitable criteria for non-bit-exact metadata conformance
- Invitation to IVAS codec implementors to share non-bit-exact results
- Goal: Set reasonable limits for both **MASA metadata and ISM metadata** conformance
- Ensure high-quality implementations across different platforms

## Annex A: MASA Metadata Difference Tool (masaDiffTool)

### A.1 Purpose and Scope

Informative tool designed to support evaluation of non-bit-exact behavior in MASA-format metadata produced by IVAS implementations. Primary use case: comparison between reference and alternative implementations.

### A.2 Tool Functionality

**Validation and comparison features:**
- Validates both inputs use IVASMASA format
- Performs structured comparison of descriptive and spatial metadata across all frames
- Reports exact matches or identifies differing frames
- Computes aggregated difference measures (maximum and mean absolute deviations) for key spatial parameters
- Returns 0 only if metadata files are identical

**Error handling:**
- Stops comparison if metadata format doesn't match IVASMASA
- Classifies frames with mismatched numbers of spatial directions as entirely different and continues with next frame

### A.3 Tool Usage

**Command syntax:**
```
masaDiffTool [options] refMetaFile cutMetaFile
```

**Optional output formats:**

1. **Text report** (`--report <file>`): Frame-by-frame summary including spatial and descriptive metadata differences
2. **CSV report** (`--csv <file>`): Structured per-frame table with key difference indicators and summary metrics

**Standard output includes:**
- Overall metadata difference status
- Count of frames with differences
- Summary status for descriptive and spatial metadata
- Aggregated maximum and mean absolute differences across all frames for:
  - Direction
  - Energy ratios
  - Coherence measures

### A.4 Availability

- Binary version available in IVAS codec public repository (3GPP Forge)
- Will be contributed to official TS 26.252 software package during next update cycle