# Summary of S4-260119: glTF-based Representation Formats for 3D Gaussian Splats

## Introduction and Scope

This contribution addresses **Objective 2c** of the FS_3DGS_MED Study Item ("Determine relevant formats") by providing a comprehensive analysis of glTF-based representation formats for 3D Gaussian Splatting. The document identifies a gap in TR 26.958 V0.1.1, which currently only mentions PLY as a storage format without comparative analysis of the emerging glTF-based format ecosystem from Khronos and MPEG.

The contribution proposes a **two-layer architecture** combining:
- **KHR_gaussian_splatting** (Khronos) for canonical splat semantics
- **MPEG_gaussian_splatting_transport** (MPEG-I Scene Description) for distribution and streaming capabilities

## KHR_gaussian_splatting (Khronos Layer)

### Core Attribute Semantics

The Khronos extension (review draft published August 2025) defines Gaussian splats as **POINTS primitives** within standard glTF 2.0 with the following attributes:

- **POSITION** (VEC3, required): Splat center position using standard glTF base attribute
- **ROTATION** (VEC4, required): Quaternion (x,y,z,w) for local axes orientation
- **SCALE** (VEC3, required): Per-axis scale in log-space
- **OPACITY** (SCALAR, required): Opacity in range [0,1]
- **SH_DEGREE_l_COEF_n** (VEC3, conditional): Spherical harmonics coefficients organized by degree (0-3) and coefficient index for view-dependent lighting
- **COLOR_0** (VEC3/VEC4, recommended): Baseline color for fallback point-cloud rendering

### Extensibility and Backward Compatibility

Key design features:
- **Nested extensions mechanism** inside the KHR_gaussian_splatting object allows other extensions to add compression, alternative encodings, or processing without duplicating semantics
- **Graceful degradation**: Clients not recognizing the extension can still render as standard point cloud using POSITION and COLOR_0
- Provides strong anchor for MPEG and 3GPP work targeting interoperable distribution and streaming

## MPEG_gaussian_splatting_transport (MPEG Layer)

### Architecture Approach

The MPEG extension is carried as a **nested extension** inside `KHR_gaussian_splatting.extensions`, avoiding semantic duplication and adding only transport-level features.

### Transport-Level Features

#### Alternative SH Layouts

Two MPEG-specific SH coefficient storage modes alongside Khronos default:

1. **mpegProgressive layout**: 
   - Groups coefficients by SH degree (degree 1, 2, 3 as separate SCALAR accessors)
   - Efficient for progressive refinement
   - Receiver can render with only SH degree 0 data and incrementally fetch higher degrees
   - DC (degree 0) term reconstructed from COLOR_0.rgb or carried via KHR SH_DEGREE_0_COEF_0

2. **mpegPerChannel layout**:
   - Separates coefficients by color channel (R, G, B)
   - More efficient for certain compression schemes

#### Progressive Download

- Optional progressive ordering signaled by listing accessor indices in `progressive.stages`
- Ordered from lower to higher fidelity
- Receiver may initially fetch only first stage and progressively refine without re-decoding previous data

#### Timed Delivery for 4D Splats

- Dynamic 4D Gaussian splat sequences supported using existing MPEG timed media mechanisms
- Accessor treated as time-varying if and only if it carries `MPEG_accessor_timed` extension
- Timed accessors backed by circular buffers as defined by MPEG-I Scene Description

## Two-Layer Architecture Benefits for 3GPP

### Architectural Summary

- **Layer 1 (Khronos)**: Canonical splat semantics (geometry, appearance, SH lighting) and fallback point-cloud path
- **Layer 2 (MPEG)**: Progressive download, timed delivery, and alternative SH layouts as nested extension

### 3GPP Service Integration Advantages

1. **Alignment with existing 3GPP specifications**: glTF already adopted by TS 26.118 (Immersive teleconferencing) and TS 26.119 (MeCAR)

2. **5GMS adaptive delivery mapping**: Progressive download and timed delivery map naturally to 5G Media Streaming

3. **Bandwidth-adaptive quality**: Progressive SH degree layout enables network/receiver control of SH levels to fetch, analogous to spatial/temporal layer selection in scalable video codecs

4. **Future-proof extensibility**: Clear path for future compression extensions (e.g., from ongoing MPEG Gaussian Splat Coding exploration) and tiled spatial delivery without breaking backward compatibility

## Format Comparison

### PLY
- De facto training output format
- Raw float32 attributes without compression
- Very large files (typically 200+ MB for single scene at SH degree 3)
- **Limitations**: No extensibility mechanism, no progressive delivery support, no scene graph, no standard metadata support (camera parameters, animation)

### SPZ (Splat Zip)
- Developed by Niantic as compact binary container
- Applies quantization and packing (~90% size reduction vs PLY)
- Extension under development in Khronos
- Superior compression schemes (e.g., Qualcomm's L-GSC) also being considered

### glTF + KHR_gaussian_splatting + MPEG transport
- Full scene graph support (nodes, transforms, animations)
- Standard extensibility
- Backward-compatible fallback
- MPEG transport layer for progressive and timed delivery
- Signaling and usage of different compression schemes through proper extensions
- **Recommended as primary format path for 3GPP**

## Proposals for TR 26.958

The contribution proposes to include the following in TR 26.958 Section 4 and new subsection under Section 11:

1. **Document KHR_gaussian_splatting** as emerging industry baseline for 3DGS representation in glTF, including:
   - Attribute semantics
   - SH coefficient organization
   - Backward-compatible fallback via POINTS
   - Extensibility mechanism

2. **Document MPEG_gaussian_splatting_transport** being developed within MPEG-I Scene Description, including:
   - Progressive download
   - Timed delivery for dynamic 4D Gaussian splat sequences
   - Alternative SH coefficient layouts (mpegProgressive and mpegPerChannel)

3. **Document two-layer architecture** (Khronos semantics + MPEG transport) and its suitability for 3GPP service integration, noting alignment with glTF-based approach in TS 26.118 and TS 26.119