Summary of S4-260119: glTF-based Representation Formats for 3D Gaussian Splats

Introduction and Scope

This contribution addresses Objective 2c of the FS_3DGS_MED Study Item ("Determine relevant formats") by providing a comprehensive analysis of glTF-based representation formats for 3D Gaussian Splatting. The document identifies a gap in TR 26.958 V0.1.1, which currently only mentions PLY as a storage format without comparative analysis of the emerging glTF-based format ecosystem from Khronos and MPEG.

The contribution proposes a two-layer architecture combining:
- KHR_gaussian_splatting (Khronos) for canonical splat semantics
- MPEG_gaussian_splatting_transport (MPEG-I Scene Description) for distribution and streaming capabilities

KHR_gaussian_splatting (Khronos Layer)

Core Attribute Semantics

The Khronos extension (review draft published August 2025) defines Gaussian splats as POINTS primitives within standard glTF 2.0 with the following attributes:

• POSITION (VEC3, required): Splat center position using standard glTF base attribute
• ROTATION (VEC4, required): Quaternion (x,y,z,w) for local axes orientation
• SCALE (VEC3, required): Per-axis scale in log-space
• OPACITY (SCALAR, required): Opacity in range [0,1]
• SH_DEGREE_l_COEF_n (VEC3, conditional): Spherical harmonics coefficients organized by degree (0-3) and coefficient index for view-dependent lighting
• COLOR_0 (VEC3/VEC4, recommended): Baseline color for fallback point-cloud rendering

Extensibility and Backward Compatibility

Key design features:
- Nested extensions mechanism inside the KHR_gaussian_splatting object allows other extensions to add compression, alternative encodings, or processing without duplicating semantics
- Graceful degradation: Clients not recognizing the extension can still render as standard point cloud using POSITION and COLOR_0
- Provides strong anchor for MPEG and 3GPP work targeting interoperable distribution and streaming

MPEG_gaussian_splatting_transport (MPEG Layer)

Architecture Approach

The MPEG extension is carried as a nested extension inside KHR_gaussian_splatting.extensions, avoiding semantic duplication and adding only transport-level features.

Transport-Level Features

Alternative SH Layouts

Two MPEG-specific SH coefficient storage modes alongside Khronos default:

1. mpegProgressive layout:
2. Groups coefficients by SH degree (degree 1, 2, 3 as separate SCALAR accessors)
3. Efficient for progressive refinement
4. Receiver can render with only SH degree 0 data and incrementally fetch higher degrees

5. DC (degree 0) term reconstructed from COLOR_0.rgb or carried via KHR SH_DEGREE_0_COEF_0

6. mpegPerChannel layout:

7. Separates coefficients by color channel (R, G, B)
8. More efficient for certain compression schemes

Progressive Download

• Optional progressive ordering signaled by listing accessor indices in progressive.stages
• Ordered from lower to higher fidelity
• Receiver may initially fetch only first stage and progressively refine without re-decoding previous data

Timed Delivery for 4D Splats

• Dynamic 4D Gaussian splat sequences supported using existing MPEG timed media mechanisms
• Accessor treated as time-varying if and only if it carries MPEG_accessor_timed extension
• Timed accessors backed by circular buffers as defined by MPEG-I Scene Description

Two-Layer Architecture Benefits for 3GPP

Architectural Summary

• Layer 1 (Khronos): Canonical splat semantics (geometry, appearance, SH lighting) and fallback point-cloud path
• Layer 2 (MPEG): Progressive download, timed delivery, and alternative SH layouts as nested extension

3GPP Service Integration Advantages

1. Alignment with existing 3GPP specifications: glTF already adopted by TS 26.118 (Immersive teleconferencing) and TS 26.119 (MeCAR)

2. 5GMS adaptive delivery mapping: Progressive download and timed delivery map naturally to 5G Media Streaming

3. Bandwidth-adaptive quality: Progressive SH degree layout enables network/receiver control of SH levels to fetch, analogous to spatial/temporal layer selection in scalable video codecs

4. Future-proof extensibility: Clear path for future compression extensions (e.g., from ongoing MPEG Gaussian Splat Coding exploration) and tiled spatial delivery without breaking backward compatibility

Format Comparison

PLY

• De facto training output format
• Raw float32 attributes without compression
• Very large files (typically 200+ MB for single scene at SH degree 3)
• Limitations: No extensibility mechanism, no progressive delivery support, no scene graph, no standard metadata support (camera parameters, animation)

SPZ (Splat Zip)

• Developed by Niantic as compact binary container
• Applies quantization and packing (~90% size reduction vs PLY)
• Extension under development in Khronos
• Superior compression schemes (e.g., Qualcomm's L-GSC) also being considered

glTF + KHR_gaussian_splatting + MPEG transport

• Full scene graph support (nodes, transforms, animations)
• Standard extensibility
• Backward-compatible fallback
• MPEG transport layer for progressive and timed delivery
• Signaling and usage of different compression schemes through proper extensions
• Recommended as primary format path for 3GPP

Proposals for TR 26.958

The contribution proposes to include the following in TR 26.958 Section 4 and new subsection under Section 11:

1. Document KHR_gaussian_splatting as emerging industry baseline for 3DGS representation in glTF, including:
2. Attribute semantics
3. SH coefficient organization
4. Backward-compatible fallback via POINTS

5. Extensibility mechanism

6. Document MPEG_gaussian_splatting_transport being developed within MPEG-I Scene Description, including:

7. Progressive download
8. Timed delivery for dynamic 4D Gaussian splat sequences

9. Alternative SH coefficient layouts (mpegProgressive and mpegPerChannel)

10. Document two-layer architecture (Khronos semantics + MPEG transport) and its suitability for 3GPP service integration, noting alignment with glTF-based approach in TS 26.118 and TS 26.119