Summary of S4-260169: Pseudo-CR on 3DGS Delivery Workflows Based on Capability Negotiation

Document Overview

This contribution from Tencent proposes updates to TR 26.958 v0.1.1 to define adaptive delivery workflows for 3D Gaussian Splats (3DGS) content in mobile environments. The document addresses the heterogeneity in both 3DGS scene complexity and UE capabilities through capability negotiation mechanisms.

Motivation and Problem Statement

The contribution identifies a critical gap in the current study: static delivery workflows for 3DGS content pose significant risks including:
- Poor Quality of Experience (QoE) when content complexity exceeds UE rendering capabilities
- Device overheating and thermal throttling
- Inefficient resource utilization across diverse mobile devices

The heterogeneity exists on two dimensions:
- Content complexity: Ranging from simple objects (thousands of primitives) to massive scenes (millions of primitives)
- Device capabilities: Significant variation in GPU power, thermal limits, memory, and battery constraints

Main Technical Contributions

Adaptive Delivery Framework (Clause 9.2)

The contribution proposes updating clause 9.2 with a comprehensive adaptive workflow that introduces:

1. Capability Reporting Mechanism: UEs report both static and dynamic capabilities to the server
2. Static capabilities: Maximum visible Gaussians at target frame rate (e.g., 30fps), highest supported Spherical Harmonics (SH) degree (0-3), maximum memory, supported quantization/compression formats, GPU rendering capacity, CPU performance class, native screen resolution/frame rate, memory bandwidth

3. Dynamic state: Current thermal status (throttling level), battery level, available GPU/CPU compute headroom, real-time battery charge

4. Rendering Budget Concept: A negotiated constraint that ensures target frame rates and maximizes session duration based on device capabilities

Two Negotiation Modes

The contribution defines two distinct approaches aligned with TR 26.928 principles:

Server-Centric Decision Mode (Clause 9.2.2.2)

In this approach, the UE acts as a data provider while the server makes adaptation decisions:

Workflow Steps:
1. Hardware assessment: UE evaluates capabilities via system checks (potentially using OpenXR APIs)
2. Capability reporting: UE transmits comprehensive capability report (CPU, GPU, Memory constraints)
3. Server decision: Server analyzes report and determines optimal delivery strategy
4. Content adaptation: Server processes 3DGS model through:
- Pruning low-opacity or spatially insignificant splats
- LOD selection from pre-generated levels
- SH degree reduction (stripping high-order coefficients, transmitting only Direct Color components)
- Quantization adjustments
5. Data delivery: Server streams optimized 3DGS payload
6. Local adaptation: UE performs final on-device optimizations (further pruning/merging) to fit runtime constraints
7. Rendering: UE executes rendering pipeline

Key characteristics:
- Server employs internal logic or lookup tables to map raw metrics to rendering budget
- Server determines primitive count limits (e.g., N primitives for specific GPU under thermal stress)
- Reduces both network bandwidth and client rendering load

Client-Centric Decision Mode (Clause 9.2.2.3)

In this approach, the UE determines its own requirements and explicitly requests specific content characteristics:

Workflow Steps:
1. Hardware analysis: UE performs internal audit of hardware resources and API support
2. Format determination: UE calculates optimal 3DGS representation format (point budget, SH degree) based on continuous self-assessment of frame time, thermal headroom, and hardware capabilities
3. Content request: UE explicitly specifies required format parameters (quantization levels, SH orders, point budget)
4. Server-side adaptation: Server processes source content to match UE-specified constraints
5. Data delivery: Server streams optimized payload
6. Local refinement: UE applies final local adaptations
7. Rendering: UE executes rendering pipeline

Key characteristics:
- Decision-making responsibility delegated to UE
- UE continuously monitors performance metrics
- Server acts as content filter/selector fulfilling explicit UE requests

Use Case Alignment

The proposed workflows address requirements from:
- Clause 5.2: Static 3DGS scene delivery
- Clause 5.4: Dynamic 3DGS content delivery

Technical Benefits

The contribution ensures:
- Frame rate stability through capability-aware content delivery
- Thermal management by preventing device overheating
- Prevention of application crashes and frame drops
- Optimized battery consumption
- Maximized session duration
- Content complexity aligned with hardware processing limits

Proposed Changes

The document proposes modifications to Clause 9.2 of TR 26.958, specifically:
- Adding new clause 9.2.1 (Overview)
- Adding new clause 9.2.2 (Workflow with capability negotiation)
- Adding new clause 9.2.2.1 (Objectives)
- Adding new clause 9.2.2.2 (Server-centric 3DGS adaptation) with Figure 2
- Adding new clause 9.2.2.3 (Client-centric 3DGS adaptation) with Figure 3