[FS_3DGS_MED] Pseudo-CR on objective metrics for 3DGS
This change request proposes the adoption of a standardized objective quality evaluation methodology for 3D Gaussian Splatting (3DGS) content. The contribution addresses the current gap in TR 26.958, which contains only placeholders for metrics and reference implementations. The proposal leverages the mpeg-gsc-metrics software tool recently developed by MPEG for computing objective quality metrics.
The CR identifies three key requirements for the study:
The proposed software is a fork of MPEG metrics software intended for storage in the 3GPP git repository to facilitate updates and future experiments.
The CR introduces a comprehensive objective metrics section defining:
Supported Metrics:
- PSNR and MSE: Computed in both RGB and YUV color spaces with weighted averages
- Object Masked (OM) Metrics: PSNR and SSIM variants computed only on valid pixels defined by union of object masks
- Perceptual Metrics: SSIM and IVSSIM
- Geometric Statistics: Occupancy rate measuring valid pixel coverage percentage
Dual-Mode Rasterizer:
- CPU rasterizer: Software-based implementation ensuring bit-exact rendering regardless of hardware/OS (recommended for normative results)
- GPU rasterizer: OpenGL-based accelerated rendering for visual inspection and rapid experiments
Evaluation Process:
1. Viewpoint generation from original PLY file or explicit definition
2. Rendering using standardized rasterizer (CPU or GPU)
3. Metric computation on rendered pairs
The CR adds detailed usage documentation for the 3DGS-Metrics command-line tool:
12.4.1 Basic Metric Computation:
- Simple command-line interface for comparing source and decoded PLY files
12.4.2 Evaluation Using Embedded Camera Parameters:
- --useCameraPosition option enables rendering using camera parameters stored in PLY header comments
- Parameters typically inserted by content preparation tools using COLMAP photogrammetry data
- Ensures exact camera intrinsics and extrinsics without external configuration
12.4.3 Evaluation with Loaded Viewpoints:
- Support for external viewpoint files specifying camera poses
- CPU rendering option for bit-exact results
12.4.4 Video Generation:
- -s flag enables generation of rendered video sequences (Source, Decode, Butterfly comparison)
- Facilitates visual inspection alongside metric computation
12.4.5 Output Results Format:
- Detailed per-frame and global average statistics
- Comprehensive reporting of MSE, PSNR, and SSIM in RGB and YUV color spaces
- Example output provided for Bartender sequence at 1920x1080 resolution showing:
- RGB PSNR (avg): 49.79 dB
- YUV PSNR (avg): 53.96 dB
- SSIM (avg): 0.998386
- 100% occupancy
The CR proposes adopting the 3DGS metrics software as the reference tool for objective quality evaluation to ensure all contributions are measured against the same baseline. This standardization will facilitate technical work by providing consistent, comparable, and reproducible results across different proponents within the study.