# Summary of 3GPP Technical Document S4-260143

## Document Information
- **Type:** Change Request (CR 0002 rev 3)
- **Specification:** TS 26.966 v19.0.0
- **Work Item:** FS_AVFOPS_MED (Feasibility Study on Audio Video File Operations for Media)
- **Category:** B (addition of feature)
- **Release:** Rel-20
- **Source:** Xiaomi Communications

## Purpose and Scope

This CR proposes adding a new scenario (Scenario #6) on **Refocusable Video** to TR 26.966, addressing objective 1 of identifying relevant new representation formats not yet documented in TS 26.265.

## Main Technical Contributions

### 5.7.1 Overview - Use Case Description

The CR introduces the concept of refocusable video, which enables post-capture modification of depth of field effects (bokeh). Key points:

- Traditional portrait photography achieves bokeh through lens selection
- Digital photography can simulate this effect pre-encoding
- Growing user expectation (especially prosumers) to refocus already captured content
- Technical approach: record sharp video + depth map sequence, then generate bokeh effect per frame during playback/editing

### 5.7.2 Previous Work in 3GPP

Identifies gap: **coded representation of depth maps as part of video bitstream has not been addressed in 3GPP specifications**.

### 5.7.3 Review of Related Work

Comprehensive survey of depth map representation across multiple standards bodies:

#### 5.7.3.1 ISO/IEC 23091-2:2025 CICP Video

- MV-HEVC enables depth map encoding via auxiliary layers
- Auxiliary layer identified as depth auxiliary layer containing depth representation information SEI message
- New color primaries code point (value **130**) defined to indicate decoded picture represents depth map
- HasChromaticityCoordinates = 0 (one color component)
- Can be carried in VUI parameters for AVC/HEVC without relying on auxiliary layer design

#### 5.7.3.2 ISO/IEC 23000-22 MIAF and ISO/IEC 23008-12 HEIF

- MIAF builds on HEIF for multiple images, groups, sequences with defined relationships
- Depth maps defined as auxiliary image items
- Identified using auxiliary image item type: `urn:mpeg:mpegB:cicp:systems:auxiliary:depth`
- Depth map interpretation out of scope; recommends including depth representation information SEI message as item property for HEVC-encoded auxiliary items

#### 5.7.3.3 SMPTE ST 2087:2016 Depth Map Representation

Defines comprehensive depth map data representation with key definitions:

**Terminology:**
- **Reference Camera:** Camera corresponding to viewpoint (can be virtual)
- **Depth Map:** Array of depth values corresponding to image pixels
- **Depth Value:** Distance in meters from reference camera to object surface, measured parallel to optical axis
- **Relative Depth Value:** Offset and scaled representation of depth value

**Two representations specified:**

1. **32-bit floating point:**
   - IEEE 754 single-precision format
   - Unit: meter (1.0 = 1 meter)
   - Max value: positive infinity (+INF, 0x7F800000)
   - Unknown value: NaN
   - Co-located sample mapping

2. **16-bit floating point:**
   - IEEE 754 half-precision format
   - Relative depth values (unitless)
   - Max value: positive infinity (+INF, 0x7C00)
   - Unknown value: NaN
   - Requires metadata: DepthScaleFactor and DepthOffset

#### 5.7.3.4 ISO/IEC 23008-2 HEVC / ITU-T H.265

- MV-HEVC enables depth map encoding as auxiliary layer
- Auxiliary layer identified as depth auxiliary layer with depth representation information SEI message
- References Clause 6.9 Solution #4.1

#### 5.7.3.5 ISO/IEC 14496-12 ISOBMFF

- MPEG Systems WG03 developing Amendment 2 of 8th edition
- Specifies storage of depth map video sequences in auxiliary video track linked to main video track
- Covers: media handler, track referencing, metadata box for depth map interpretation

#### 5.7.3.6 SMPTE ST 268-1:2014 DPX Format

Digital Picture Exchange Format v2.0 for moving pictures:

**Depth component support:**
- Code value **8**: Depth (Z) component

**Transfer characteristics:**
- Code **11**: Z (depth) – linear
- Code **12**: Z (depth) – homogeneous (requires distance to screen and angle of view in user-defined section)

### 5.7.4 Functional Requirements

Outlines analysis framework based on:

1. **Hardware impact assessment:**
   - Option a: Reference existing hardware product-grade support
   - Option b: Describe expected hardware implementation impact with justifications

2. **Codec capabilities:** TBD

## References Added

The CR adds 9 new normative/informative references covering:
- Android AOSP camera bokeh documentation
- JVET documents on CICP extensions
- ISO/IEC standards (MIAF, ISOBMFF amendments)
- SMPTE standards (RP 157, ST 268-1, ST 2087)
- Google Dynamic Depth specification
- Android MP4-AT file format

## Impact Assessment

- **Specifications affected:** Only TR 26.966
- **Other specs:** None
- **Test specifications:** None
- **O&M specifications:** None