# Comprehensive Summary of S4-260189: AI Intermediate Data Format

## 1. Introduction and Scope

This contribution proposes defining an intermediate data carriage format for AI/ML split inferencing, derived from TR 26.927. The document introduces:

- A description of intermediate data
- Definition of intermediate data structure
- An example format structure (proposed as an Annex) including:
  - AI Parameter Set (AIPS) specifying AI-related parameters
  - TLV encapsulation for both AIPS and intermediate data

## 2. Technical Background and Motivation

### 2.1 Split Inferencing Requirements

Split inferencing, approved and mandated in 5G, is a key objective of the work item. The solution must support:

- Different input data types producing intermediate data
- Multiple media modalities (video, audio, text) without restriction to one
- An agnostic transport format for 5G use cases

### 2.2 Source and Derivation

The proposed format is derived from:

- User-plane data structure in Clause 6.8 of TR 26.927
- Addition of a **partition identifier** (previously "split-point identifier") from Clause 6.6 of TR 26.927
- The partition identifier enables selection of pre-configured partitioning negotiated during configuration phase

### 2.3 Dynamic Nature of Tensor Characteristics

Tensor characteristics are not static and may change dynamically based on:

- Resolution of input inference
- Content of input inference

These characteristics must be conveyed through the user plane for accurate interpretation at the receiving end.

## 3. Main Technical Contributions

### 3.1 Intermediate Data Definition (Clause X.X.1)

**Key Definition**: Intermediate data refers to output tensor(s) computed by a sub-model executing an inference subtask up to a defined and negotiated partitioning, transferred between endpoints (device, edge, server) to serve as input to a subsequent sub-model.

**Characteristics**:
- May be compressed and/or encoded before transmission
- Processing shall not alter semantics required by receiving sub-model
- Non-persistent, dynamic, and context-dependent
- Characteristics (shape, size, format) vary as function of:
  - Input data
  - Selected inference partitioning
  - Runtime configuration

### 3.2 Intermediate Data Structure (Clause X.X.2)

**Configuration Stage**: Structure defined and exchanged at configuration stage, referred to as partitioning configuration.

**Dynamic Factors**:
- Input media size/resolution changes may alter tensor shape
- Selected partitioning identifies active partitioning among pre-configured options
- Selected compression profile (algorithm and parameters) optimized for efficiency

**Required Information in Format**:
- Tensor identifier
- Inferred tensor length (derived from current tensor shape)
- Partitioning identifier (referencing negotiated configuration)
- Compression profile identifier (indicating compression method)

**Solution**: AI Parameter Set (AIPS) defined to capture information applicable to all tensors and associated data.

### 3.3 AI Parameter Set (AIPS) Definition (Annex X.X.1-3)

**Purpose**: Carries metadata (tensor metadata) associated with intermediate data payload.

**AIPS Lifetime**:
- **Starts**: When decoder first receives and parses AIPS TLV unit
- **Ends**: When:
  - New AIPS with same or different `ai_parameter_set_id` is received
  - New session begins
  - Decoder is reset
  - Number of tensors or tensor shape changes

**AIPS Fields** (Table X.X.13-1):

| Field | Meaning |
|-------|---------|
| `ai_parameter_set_id` | Unique ID of AIPS |
| `split_point_id` or `partition_id` | Key identifier of split point/partition |
| `num_tensors` | Number of tensors |
| For each tensor: | |
| - `tensor_id` | Tensor identifier |
| - `dtype` | Data type of tensor data |
| - `rank` | Number of dimensions |
| - For each dimension: `dimension` | Size of dimension |
| - `compression_profile_id` | Compression profile identifier |

### 3.4 TLV Encapsulation (Clause X.X.2-4)

**TLV Message Components**:
- **Type**: Indicates payload information
- **Length**: Value of payload
- **Payload**: Data

**TLV Unit Types** (Table X.X.24-1):

| Type Value | Description |
|------------|-------------|
| 0 | Reserved |
| 1 | AI Parameter Set data (AIPS) |
| 2 | Intermediate data |
| 3-255 | Undefined |

**Encapsulation Scenarios**:

1. **AIPS Data Encapsulation** (X.X.24.2): TLV unit encapsulating AIPS data as defined in clause 1.3

2. **Single Tensor Encapsulation** (X.X.24.3): 
   - TLV unit value comprises AIPS identifier and tensor data
   - Tensor data includes: tensor identifier, tensor length (optional), tensor payload data
   - Tensor payload contains flattened byte array, possibly compressed per AIPS compression profile ID

3. **Multiple Tensors Encapsulation** (X.X.24.4): TLV unit encapsulating more than one tensor data

## 4. Key Changes from Previous Version

**Terminology Updates**:
- "Split point" terminology changed to "partitioning" throughout
- "Head sub-model" and "Tail sub-model" terminology refined to "sub-model" and "subsequent sub-model"

**Structural Additions**:
- Addition of partition identifier (highlighted as new in original document)
- Formalization of AIPS lifetime management
- Complete TLV encapsulation framework

## 5. Proposal for Integration

The document proposes:

1. Incorporate changes 1 and 2 into a base CR
2. Include change 3 (AIPS and TLV details) in a dedicated annex for illustration purposes