# Summary of S4-260230: Experimental Approach and Test Setup for Media Transmission for AI Inferencing

## Document Overview

This Change Request (CR) to TR 26.823 v0.2.0 addresses the currently empty Clause 6.5.1 by providing detailed experimental approaches and test setups for evaluating media transmission for AI inferencing scenarios. The contribution is part of the Study on dynamically changing traffic characteristics and usage of enhanced QoS support in 5GS for media applications and services.

## Main Technical Contributions

### Two Experimental Approaches Proposed

The document proposes two distinct experimental approaches corresponding to the AI inferencing in XR service scenario (Clause 5.6):

1. **Experimental approach #1**: Based on commercially available generative AI applications
2. **Experimental approach #2**: Based on a standalone generative AI platform for traffic measurement associated with QoE metrics

---

## Experimental Approach #1: Using Commercially Available Generative AI Applications

### Experimental Methodology

- **Black box approach**: Uses commercially available generative AI applications (e.g., OpenAI ChatGPT™, META AI™, Google Gemini™) to measure and characterize media transmission traffic in uplink and downlink
- **Focus**: Interactive or delay-bound AI inference responses as described in typical implementation and end-to-end procedures
- **Example use cases**: 
  - Meta AI glasses "ask Meta AI about what you see" functionality (voice + periodic images uplink, audio response downlink)
  - OpenAI ChatGPT™ voice mode
  - Google Gemini™ live mode with camera feed

### Measurement Procedure

- **Baseline measurement**: First conducted on wired network (ideal network conditions)
- **5G network testing**: Uses test channels or emulated 5G network for deterministic, controllable, and reproducible conditions
- **Network conditions considered**: Nominal, cell-edge, multi-UE scenarios with system load

### Test Setup

- **Observation Points (OPs)**:
  - **UE side**: 5G network emulator ingress of AI inference input data and egress of AI inference output data
  - **UPF side**: 5G network emulator egress of AI inference input data and ingress of AI inference output data
- **Measurement tool**: Open-source network protocol analyzer (e.g., Wireshark) for IP packet analysis
- **Purpose**: Measurements at both ingress and egress highlight network performance impact on traffic characteristics

### Limitation

**NOTE**: Due to black box nature, QoE metrics identified in Clause 5.6.3 cannot be measured due to lack of appropriate Observation Points.

---

## Experimental Approach #2: Using Standalone Generative AI Platform

### Experimental Methodology

- **White box approach**: Standalone client/server generative AI platform mimicking commercially available applications
- **Client side**: Generation and transmission of AI inference input data
- **Server side**: AI inference, generation and transmission of AI inference output data
- **Key advantage**: Control over open-source AI model and generation/reception of AI inference data enables QoE metrics measurement (Clause 5.6.3) in addition to traffic characteristics

### QoE Measurement Capabilities

- **Per-packet metadata insertion**: Enables latency QoE metrics measurement (e.g., time-to-first-token, end-to-end latency)
- **Packet marking**: Uplink packets (questions) and corresponding downlink packets (inference responses) marked with metadata for correlation

### Measurement Procedure

Similar to Approach #1:
- **Baseline**: Wired network (ideal conditions)
- **5G testing**: Test channels or emulated 5G network
- **Network conditions**: Nominal, cell-edge, multi-UE scenarios

### Test Setup

#### Client Side (Custom Application, e.g., Unity 3D)
- Generates and transmits input data (prompt/text, image) with metadata
- Receives AI inference output data with metadata
- Renders AI inference results

#### Server Side (Custom AI-enabled Application, e.g., Unity 3D)
- Receives AI inference input data with metadata
- Performs AI inferencing using open-source standalone Multi-Modal LLM (e.g., Llava model from LlamaSharp based on llama.cpp)
- Collects QoE metrics
- Transmits AI inference output data with metadata

#### Observation Points
- **UE and UPF side OPs**: For uplink/downlink traffic characteristics (as in Approach #1)
- **Client and server-side OPs**: For QoE metrics measurement

### Variant Configuration

**NOTE**: Test setup may be extended to mimic advanced AI-enabled AR devices (e.g., lightweight AR+AI glasses requiring remote rendering) by adding XR data (e.g., periodic pose information) to AI inference input data in uplink to evaluate impact on traffic characteristics.

---

## Key Technical Differentiators

| Aspect | Approach #1 (Commercial Apps) | Approach #2 (Standalone Platform) |
|--------|-------------------------------|-----------------------------------|
| **Control** | Black box | White box |
| **QoE Metrics** | Not measurable | Measurable |
| **Traffic Characteristics** | Measurable | Measurable |
| **Flexibility** | Limited | High (customizable) |
| **Realism** | High (actual commercial apps) | Medium (mimics commercial apps) |
| **Metadata Support** | No | Yes (per-packet) |