Media Coding Standards and AI Framework Development Overview

Neural Network Watermarking Use Cases and Functional Requirements Leonardo Chiariglione, Mihai Mitrea 2022/07/12 T15UTC .

MPAI: AI-based data coding Standards Media coding standards have made possible the success of digital media Moving Picture, Audio, and Data Coding by Artificial intelligence (MPAI) Targets AI-enabled data coding standards. Data coding: transforms data to a format more suitable for an application. MPAI develops standards for: Components (AI Modules AIM): their function, I/O data Organised in workflows (AI Workflows AIW): their function, I/O data, AIM topology Executed in an MPAI specified environment (AI Framework AIF): metadata, API 3/19/202 5 2

MPAI standards bottom up AI Workflow (AIW) AI module (AIM) AI AI Text Module (AIM) Module AIM Speech Inputs Speech analysis Outputs Emotion AI AI AIM Storage Module AIM Module AIM Access 3/19/2025 3

MPAI-AIF: AI Framework AI AI Module (AIM) Module (AIM) Outputs Inputs User Agent AI Workflow (AIW) AI AIM Storage AI Module (AIM) Module (AIM) Controller Global Storage MPAI Store Communication Access MPAI standards specify components And their integration (AIW-AIM-AIF)

Goal: make standards accessible & timely available Before initiating a standard, Active Principal Members develop & adopt the Framework Licence (FWL), a licence without values: $, %, dates etc. declaring that the eventual licence will be issued 1. At a price comparable with similar standard technologies. 2. Not after products are on the market. During the development, any Member making a contribution declares it will make its licence available according to the FWL. After the development, Members holding IP in the standard select the preferred patent pool administrator. 3/19/202 5 5

How MPAI develops standards All Principal Members Stage 0 Stage 1 Stage 2 Stage 3 Proposal Commercial Requirements Functional Requirements Interest Collection Use Cases Standard MPAI Standard Call for Technologies Community Comments Standard Development Stage 7 Stage 6 Stage 5 Stage 4 Principal Members All All Members 3/19/202 5 6

MPAI standards developed so far Standard Name Acronym MPAI-AIF MPAI-CAE MPAI-MMC MPAI-CUI V 1.1 1.3 1.2 1.0 AI Framework Context-Based Audio Enhancement Multimodal Conversation Compression and Understanding of Financial Data Governance of the MPAI Ecosystem MPAI-GME 1.0 The next round of MPAI standards Project name Acronym MPAI-AIF MPAI-MMC 2.0 MPAI-NNW V. AI Framework Multimodal Conversation Neural Network Watermarking 2.0 1.0 3/19/202 5 7

Watermarking is useful for Neural Networks Machine learning is a costly field: Buying an AI solution ranges from $ 6000 to $300.000 Renting a pre-built module may cost around $ 40.000/year An AI solution could: Use multiple alternative Neural Networks to provide an inference identifying the one that actually produced the inference is important Be shared among multiple users keeping a track of this process is useful Be altered without knowing it, embedded on a hardware or maliciously attackerd identifying such modifications ensures correct functioning Ensuring traceability and integrity of Neural Networks becomes mandatory 8 3/19/2025

Neural Network Watermarking Watermarking regroups tools allowing to imperceptibly and persistently insert some metadata into an original content Watermark insertion Watermark detection Fidelity Original watermark Recovered watermark Watermarking Watermarked NN Robustness Data payload Original NN Neural Network Watermarking is an incremental challenge: Neural Networks are no longer a static content but a dynamic one (function) The evaluation of the impact of the watermark is much harder to evaluate The polynomial interpolation analogy 9 3/19/2025

Problem statement A rich state-of-the-art despite its 4-year history Two typologies of solutions: White box: the watermarking is applied to a Neural Network (parameters) Black box: the watermarking is applied to the inference generated by a Neural Network 10 3/19/2025

Fidelity Problem statement Watermarking Robustness Data payload A rich state-of-the-art despite its 4-year history Two typologies of solutions: White box: the watermarking is applied to a Neural Network (parameters) Black box: the watermarking is applied to the inference generated by a Neural Network Fidelity depending on the Task: Mostly classification task in the literature 11 3/19/2025

Fidelity Problem statement Watermarking Robustness Data payload A rich state-of-the-art despite its 4-year history Two typologies of solutions: White box: the watermarking is applied to a Neural Network (parameters) Black box: the watermarking is applied to the inference generated by a Neural Network Fidelity depending on the Task: Mostly classification task in the literature Robustness against specific modifications: Pruning, fine-tuning, model compression, knowledge distillation 12 3/19/2025

Fidelity Problem statement Watermarking Robustness Data payload A rich state-of-the-art despite its 4-year history Two typologies of solutions: White box: the watermarking is applied to a Neural Network (parameters) Black box: the watermarking is applied to the inference generated by a Neural Network Fidelity depending on the Task: Mostly classification task in the literature Robustness against specific modifications: Pruning, fine-tuning, model compression, knowledge distillation Data payload evaluation differ to one to another 13 3/19/2025

Fidelity Problem statement Watermarking Robustness Data payload A rich state-of-the-art despite its 4-year history Two typologies of solutions: White box: the watermarking is applied to a Neural Network (parameters) Black box: the watermarking is applied to the inference generated by a Neural Network Fidelity depending on the Task: Mostly classification task in the literature Robustness against specific modifications: Pruning, fine-tuning, model compression, knowledge distillation Data payload evaluation differ to one to another MPAI-NNW will provide tools to enable watermarking technology providers to qualify their products 14 3/19/2025

Use cases Related to: Identifying the ownership of an Neural Network Identifying an Neural Network (e.g. a DOI for NN) Verifying the integrity of an Neural Network Structured at two levels: Watermarking the Neural Network model Identify the ownership of an NN - for NN customer and NN owner - for NN customer, NN owner and NN end-user Watermarking the Neural Network inference Use cases related to Neural Network Watermarking: - watermarking of the NN model - watermarking of NN inference Identify an NN (e.g. DOI for NN) Verify the integrity of an NN 15 3/19/2025

Scope of the MPAI-NNW standard MPAI-NNW enables watermarking technology providers to qualify their products, according to three items: The injection of the watermark without deteriorating the performance of the Neural Network A testing dataset to be used for the watermarked and unwatermarked NN An evaluation methodology to assess any change of the performance, induced by the watermark The watermark detector to ascertain the presence and the watermark decoder to successfully retrieve the payload of the inserted watermark Performance criteria for the Watermark detector or decoder, e.g., relative numbers of missed detection and false alarm or percentage of the retrieved payload. A list of potential Modification types expected to be applied to the watermarked NN as well as of their ranges The injection, and detector/decoder computational cost Execution time on a given processing environment 16 3/19/2025

Overview of MPAI-NNW Requirements NNW - Evaluate performances of watermarking NN Modification of performance induced by the watermarking process 0 1 0 0 0 1 1 1 injector Processing cost of the insertion injector detector detector Watermarked NN Trained NN decoder decoder Yes MPAI Tester Performance criteria for the detector/decoder Watermark provider detector No Processing cost of the detection/decoding phase decoder 0 1 0 0 0 1 1 1 17 3/19/2025

Requirement related to watermark injection Measure the impact of the watermark on the performance Conformance testing dataset & evaluation methodology to assess the deterioration (if any) of the quality of the inference induced by the watermark To respondent: List of tasks to be performed by the Neural Network Comment on the process described in NXXX, section 5.1 Methods to measure the quality of the inference produced by the Neural Network 18 3/19/2025

Requirements related to watermarking detection and decoding Measure Detection and Decoding capability Explain the process of testing the detector/decoder of a watermark technology List potential attacks that will be considered in the standard To respondent: List of potential modifications to be applied to a watermarked Neural Network Parameters and ranges of proposed modifications Suitable distance to evaluate the Symbol Error Rate between the original and retrieved watermarks 19 3/19/2025

Requirements related to processing cost Measure the processing cost The processing cost of injection, detection and decoding help qualify a Neural Network Watermarking method. To respondent: Set of testing environment CPU types, number of cores, frequency, memory; GPU types, frequency, memory Set of values characterizing the processing of Neural Networks training Execution time (in second), CPU footprint (in MB) and GPU footprint (in MB) 20 3/19/2025

Next Steps MPAI intends to develop three standards MPAI-AIF V2 - MPAI-MMC V2 - MPAI-NNW V1 with the following process Year 2022 Month June July July September 13 October October Spring Day 22 18 19 Who Does Approves Deliver Publishes CfT, UCFR, FWL Notify Intention to submit proposal Submit proposals Kicks off Evaluations Approves Technical Specification What MPAI Principal Members MPAI Respondents Respondents MPAI MPAI UCFR Framework Licence 10 12 2023 21 3/19/2025

Join the fun, build the future! https://www.mpai.community/ More about this call at http://nnw.mpai.community/ 3/19/202 5 22