Understanding Hyper-V IOMMU Architecture & Processor Handling
Dive into the detailed architecture of Hyper-V IOMMU, covering topics such as DMA, interrupts, device assignment, security issues, IOMMU overview, and more. Explore how Direct Memory Access (DMA) and interrupts play a crucial role in data transfer and processor handling within the system. Discover the terminology used, including System Physical Address (SPA) and Guest Physical Address (GPA), to comprehend the virtual and physical memory management. Gain insights into processor interrupt handling, Local APIC units, and Inter-Processor Interrupts (IPI) functioning.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
AGENDA [Brief] Overview of DMA and Interrupts Hyper-V Current I/O Architecture Device Assignment I/O architecture Security Issues IOMMU Overview Hypervisor IOMMU Architecture Additional Use Cases References Questions
TERMINOLOGY System Physical Address (SPA) An address that refers to the physical memory in the system Guest Physical Address (GPA) An address that refers to the guest view of physical memory Logical Processor (LP) - Physical CPU which can be a thread (SMT) or core Virtual Processor (VP) Virtual CPU for the partitions Root partition A partition that manages the child partitions
DIRECT MEMORY ACCESS (DMA) Allows devices to read/write data to/from memory Uses physical addressing Can be coherent or non coherent Allows processor to work in parallel while data is accessed by the device
DMA (CONT.) Device A requests DMA at 0x21000 DMA Request Device A Address: 0x21000 DMA Request Physical Address: 0x21000
INTERRUPTS Interrupts are events that indicate that a condition exists somewhere in the system, the processor or within the currently executing program that requires the attention of a processor (Intel Manual) Interrupts can be generated either by the processor, devices or the currently executing program On all modern PC systems, Interrupts are processed through the Local APIC unit on each processor
PROCESSOR INTERRUPT HANDLING Each processor has one Local APIC unit Each Local APIC unit has a unique id in the system Local APIC receives interrupt requests from internal and external sources and sends them to processor core for handling Local APIC can send interrupt requests to other logical processors in the system. This is called Inter Processor Interrupt (or IPI) Interrupt requests are sent by writing to the ICR register on the APIC page
INTERRUPT HANDLING (CONT) Interrupts are identified from various fields in the interrupt request shown below
DEVICE INTERRUPTS Two ways for a device to generate an interrupt request: I/O APIC Message Signaled Interrupt (MSI)
DEVICE INTERRUPTS I/O APIC This is used by devices that are connected to I/O APIC in the system The interrupt requests comes to the I/O APIC which sends the interrupt request to Local APIC Multiple devices can be connected to same I/O APIC input, thus sharing the interrupt
DEVICE INTERRUPTS MSI Devices generate an MSI interrupt by writing MSI Data to a special address called MSI Address Each device can generate its own interrupts and there is no need to share interrupts between multiple devices This is much more flexible method and is required on the PCIe (PCI Express) devices
HYPER-V CURRENT I/O ARCHITECTURE Root Partition Child Partition Networking Stack Networking Stack VSP VSC NIC Driver VMBUS Hypervisor NIC
LIMITATIONS Significant cross-partition communication overhead Hardware features are not utilized (such as RSS) Extra memory copy overhead Root partition bottleneck
DEVICE ASSIGNMENT I/O ARCHITECTURE Root Partition Child Partition Networking Stack Networking Stack NIC Driver NIC Driver Hypervisor NIC NIC
SECURITYISSUES The child partition is untrusted The device assigned to the child partition has ability: to do DMA to generate interrupt requests This means the child partition can program the device to: Read/Write the hypervisor, root partition or other partition s memory Spoof interrupts from other devices DoS attack by causing interrupt storm Effectively take over the whole system
WHATISNEEDEDFORSAFEDEVICE ASSIGNMENT Ability to restrict devices to access its assigned child partition s memory only Ability to prevent interrupt spoofing or storm attack from such devices The answer: IOMMU
WHAT IS AN IOMMU? Hardware that allows DMA requests to be programmed in terms of GPA and verified using I/O page tables before accessing memory DMA remapping Hardware that allows interrupt requests to be verified using interrupt remapping tables and targeted to the specified processors Interrupt remapping Intel VT-d2 and AMD IOMMU
DMA REMAPPING Provides translation of GPA to SPA for DMA requests Uses I/O page tables for translation Supports Domains Domain is a container for one or more devices that share one I/O page table I/O page tables are used to ensure that a device can only access memory that belongs to its assigned domain I/O page table can only be accessed by the privileged software thus preventing any tampering by untrusted software
DMA REMAPPING (CONT.) Device A requests DMA at 0x21000 DMA Request Device A Address: 0x21000 DMA Request Address: 0xCF000 IOMMU GPA 0x21000 0x22000 0x23000 SPA 0xCF000 0xD0000 0xD1000
DMA REMAPPING (CONT.) Device B requests DMA (uses different translation tables) DMA Request Device B Address: 0x21000 DMA Request Address: 0xEA000 IOMMU GPA 0x21000 0x22000 0x23000 0x23000 SPA 0xEA000 0xEB000 0xEC000 0xD1000 GPA 0x21000 0x22000 SPA 0xCF000 0xD0000
DMA REMAPPING (CONT.) Device C requests DMA DMA Request Device C Address: 0x21000 IOMMU GPA 0x21000 0x22000 0x23000 SPA - - -
DMA REMAPPING Enables DMAs to be programmed in terms of GPA Allows for restricting DMA access to a subset of physical memory
INTERRUPT REMAPPING Allows software to setup an interrupt remapping table (IRT) Interrupt requests from devices are intercepted and routed based on the IRT Restricts the interrupt requests from a device to the assigned vector on assigned processor
INTERRUPT REMAPPING (CONT.) Interrupt Request LP 2, Vector 0x82 Interrupt Request Index* (2) Device: BDF-A Logical Processor 2 IOMMU Device ID BDF-A BDF-A BDF-B BDF-C Interrupt Index 1 2 3 4 Target LP 1, Vector 0x80 LP 2, Vector 0x82 LP 1, Vector 0x81 LP 2, Vector 0x91 *Interrupt Index is calculated from MSI Data/Address pair or IOAPIC RTE.
HYPERVISOR IOMMU ARCHITECTURE Introduces two new concepts Device Domain and Device Interrupt Provides support for DMA remapping and interrupt remapping to multiple partitions Abstracts hardware differences between AMD IOMMU and Intel VT-d2 implementation
DEVICE DOMAIN Device domain is a container for one or more devices Contains one I/O page table Associated with a partition Devices can be attached to (and detached from) a domain All DMA requests from the attached device goes through the domain I/O page table Uses PCI Segment/Bus/Device/Function as the device identifier One identity mapped domain per partition
DEVICE DOMAIN Root Partition Child Partition Hypervisor I/O Page Table (For Child Partition setup by the Hypervisor) Device (Assigned to child partition) IOMMU
DEVICE INTERRUPTS Hardware Interrupt Remapping Hardware interrupt remapping ensures device interrupt come on a specific physical processor and interrupt vector Ensures that a device can t spoof an interrupt request Software Interrupt Remapping Software Interrupt Remapping maps a specific hardware interrupt from an LP to a VP Routing an interrupt from a device to specific VP, vector pair requires both software and hardware interrupt remapping
DEVICE INTERRUPTS Child Child Root Partition Partition (1) Partition (2) Virtual Processor Virtual Processor Virtual Processor Virtual Processor Hypervisor Logical Processor 0 Logical Processor 1 Software Interrupt Remapping Table Device (Assigned to child partition 1) IOMMU Hardware Interrupt Remapping Table
REFERENCES AMD IOMMU specifications http://www.amd.com/us- en/assets/content_type/white_papers_and_tech_docs/ 34434.pdf Intel VT-d2 specifications ftp://download.intel.com/technology/computing/vptech /Intel(r)_VT_for_Direct_IO.pdf
