OpenFabrics Development and Evolution in InfiniBand Technology
Explore the progression of OpenFabrics technology through the phases of OpenIB and iWarp, leading to the present challenges of multiple incompatible versions and vendor-specific APIs. Learn about proposed alternatives for scalable fabric interfaces and the implications of incorporating interface sets under a single framework.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
OpenFabrics 2.0 or libibverbs 1.0 Sean Hefty Intel Corporation
Pre-OpenIB Multiple IB vendors each with a proprietary software stack Multiple versions of the verbs interface Binary incompatible Formation of OpenIB Focused on creating a single, open source software stack for InfiniBand Apps have 1 API to target, distros have 1 solution to support Greatly increased adoption 2 www.openfabrics.org
OpenIB Resulting software stack was hardware facing Verbs is a hardware semantic Was never intended to be an API Application developers have always disliked it OpenIB expanded to include iWarp Changed name to OpenFabrics Folded iWarp support under InfiniBand API 3 www.openfabrics.org
OpenFabrics Today The fundamental stack is largely unchanged The result: There are multiple, binary incompatible versions of the verbs interface There are vendor-specific APIs FCA, MXM, PSM, UCCS Sounds a lot like the pre-OpenIB days 4 www.openfabrics.org
Why Not Just Extend Verbs? Author s prediction: will not succeed OIB created a single verbs solution, which later split The split was a direct result of needing extensions Additional APIs have been already been developed that are NOT part of verbs or extended verbs Existence proof against There are fundamental issues with the API that will prohibit scaling to very large systems 5 www.openfabrics.org
Proposed Alternative Develop application facing APIs and let each vendor determine the best way to support those APIs There is no intent to disadvantage any hardware solution Define mechanisms for migrating providers and apps to the new framework 6 www.openfabrics.org
(Scalable) Fabric Interfaces Fabric Interfaces Control Interface Message Queue Atomics RDMA Collective Operations Active Messaging Tag CM Services Matching Q: What is implied by incorporating interface sets under a single framework? Objects exist that are usable between the interfaces Isolated interfaces turn the framework into a complex dlopen Interfaces are composable May be used together Cohesive interfaces, not merely a union 7 www.openfabrics.org
Migrating Providers from Verbs to FI RDMA Message Queue CM Services FI Providers can integrate into framework with minimal changes libfabric Verbs ibverbs abstraction layer RDMA CM libibverbs Verbs Provider Verbs Provider Providers leverage abstraction layer and RDMA CM capabilities Add optimize data transfers and completion processing calls to avoid translation overhead 8 www.openfabrics.org
Migrating Apps from Verbs to FI Expose verbs interfaces directly from FI Use macros to convert libibverbs exported calls to FI calls Or layer libibverbs over libfabric Applications must recompile Minimal benefit to app 9 www.openfabrics.org
Migrating Apps from Verbs to FI Define verbs compatibility mode Allows mapping objects between interfaces E.g. QP fabric socket CQ EC Restricts implementation Mapping must be documented Allow software to adopt new interfaces selectively E.g. send/recv/ec_read SFI verbs libibverbs libfabric Verbs Provider SFI Provider Dual-Provider Library 10 www.openfabrics.org
