Intel Gaudi 3 Inference Reference Design 4-Card PCIe

The Intel Gaudi 3 Inference Reference Design 4-Card PCIe (HLRD-325B-4) is a validated, production-ready inference platform integrating four Intel Gaudi 3 AI accelerators in a PCIe form-factor configuration. Built on Intel's third-generation Gaudi architecture, Gaudi 3 is fabricated on a 5nm process node and delivers substantial improvements in compute throughput and memory bandwidth over its predecessor. Each Gaudi 3 accelerator features 64 Tensor Processor Cores (TPCs) and a Matrix Multiplication Engine (MME) optimized for both FP8 and BF16 precision, enabling high-throughput execution of large language model (LLM) inference, generative AI, and deep learning recommendation workloads at enterprise scale.

The reference design leverages Gaudi 3's integrated RDMA-capable 21-port Ethernet fabric — 24 × 100GbE ports per card — to enable direct card-to-card communication without requiring an external network switch in scale-up configurations. Each accelerator is equipped with 128 GB of HBM2e memory, providing the high-capacity, high-bandwidth memory footprint required for hosting large generative models in their entirety. The 4-card PCIe assembly is validated as a complete subsystem, reducing integration risk for datacenter operators deploying inference infrastructure based on standard PCIe server platforms.

Designed for enterprise datacenters and cloud service providers, the HLRD-325B-4 is supported by Intel's open software ecosystem including the Intel Gaudi Software Suite and compatibility with industry-standard frameworks such as PyTorch and Hugging Face Transformers via the Optimum Habana integration. This reference design is particularly well-suited for organizations seeking a scalable, standards-based PCIe inference solution for production AI services without proprietary interconnect lock-in.

Ideal for

• Large language model (LLM) inference serving for enterprise generative AI applications, including multi-billion parameter transformer models
• High-throughput text generation and summarization pipelines in financial services, legal, and media organizations
• Retrieval-augmented generation (RAG) inference backends requiring large on-accelerator memory capacity to host both retriever and generator models simultaneously
• Deep learning recommendation model (DLRM) inference for e-commerce, digital advertising, and content personalization platforms
• Multi-modal AI inference workloads combining vision and language models in healthcare imaging, manufacturing quality control, and document processing
• Enterprise AI inference infrastructure consolidation in datacenter environments standardized on PCIe server architectures, avoiding proprietary fabric requirements

Technical specifications

Available from Omnixon Global. Submit an RFQ and our team will confirm configuration and availability for your order.