Gigabyte G492-Z51 GPU Server Review
May 8, 2023If you’re looking for a high-performance GPU enabled AI, AI Training and Inference, high-performance computing beast, then the Gigabyte G492-Z51 GPU Server is for you (SHOP HERE)! This platform can support up to 10x double-wide GPUs, is an Nvidia-Certified system, features 3rd gen AMD EPYC processors, up to 8TB of memory at full capacity, and has an interesting hybrid storage condition.
This would make a great rig for cryptocurrency mining, too. The Gigabyte G492-Z51 GPU Server featuring a next generation motherboard can support a variety of high-performance applications like scientific simulation, financial analysis, and other mission critical workloads. It’s compatible with the whole family of AMD EPYC processors from Naples, to Rome and with rev.A00, Milan. And then there’s Milan X with 3D v-Cache technology.
On the front of the system there are 12x 3.5-inch hot-swappable drive bays with 8x amber HDD trays supporting NVMe or SATA and those 4x blue HDD trays are earmarked for SATA or SAS exclusively. All hot swappable and supporting HDDs or SSDs. For SAS support you will need a SAS HBA/RAID card.
Also located on the front 2x low profile PCIe Gen 4.0 x16 slots and some ports typically found on the back of the system like 2x 10Gb Ethernet Base-T LAN ports alongside the dedicated management LAN port. Then 2x USB 3.0 ports and a VGA port. Next, a few familiar items, the power button and ID button both with diagnostic LED. A reset button, non-maskable interrupt button, and then system status LED and HDD LED.
Using that mLAN port on the front of the system, administrators can access the system remotely to perform management tasks, get system status, inventory, and a host of other data. Gigabyte provides a few management options including the Gigabyte Management Console which comes preloaded on the system and can be used to manage this one server or a small server cluster. For multiple servers, Gigabyte offers the Gigabyte Server Manager or GSM. That suite of tools leverages the Management Console in each Gigabyte server using the GSM Agent and provides more tools to manage your network. Both offered at our favorite price for goods and services—free…
Turning the system around to the back of this system, there’s a low-profile PCIe slot on top with 3x 80 PLUS Platinum 2200W redundant PSUs below. Just under that last PSU, there’s an OCP mezzanine slot for an optional OCP 3.0 card that plugs in once you remove that cover plate.
The 3.0 OCP cards are much easier on maintenance too and can easily be removed from the back of the chassis instead of having to take off the cover panel and remove a few risers to access the 2.0 versions. Taking up the majority of the back panel are PCIe cover plates for up to 10x GPUs, two of which are mounted horizontally with one behind each of the dual external fan modules. All PCIe slots on this system are PCIe 4.0 compatible assuming it’s running 2nd or 3rd gen AMD CPUs.
With the cover off you can see the two PCIe x16 slots perched on top with the external fans. Those slots can be used for a double-wide GPU or high-performance I/O card. There are several options for I/O with Intel based and Mellanox-based card options offering 10Gb/s, 25Gb/s, 40Gb/s, 100Gb/s and 200GB/s per port with single, dual, and quad port options. Although for those 100 and 200Gb/s cards they’re only available in dual or single port options. The supported PCIe bandwidth is also a consideration with options for PCIe 3.0 and PCIe 4.0. PCIe 4.0 offers twice the bandwidth compared to PCIe 3.0, so theoretically, much faster. In addition to those slots above, an optional OCP 3.0 card can be placed in the x16 mezzanine card slot below the PSUs, also providing up to 4x ports at 100Gb/s.
This is a dual-root system with half the 10x GPUs running through CPU1 and the other half connected to CPU2. CPU to CPU communication over AMD’s Infinity fabric fills in the communication gap. The infinity fabric provides CPU to CPU communication bandwidth at up to 18 Giga-transfers per second. The architecture of this system also allows direct GPU to GPU access, direct transfers, and GPU Direct RDMA, or remote direct memory access. That means two networked computers can exchange data in main memory without involving the processors, and that is where those high-speed I/O controllers come in handy! GPUs supported on this system include the Turing based T4 and several Ampere-based GPUs, including the A100 either the 40GB or 80GB versions plus the A10. Also supported are a few RTX cards for graphic support like the Ampere based RTX 8000, the RTX 5000, and RTX 4000. This platform is Nvidia Certified but it’ll also support a number of other GPU accelerators from AMD, Xilinx, and Qualcomm.
Even with EPYC processors delivering 128 PCIe 4.0 lanes, there is still not enough PCIe bandwidth for all 10x potential GPUs and those 8x NVMe drive bays that also depend on the PCIe Bus for data transfers. Not to mention the OCP mezzanine card slot and that other low-profile PCIe slot on top. In order to handle all of the PCIe traffic, PCIe 4.0 switches are used, which provide queuing of data packets. Switches enable the data to be delivered faster by bifurcating the PCIe lanes. Expanding some and reducing others depending on the current workload. All of which takes microseconds, nanoseconds? resulting in superfast performance.
Revision A00 enables the Gigabyte G492-Z51 GPU server to support 3rd generation AMD EPYC CPUs. A definite performance increase over the initial Rev.100 configuration that only supported 1st and 2nd generation AMD EPYC processors. Of course, now we also get compatibility with another CPU architecture—Milan-X which still features the 7 nanometer Zen 3 cores as Milan without the X. Milan-X offers true 3D Die stacking. When Moore’s Law seemed to be at the end of its proverbial rope just a few years ago, you may have been wondering how these manufacturers were going to continue busting out significant CPU upgrades. This 3-D stacking technology appears to just stack those smaller units on top of each other for greater density and performance. Skyrise buildings worked for inner city development with skyrocketing real estate prices, why not computing too?
Sticking to the newest processors, you can install either the Milan processors with up to 64 physical cores and 128 virtual threads, PCIe 4.0, and an L3 Cache limit of 256 MegaBytes, each. Or you can go Milan X with the same core count, and PCIe 4.0 support but now with up to 768 MegaBytes of L3 Cache for improved performance on highly technical workloads. Think Electronic Design Automation, Computational Fluid Dynamics, and Finite Element Analysis software and solutions. Apparently, these chips are also great for applications in simulation and computer rendering for 3D models so you have to be thinking great for game developers too. You do have to wonder if this 3D V-Cache technology can be applied in other ways. Like maybe 3D v-Core Technology where they attach another level of 64 cores on top of the chiplets for 128 physical cores and 256 virtual threads, per CPU.
Providing the memory for these CPUs are 3DS Registered and Load-Reduced memory modules. Yes, you can go with standard RDIMMs and LRDIMMs but for the highest density on your memory, 3DS modules offer of up to 256GB, each. With 256GB modules installed in all slots, that’s a maximum capacity of up to 8TB of memory. Standard RDIMMs and LRDIMMs will only provide half that with a supported memory module capacity of only 128GB for a total capacity of up to 4TB with all slots loaded.
For a PCIe GPU-based, high-performance powerhouse, the Gigabyte G492-Z51 GPU Server certainly ticks all the boxes! Support for dual 3rd gen AMD processors, 10x top-of-the-line GPUs, PCIe 4.0 compatibility, support for up to 8TB of memory, and a few PCIe slots loft over for some high-speed I/O traffic. That’s a WIN.
You can find this system at IT Creations along with Gigabyte’s other 2U and even 1U GPU server options. And we can also build it out to your specifications, and test it, so you can drop it into your network right after you receive it. Check out our website for parts and components too!
