Massive artificial intelligence models often surpass the physical memory capacities of individual chips, forcing teams to link multi-system hardware to achieve viable local inference. By coordinating two networked workstations with specialized engines, operators can eliminate slow CPU offloading to handle heavy enterprise workloads directly on high-speed accelerator memory. This distributed approach provides a scalable roadmap for organizations deploying cutting-edge mixture-of-experts architectures within their own infrastructure. In this setup, ConnectX-8 RDMA links two DGX Stations using tensor parallelism to fully accommodate the 464-gigabyte checkpoint on GPU memory.
Key points
Connecting two systems via ConnectX-8 RDMA allows model serving entirely without relying…: Connecting two systems via ConnectX-8 RDMA allows model serving entirely without relying on CPU weight offloading.
Station A acts as rank 0 to host an OpenAI-compatible API endpoint while Station B operat…: Station A acts as rank 0 to host an OpenAI-compatible API endpoint while Station B operates as a rank 1 worker.
Both interconnected stations process every incoming query simultaneously before Station A…: Both interconnected stations process every incoming query simultaneously before Station A returns the final completed answers.
Sources checked by our analysis pipeline: 1
Running a frontier-scale model locally means keeping all its weights in GPU memory; the largest models no longer fit on one machine. Two NVIDIA DGX Stations connected over ConnectX-8 RDMA serve one model through a single endpoint, with tensor parallelism splitting the model weights and computation across both GPUs.
The NVIDIA GLM-5.3 NVFP4 model, a quantized version of Z.ai’s mixture-of-experts (MoE) model, has 753 billion parameters, with 40 billion activated per token. Sparse activation reduces computation, but all model weights still need to remain available because each token can be routed to different experts. At about 464 GB, the checkpoint exceeds the 252 GB of HBM3e on a single GB300 GPU. This tutorial uses two DGX Stations to keep the weights in GPU memory, with CPU weight offload disabled.
Before starting, connect the two Stations with the Connect Two DGX Stations playbook.
This post shows how to serve GLM-5.3 locally across two NVIDIA DGX Stations with vLLM, send a request, and measure behavior under concurrent load. Tensor parallelism (TP=2) splits the model’s tensors across the two GB300 GPUs. Both Stations…
The debut of this new offering marks an expansion of the platform's branded ecosystem. While the name points to potential financial or asset management integration, the creators have not yet specified its actual use case. As a result, users and industry observers must wait for subsequent announcements to understand the full operational scope.
The collaboration targets enterprises seeking to deploy advanced artificial intelligence without compromising strict corporate controls and data security. By merging analytical systems with frontier intelligence models, the platform applies approved internal definitions and regulatory compliance to automated business queries. This setup allows organizations to safely scale generative tools across employee analysis, engineering workflows, and autonomous system development.
Academic and industry researchers from institutions including Carnegie Mellon, NVIDIA, and Johns Hopkins have introduced a standardized evaluation suite to measure how artificial intelligence handles flawed user inquiries. The initiative aims to push assistant models beyond blind compliance by testing whether they can diagnose an end user's true objective. This capability is critical for preventing automated systems from reinforcing misguided or counterproductive workflows.