Nvidia teams up with Microsoft to put neural shading into DirectX, giving devs access to AI tensor cores

Nvidia and Microsoft announced on Thursday that they would be adding neural shading support to the Microsoft DirectX preview this April. Neural shading will use cooperative vectors and Nvidia's Tensor cores (matrix operations units) to speed up graphics rendering in games that support the technology. It will better allow for the generic use, via HLSL (high level shading language) of traditional rendering techniques alongside AI enhancements.

While real-time computer graphics and graphics processing units (GPUs) have come a long way, the graphics rendering pipeline itself has evolved slower than hardware. In particular, while Nvidia's GPUs have featured Tensor cores (primarily aimed at AI compute) for over seven years now, they have only been used so far for things like upscaling (Nvidia's DLSS), ray reconstruction (DLSS 3.5) and denoising, and frame generation (at least for DLSS 4).

This is going to change with the so-called neural rendering — a broad term that describes a real-time graphics rendering pipeline enhanced with new methods and capabilities enabled by AI. A specific subset of neural rendering focused on enhancing the shading process in graphics is called neural shading. Its main purpose is to improve the appearance of materials, lighting, shadows, and textures by integrating AI into the shading stage of the graphics pipeline. The addition of cooperative vectors — which allow small neural networks to run in different shader stages, like within a pixel shader, without monopolizing the GPU — is a key enabler for neural shading.

Cooperative vectors rely on matrix-vector multiplication, so they need specialized hardware, such as Nvidia's Tensor cores, to operate. To that end, they can potentially work on Intel's XMX hardware as long as they meet Microsoft's requirements. They may also work on AMD's RDNA 4 AI accelerators, though RDNA 3 seems more doubtful (as it lacks AI compute throughput compared to the competition). Still, Microsoft is working with AMD, Intel, Nvidia, and Qualcomm to ensure cross-vendor support for cooperative vectors over time. 

"Microsoft is adding cooperative vector support to DirectX and HLSL, starting with a preview this April," said Shawn Hargreaves, Direct3D development manager at Microsoft. "This will advance the future of graphics programming by enabling neural rendering across the gaming industry. Unlocking Tensor Cores on Nvidia RTX GPUs will allow developers to fully leverage RTX Neural Shaders for richer, more immersive experiences on Windows."

Our understanding — we've reached out to AMD, Intel, and Nvidia for additional comment but haven't heard back — is that the upcoming DirectX preview with cooperative vectors will start as an Nvidia exclusive. Once driver support is available from AMD and Intel, it should work on their GPUs as well. But we'll need to wait to find out not only whether it works, but how well it works — both in terms of image fidelity as well as performance.